Paul Eggert [Tue, 5 Aug 2025 22:45:35 +0000 (15:45 -0700)]
cp: port better to old limited hosts
Port better ancient platforms where OFF_T_MAX is only 2**31 - 1,
but some devices have more than that many bytes.
* src/copy-file-data.c (copy_file_data): Byte count is now
count_t, not off_t. All callers changed. Since we need to check
for overflow anyway, also check for too-small calls to fadvise.
Paul Eggert [Tue, 5 Aug 2025 21:45:18 +0000 (14:45 -0700)]
cp: refactor out data copying
* po/POTFILES.in, src/local.mk (copy_sources): Add the new file.
* src/copy.c: Move the #includes of alignalloc.h, buffer-lcm.h,
fadvise.h, full-write.h, ioblksize.h to copy-file-data.c.
(enum copy_debug_val, struct copy_debug): Move these decls to copy.h.
(punch_hole, create_hole, is_CLONENOTSUP, sparse_copy)
(write_zeros, lseek_copy, HAVE_STRUCT_STAT_ST_BLOCKS)
(enum scantype, struct scan_inference, infer_scantype):
Move to copy-file-data.c.
(copy_reg): Move the data-copying part of this function
to the new function copy_file_data in copy-file-data.c.
* src/copy-file-data.c: New file, taken from part of copy.c.
Paul Eggert [Tue, 5 Aug 2025 19:47:16 +0000 (12:47 -0700)]
cp: refactor src/copy.c
This is in preparation for splitting this large module.
* src/copy.c (sparse_copy, lseek_copy): New arg DEBUG, used to
identify copy debug info instead of using a static var. All
callers changed.
(lseek_copy, infer_scantype): New args SRC_POS and POS.
Callers changed.
(copy_file_data): New function, with contents taken from copy.
(copy_reg): Call it.
doc: NEWS: mention fold can operate on very long lines
* NEWS: Before commit fb9016d50 (fold: use fread instead of getline,
2025-08-24), fold required that the maximum line size in a file fit into
memory. Document that this is no longer the case.
fold: fix out of bounds write with zero width characters
* src/fold.c (fold_file): Prefer putchar ('\n') to copying characters.
If we do not have room in the output buffer print it since it is not a
full line of text.
* tests/fold/fold-zero-width.sh: New test case.
* tests/local.mk (all_tests): Add it.
cksum,wc: support disabling hardware acceleration at runtime
This is useful to give better test coverage at least,
and may be useful for users to tune their environment.
* bootstrap.conf: Reference the cpu-supports gnulib module.
* src/cksum.c: Use cpu_supports() rather than __builtin_cpu_supports().
* src/wc.c: Likewise.
* tests/cksum/cksum.sh: Adjust to testing all implementations.
* tests/wc/wc-cpu.sh: A new test to do likewise.
* tests/local.mk: Reference the new wc test.
maint: basenc: refactor all encodings to use finalize
Finalize was required for base58, but it's a more general mechanism
which simplifies the logic for all encodings
* src/basenc.c (do_decode): Always call base_decode_ctx_finalize(),
rather than the awkward double loop at end of buffer.
* tests/basenc/basenc.pl: Add basenc finalization tests.
maint: cleanup libraries unnecessarily added to fold
* src/local.mk (src_fold_LDADD): Remove $(MBRTOWC_LIB) since it is
already added to LDADD. Remove $(LIBC32CONV) and $(LIBUNISTRING) which
were for an uncommitted patch which used Gnulib's mbfile module.
nohup: avoid FORTIFY runtime failure on Bionic libc
The meaning of non-file permission umask bits is implementation defined.
On Bionic libc, attempting to set them triggers a FORTIFY runtime check.
$ nohup true
FORTIFY: umask: called with invalid mask -601
Aborted nohup true
* src/nohup.c: (main) Avoid setting non-permission bits in umask.
Just clear the umask to ensure we create nohup.out with u+rw,
as we restore the original umask before the exec().
* tests/misc/nohup.sh: Add a test case.
* NEWS: Mention the bug fix.
basenc: ensure partial padding with newlines induces an error
* src/basenc.c (has_padding): A more robust helper to
identify padding in the presence of trailing newlines.
(do_decode): Use has_padding() rather than just looking
at the last character.
* tests/basenc/base64.pl: Fully test commit v9.4-53-g378dc38f4
by ensuring partially padded data is diagnosed.
baddecode9 is the case fixed in this commit.
* NEWS: Mention the bug fix.
This implicitly tests the previous commit to
adjust how date(1) handles multiple named format options.
Currrently it tests the following are supported:
chown --quiet --silent
date --rfc-email --rfc-822 --rfc-2822
date --uct --utc --universal
dircolors --bourne-shell --sh
dircolors --csh --c-shell
head --quiet --silent
* tests/misc/option-aliases.sh: A new test to ensure all
option aliases supported by a command are supported.
* Reference the new test.
tests: fold: check if multi-byte spaces are treated as blank
This avoids a test failure on FreeBSD 14, MacOS 15, and musl.
Fix suggested by Pádraig Brady in:
<https://bugs.gnu.org/79301#32>.
* tests/fold/fold-spaces.sh (isblank): New function. Only run the tests
if the character is treated as blank.
Fixes https://bugs.gnu.org/79301
To make the interface more concise and consistent,
while being backwards compatible.
* src/digest.c (main): Continue to support -a "sha###" but
also support -a "sha2" and treat it like "sha3", except in...
(output_file): ... maintain the legacy tags for better compatability.
* doc/coreutils.texi (cksum invocation): Document the -a sha2 option.
* tests/cksum/cksum-base64.pl: Adjust as per modified --help.
* tests/cksum/cksum-c.sh: Add new supported SHA2-### tagged variant.
* NEWS: Mention the new feature.
Collin Funk [Sun, 31 Aug 2025 23:56:08 +0000 (16:56 -0700)]
cksum: add support for SHA-3
* src/digest.c: Include sha3.h.
(BLAKE2B_MAX_LEN): Rename to
DIGEST_MAX_LEN since it is also used for SHA-3.
(sha3_sum_stream): New function.
(enum Algorithm, algorithm_args, algorithm_args, algorithm_types)
algorithm_tags, algorithm_bits, cksumfns, cksum_output_fns): Add entries
for SHA-3.
(usage): Mention that SHA-3 is supported. Mention requirements for
--length with SHA-3.
(split_3): Use DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Determine the
length of the digest for SHA-3. Make sure it is 224, 256, 384, or 512.
(digest_file): Set the digest length in bytes. Use DIGEST_MAX_LEN
instead of BLAKE2B_MAX_LEN. Always append the digest length to SHA3 in
the output.
(main): Allow the use of --length with 'cksum -a sha3'. Use
DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Make sure it is 224, 256,
384, or 512.
* tests/cksum/cksum-base64.pl (@pairs): Add expected sha3 output.
(fmt): Modify the output to use SHA3-512 since that is the default.
(@Tests): Modify arguments for sha3 to use --length=512.
* tests/cksum/cksum-sha3.sh: New test, based on tests/cksum/b2sum.sh.
* tests/local.mk (all_tests): Add the test.
* bootstrap.conf: Add crypto/sha3.
* gnulib: Update to latest commit.
* NEWS: Mention the change.
* doc/coreutils.texi (cksum general options): Mention sha3 as a
supported argument to the -a option. Mention that 'cksum -a sha3'
supports the --length option. Mention that SHA-3 is considered secure.
fold: check that characters are not non-breaking spaces when -s is used
NetBSD 10 and Solaris 11.4 treat non-breaking spaces as blank
characters unlike glibc.
* src/system.h: Include uchar.h.
(c32isnbspace): New function based on iswnbspace from src/wc.c.
* src/fold.c (fold_file): Use it.
* src/wc.c (iswnbspace): Remove function.
(maybe_c32isnbspace): New function.
(wc, main): Use it.
Fixes https://bugs.gnu.org/79300
maint: prefer issymlink to readlink with a small buffer
* bootstrap.conf (gnulib_modules): Add issymlink and issymlinkat.
* src/copy.c: Include issymlink.h.
(copy_reg): Use issymlink instead of readlinkat.
* src/rmdir.c: Include issymlink.h.
(main): Use issymlink instead of readlink.
* src/tail.c: Include issymlink.h.
(recheck, any_symlinks): Use issymlink instead of readlink.
* src/test.c: Include issymlink.h.
(unary_operator): Use issymlink instead of readlink.
seq: be more accurate with large integer start values
* src/seq.c (main): Avoid possibly innacurate conversion
to long double, for all digit start values.
* tests/seq/seq-long-double.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes https://bugs.gnu.org/79369
Pádraig Brady [Sun, 31 Aug 2025 13:29:56 +0000 (14:29 +0100)]
ls: fix alignment with locale formatted --size
Fix allocated size alignment in locales with multi-byte grouping chars.
Tested with: LC_ALL=sv_SE.utf8 ls --size --block-size=\'k
* src/ls.c (print_file_name_and_frills): Don't rely on
printf("%*s", width, string) to pad multi-byte strings appropriately.
Instead work out the padding required and use:
printf("%*s%s", padding, "", string) to pad multi-byte appropriately.
* tests/ls/block-size.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/79347
Pádraig Brady [Sat, 30 Aug 2025 11:08:24 +0000 (12:08 +0100)]
b2sum: --length: fix upper bound check
* src/digest.c (main): Don't saturate -l to BLAKE2B_MAX_LEN,
so that the subsequent bounds check is performed.
* tests/cksum/b2sum.sh: Add a test case.
* NEWS: Mention the fix introduced in commit v9.5-71-gf2c84fe63
Collin Funk [Thu, 28 Aug 2025 01:33:37 +0000 (18:33 -0700)]
fold: fix handling of invalid multi-byte characters
* src/fold.c (fold_file): Continue the loop when we have buffered bytes
but nothing left to read from the file.
(adjust_column): Don't assume that the character is printable.
* tests/fold/fold-characters.sh: Add a new test case.
(bad_unicode): New function.
Collin Funk [Tue, 26 Aug 2025 06:15:21 +0000 (23:15 -0700)]
fold: don't truncate multibyte characters at the end of the buffer
* src/fold.c (fold_file): Replace invalid characters with the original
byte read. Copy multibyte sequences that may not yet be read to the
start of the buffer before reading more bytes.
* tests/fold/fold-characters.sh: Add a test case.
Collin Funk [Sun, 24 Aug 2025 19:05:41 +0000 (12:05 -0700)]
fold: use fread instead of getline
* src/fold.c: Include ioblksize.h.
(fold_file): Use two IO_BUFSIZE-sized buffers. Use fread instead of
getline. Check for if we reached the end of file.
Pádraig Brady [Sun, 24 Aug 2025 10:41:01 +0000 (11:41 +0100)]
tests: nproc: fix false failure on some systems
* tests/nproc/nproc-quota.sh: Also simulate sched_getscheduler()
as this will not be called on older or non linux, or
may return ENOSYS on Alpine.
Fixes https://bugs.gnu.org/79299
Collin Funk [Thu, 21 Aug 2025 04:13:52 +0000 (21:13 -0700)]
fold: add the --characters option
* src/fold.c: Include mcel.h.
(count_bytes): Remove variable.
(counting_mode, last_character_width): New variables.
(shortopts, long_options): Add the option.
(adjust_column): If --characters is in used account for number of
characters instead of their width.
(fold_file): Use getline and iterate over the result with mcel functions
to handle multibyte characters.
(main): Check for the option.
* src/local.mk (src_fold_LDADD): Add $(LIBC32CONV), $(LIBUNISTRING), and
$(MBRTOWC_LIB).
* tests/fold/fold-characters.sh: New file.
* tests/fold/fold-spaces.sh: New file.
* tests/fold/fold-nbsp.sh: New file.
* tests/local.mk (all_tests): Add the tests.
* NEWS: Mention the new option.
* doc/coreutils.texi (fold invocation): Likewise.
Paul Eggert [Sat, 23 Aug 2025 00:34:04 +0000 (17:34 -0700)]
cp: improve hole handling on squashfs
Better fix for problem reported by Jeremy Allison
<https://bugs.gnu.org/79267>.
* src/copy.c (struct scan_inference): New type, replacing
union scan_inference. All uses changed. This is so
infer_scantype can report the first hole's offset when known.
(lseek_copy): 5th arg is now struct scan_inference const *,
not just off_t. All uses changed.
(infer_scantype): If SEEK_SET+SEEK_HOLE do not find a hole,
fall back on ZERO_SCANTYPE.
Paul Eggert [Fri, 22 Aug 2025 17:37:50 +0000 (10:37 -0700)]
cp: go back to copy_file_range optimization
This reverts part of the previous change.
* src/copy.c (lseek_copy): When calling sparse_copy, do not
ask it to scan for zeros unless --sparse=always, so that it
can use copy_file_range which can be far more efficient.
Paul Eggert [Thu, 21 Aug 2025 22:02:10 +0000 (16:02 -0600)]
cp: always punch holes that we make
Problem reported by Jeremy Allison <https://bugs.gnu.org/79267>.
* src/copy.c (create_hole, sparse_copy): Omit arg PUNCH_HOLES,
as we always punch holes now. All uses changed.
(lseek_copy): When calling sparse_copy, scan for holes when
sparse_mode == SPARSE_AUTO, as that means we are making holes.
(copy_reg): Always punch any hole made at end.
Collin Funk [Tue, 19 Aug 2025 18:39:04 +0000 (11:39 -0700)]
maint: prefer https to http
* doc/sort-version.texi (Other version/natural sort implementations):
Use https in documentation link.
* tests/chmod/symlinks.sh: Use https in license text.
Pádraig Brady [Tue, 19 Aug 2025 15:49:22 +0000 (16:49 +0100)]
nproc: honor cgroup v2 CPU quotas
* NEWS: Mention the new feature.
* doc/coreutils.texi (nproc invocation): Mention that
cgroup CPU quotas can limit the reported number.
* gnulib: Update to new nproc gnulib implementation:
https://github.com/coreutils/gnulib/commit/9b07115f4a
Collin Funk [Fri, 15 Aug 2025 02:43:52 +0000 (19:43 -0700)]
maint: avoid syntax-check failure from previous commit
* src/tsort.c: Don't include long-options.h since the previous commit
removed the call to parse_gnu_standard_options_only. This avoids a
sc_prohibit_long_options_without_use syntax-check failure.
Paul Eggert [Thu, 14 Aug 2025 16:17:51 +0000 (09:17 -0700)]
tsort: add do-nothing -w option
This is for conformance to POSIX.1-2024
* src/tsort.c: Include getopt.h.
(main): Accept and ignore -w. Do not bother altering
the usage message, as the option is useless.
* tests/misc/tsort.pl (cycle-3): New test.
Collin Funk [Sun, 10 Aug 2025 00:53:29 +0000 (17:53 -0700)]
realpath: support the -E option required by POSIX
* src/realpath.c (longopts): Add the option.
(main): Likewise.
(usage): Add the option to the --help message.
* tests/misc/realpath.sh: Add a simple test.
* doc/coreutils.texi (realpath invocation): Mention the new option.
* NEWS: Likewise.
Pádraig Brady [Sun, 10 Aug 2025 15:15:37 +0000 (16:15 +0100)]
doc: --base58: add example usage to info
* doc/coreutils.texi (basenc invocation): Add an example
using --base58 to generate a unique ID. This also demonstrates
compound usage of the basenc command, to convert to/from binary.
A 58 character encoding that:
- avoids visually ambiguous 0OIl characters
- uses only alphanumeric characters
Described at:
- https://tools.ietf.org/html/draft-msporny-base58-03
This implementation uses GMP (or gnulib's gmp fallback).
Performance is good in comparison to other implementations.
For example when using libgmp on an i7-5600U system,
encoding is 530 times faster, and decoding 830 times faster
than the implementation using arbitrary precision ints in cpython 3.13.
Memory use is proportional to the size of input.
Encoding benchmarks:
$ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
real 0m0.018s
./configure --quiet --without-libgmp && make -j $(nproc)
$ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
real 0m3.431s
# dnf install python3-base58
$ time yes | head -c65535 | base58 >file.enc # cpython 3.13
real 0m9.700s
Decoding benchmarks:
$ time src/basenc --base58 -d <file.enc >/dev/null
real 0m0.010s
$ ./configure --without-libgmp && make # gnulib gmp
$ time src/basenc --base58 -d <file.enc >/dev/null
real 0m0.145s
$ time base58 -d <file.enc >/dev/null # cpython 3.13
real 0m8.302s
* src/basenc.c (base_decode_ctx_finalize, base_encode_ctx_init,
base_encode_ctx, base_encode_ctx_finalize): New functions to
provide more general processing functionality.
(base58_{de,en}code_ctx{_init,,_finalize}): New functions to
accumulate all input before calling ...
(base58_{de,en}code): ... the GMP based encoding/decoding routines.
(do_encode, do_decode): Call the ctx variants if enabled.
* doc/coreutils.texi (basenc invocation): Describe the new option,
and indicate the main use case being interactive user use.
* src/local.mk: Link basenc with GMP.
* tests/basenc/basenc.pl: Add test cases.
* NEWS: Mention the new feature.
Pádraig Brady [Thu, 7 Aug 2025 19:39:24 +0000 (20:39 +0100)]
basenc: fix stripping of '=' chars in some encodings
* src/basenc.c (do_decode): With -i ensure we strip '=' chars
if there is no padding for the chosen encoding.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the bug fix.
Collin Funk [Sat, 9 Aug 2025 03:32:18 +0000 (20:32 -0700)]
maint: prefer attribute.h in .c files
* src/basenc.c (base16_encode, z85_encoding, do_decode): Use
ATTRIBUTE_NONSTRING instead of ATTRIBUTE_NONSTRING.
* src/basenc.c (sc_prohibit-_gl-attributes): New rule for
'make syntax-check'.
Paul Eggert [Thu, 7 Aug 2025 22:42:23 +0000 (15:42 -0700)]
cp: omit some needless lseek calls
The sparse code sometimes issued multiple lseeks against the
same file without doing anything in betwee. Optimize them away
by keeping track of the last hole output, in a way that
crosses the sparse_copy function call boundary.
* src/copy.c (sparse_copy): New arg hole_size, replacing old args
scan_holes and last_write_made_hole. All callers changed.
(sparse_copy, lseek_copy): Do not create hole at
end; let the caller deal with it. All callers changed.
(lseek_copy): New args hole_size and total_n_read. Caller changed.
(copy_reg): Create hole at end for both lseek_copy and sparse_copy.
Paul Eggert [Wed, 6 Aug 2025 23:25:01 +0000 (16:25 -0700)]
cp: --sparse=always was missing some holes
The sparse code assumed that st_blksize was the minimum hole size.
However, st_blksize is an optimum I/O buffer size, not the file
system fundamental block size. Use ST_NBLOCKSIZE instead;
although it may underestimate the true block size that just slows
‘cp’ down a bit, without introducing bugs.
* src/copy.c (sparse_copy): Arg scan_holes replaces
the old hole_size arg. All callers changed.
(lseek_copy): Remove hole_size arg; no longer needed.
Caller changed.
Collin Funk [Wed, 6 Aug 2025 01:46:04 +0000 (18:46 -0700)]
maint: use consistent references to standard files in messages
* cfg.mk (sc_standard_outputs): Add a grep command for source files.
* src/du.c (main): Use standard input instead of stdin, standard output
instead of stdout, and standard error instead of stderr in messages.
* src/nohup.c (main): Likewise.
* src/sort.c (main): Likewise.
* src/split.c (main): Likewise.
* src/stdbuf.c (main): Likewise.
* src/wc.c (main): Likewise.
* tests/du/files0-from.pl (@Tests): Adjust test case to new messages.
* tests/sort/sort-files0-from.pl: Likewise.
* tests/wc/wc-files0-from.pl: Likewise.
'make syntax-check' complains:
src/tail.c
maint.mk: the above files include safe-read.h but don't use it
make: *** [maint.mk:737: sc_prohibit_safe_read_without_use] Error 1
The removal was missed for tail.c in recent commit d3c7072a0950.
Paul Eggert [Sun, 3 Aug 2025 22:04:01 +0000 (15:04 -0700)]
maint: prefer same-inode.h
This does not change behavior on POSIX platforms; it’s mostly to
make it clearer when we’re looking for file identity.
* src/cat.c (main):
* src/copy.c (struct dir_list, is_ancestor, copy_internal):
* src/tail.c (struct File_spec, record_open_fd, recheck)
(tail_forever_inotify, tail_file):
* src/test.c (binary_operator):
Use psame_inode, PSAME_INODE, or SAME_INODE instead of comparing
device and inode numbers by hand.
Paul Eggert [Sun, 3 Aug 2025 15:29:52 +0000 (08:29 -0700)]
tail: refactor ‘failable’
* src/tail.c (recheck, tail_file): Do not mark a file as tailable
merely because --retry is not in effect. Simplify internal logic.
This should not change behavior; it’s just for clarity and to
make the code match the comments better.
Paul Eggert [Sun, 3 Aug 2025 15:19:53 +0000 (08:19 -0700)]
tail: fix race between read and fstat
* src/tail.c (get_file_status): Remove, since after the changes
described below it would be called in just one place and it’s a
bit clearer to inline by hand.
(tail_file): Don’t call fstat after reading the file, as that
misses changes arriving between read and fstat. Instead, reuse
the fstat done before reading the file.
Paul Eggert [Sat, 2 Aug 2025 23:55:09 +0000 (16:55 -0700)]
tail: track errno more accurately
This matters only in some obscure cases hard to test for.
* src/tail.c (file_lines, pipe_lines, pipe_bytes, start_bytes)
(start_lines, tail_bytes, tail_lines, tail): New return
convention, which reports errno. All callers changed.
(recheck): Don’t lose track of errno if a regular file is
replaced by a symlink.
(get_file_status): Set errno to 0 on success.
(tail_file): Be more careful about f->errnum.
It is now -1 only if the failure was not due to
a system call failing.
Paul Eggert [Sat, 2 Aug 2025 19:58:06 +0000 (12:58 -0700)]
tail: prefer readlink to lstat+S_ISLNK
When not already calling lstat for some other reason,
prefer readlink to lstat+S_ISLNK,
as readlink does not suffer from EOVERFLOW issues.
* src/rmdir.c (main):
* src/tail.c (recheck, any_symlinks):
* src/test.c (unary_operator):
Paul Eggert [Fri, 1 Aug 2025 20:46:33 +0000 (13:46 -0700)]
tail: fix unlikely races with >=2 --pids
Also, fix commentary to talk about “nonexistent” rather than
“dead” processes, since the code looks for the former not the
latter and the difference matters for zombies.
* src/tail.c (some_writers_exist): Rename from writers_are_dead,
negate the sense, don’t have a special and counterintuitive case
for !nbpids, remove PIDs found not to exist, and avoid some
though not all unlikely races when kernels reuse PIDs.
(tail_forever): Optimize via blocking I/O even if --pid was used,
so long as all the writers no longer exist.
(tail_forever, tail_forever_inotify): Simplify the writers_dead
logic; there is no need to have a local var to track this, since
we can use pids and nbpids now.
(parse_options): Also free and clear pids if !forever.
Paul Eggert [Fri, 1 Aug 2025 14:58:08 +0000 (07:58 -0700)]
tail: prefer < 0 to == -1
* src/tail.c (valid_file_spec, recheck, writers_are_dead)
(tail_forever, check_fspec, tail_forever_inotify, tail_file)
(parse_options, main): Be a bit more systematic about checking
for sign, rather than for exact equality or inequality,
when the sign is enough. Makes the code a bit clearer
now that -2 sometimes means success.
Paul Eggert [Fri, 1 Aug 2025 01:45:12 +0000 (18:45 -0700)]
tail: record file offset more carefully
* src/tail.c (struct File_spec): New member read_pos, replacing
size, since the value was really a read position not a size.
All uses changed.
(xlseek): Move defn up.
(record_open_fd): If the read_pos (formerly) size arg is unknown,
compute it here if it is a regular file.
(file_lines): Return the resulting read pos (or -1 on failure)
instead of storing it via a pointer. Caller changed.
Simplify by using SEEK_CUR instead of SEEK_SET when that is easy.
Avoid reading the same data twice when there are not enough
lines in the file.
(pipe_lines): Return -2 on success, -1 on failure, rather than
updating a read pos via a pointer (which was weird for pipes anyway).
Caller changed.
(pipe_bytes, tail_bytes, tail_lines, tail):
Return -1 on failure, a file offset if known, and < -1 otherwise,
instead of storing a file offset via a pointer. Caller changed.
(pipe_bytes): Take initial file offset as an arg, or -1 if unknown.
(start_bytes, start_lines): Return -1 (not 1) on error, -2 (not
-1) on EOF, and do not accept pointer to read pos as an arg since
neither we nor our caller know the read pos. Callers changed.
(recheck): Do not assume a newly-opened file is at offset zero,
as this is not always true on Solaris.
(tail_forever, check_fspec): Use dump_remainder result only
on regular files, to prevent (very unlikely) overflow.
(tail_file): Remove no-longer-needed TAIL_TEST_SLEEP code.
Paul Eggert [Thu, 31 Jul 2025 19:31:58 +0000 (12:31 -0700)]
maint: prefer 'read' to 'safe_read'
In the old days, safe_read acted more like what full_read does now.
When that went away, some code that invoked safe_read should
have gone back to plain 'read' but I guess we never got around to it.
Simplify this code by going back to plain 'read'.
Use safe_read only in csplit.c, which has a signal handler
and where 'read' can therefore fail with EINTR.
Although safe_read also checks for oversize buffers,
that is better done via io_blksize.
* src/cat.c (simple_cat, cat):
* src/head.c (copy_fd, elide_tail_lines_pipe)
* src/tac.c (tac_seekable, copy_to_temp):
(elide_tail_lines_seekable, head_lines):
* src/tail.c (dump_remainder, file_lines, pipe_lines)
(pipe_bytes, start_bytes, start_lines, tail_forever_inotify):
* src/tr.c (plain_read):
Use plain 'read', not safe_read, since there is no
need to worry about signals or oversize requests.
Also, there is no longer a need to include safe-read.h.
* src/ioblksize.h: Include sys-limits.h, for SYS_BUFSIZE_MAX.
(io_blksize): Max out at SYS_BUFSIZE_MAX.
Paul Eggert [Wed, 30 Jul 2025 22:11:31 +0000 (15:11 -0700)]
tail: prefer signed types to size_t, blksize_t
* src/tail.c (struct File_spec, xwrite_stdout, file_lines)
(pipe_lines, pipe_bytes, start_bytes, any_live_files)
(tail_forever, any_remote_file, any_non_remote_file)
(any_symlinks, any_non_regular_fifo, tailable_stdin)
(tail_forever_inotify, ignore_fifo_and_pipe, main):
Prefer a signed type to size_t, if possible.
Ordinarily this is idx_t, but use int when the value
must fit in int anyway.
(file_lines): Similarly for blksize_t, which had no business
being here anyway.
(main): Check for overflow in the oddball case where ptrdiff_t is
narrower than int.
Paul Eggert [Wed, 30 Jul 2025 15:46:04 +0000 (08:46 -0700)]
tail: prefer intmax_t to uintmax_t
Signed types let us debug better, by using -fsanitize=undefined.
* doc/local.mk (doc/constants.texi):
Adjust change from macro to enum.
* src/tail.c (COPY_TO_EOF, COPY_A_BUFFER)
(DEFAULT_MAX_N_UNCHANGED_STATS_BETWEEN_OPENS):
Now enum constants, not macros.
(COPY_TO_EOF, COPY_A_BUFFER): Now negative, not positive.
(count_t): New typedef. Use it instead of uintmax_t.
(COUNT_MAX): New macro; use it instead of UINTMAX_MAX.
(struct File_spec, max_n_unchanged_stats_between_opens)
(dump_remainder, file_lines, pipe_lines, pipe_bytes)
(start_bytes, start_lines, tail_forever, check_fspec)
(tail_forever_inotify, tail_bytes, tail_lines, tail, tail_file)
(parse_obsolete_option, parse_options, main):
Prefer count_t to uintmax_t.
Paul Eggert [Wed, 30 Jul 2025 14:49:14 +0000 (07:49 -0700)]
tail: don’t output more lines than requested
* src/tail.c (file_lines): Fix an unlikely bug where ‘tail -n N’
could output more than N lines if standard input is a largish
regular file with large initial offset that starts with (say) N-1
lines after the initial offset, but grows to N+1 lines between the
fstat and read calls. In this case ‘tail -n N’ now outputs N-1
lines, not N+1; that is, it pretends the file grew after ‘tail’
read it. That is better than outputting more than N lines.
Paul Eggert [Tue, 29 Jul 2025 20:22:41 +0000 (13:22 -0700)]
tail: prettyname cleanup
* src/tail.c: Use ‘prettyname’ consistently as the identifier
for a prettified file name, as opposed to ‘pretty_filename’,
‘pretty_name’, and ‘name’. This makes the code easier to follow.
(struct File_spec): New member prettyname.
(pretty_name): Remove.
All uses of pretty_name (f) replaced by f->prettyname.
(close_fd, fremote): Accept struct File_spec, not name.
All callers changed.
(main): Initialize the new prettyname member.
This is simpler/smaller than calling pretty_name everywhere.
Paul Eggert [Tue, 29 Jul 2025 16:14:37 +0000 (09:14 -0700)]
tail: refactor SEEK_END and lines
* src/tail.c (tail_lines): Refactor to simplify the confusing
code for using SEEK_END when counting lines. The old code
had a ‘end_pos != 0’ expression that was always true.
Paul Eggert [Tue, 29 Jul 2025 15:24:58 +0000 (08:24 -0700)]
tail: refactor to skip stat call on failure
* src/tail.c (tail_bytes): New function.
(tail_bytes, tail_lines, tail): Accept struct stat pointer from
caller instead of calling fstat ourselves. All callers changed.
(tail_file): Skip a call to fstat if fstat already failed.
* tests/tail/follow-stdin.sh: Adjust to match new behavior
on failure, which omits a redundant diagnostic.
Paul Eggert [Tue, 29 Jul 2025 06:39:30 +0000 (23:39 -0700)]
tail: speed up -c N for huge N
When the user specifies -c N where 2**63 <= N, don’t give
up and use the slow method (which will exhaust memory if
the file is large). Instead, treat it as N = 2**63 - 1,
since that has equivalent effect.
* src/tail.c (tail_bytes): With -c N and large N, adjust
the code so that lseeks can still be used without
affecting correctness. Formerly the code gave up and
did a sequential pass through the whole input, which
could easily exhaust memory.
Paul Eggert [Mon, 28 Jul 2025 18:24:38 +0000 (11:24 -0700)]
tail: allow >=2**64 in traditional form
This better matches the treatment of POSIX form,
e.g., ‘tail +Nc’ is now like ‘tail -c +N’ even when N is large.
* src/tail.c: Don’t include xstrtol.h.
(parse_obsolete_option): Treat numbers greater than UINTMAX_MAX as
if they are UINTMAX_MAX. Parse the number by hand with saturating
arithmetic; nowadays that’s simpler than using xstrtoumax. There
is no need for a diagnostic now, as the error cannot happen any more.
* tests/tail/tail.pl (obs-plus-c3): New test.
Paul Eggert [Wed, 23 Jul 2025 05:05:17 +0000 (22:05 -0700)]
tail: simplify -c+ on regular files
* src/tail.c (tail_bytes): Simplify the -c+N case by treating
regular files like other files; if the lseek fails for whatever
reason, fall back on the unoptimized version instead of reporting
a fatal error.
Paul Eggert [Mon, 14 Jul 2025 23:41:05 +0000 (16:41 -0700)]
maint: don’t lseek memory objects
This affects behavior only on memory objects, which are not in Linux.
Formerly the code would use lseek on these objects,
but POSIX says the result of lseek is unspecified on them,
and in QNX lseek has undefined behavior on typed memory objects.
* src/head.c (elide_tail_bytes_file, elide_tail_lines_file):
Omit unnecessary uses of presume_input_pipe.
Improve some out-of-date comments.
(head): Do not assume a file is seekable merely because its
st_size is usable. Instead, seek only on regular files.
* src/od.c (skip): Do not seek on memory objects.
Paul Eggert [Mon, 14 Jul 2025 20:32:27 +0000 (13:32 -0700)]
wc: don’t assume opening a file puts you at start
This is not true on Solaris when opening /dev/stdin; it could be
at a nonzero file offset. Arguably Linux should do likewise.
* src/wc.c (wc): Omit last arg, and act as if it is always -1.
All uses changed.
Paul Eggert [Mon, 14 Jul 2025 18:40:33 +0000 (11:40 -0700)]
head: simplify head_bytes
* src/head.c (diagnose_read_failure): New function.
Use it when possible, to simplify callers.
(head_bytes): Use copy_fd and diagnose_copy_fd_failure instead of
reinventing the wheel.
Collin Funk [Sun, 3 Aug 2025 19:33:55 +0000 (12:33 -0700)]
readlink: improve documentation for POSIXLY_CORRECT in --help
* NEWS: Improve wording to not imply POSIXLY_CORRECT=1 readlink -q will
be verbose.
* src/readlink.c (usage): Mention the affect of POSIXLY_CORRECT on
-s (--silent), -q (--quiet), and -v (--verbose) in the help message.
(main): Remove spurious newline added by previous commit.
* doc/coreutils.texi (readlink invocation): Mention that -s (--silent)
and -q (--quiet) are not the default when POSIXLY_CORRECT is set.
projects / thirdparty / coreutils.git / log
