Note the strncat man page says that:
"src does not need to be null-terminated
if it contains n or more bytes."
And the POSIX spec says that the second (source) parameter
is an array not a string.
So I think it's incorrect for strncat to require src be a string type.
This constraint seems to be being added to the gcc builtin strncat,
as specifiying -fno-builtin also avoids the warning.
Note specifying any optimization level also avoids the warning.
* src/who.c (make_id_equals_comment): Avoid the issue by using
stpcpy + stzncpy, instead of strcpy + strncat.
This pattern is used elsewhere in who.c
Pádraig Brady [Tue, 27 Oct 2020 20:00:43 +0000 (20:00 +0000)]
maint: cleanup operation of fs-magic-compare
* src/local.mk: Ensure we map 2 hex digits to 4,
so that we don't output already handled Z3FOLD file system (0x33).
Also hide the generation command for src/fs.h.
Pádraig Brady [Tue, 27 Oct 2020 14:10:14 +0000 (14:10 +0000)]
doc: make blank lines before --help consistent
* src/basenc.c (usage): Remove extraneous blank line,
to be consistent with other utilities that have options.
* src/realpath.c: Likewise.
* src/runcon.c: Likewise.
Addresses https://bugs.gnu.org/44248
Jim Meyering [Sun, 25 Oct 2020 17:04:19 +0000 (10:04 -0700)]
maint: avoid new sort.c warning from upcoming GCC11
gcc version 11.0.0 20201025 (experimental) warns that
src/sort.c:1655:1: warning: function might be candidate for attribute \
'pure' if it is known to return normally [-Wsuggest-attribute=pure]
* src/sort.c (limfield): Mark as pure.
Pádraig Brady [Sun, 25 Oct 2020 16:40:35 +0000 (16:40 +0000)]
nl: only fail if need to output overflowed numbers
Previously we would have failed immediately upon internal overflow,
which didn't output the full line being processed, and assumed
there would be another numbered line.
* src/nl.c (line_no_overflow): A new global to track overflow.
(print_lineno): Only fail if about to output an overflowed number.
(reset_lineno): A new function to refactor resetting of the number,
and which also clears line_no_overflow.
* tests/misc/nl.sh: Add a test case.
Pádraig Brady [Sun, 25 Oct 2020 13:10:32 +0000 (13:10 +0000)]
maint: sync help2man to latest version
* man/help2man: sync to changes from version 1.47.16.
Note this doesn't materially change the generated man pages.
Addresses https://bugs.gnu.org/44105
Paul Eggert [Mon, 19 Oct 2020 17:47:32 +0000 (10:47 -0700)]
build: update gnulib submodule to latest
* gl/lib/randperm.c, src/cp-hash.c, src/ls.c, src/sort.c, src/tail.c:
Change all instaces of hash_delete to hash_remove to accommodate
change to Gnulib API.
Grigorii Sokolik [Sat, 17 Oct 2020 15:54:17 +0000 (16:54 +0100)]
maint: remove already handled FIXME in tail.c
* src/tail.c: Remove FIXME to follow a file name in a recreated
directory. The comment was added in commit v8.5-191-g61b77891c
while the fix (albeit not using inotify) was added in
commit v8.27-21-gba5fe2d4b
* src/stat.c (usage): Replace a mistaken semicolon with a colon,
and replace mistaken backticks with single quotes. Also reorder
some words, for clarity.
Fixes https://bugs.gnu.org/43707
Pádraig Brady [Wed, 12 Aug 2020 17:40:04 +0000 (18:40 +0100)]
doc: clarify timeout --foreground description
* doc/coreutils.texi (timeout invocation): Avoid any implication
that `timeout --foreground` could be used to retroactively
timeout commands not already invoked by timeout(1).
Fixes bug https://bugs.gnu.org/42831
csplit: fix regex suppression with specific match count
* src/csplit.c (process_regexp): Process the line suppression
in all invocations so that the last match is suppressed.
Previously with a non infinite match count,
the last regex pattern was not suppressed.
* NEWS: Mention the bug fix.
* tests/misc/csplit-suppress-matched.pl: Add a test case.
Fixes https://bugs.gnu.org/42764
* doc/coreutils.texi (timeout invocation): Document that the the
duration of --kill-after=DURATION begins when sending the initial
signal. Also mention that -k does not have any effect if timeout's
duration is 0.
Paul Eggert [Sun, 19 Jul 2020 01:45:06 +0000 (18:45 -0700)]
factor: port to --without-libgmp
* src/factor.c (mp_factor_using_division): Use mpz_fdiv_q_2exp
instead of its no-longer-documented mpz_div_2exp alias.
(print_factors): Use mpz_out_str instead of gmp_printf.
Paul Eggert [Fri, 10 Jul 2020 22:54:51 +0000 (15:54 -0700)]
build: be less aggressive about -fanalyzer
* configure.ac: Don’t enable -fanalyzer unless configured with the
new --enable-gcc-warnings=expensive option. See thread at:
https://lists.gnu.org/r/coreutils/2020-07/msg00011.html
Paul Eggert [Wed, 8 Jul 2020 15:05:06 +0000 (08:05 -0700)]
factor: treat ' +bignum' like non-bignum
* src/factor.c (strto2uintmax): Instead of here ...
(print_factors): ... skip spaces and '+' here, so that
bignums are treated like non-bignums.
* tests/misc/factor.pl (bug-gmp-plus_2_sup_128_plus_1): New test.
Paul Eggert [Tue, 7 Jul 2020 17:39:10 +0000 (10:39 -0700)]
maint: use Gnulib libgmp module
This lets use assume multiple-precision arithmetic on all
platforms, simplifying the code.
* bootstrap.conf (gnulib_modules): Add libgmp.
* configure.ac: Don’t call cu_GMP, as this is now done by Gnulib.
* m4/gmp.m4: Remove.
* src/expr.c, src/factor.c: Use gmp.h unconditionally.
* src/factor.c: Use the simpler ‘#ifndef mpz_inits’ to
determine whether there is an mpz_inits macro.
Pádraig Brady [Fri, 26 Jun 2020 18:57:09 +0000 (19:57 +0100)]
tests: fix false failure with valgrind and reflink
* tests/cp/fiemap-FMR.sh: Avoid FICLONE ioctl,
which would avoid the point of the test (fiemap testing).
Also it avoids a valgrind bug with this ioctl:
https://bugs.kde.org/show_bug.cgi?id=397605
Paul Eggert [Thu, 25 Jun 2020 23:31:44 +0000 (16:31 -0700)]
cp: use SEEK_DATA/SEEK_HOLE if available
If it works, prefer lseek with SEEK_DATA and SEEK_HOLE to FIEMAP,
as lseek is simpler and more portable (will be in next POSIX).
Problem reported in 2011 by Jeff Liu (Bug#8061).
* NEWS: Mention this.
* src/copy.c (lseek_copy) [SEEK_HOLE]: New function.
(enum scantype): New constants ERROR_SCANTYPE, LSEEK_SCANTYPE.
(union scan_inference): New type.
(infer_scantype): Last arg is now union scan_inference *,
not struct extent_scan *. All callers changed.
Prefer SEEK_HOLE to FIEMAP if both work, since
SEEK_HOLE is simpler and more portable.
(copy_reg): Do the fdadvise after initial scan, in case the scan
fails. Report an error if the initial scan fails.
(copy_reg) [SEEK_HOLE]: Use lseek_copy if scantype says so.
Paul Eggert [Wed, 24 Jun 2020 02:18:04 +0000 (19:18 -0700)]
cp: refactor extent_copy
* src/copy.c (extent_copy): New arg SCAN, replacing
REQUIRE_NORMAL_COPY. All callers changed.
(enum scantype): New type.
(infer_scantype): Rename from is_probably_sparse and return
the new type. Add args FD and SCAN. All callers changed.
Bernhard Voelker [Fri, 19 Jun 2020 18:33:55 +0000 (20:33 +0200)]
stat,tail: add support for the VBOXSF file system
* src/stat.c (human_fstype): Add case for the 'vboxsf' file system type
which is used for VirtualBox Shared Folders mounted in VirtualBox guest
VMs.
* NEWS: Mention the Improvement.
Fixes https://bugs.gnu.org/41935
doc: timeout: improve documentation of the exit status
* doc/coreutils.texi (timeout invocation): Document that the exit
status is 137 when the KILL signal is used, regardless of whether that
signal is sent to COMMAND or timeout.
* src/timeout.c (usage): Likewise. Also split out and expand
on the possible exit status values to a separate table.
Paul Eggert [Mon, 1 Jun 2020 22:53:27 +0000 (15:53 -0700)]
maint: use getrandom, not getentropy
This makes for one Gnulib module less, and at runtime there’s
typically just one getrandom syscall instead of several for large
nonces.
* gl/lib/randread.c: Include sys/random.h instead of sys/time.h
and unistd.h.
(get_nonce): Use getrandom, not getentropy.
* gl/modules/randread (Depends-on):
Depend on getrandom, not getentropy.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Say "getrandom" rather than "getentropy" in (unlikely) diagnostic.
Paul Eggert [Mon, 1 Jun 2020 05:21:34 +0000 (22:21 -0700)]
maint: use getentropy and new tempname modules
Update gnulib submodule to latest and use its new features.
Gnulib’s new getentropy module means coreutils can now assume
getentropy instead of approximating it, badly in some cases.
Gnulib’s improvements to the tempname module mean coreutils no
longer needs to maintain private patches.
* bootstrap.conf (gnulib_modules): Remove gettimeofday.
* gl/lib/randread.c (NAME_OF_NONCE_DEVICE): Remove.
(get_nonce): Return success indicator. Remove bytes_bound arg.
All callers changed. Rewrite by using getentropy instead of
reading the nonce device and falling back on gettimeofday.
Fail if getentropy fails.
(randread_new): Return NULL (setting errno) if get_nonce fails.
All callers changed.
* gl/lib/tempname.c.diff, gl/lib/tempname.h.diff:
* gl/modules/tempname.diff: Remove.
* gl/modules/randread (Depends-on):
Depend on getentropy, not gettimeofday.
* src/ptx.c (swallow_file_in_memory):
* src/shuf.c (read_input):
Adjust to read_file changes in Gnulib.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Diagnose the new form of randread_new failures: randread_new can
fail now when !random_source, meaning getentropy failed.
Bernhard Voelker [Sat, 23 May 2020 00:46:22 +0000 (02:46 +0200)]
tests: fix removed-directory test
The previous attempt to skip that test on NFS (commit 4181fc518362)
made the test fail; it introduced two problems:
a) In the good case, i.e., when the subshell returns with exit status 0,
the test ran into framework_failure_.
b) As the subshell also runs with 'set -x', the later comparison of
/dev/null with 'err' would fail.
* tests/ls/removed-directory.sh: Revert to the style without subshell,
and add 'test -d .' to verify that 'ls' can read the removed dir.
Paul Eggert [Thu, 21 May 2020 00:31:18 +0000 (17:31 -0700)]
date: document +%-N change
Suggested by Kamil Dudka in:
https://lists.gnu.org/r/bug-gnulib/2020-05/msg00205.html
* NEWS: Mention the change for coreutils 8.23.
* doc/coreutils.texi (Padding and other flags):
Document it.
Paul Eggert [Wed, 20 May 2020 23:40:26 +0000 (16:40 -0700)]
maint: omit unnecessary pragmas and fix tsort.c
* src/chown-core.c, src/comm.c:
* src/tsort.c (record_relation):
Remove GCC 10 pragmas that are not needed in GCC 10.1.0 (the first
public GCC 10 release) and that in some cases cause diagnostics
with GCC 10.1.0. The tsort.c change fixes a bug that was
inadvertantly introduced when these pragmas were added.
Pádraig Brady [Sun, 10 May 2020 13:22:06 +0000 (14:22 +0100)]
maint: avoid warnings from GCC's -fanalyzer
* src/env.c (build_argv): Add an assert() to avoid:
warning: use of NULL 'n' where non-null expected
[CWE-690] [-Wanalyzer-null-argument]
note: argument 1 of 'getenv' must be non-null
* src/dd.c (alloc_ibuf): Don't discard the allocated pointer, to avoid:
[CWE-401] [-Wanalyzer-malloc-leak]
(alloc_obuf): Likewise.
(cleanup): Deallocate the now tracked buffers which
also avoids "possibly lost" warnings from valgrind.
* src/tsort.c (search_item): Add asserts to avoid:
[CWE-690] [-Wanalyzer-null-dereference]
(record_relation): An assert doesn't suffice here,
so disable the warning for this function.
* src/comm.c: Suppress the following false positive for the whole file:
[CWE-457] [-Wanalyzer-use-of-uninitialized-value]
* src/chown-core.c: Suppress the following false positive for the file:
[CWE-415] [-Wanalyzer-double-free]
Jason Kim [Sun, 19 Apr 2020 08:40:16 +0000 (01:40 -0700)]
ls: allow --classify to be ignored for non tty output
Have the `ls` `--classify` option take an optional argument for when to
classify ("always", "auto", "never"), just like the optional argument
for `--color`. When the optional argument is not specified, default to
"always" for backwards compatibility.
* src/ls.c (usage): Update help text.
(decode_switches): Support an optional argument for --classify.
* tests/ls/classify.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Paul Eggert [Sat, 7 Mar 2020 18:29:51 +0000 (10:29 -0800)]
ls: improve removed-directory test
* tests/ls/removed-directory.sh: Remove host_triplet test.
Skip this test if one cannot remove the working directory.
From a suggestion by Bernhard Voelker (Bug#39929).
Paul Eggert [Fri, 6 Mar 2020 01:25:29 +0000 (17:25 -0800)]
ls: restore 8.31 behavior on removed directories
* NEWS: Mention this.
* src/ls.c: Do not include <sys/sycall.h>
(print_dir): Don't worry about whether the directory is removed.
* tests/ls/removed-directory.sh: Adjust to match new (i.e., old)
behavior.
Pádraig Brady [Wed, 4 Mar 2020 15:56:52 +0000 (15:56 +0000)]
tests: don't rely on system env(1) being present
* tests/misc/env-S.pl: `env -i env` will call the system env
due to the path being cleared, so pass the absolute path
of our env binary under test to avoid that. This was seen
to be an issue on Guix where /usr/bin/env was not available.
Pádraig Brady [Wed, 4 Mar 2020 12:24:49 +0000 (12:24 +0000)]
basenc: avoid undefined behaviour in z85 processing
* src/basenc.c (z85_decode_ctx_init): Ensure we're working
with unsigned, as otherwise ubsan triggers with:
src/basenc.c:767:18: runtime error: signed integer overflow:
43 * 52200625 cannot be represented in type 'int'
(z85_encode): Likewise to avoid the usban error:
src/basenc.c:630:26: runtime error:
left shift of 134 by 24 places cannot be represented in type 'int'
Pádraig Brady [Sun, 1 Mar 2020 22:28:29 +0000 (22:28 +0000)]
tests: avoid a false failure on OpenIndiana 11
* tests/misc/timeout-parameters.sh: Split the large timeout
handling to ...
* tests/misc/timeout-large-parameters.sh: ... here, so that
the 3 second delay is contained in its own test, and if
the test is skipped due invalid handling within timeout(1),
it will be more apparent.
Also adjust the check so we skip whenever the kernel timer
fires immediately, to handle the buggy OpenIndiana 11 kernel also.
Reported by Bruno Haible.
Pádraig Brady [Sun, 1 Mar 2020 21:49:16 +0000 (21:49 +0000)]
tests: use bash in some scripts to avoid false failures
* init.cfg (require_bash_as_SHELL_): A new function to replace
SHELL for the current test, with bash if available.
This is useful on OpenIndiana 11 where /bin/sh was seen
to have races in handling of SIGPIPE.
* tests/misc/seq-epipe.sh: Use the new function to enforce bash.
* tests/misc/env-signal-handler.sh: Likewise.
Reported by Bruno Haible
Pádraig Brady [Sun, 1 Mar 2020 12:36:35 +0000 (12:36 +0000)]
tests: improve test coverage for ls stat checks
* tests/ls/stat-free-color.sh: Check for the availability
of various stat calls individually, and add statx() and fstatat64()
to the list to check. Fix the stat counting logic to
ignore lines like "+++ exited with 0 +++".
* tests/ls/stat-free-symlinks.sh: Check syscalls other than stat().
Bruno Haible [Sun, 1 Mar 2020 02:03:59 +0000 (03:03 +0100)]
tests: enable 4 more tests to be executed on FreeBSD
* init.cfg (gcc_shared_libs_): New variable.
(gcc_shared_): Use it, instead of hardcoding -ldl.
(require_gcc_shared_): Determine the suitable value
for gcc_shared_libs_.
Pádraig Brady [Sat, 29 Feb 2020 20:23:44 +0000 (20:23 +0000)]
tests: avoid false failures on darwin 19.2.0
With these adjustments, all tests pass on macOS Catalina.
* tests/dd/sparse.sh: Adjust so that systems like apfs that
don't create holes < 16 MiB do not fail erroneously.
* tests/touch/trailing-slash.sh: Darwin was seen to dereference
symlinks to files when given a trailing slash, so avoid
that particular case.
* configure.ac: Reenable distribution of gzip-compressed
tarballs, for Guix bootstrapping reasons as discussed at:
https://lists.gnu.org/r/coreutils/2020-02/msg00042.html
* THANKS.in: Remove me, as now a committer.
* NEWS (Build-related): Mention this.
Colin Watson [Tue, 11 Feb 2020 10:45:46 +0000 (10:45 +0000)]
ls: issue error message on removed directory
If the current directory has been removed, then "ls" confusingly
produced no output and no error message, indistinguishable from
running on an empty directory.
* src/ls.c (print_dir): Report ENOENT on GNU/Linux if readdir
finds no directory entries at all, not even "." or "..",
and a recheck with the getdents syscall returns ENOENT.
We recheck with getdents() as POSIX states that
"The directory entries for dot and dot-dot are optional".
* tests/ls/removed-directory.sh: New file.
* tests/local.mk (all_tests): Add new test.
* NEWS: Mention the change in behavior.
Reported by Owen Thomas.
Pádraig Brady [Tue, 25 Feb 2020 13:30:39 +0000 (13:30 +0000)]
b2sum: sync better with upstream
* src/blake2/blake2-impl.h: Sync load16() implementation,
which doesn't change code generation.
Also leverage (builtin) memcpy to more efficiently
move data on little endian systems,
giving a 2% win with GCC 9.2.1 on an i3-2310M.
Pádraig Brady [Tue, 25 Feb 2020 11:33:04 +0000 (11:33 +0000)]
factor: sync longlong.h adjustments from upstream
* src/longlong.h: Sync changes from:
https://gmplib.org/repo/gmp/log/tip/longlong.h
mips64: Provide r6 asm code as default expression yields.
arm32: Define sub_ddmmss separately for non-thumb (no rsc instruction).
powerpc: Add "CLOBBER" descriptions for some registers.
x86: Fix criterion for when to use mulx in umul_ppmm.
Pádraig Brady [Sun, 23 Feb 2020 13:20:08 +0000 (13:20 +0000)]
uniq: avoid strcoll() to improve performance and consistency
strcoll() is only significant to uniq(1) if it returns 0,
and it generally only does so with buggy locales or mismatched
locales and data. Some systems may have strcoll()
return 0 for equivalent normalized unicode forms,
but for consistency across platforms strcoll() is avoided.
The various cases are defined in the new test.
This is consistent with newer POSIX standards as discussed at:
https://www.austingroupbugs.net/view.php?id=963
* src/uniq.c: s/xstrcoll/memcmp/.
* tests/local.mk: Reference the new test.
* tests/misc/uniq-collate.sh: Add a new test.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/38627
Pádraig Brady [Sat, 15 Feb 2020 11:16:35 +0000 (11:16 +0000)]
doc: clarify that '%a' stat format outputs mode bits
* src/stat.c (usage): Mention permission bits rather than
"access" so there is no confusion with ACLs etc.
Also indicate we output the file type with '%A'.
* doc/coreutils.texi (stat invocation): Likewise.
Also indicate '%A' is similar to `ls -ld` output.
Addresses https://bugs.gnu.org/39613
Jim Meyering [Thu, 6 Feb 2020 17:00:26 +0000 (09:00 -0800)]
build: suppress new FP warning from gcc-10.0.1
* configure.ac (GNULIB_WARN_CFLAGS): Add -Wno-return-local-addr
to avoid FP warning about careadlinkat.c. Discussed starting in
https://lists.gnu.org/r/coreutils/2020-02/msg00006.html
Pádraig Brady [Sun, 2 Feb 2020 16:56:18 +0000 (16:56 +0000)]
build: update to latest gnulib
Pick up recent build fixes to avoid sysctl.h inclusion on glibc systems,
restrict the max file size supported by read-file to PTRDIFF_MAX,
and to avoid a -Werror=unused failure in test-canonicalize.
Pádraig Brady [Tue, 4 Feb 2020 00:37:23 +0000 (00:37 +0000)]
tests: avoid false failure due to varying /proc/kallsyms
* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357
Pádraig Brady [Fri, 31 Jan 2020 01:46:40 +0000 (20:46 -0500)]
rmdir: fix --ignore-fail-on-non-empty with permissions errors
Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty`
had reversed the failure status for directories that failed
to be removed for permissions reasons. I.E. it would have
returned a failure status for such non empty dirs, and vice versa.
* src/rmdir.c (errno_may_be_non_empty): Rename from the
more confusing errno_may_be_empty(), and remove the EEXIST
case (specific to Solaris), which is moot here since
handled in errno_rmdir_non_empty().
(ignorable_failure): Fix the logic error so that
_non_ empty dirs are deemed to have ignorable failures.
(main): Fix clobbering of errno by is_empty_dir().
(remove_parents): Likewise.
* tests/rmdir/ignore.sh: Add a test case.
* THANKS.in: Add reporter who fixed the errno handling.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/39364
Pádraig Brady [Sun, 2 Feb 2020 16:24:18 +0000 (16:24 +0000)]
build: avoid vector performance warnings in randperm
* configure.ac: Add -Wno-vector-operation-performance to suppress the
following gcc-9.2 error in gl/lib/randperm.c:
error: vector operation will be expanded piecewise
Pádraig Brady [Thu, 2 Jan 2020 16:20:13 +0000 (16:20 +0000)]
ls: support --time=creation to show/sort birth time
* src/ls.c (usage): Reorganize help for --time,
and add description for --time=birth.
(do_statx): Store btime in mtime if available.
(get_stat_btime): A new function to read the creation time
from the appropriate stat structure member.
(cmp_btime): A new function to compare birth time.
(print_long_format): Output '?' when birth time unavailable.
* doc/coreutils.texi: Document --time={birth,creation}.
* tests/local.mk: Reference the new test.
* tests/ls/birthtime.sh: Add a new test.
* NEWS: Mention the new feature.
Chris Meyering [Thu, 30 Jan 2020 06:34:48 +0000 (22:34 -0800)]
build: rearrange yes(1) code to prevent GCC 10 warning
* src/yes.c (main): Convert for loop to do-while in order to indicate
that the loop will be run at least once.
This avoids the following warning after the second loop:
src/yes.c:110:20: error: writing 1 byte into a region of size 0