]> git.ipfire.org Git - thirdparty/coreutils.git/log
thirdparty/coreutils.git
5 years agobuild: update to latest gnulib
Pádraig Brady [Sun, 2 Feb 2020 16:56:18 +0000 (16:56 +0000)] 
build: update to latest gnulib

Pick up recent build fixes to avoid sysctl.h inclusion on glibc systems,
restrict the max file size supported by read-file to PTRDIFF_MAX,
and to avoid a -Werror=unused failure in test-canonicalize.

5 years agotests: avoid false failure due to varying /proc/kallsyms
Pádraig Brady [Tue, 4 Feb 2020 00:37:23 +0000 (00:37 +0000)] 
tests: avoid false failure due to varying /proc/kallsyms

* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357

5 years agormdir: fix --ignore-fail-on-non-empty with permissions errors
Pádraig Brady [Fri, 31 Jan 2020 01:46:40 +0000 (20:46 -0500)] 
rmdir: fix --ignore-fail-on-non-empty with permissions errors

Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty`
had reversed the failure status for directories that failed
to be removed for permissions reasons.  I.E. it would have
returned a failure status for such non empty dirs, and vice versa.

* src/rmdir.c (errno_may_be_non_empty): Rename from the
more confusing errno_may_be_empty(), and remove the EEXIST
case (specific to Solaris), which is moot here since
handled in errno_rmdir_non_empty().
(ignorable_failure): Fix the logic error so that
_non_ empty dirs are deemed to have ignorable failures.
(main): Fix clobbering of errno by is_empty_dir().
(remove_parents): Likewise.
* tests/rmdir/ignore.sh: Add a test case.
* THANKS.in: Add reporter who fixed the errno handling.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/39364

5 years agobuild: avoid vector performance warnings in randperm
Pádraig Brady [Sun, 2 Feb 2020 16:24:18 +0000 (16:24 +0000)] 
build: avoid vector performance warnings in randperm

* configure.ac: Add -Wno-vector-operation-performance to suppress the
following gcc-9.2 error in gl/lib/randperm.c:
  error: vector operation will be expanded piecewise

5 years agobuild: avoid including sysctl.h on glibc
Pádraig Brady [Sun, 2 Feb 2020 15:52:59 +0000 (15:52 +0000)] 
build: avoid including sysctl.h on glibc

* src/uname.c: Avoid unneeded header with GLIBC,
which has been deprecated since glibc-2.30.
* src/uptime.c: Likewise.

5 years agols: support --time=creation to show/sort birth time
Pádraig Brady [Thu, 2 Jan 2020 16:20:13 +0000 (16:20 +0000)] 
ls: support --time=creation to show/sort birth time

* src/ls.c (usage): Reorganize help for --time,
and add description for --time=birth.
(do_statx): Store btime in mtime if available.
(get_stat_btime): A new function to read the creation time
from the appropriate stat structure member.
(cmp_btime): A new function to compare birth time.
(print_long_format): Output '?' when birth time unavailable.
* doc/coreutils.texi: Document --time={birth,creation}.
* tests/local.mk: Reference the new test.
* tests/ls/birthtime.sh: Add a new test.
* NEWS: Mention the new feature.

5 years agobuild: rearrange yes(1) code to prevent GCC 10 warning
Chris Meyering [Thu, 30 Jan 2020 06:34:48 +0000 (22:34 -0800)] 
build: rearrange yes(1) code to prevent GCC 10 warning

* src/yes.c (main): Convert for loop to do-while in order to indicate
that the loop will be run at least once.
This avoids the following warning after the second loop:
src/yes.c:110:20: error: writing 1 byte into a region of size 0

5 years agomaint: add lib/iconv_open-zos.h to .gitignore
Emil Engler [Wed, 1 Jan 2020 22:59:33 +0000 (23:59 +0100)] 
maint: add lib/iconv_open-zos.h to .gitignore

* .gitignore: Add file newly generated by gnulib commit 49e78fc

5 years agobuild: auto enable use of openssl with >= version 3
Pádraig Brady [Wed, 1 Jan 2020 17:43:53 +0000 (17:43 +0000)] 
build: auto enable use of openssl with >= version 3

* configure.ac: Set --with-openssl=auto-gpl-compat as the default,
so that openssl is used for md5sum etc., with openssl >= 3,
which is newly licensed under ASL v2.
* gnulib: Update to include "auto-gpl-compat" support.

5 years agomaint: adjust to split out xstrtol-error gnulib module
Pádraig Brady [Wed, 1 Jan 2020 14:36:12 +0000 (14:36 +0000)] 
maint: adjust to split out xstrtol-error gnulib module

* bootstrap.conf: Depend on the new module split from xstrtol.
* src/df.c: Include "xstrtol-error.h" for xstrtol_fatal.
* src/du.c: Likewise.
* src/ls.c: Likewise.
* src/od.c: Likewise.
* src/pr.c: Likewise.
* src/sort.c: Likewise.

5 years agomaint: update all copyright year number ranges
Pádraig Brady [Wed, 1 Jan 2020 14:16:56 +0000 (14:16 +0000)] 
maint: update all copyright year number ranges

Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.

5 years agodoc: add example to demonstrate sub-second sleep times
Bernhard Voelker [Sun, 8 Dec 2019 16:29:57 +0000 (17:29 +0100)] 
doc: add example to demonstrate sub-second sleep times

* doc/coreutils.texi (sleep invocation): Add an example to demonstrate
how to use the floating-point and the scientific notation to sleep
for sub-second times, e.g. milli-, micro- and nanoseconds.

Inspired by Stephane Chazelas in:
  https://lists.gnu.org/r/coreutils/2019-12/msg00005.html

5 years agomaint: fix syntax-check failure from recent adjustment
Pádraig Brady [Mon, 2 Dec 2019 16:17:45 +0000 (16:17 +0000)] 
maint: fix syntax-check failure from recent adjustment

* cfg.mk (old_NEWS_hash): Regenerate after commit v8.31-56-gc1e1965.

5 years agochcon: do not validate security context if SELinux is disabled
Kamil Dudka [Mon, 2 Dec 2019 13:02:02 +0000 (14:02 +0100)] 
chcon: do not validate security context if SELinux is disabled

* src/chcon.c (main): Skip call of security_check_context()
in case SELinux is disabled to avoid unnecessary failure.

Bug: https://bugzilla.redhat.com/1777831

5 years agodoc: remove colon from node name
Paul Eggert [Tue, 12 Nov 2019 00:53:49 +0000 (16:53 -0800)] 
doc: remove colon from node name

* doc/sort-version.texi (Minus/Hyphen and Colon characters):
Rename from “Minus/Hyphen @samp{-} and Colon @samp{:} characters”,
as texi2any 6.6 complains about colons in node names.

5 years agoshred: modernize documentation
Paul Eggert [Tue, 12 Nov 2019 00:52:47 +0000 (16:52 -0800)] 
shred: modernize documentation

* doc/coreutils.texi (shred invocation):
Modernize discussion to today’s technology (Bug#38168).
* src/shred.c (usage): Omit lengthy duplication of the manual’s
discussion of file systems and storage devices, as that became out
of sync with the manual.  Instead, just cite the manual.

5 years agoall: improve parsing of numeric arguments
Paul Eggert [Tue, 22 Oct 2019 21:55:24 +0000 (14:55 -0700)] 
all: improve parsing of numeric arguments

This addresses a longstanding "update all callers" FIXME in
lib/xstrtol.c, by having programs check that numbers do not
have unknown suffixes.  The problem was also reported for
'shuf' by my student Maggie Huang while reimplementing a shuf
subset in Python as an exercise in UCLA Computer Science 35L:
https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html
This patch also improves the portability of the code to unusual
platforms where ULONG_MAX < SIZE_MAX.
* NEWS: Mention user-visible changes.
* src/chgrp.c (parse_group):
* src/chroot.c (parse_additional_groups):
* src/du.c (main):
* src/install.c (get_ids):
* src/join.c (string_to_join_field):
* src/ls.c (decode_switches):
* src/md5sum.c (split_3):
* src/shuf.c (main):
* src/sort.c (specify_nthreads):
* src/uniq.c (size_opt, main):
Use uintmax_t instead of unsigned long, for portability
to oddball platforms where unsigned long is not wide enough.
* src/du.c (main):
* src/expr.c (mpz_init_set_str) [!HAVE_GMP]:
* src/install.c (get_ids):
* src/ls.c (decode_switches):
* src/mknod.c (main):
* src/ptx.c (main):
* src/shuf.c (main):
* src/sort.c (specify_nmerge, specify_nthreads):
Reject numbers with suffixes.
* src/md5sum.c (split_3): Simplify.

5 years agostdbuf: improve size checking
Paul Eggert [Tue, 22 Oct 2019 21:31:52 +0000 (14:31 -0700)] 
stdbuf: improve size checking

* bootstrap.conf (gnulib_modules): Add minmax.
* src/libstdbuf.c: Include stdint.h, minmax.h.
(apply_mode): Don’t assume SIZE_MAX <= ULONG_MAX.
Improve checking for invalid sizes.

5 years agoshuf: improve randperm overflow checking
Paul Eggert [Tue, 22 Oct 2019 19:58:07 +0000 (12:58 -0700)] 
shuf: improve randperm overflow checking

* gl/lib/randperm.c: Include randperm.h first, since it’s the API.
Include stdint.h, count-leading-zeros.h, verify.h.
(floor_lg): Rename from ceil_log (which was not actually
implementing the ceiling!) and implement the floor using
count_leading_zeros.
(randperm_bound): Use floor_lg, not ceil_log.  Use uintmax_t
instead of size_t in case the size gets large on a 32-bit host.
* gl/modules/randperm (Depends-on): Add count-leading-zeros, stdint.

5 years agobuild: don’t worry about logical-op checking
Paul Eggert [Tue, 22 Oct 2019 19:52:52 +0000 (12:52 -0700)] 
build: don’t worry about logical-op checking

* configure.ac: Remove code tailoring --enable-gcc-warnings
to GCC 4.7 and earlier, as developers no longer need to worry
about GCCs that old.

5 years agobuild: re-enable type-limits checking
Paul Eggert [Tue, 22 Oct 2019 18:34:56 +0000 (11:34 -0700)] 
build: re-enable type-limits checking

* configure.ac: When --enable-gcc-warnings is used, omit
-Wno-type-limits.  The need for -Wno-type-limits has passed, now
that intprops.h uses builtin primitives for GCC 5 and later, given
that recent GCCs issue type-limits warnings only for non-constant
expressions.  --enable-gcc-warnings is not intended for use with
old compilers, so we can drop -Wno-type-limits now.

5 years agoshuf: fix bug with ‘-r -n 0’
Paul Eggert [Mon, 21 Oct 2019 20:03:46 +0000 (13:03 -0700)] 
shuf: fix bug with ‘-r -n 0’

‘shuf -r -n 0 file’ would mistakenly read from standard input.
Problem reported by my student Jingnong Qu while reimplementing a
shuf subset in Python as an exercise in UCLA Computer Science 35L:
https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html
* NEWS: Mention the fix.  Also, ASCIIfy a previous item.
* src/shuf.c (main): Fix bug.
* tests/misc/shuf.sh: Add a test case for the bug.

5 years agols: use statx instead of stat when available
Jeff Layton [Thu, 19 Sep 2019 15:59:45 +0000 (11:59 -0400)] 
ls: use statx instead of stat when available

statx allows ls to indicate interest in only certain inode metadata.
This is potentially a win on networked/clustered/distributed
file systems. In cases where we'd have to do a full, heavyweight stat()
call we can now do a much lighter statx() call.

As a real-world example, consider a file system like CephFS where one
client is actively writing to a file and another client does an
ls --color in the same directory. --color means that we need to fetch
the mode of the file.

Doing that with a stat() call means that we have to fetch the size and
mtime in addition to the mode. The MDS in that situation will have to
revoke caps in order to ensure that it has up-to-date values to report,
which disrupts the writer.

This has a measurable affect on performance. I ran a fio sequential
write test on one cephfs client and had a second client do "ls --color"
in a tight loop on the directory that held the file:

Baseline -- no activity on the second client:

WRITE: bw=76.7MiB/s (80.4MB/s), 76.7MiB/s-76.7MiB/s (80.4MB/s-80.4MB/s),
       io=4600MiB (4824MB), run=60016-60016msec

Without this patch series, we see a noticable performance hit:

WRITE: bw=70.4MiB/s (73.9MB/s), 70.4MiB/s-70.4MiB/s (73.9MB/s-73.9MB/s),
       io=4228MiB (4433MB), run=60012-60012msec

With this patch series, we gain most of that ground back:

WRITE: bw=75.9MiB/s (79.6MB/s), 75.9MiB/s-75.9MiB/s (79.6MB/s-79.6MB/s),
       io=4555MiB (4776MB), run=60019-60019msec

* src/stat.c: move statx to stat struct conversion to new header...
* src/statx.h: ...here.
* src/ls.c: Add wrapper functions for stat/lstat/fstat calls,
and add variants for when we are only interested in specific info.
Add statx-enabled functions and set the request mask based on the
output format and what values are needed.
* NEWS: Mention the Improvement.

5 years agotruncate: avoid integer-overflow assumptions
Paul Eggert [Thu, 3 Oct 2019 19:41:22 +0000 (12:41 -0700)] 
truncate: avoid integer-overflow assumptions

* src/truncate.c (do_ftruncate): Simplify overflow checking,
and don’t rely on theoretically-nonportable assumptions
like assuming that OFF_MAX < UINTMAX_MAX.

5 years agonumfmt: avoid unlikely integer overflow
Paul Eggert [Thu, 3 Oct 2019 19:38:15 +0000 (12:38 -0700)] 
numfmt: avoid unlikely integer overflow

* src/numfmt.c (parse_format_string): Report overflow if
pad < -LONG_MAX, since that can’t be negated.

5 years agonl: fix integer-overflow bug
Paul Eggert [Thu, 3 Oct 2019 19:37:12 +0000 (12:37 -0700)] 
nl: fix integer-overflow bug

Problem reported by Roland Illig (Bug#37585)
* src/nl.c (print_lineno): Don’t rely on undefined behavior when
checking for integer overflow.

5 years agocp: simplify integer overflow checking
Paul Eggert [Thu, 3 Oct 2019 19:35:44 +0000 (12:35 -0700)] 
cp: simplify integer overflow checking

* src/copy.c (sparse_copy): Use INT_ADD_WRAPV instead
of doing overflow checking by hand.

5 years agoseq: use faster processing for integer steps from 2 to 200
Pádraig Brady [Tue, 3 Sep 2019 09:14:48 +0000 (10:14 +0100)] 
seq: use faster processing for integer steps from 2 to 200

* src/seq.c: (seq_fast): Accept STEP as a parameter and use that
to skip the output of generated numbers.
(main): Relax to using seq_fast for integer steps between 1 and 200.
For larger steps the throughput was faster using the standard
incrementing procedure.
(cmp): Use the equivalent but faster memcmp for equal len strings.
* tests/misc/seq.pl: Update fast path cases.
Addresses https://bugs.gnu.org/37241

5 years agomaint: use consistent header ordering and spacing in NEWS
Pádraig Brady [Wed, 4 Sep 2019 01:39:04 +0000 (02:39 +0100)] 
maint: use consistent header ordering and spacing in NEWS

* NEWS: Move "Changes in behavior" before "New features",
and ensure there is only a single blank line between sections.

5 years agobuild: update gnulib submodule to latest
Paul Eggert [Thu, 15 Aug 2019 09:41:16 +0000 (02:41 -0700)] 
build: update gnulib submodule to latest

5 years agoscripts: document how to build older versions on newer systems
Assaf Gordon [Sat, 3 Aug 2019 02:22:33 +0000 (20:22 -0600)] 
scripts: document how to build older versions on newer systems

Based on https://lists.gnu.org/r/coreutils/2019-08/msg00011.html .

* scripts/build-older-versions/README.older-versions: Documentation
* scripts/build-older-versions/build-older-versions.sh: Helper script.
* scripts/build-older-versions/.gitignore: Ignore build directory.
* scripts/build-older-versions/coreutils-5.0-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-5.97-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-6.10-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-6.11-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-6.12-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-7.2-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-8.13-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-8.17-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-8.18-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-8.24-on-glibc-2.28.diff,
scripts/build-older-versions/coreutils-8.4-on-glibc-2.28.diff: Patches.

5 years agobuild: adjust for recent gnulib pthread changes
Bruno Haible [Mon, 12 Aug 2019 03:29:00 +0000 (21:29 -0600)] 
build: adjust for recent gnulib pthread changes

Discussed in https://lists.gnu.org/r/coreutils/2019-08/msg00030.html .

* bootstrap.conf (gnulib_modules): Replace 'pthread' with
pthread-* modules.
* src/sort.c: Remove GNULIB_defined_pthread_functions conditional.

5 years agodate: mention military timezone changes from gnulib
Assaf Gordon [Sat, 10 Aug 2019 02:16:06 +0000 (20:16 -0600)] 
date: mention military timezone changes from gnulib

Gnulib commits f1f10d47be8762e4ca17c8957a0520b08d28abfb and
0673d8ab42c9bb0cf618a21b537cdd8fb976fb73 negated the meaning of
military timezones parsed in gnu date.
See https://lists.gnu.org/r/bug-gnulib/2019-08/msg00005.html and
https://lists.gnu.org/r/coreutils/2019-08/msg00021.html

* NEWS: Mention this user-visible change.
* tests/misc/date.pl: Add tests for the new behavior.

5 years agomaint: add lib/argmatch.h to po/POTFILES.in
Bernhard Voelker [Mon, 22 Jul 2019 06:53:28 +0000 (08:53 +0200)] 
maint: add lib/argmatch.h to po/POTFILES.in

* po/POTFILES.in (lib/argmatch.h): Add to avoid sc_po_check error:
    "maint.mk: you have changed the set of files with translatable \
     diagnostics;"

5 years agognulib: update to latest
Assaf Gordon [Sat, 10 Aug 2019 01:51:42 +0000 (19:51 -0600)] 
gnulib: update to latest

6 years agodoc: clarify that truncate creates sparse files
Pádraig Brady [Thu, 8 Aug 2019 12:31:41 +0000 (13:31 +0100)] 
doc: clarify that truncate creates sparse files

* src/truncate.c (usage): Explicitly mention "sparse".
* doc/coreutils.texi (truncate invocation): Likewise.
Addresses https://bugs.gnu.org/36963

6 years agodircolors: recognize the WebP image format
Mike Swanson [Fri, 2 Aug 2019 20:50:17 +0000 (13:50 -0700)] 
dircolors: recognize the WebP image format

* src/dircolors.hin: Add .webp for the WebP image format.
Fixes https://bugs.gnu.org/36899

6 years agomaint: fix error in syntax-check checking
Bernhard Voelker [Tue, 6 Aug 2019 23:11:01 +0000 (16:11 -0700)] 
maint: fix error in syntax-check checking

The previous commit introduced a bug into the following syntax-check,
and thus effectively turned it off:

  $ make sc_prohibit_test_calls_print_ver_with_irrelevant_argument; \
      echo $?
  prohibit_test_calls_print_ver_with_irrelevant_argument
  fatal: cannot change to 'grep': No such file or directory
  0

* cfg.mk (sc_prohibit_test_calls_print_ver_with_irrelevant_argument):
Remove changing directory, and pass $(srcdir) as argument to 'git -C'.

6 years agomaint: fix issues in syntax-check
Akim Demaille [Sun, 4 Aug 2019 18:02:01 +0000 (11:02 -0700)] 
maint: fix issues in syntax-check

* cfg.mk (sc_prohibit_colon_redirection): Don't expect `|` to denote
the pipe character in git grep.
(sc_tests_executable)
(sc_case_insensitive_file_names)
(sc_some_programs_must_avoid_exit_failure)
(sc_prohibit_test_background_without_cleanup_)
(sc_prohibit_test_calls_print_ver_with_irrelevant_argument)
(sc_prohibit_test_ulimit_without_require_)
(sc_prohibit_test_background_without_cleanup_)
(sc_THANKS_in_duplicates)
*sc_prohibit_test_calls_print_ver_with_irrelevant_argument):
Don't expect builddir to be a descendant of srcdir.
(sc_strftime_check): Don't check file size against 0 when "N\nq\n" was
already put in the file.
* THANKS.in: Remove me.

6 years agoseq: fix superfluous output line
Assaf Gordon [Thu, 1 Aug 2019 23:01:21 +0000 (17:01 -0600)] 
seq: fix superfluous output line

Under certain circumstances seq prints an extra line when the output
format has custom format with characters following the printed numbers:

    $ seq -f "%g " 1000000 1000000
     1e+06
     1e+06

This is due to the "print_extra_number" logic using strings to determine
whether a 'extra number' is needed, but only one string was trimmed
when using a custom printf format.

Prompted by https://lists.gnu.org/r/coreutils/2019-08/msg00001.html

* NEWS: Mention fix.
* src/seq.c (print_numbers): Trim the 'x0_str' string before comparing
it to the previous 'x_str' string.
* tests/misc/seq-extra-number.sh: Add this scenario.
* tests/local.mk (all_tests): Add new test.

6 years agodoc: improve new version sort chapter
Bernhard Voelker [Fri, 19 Jul 2019 00:11:03 +0000 (02:11 +0200)] 
doc: improve new version sort chapter

* doc/sort-version.texi: Fix some typos, avoid overly long lines in
the generated PDF, enclose some sample strings in @samp{...} for better
readability, etc.  This also avoids an sc-avoid-builtin error:
s/builtin/built-in/

6 years agodoc: add "version sort ordering" chapter
Assaf Gordon [Wed, 10 Jul 2019 01:36:10 +0000 (19:36 -0600)] 
doc: add "version sort ordering" chapter

* doc/sort-version.texi: New file.
* doc/local.mk (doc_coreutils_TEXINFOS): Add new file.
* doc/coreutils.texi: @include new file, replace previous "Details about
version sort" section.

6 years agostat: don't explicitly request file size for filenames
Andreas Dilger [Thu, 27 Jun 2019 08:25:55 +0000 (02:25 -0600)] 
stat: don't explicitly request file size for filenames

When calling 'stat -c %N' to print the filename, don't explicitly
request the size of the file via statx(), as it may add overhead on
some filesystems.  The size is only needed to optimize an allocation
for the relatively rare case of reading a symlink name, and the worst
effect is a somewhat-too-large temporary buffer may be allocated for
areadlink_with_size(), or internal retries if buffer is too small.

The file size will be returned by statx() on most filesystems, even
if not requested, unless the filesystem considers this to be too
expensive for that file, in which case the tradeoff is worthwhile.

* src/stat.c: Don't explicitly request STATX_SIZE for filenames.

6 years agood: use fseek on non-regular files
Paul Eggert [Thu, 20 Jun 2019 01:46:57 +0000 (18:46 -0700)] 
od: use fseek on non-regular files

Problem reported by Szőts Ákos (Bug#36291).
* NEWS: Mention this.
* src/od.c (skip): Try fseek even on files that do not have usable
sizes, falling back on fread if fseek fails.

6 years agodoc: mention ls -l user/group justification
Paul Eggert [Tue, 18 Jun 2019 07:31:43 +0000 (00:31 -0700)] 
doc: mention ls -l user/group justification

* doc/coreutils.texi (What information is listed):
Document justification of user and group columns in ls -l output
(Bug#36220).

6 years agostat: fix enabling of statx logic
Jeff Layton [Fri, 14 Jun 2019 18:37:43 +0000 (14:37 -0400)] 
stat: fix enabling of statx logic

* src/stat.c: STATX_INO isn't defined until stat.h is included.
Move the test down so it works properly.

6 years agotests: avoid false-positive in date-debug test
Assaf Gordon [Mon, 13 May 2019 22:37:40 +0000 (16:37 -0600)] 
tests: avoid false-positive in date-debug test

When debugging an invalid date due to DST switching, the intermediate
'normalized time' should not be checked - its value can differ between
systems (e.g. glibc vs musl).

Reported by Niklas Hambüchen in
https://lists.gnu.org/r/coreutils/2019-05/msg00031.html
Analyzed by Rich Felker in
https://lists.gnu.org/r/coreutils/2019-05/msg00039.html

* tests/misc/date-debug.sh: Replace the exact normalized time
with 'XX:XX:XX' so different values would not trigger test failure.

6 years agostat: Use statx where available and support --cached
Jeff Layton [Tue, 28 May 2019 12:21:42 +0000 (08:21 -0400)] 
stat: Use statx where available and support --cached

* src/stat.c: Drop statbuf argument from out_epoch_sec().
Use statx() rather than [lf]stat() where available,
so a separate call is not required to get birth time.
Set STATX_* mask bits only for things we want to print,
which can be more efficient on some file systems.
Add a new --cache= command-line option that sets the appropriate hint
flags in the statx call.  These are primarily used with network
file systems to indicate what level of cache coherency is desired.
The new option is available unconditionally for better portability,
and ignored where not implemented.
* doc/coreutils.texi: Add documention for --cached.
* man/stat.x (SEE ALSO): Mention statx().
* NEWS: Mention the new feature.

6 years agodoc: fix description of tail -f on truncated files
Pádraig Brady [Sun, 9 Jun 2019 11:39:27 +0000 (12:39 +0100)] 
doc: fix description of tail -f on truncated files

* doc/coreutils.texi (tail invocation): Update to match
the new behavior following commit v8.23-189-gb28ff6a

6 years agosplit: fix failure for certain number of specified files
Pádraig Brady [Sat, 8 Jun 2019 21:49:01 +0000 (22:49 +0100)] 
split: fix failure for certain number of specified files

* src/split.c (set_suffix_length): Use a more standard
zero based logN calculation for the number of units.
* tests/split/suffix-auto-length.sh: Add a test case.
* THANKS.in: Mention the reporter.
* NEWS: Mention the fix.
Fixes https://bugs.gnu.org/35291

6 years agodd: be more careful about signal handling
Paul Eggert [Thu, 30 May 2019 20:53:54 +0000 (13:53 -0700)] 
dd: be more careful about signal handling

Problem reported by Hans Henrik Bergan (Bug#36007).
* NEWS: Mention this.
* src/dd.c (iclose, ifdatasync, ifstat, ifsync):
New functions, which are more careful about SIGINT.
(cleanup): Use iclose instead of close.
(finish_up): Process signals first.
(skip, dd_copy, main): Use ifstat instead of fstat.
(dd_copy): Use ifdatasync and ifsync instead of fdatasync and fsync.

6 years agomaint: fix version number in NEWS
Paul Eggert [Wed, 29 May 2019 23:44:52 +0000 (16:44 -0700)] 
maint: fix version number in NEWS

6 years agocp: fix /dev/stdin problem on Solaris
Paul Eggert [Tue, 28 May 2019 19:42:24 +0000 (12:42 -0700)] 
cp: fix /dev/stdin problem on Solaris

Problem reported by Jakub Kulik (Bug#35713).
* NEWS: Mention this.
* configure.ac (DEV_FD_MIGHT_BE_CHR): New macro.
* src/copy.c (DEV_FD_MIGHT_BE_CHR): Default to false.
(follow_fstatat): New function.
(copy_internal): Use it.
* src/copy.h (XSTAT): Remove; no longer used.

6 years agodoc: clarify dd sparse detection is by *output* block
Kevin Locke [Thu, 23 May 2019 03:07:50 +0000 (21:07 -0600)] 
doc: clarify dd sparse detection is by *output* block

The wording of the dd --help text suggests that output will be skipped
for sparse *input* blocks (i.e. that NUL-checking is done on input
blocks) while the code actually checks/skips all-NUL *output* blocks.[1]

* src/dd.c (usage): Update the --help text to clarify the above.
* tests/dd/sparse.sh: Ensure sparseness is controlled with obs.

[1]: https://superuser.com/a/1136358

6 years agodoc: fix typo in sort set operations example
Martin Castillo [Wed, 22 May 2019 13:43:10 +0000 (14:43 +0100)] 
doc: fix typo in sort set operations example

* doc/coreutils.texi (sort invocation): Add a missing -u
option to uniq.
Addresses https://bugs.gnu.org/35849

6 years agob2sum: port blake2b-ref.c to HP-UX aCC
Paul Eggert [Fri, 17 May 2019 14:57:39 +0000 (07:57 -0700)] 
b2sum: port blake2b-ref.c to HP-UX aCC

Continue the fix for Bug#35650.
* src/blake2/blake2b-ref.c [HAVE_CONFIG_H]: Include <config.h>.

6 years agob2sum: sync better with upstream
Paul Eggert [Wed, 15 May 2019 19:57:53 +0000 (12:57 -0700)] 
b2sum: sync better with upstream

* src/blake2/b2sum.c: Reorder source code to minimize diffs from:
https://github.com/BLAKE2/BLAKE2/blob/master/b2sum/b2sum.c

6 years agob2sum: port to HP-UX aCC
Paul Eggert [Wed, 15 May 2019 19:42:23 +0000 (12:42 -0700)] 
b2sum: port to HP-UX aCC

Its support for the -include option is flaky.  Problem reported by
Michael Osipov (Bug#35650).  Plus, we could run into other
compilers that don’t support any option like -include.  Change the
code so that -include is not needed.  Although this causes us to
depart from the upstream version, we’re already doing that for
other reasons.
* configure.ac (USE_XLC_INCLUDE): Remove, as there’s no
guarantee a compiler will support something like -include.
* src/blake2/b2sum.c [HAVE_CONFIG_H]: Include <config.h>.
* src/local.mk (src_b2sum_CPPFLAGS): Add -DHAVE_CONFIG_H.
Do not use -include or a substitute.

6 years agostdbuf: port configure-time checking to HP-UX aCC
Paul Eggert [Tue, 14 May 2019 16:20:56 +0000 (09:20 -0700)] 
stdbuf: port configure-time checking to HP-UX aCC

Problem reported by Michael Osipov (Bug#35650).
* configure.ac: Use AC_LANG_WERROR to pay attention to compiler
and linker warnings when testing whether stdbuf will work.

6 years agob2sum: port to HP-UX C
Paul Eggert [Sat, 11 May 2019 20:48:16 +0000 (13:48 -0700)] 
b2sum: port to HP-UX C

* src/blake2/blake2.h (BLAKE2_PACKED):
Don’t assume __attribute__ ((packed)) works on non-Microsoft
compilers.  Instead, assume it works only if we have good
reason to assume so, and fall back on Microsoft (or not packing)
otherwise.  In practice, not packing is good enough and the
BLAKE2_PACKED macro is mostly just for documentation.

6 years agocp: port fiemap.h to C99
Paul Eggert [Sat, 11 May 2019 20:09:39 +0000 (13:09 -0700)] 
cp: port fiemap.h to C99

* src/extent-scan.c (extent_scan_read): Adjust to change in
struct fiemap.
* src/fiemap.h (struct fiemap): Use FLEXIBLE_ARRAY_MEMBER
to port to C99.

6 years agobasenc: port to C99
Paul Eggert [Sat, 11 May 2019 20:01:37 +0000 (13:01 -0700)] 
basenc: port to C99

* src/basenc.c: Various minor style cleanups.
(struct base_decode_context): Do not use anonymous unions, as
they’re not in C99.  Use a named union instead.  All uses changed.

6 years agomaint: adjust to recent verify_true removal
Paul Eggert [Sat, 11 May 2019 16:32:04 +0000 (09:32 -0700)] 
maint: adjust to recent verify_true removal

* src/system.h (X2NREALLOC, X2REALLOC, DECIMAL_DIGIT_ACCUMULATE):
Use verify_expr instead of verify_true, which has been removed.
(DECIMAL_DIGIT_ACCUMULATE): Remove unnecessary size check.

6 years agobuild: update gnulib submodule to latest
Paul Eggert [Sat, 11 May 2019 15:34:59 +0000 (08:34 -0700)] 
build: update gnulib submodule to latest

6 years agognulib: update to the latest
Bernhard Voelker [Wed, 10 Apr 2019 07:13:27 +0000 (09:13 +0200)] 
gnulib: update to the latest

* gnulib: Update to latest, mainly for:
  > mountlist: make parsing /proc/self/mountinfo more robust
* NEWS: Mention the fix.

Fixes https://bugs.gnu.org/33468

6 years agofactor: output immediately if stdout is a tty but stdin is not
Shugo Maeda [Sun, 31 Mar 2019 22:43:58 +0000 (15:43 -0700)] 
factor: output immediately if stdout is a tty but stdin is not

* src/factor.c (lbuf_putc): Use line buffered mode if the standard
output is a terminal in the same way as the stdio library.
User programs might use pty only for the standard out
like the example of Ruby's PTY module:
https://docs.ruby-lang.org/en/2.6.0/PTY.html#module-PTY-label-Example
* NEWS: Mention the fix.
Fixes https://bugs.gnu.orv/35046

6 years agomaint: fix syntax check failure
Pádraig Brady [Sat, 30 Mar 2019 22:07:14 +0000 (15:07 -0700)] 
maint: fix syntax check failure

* src/ln.c: Remove leading TAB.

6 years agomaint: tee: use STDIN_FILENO rather than 0
Martin Castillo [Sat, 30 Mar 2019 22:04:06 +0000 (15:04 -0700)] 
maint: tee: use STDIN_FILENO rather than 0

* src/tee.c (tee_files): Use the name rather than the value.
Addresses https://bugs.gnu.org/35041

6 years agodd: improve doc of stderr output
Paul Eggert [Wed, 20 Mar 2019 03:08:41 +0000 (20:08 -0700)] 
dd: improve doc of stderr output

* doc/coreutils.texi (dd invocation):
Document stderr output more carefully.
Say that conv=block can lose input data.

6 years agomd5sum,b2sum,sha*sum: --help: add note about binary/text mode
Kamil Dudka [Thu, 14 Mar 2019 12:48:01 +0000 (13:48 +0100)] 
md5sum,b2sum,sha*sum: --help: add note about binary/text mode

* src/md5sum.c (usage): Make it clear that there is no difference
between binary mode and text mode on GNU systems.

Bug: https://bugzilla.redhat.com/406981
Bug: https://bugzilla.redhat.com/1688740

6 years agodoc: add NEWS item for Solaris symlink fix
Paul Eggert [Sun, 17 Mar 2019 20:25:10 +0000 (13:25 -0700)] 
doc: add NEWS item for Solaris symlink fix

6 years agoln: port to symlink ("x", ".") failing with EINVAL
Paul Eggert [Sun, 17 Mar 2019 20:20:42 +0000 (13:20 -0700)] 
ln: port to symlink ("x", ".") failing with EINVAL

Problem reported by John Marino (Bug#34894).
* src/ln.c (main): Port ln -s to Solaris symlink function,
where symlink ("x", ".") fails with errno == EINVAL.

6 years agodoc: add a NEWS entry for the ln O_DIRECTORY fix
Pádraig Brady [Sat, 16 Mar 2019 21:21:40 +0000 (14:21 -0700)] 
doc: add a NEWS entry for the ln O_DIRECTORY fix

* NEWS: Mention the bugfix.

6 years agoln: port to platforms lacking O_DIRECTORY
Paul Eggert [Sat, 16 Mar 2019 18:24:19 +0000 (11:24 -0700)] 
ln: port to platforms lacking O_DIRECTORY

* src/ln.c (main): Port to older platforms lacking
support for POSIX.1-2008’s O_DIRECTORY flag (Bug#34876).

6 years agodoc: improve wording of the --kibibytes option description
Kamil Dudka [Thu, 6 Dec 2018 13:32:49 +0000 (14:32 +0100)] 
doc: improve wording of the --kibibytes option description

Bug: https://bugzilla.redhat.com/1527391 , https://bugs.gnu.org/33646

* doc/coreutils.texi (General output formatting): Improve wording of
'--kibibytes' option.

6 years agomaint: sync extra files from gnulib
Bernhard Voelker [Thu, 7 Mar 2019 07:27:22 +0000 (08:27 +0100)] 
maint: sync extra files from gnulib

Some files are physically copied from gnulib, and should get sync'ed
after each update to latest gnulib.  This was forgotten during recent
updates.

* COPYING: Merge from gnulib/doc/COPYINGv3.
* tests/init.sh: Merge from gnulib/tests/init.sh.

6 years agomaint: post-release administrivia
Pádraig Brady [Mon, 11 Mar 2019 00:08:36 +0000 (17:08 -0700)] 
maint: post-release administrivia

* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.

6 years agoversion 8.31 v8.31
Pádraig Brady [Sun, 10 Mar 2019 23:34:41 +0000 (16:34 -0700)] 
version 8.31

* NEWS: Record release date.

6 years agotests: test-N: include subsecond values in gating check
Pádraig Brady [Sun, 10 Mar 2019 07:48:06 +0000 (23:48 -0800)] 
tests: test-N: include subsecond values in gating check

* tests/misc/test-N.sh: The subsecond values for atime and mtime
were potentially seen to differ on newlyl created files.
So we include the subsecond portion when comparing stat values.

6 years agotests: wc-nbsp: fix false failures on various systems
Pádraig Brady [Sun, 10 Mar 2019 04:40:34 +0000 (20:40 -0800)] 
tests: wc-nbsp: fix false failures on various systems

* tests/misc/wc-nbsp.sh: Add gating checks for all characters,
as there are disparate classifications on various systems:
SunOS 5.10 treats \u202F, \u2060 as !iswprint()
SunOS 5.10 treats \u00A0, \u2007 as iswspace()
AIX 7.2, Darwin 17.4.0, NetBSD 7.1 treat \u2060 as !iswprint()

6 years agotests: tail-2/pipe-f: avoid false failure closing stdout
Pádraig Brady [Thu, 7 Mar 2019 06:23:21 +0000 (22:23 -0800)] 
tests: tail-2/pipe-f: avoid false failure closing stdout

* tests/tail-2/pipe-f.sh: Check closing stdout with >&-
is effective, which avoids a false failure on NetBSD 7.1
Reported by Assaf Gordon

6 years agotests: tac-2-nonseekable: ensure we don't block indefinitely
Pádraig Brady [Thu, 7 Mar 2019 05:39:15 +0000 (21:39 -0800)] 
tests: tac-2-nonseekable: ensure we don't block indefinitely

* tests/misc/tac-2-nonseekable.sh: Add a timeout to both
protect and check whether we can close stdin correctly.

6 years agotests: id/zero: avoid false failure due to sed differences
Pádraig Brady [Thu, 7 Mar 2019 05:04:26 +0000 (21:04 -0800)] 
tests: id/zero: avoid false failure due to sed differences

* tests/id/zero.sh: sed on OSX will output a \n even
if the input doesn't have a \n on the last "line".
So ensure we always have a trailing '\n' to avoid the disparity.

6 years agotests: test-N: fix false positives on some systems
Pádraig Brady [Thu, 7 Mar 2019 04:09:16 +0000 (20:09 -0800)] 
tests: test-N: fix false positives on some systems

Testing by Assaf Gordon on OSX showed the atime wasn't
being updated when explicitly set back in time.
Also Debian 8.11 / mips64 was seen to not update the
mtime when truncating an empty file.

* tests/misc/test-N.sh: Isolate from different timestamping
behaviors of various (file) systems, by correlating
the timestamps with stat(1) before using `test -N`.

6 years agodoc: replace @hashchar{} with actual hash character
Assaf Gordon [Wed, 6 Mar 2019 06:32:34 +0000 (23:32 -0700)] 
doc: replace @hashchar{} with actual hash character

Very old makeinfo-4.13 fails with:
  ./doc/coreutils.texi:2286: Unknown command `hashchar'.
  ./doc/coreutils.texi:2286: Misplaced {.
  ./doc/coreutils.texi:2286: Misplaced }.

Reported Bernhard Voelker in
https://lists.gnu.org/r/coreutils/2019-03/msg00016.html .

* doc/coreutils.texi (basenc invocation): Replace @hashchar{} with
actual hash character.  The special syntax is only required
when referring to #line directives.

6 years agobuild: avoid statx related build failure on AIX
Pádraig Brady [Wed, 6 Mar 2019 18:46:25 +0000 (10:46 -0800)] 
build: avoid statx related build failure on AIX

* src/stat.c (get_birthtime): Check also for STATX_BTIME define,
as a different statx is available on AIX 7.2.

6 years agotests: wc-nbsp.sh: avoid failure on FreeBSD
Pádraig Brady [Wed, 6 Mar 2019 18:34:16 +0000 (10:34 -0800)] 
tests: wc-nbsp.sh: avoid failure on FreeBSD

* tests/misc/wc-nbsp.sh: FreeBSD and OS X don't
treat non breaking space as printable characters.
So use wc -L to determine printability before
testing non breaking space functionality.

6 years agobuild: fix env build where SIGNUM_BOUND is not constant
Pádraig Brady [Wed, 6 Mar 2019 16:58:13 +0000 (08:58 -0800)] 
build: fix env build where SIGNUM_BOUND is not constant

* src/env.c (initialize_signals): A new function to initialize
the signals array on the heap, to avoid a build failure on
opensolaris, where SIGNUM_BOUND is not a constant.

6 years agodoc: remove older ChangeLog items
Pádraig Brady [Mon, 4 Mar 2019 09:28:42 +0000 (01:28 -0800)] 
doc: remove older ChangeLog items

* Makefile.am: Update the oldest documented version
to 8.22 which is now about 5 years old.

6 years agobuild: revert recent change with distributed man page handling
Pádraig Brady [Mon, 4 Mar 2019 09:12:04 +0000 (01:12 -0800)] 
build: revert recent change with distributed man page handling

* man/local.mk: commit f114495e added an extra check to ensure
a binary was working before using it to generate the man page.
However this was not working for the false(1) command,
and also one can generally specify that one should not
be using generated commands on the current system by passing
'cross_compiling=yes' to the configure invocation.

6 years agoenv: add --list-signal-handling to output non default handling
Pádraig Brady [Mon, 4 Mar 2019 07:50:21 +0000 (23:50 -0800)] 
env: add --list-signal-handling to output non default handling

* src/env.c (main): Output blocked or ignored signals
before a command is executed.
* doc/coreutils.texi (env invocation): Add the option.
* tests/misc/env-signal-handler.sh: Add a test case.
* NEWS: Mention the new feature.

6 years agoenv: new options --{default,ignore,block}-signal[=SIG]
Assaf Gordon [Fri, 15 Feb 2019 19:31:48 +0000 (12:31 -0700)] 
env: new options --{default,ignore,block}-signal[=SIG]

New options to set signal handlers for the command being executed.
--block-signal suggested by Paul Eggert in http://bugs.gnu.org/34488#71
--default-signal is useful to overcome the POSIX limitation that shell
must not override inherited signal state, e.g. the second 'trap' here is
a no-op:

   trap '' PIPE && sh -c 'trap - PIPE ; seq inf | head -n1'

Instead use:

   trap '' PIPE && sh -c 'env --default-signal=PIPE seq inf | head -n1'

Similarly, the following will prevent CTRL-C from terminating the
program:

   env --ignore-signal=INT seq inf > /dev/null

See https://bugs.gnu.org/34488#8

* NEWS: Mention new options.
* doc/coreutils.texi (env invocation): Document new options.
* man/env.x: Add example of --default-signal=SIG usage.
(SEE ALSO): Mention sigprocmask.
* src/env.c (signals): New global variable.
(longopts): Add new options.
(usage): Print new options.
(parse_signal_params): Parse comma-separated list of signals, store in
signals variable.
(reset_signal_handlers): Set each signal to SIG_DFL/SIG_IGN.
(parse_block_signal_params): Parse command-line options.
(set_signal_proc_mask): Call sigprocmask to block/unblock signals.
(main): Process new options.
* src/local.mk (src_env_SOURCES): Add operand2sig.c.
* tests/misc/env-signal-handler.sh: New test.
* tests/local.mk (all_tests): Add new test.

6 years agostat: print birth time on systems supporting statx
Martin Bukatovic [Sun, 3 Mar 2019 03:57:17 +0000 (19:57 -0800)] 
stat: print birth time on systems supporting statx

* configure.ac: Check for statx(), available on glibc >= 2.28.
* src/stat.c (get_birthtime): Call statx() when available.
* NEWS: Mention the improvement.

6 years agodf: support different file system encodings when not using tty
Pádraig Brady [Sun, 3 Mar 2019 22:35:18 +0000 (14:35 -0800)] 
df: support different file system encodings when not using tty

* src/df.c (replace_problematic_chars): A new wrapper to be
more conservative in our replacement when not connected to a tty.
* tests/df/problematic-chars.sh: Add a test case.

6 years agomaint: tidy up recent additions to NEWS
Pádraig Brady [Mon, 4 Mar 2019 03:21:37 +0000 (19:21 -0800)] 
maint: tidy up recent additions to NEWS

* NEWS: Move date change to improvements and fix nohup grammar.

6 years agodoc: further clarify 'yes' alternative in seq invocation
Bernhard Voelker [Tue, 26 Feb 2019 18:06:52 +0000 (19:06 +0100)] 
doc: further clarify 'yes' alternative in seq invocation

* doc/coreutils.texi (node seq invocation): Clarify to use the tool
'yes'; otherwise the reader may interpret the sentence as if one
could pass 'yes' as the INCREMENT value.

6 years agowc: treat non breaking space as a word separator
Pádraig Brady [Sun, 24 Feb 2019 05:23:47 +0000 (21:23 -0800)] 
wc: treat non breaking space as a word separator

* src/wc.c (iswnbspace): A new function to match
characters in this class.
(isnbspace): Likewise for single byte charsets.
(main): Initialize posixly_correct from the environment,
to allow disabling honoring NBSP in non C locales.
(wc): Call is[w]nbspace() along with is[w]space.
* bootstrap.conf: Ensure btowc is available.
* tests/misc/wc-nbsp.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the change in behavior.

6 years agodoc: more date +%F clarifications
Paul Eggert [Mon, 25 Feb 2019 18:20:23 +0000 (10:20 -0800)] 
doc: more date +%F clarifications

* doc/coreutils.texi (Date conversion specifiers):
Plain %F is actually like %+4Y-%m-%d.
(Padding and other flags): Mention POSIX restrictions.
* src/date.c (usage): Document recent changes.

6 years agodoc: give date +%+F example
Paul Eggert [Mon, 25 Feb 2019 16:03:04 +0000 (08:03 -0800)] 
doc: give date +%+F example

* doc/coreutils.texi (Padding and other flags):
Give example for + conversion specification.

6 years agodoc: fix typo in previous patch
Paul Eggert [Mon, 25 Feb 2019 08:19:22 +0000 (00:19 -0800)] 
doc: fix typo in previous patch