* Makefile.am: Remove our dependence on src/sort which
induces awkward dependencies for `make dist` since
THANKS will be rebuilt once src/sort is newer.
Instead we remove the problematic -f option to sort
which actually doesn't change the order given
our current input.
tests: avoid side effects of $SHELL environment variable
Since non interactive shells don't generally set $SHELL,
its value is propagated through the tests and may cause issues;
for example if $SHELL implicitly adjusts $PATH when run.
Instead we set $SHELL to that determined by the posix-shell module,
and use that consistently for all test sub scripts,
including those created thorugh the `split --filter` command.
* tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL)
which defaults to $CONFIG_SHELL and thus usually /bin/sh.
* tests/envvar-check: Remove bash environment variables with
side effects, in case /bin/bash was selected for $SHELL.
* tests/misc/help-version.sh: Remove redundant initialization of $SHELL.
* tests/install/strip-program.sh: Use $SHELL for sub script.
* tests/misc/sort-compress-hang.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars.
* tests/misc/pwd-long.sh: Likewise.
tests/misc/wc-proc.sh fails when the page size is 64K
* src/wc.c (wc): The lseek adjustment should be based on st_blksize,
rather than on the internal buffer size. This is significant on
aarch64 where st_blksize in /proc is the 64K (the page size) and
thus larger than the internal buffer.
* src/split.c (main): Even though the similar processing is done
on the internal buffer size, that's based on st_blksize and
so fine in this regard. Add an assert to enforce this.
Avoid this path for the undocumented ---io-blksize option.
Pádraig Brady [Tue, 30 Jun 2015 23:10:23 +0000 (00:10 +0100)]
numfmt: increase precision on 32 bit FreeBSD
* m4/jm-macros.m4 (HAVE_FPSETPREC): Define if needed.
* src/numfmt.c (main): Call fpsetprec() if needed.
Fixes large-15 and large-16 test failures on 32 bit FreeBSD.
Pádraig Brady [Tue, 30 Jun 2015 05:58:57 +0000 (06:58 +0100)]
factor: ensure atomic output through pipes
The new tests/misc/factor-parallel.sh test was
seen to fail on FreeBSD (derived) systems, which was
due to split(1) --filter reading partial lines
through pipes, as factor(1) was writing a little
over PIPE_BUF each time.
* src/factor.c (lbuf): A new structure to internally buffer lines.
(lbuf_alloc): A new function to allocate enough at program start.
(lbuf_putint): A new function to buffer a uintmax_t.
(lbuf_flush): A new function to write directly to standard output.
(lbuf_putc): A new function to buffer a character and if enough
lines are buffered, then output complete lines <= PIPE_BUF,
and continue to buffer the rest.
(main): Call the internal buffer allocator, and register
the final flush from the internal buffer at program exit.
Pádraig Brady [Tue, 30 Jun 2015 02:51:28 +0000 (03:51 +0100)]
tests: fix false failure on slower systems
* tests/dd/stats.sh: Wait 20s for dd to write 250MB through a fifo,
rather than 10s for 500MB. The failure was seen often on
a lightly loaded SPARC-Enterprise-T5220 running Solaris 10.
Jim Meyering [Sun, 28 Jun 2015 17:35:04 +0000 (10:35 -0700)]
maint: stdbuf.c: avoid the OS X putenv function
* bootstrap.conf (gnulib_modules): Add setenv, to make this
module dependency explicit; setenv is also used by split.
* src/stdbuf.c (set_LD_PRELOAD) [__APPLE__]: Use the OS X setenv
function, rather than putenv, per that documentation:
https://developer.apple.com/\
library/mac/documentation/Darwin/Reference/ManPages/man3/putenv.3.html
Jim Meyering [Mon, 29 Jun 2015 17:25:07 +0000 (10:25 -0700)]
build: numfmt.c: avoid a shadowing warning
* src/numfmt.c (parse_field_arg): Rename parameter s/optarg/arg/,
to avoid shadowing getopt's global variable.
Otherwise, building on OS X, with --enable-gcc-warnings, I saw this:
In file included from src/numfmt.c:19:0:
src/numfmt.c: In function 'parse_field_arg':
./lib/config.h:3109:25: error: declaration of 'rpl_optarg' shadows\
a global declaration [-Werror=shadow]
Assaf Gordon [Mon, 29 Jun 2015 11:54:41 +0000 (12:54 +0100)]
numfmt: fix printf argument order
* src/numfmt.c (double_to_human): Fix the argument order
passed to snprintf, which happened to work on amd64 with
its separate va_arg storage area for floats¹,
but would fail tests for example on i686.
Pádraig Brady [Sun, 28 Jun 2015 02:53:51 +0000 (03:53 +0100)]
maint: clarify df's use of device IDs from /proc/self/mountinfo
* src/df.c (filter_mount_list): Clarify why we still stat even
though devices IDs may already be available. Note using
/proc/self/mountinfo is still an advantage to get filtered items
with accurate device patchs in chroots and with bind mounts.
I.E. on older setups with static /etc/mtab, df will now
bypass that to get the more accuracte and dynamic info.
Pádraig Brady [Thu, 25 Jun 2015 15:39:36 +0000 (16:39 +0100)]
maint: clarify integer operations in recent commit
* src/factor.c (print_uintmaxes): Comment that the
value of n_out doesn't matter on error, and add an
explicit cast to avoid any future warnings.
Suggested by Jim Meyering RE commit v8.23-229-g4d2d6c5
Pádraig Brady [Thu, 25 Jun 2015 12:21:28 +0000 (13:21 +0100)]
tests: avoid false failures with LD_PRELOAD=libasan.so.2
The LD_PRELOAD checks by -fsanitize=address are overly strict:
https://groups.google.com/forum/#!topic/address-sanitizer/jEvOJgkDqQk
A workaround is to first export LD_PRELOAD=libasan.so.2
The tests below are adjusted so that workaround is not discarded.
* tests/cp/no-ctx.sh: Append to $LD_PRELOAD.
* tests/df/no-mtab-status.sh: Likewise.
* tests/df/skip-duplicates.sh: Likewise.
* tests/ls/getxattr-speedup.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/cp/nfs-removal-race.sh: Likewise. Also check that
LD_PRELOAD is effective to aid future maintainability
and avoid false failure if libasan.so.2 is not preloaded.
Pádraig Brady [Wed, 24 Jun 2015 18:14:47 +0000 (19:14 +0100)]
maint: avoid undefined behavior in qsort call
GCC 5.1.1 -fsanitize=undefined with glibc 2.21 is returning:
"runtime error: null pointer passed as argument 1,
which is declared to never be null"
* src/ptx.c (sort_found_occurs): Avoid the call with no entries.
Pádraig Brady [Wed, 15 Oct 2014 01:18:35 +0000 (02:18 +0100)]
factor: avoid interspersed lines for parallel runs
* src/factor.c (n_out): A new global variable to track
how much data has been written to stdout.
(print_factors_single): Use n_out to determine whether
to flush the current (and previous) lines.
* tests/misc/factor-parallel.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Pádraig Brady [Tue, 23 Jun 2015 21:51:24 +0000 (22:51 +0100)]
seq: handle exponents more consistently
src/seq.c (scan_arg): Set precision and width _after_ exponentiation.
For example, this will make '1.1e1 12' and '11 1.2e1' equivalent.
One can still set the precision by specifying extra precision on
the start value, or more naturally with a precision on a step value.
* tests/misc/seq-precision.sh: Add new cases.
Pádraig Brady [Tue, 23 Jun 2015 21:48:25 +0000 (22:48 +0100)]
seq: use consistent output format with hex integers
* src/seq.c (scan_arg): Set precision to 0 for hex constants
(while avoiding hex floats). This will use then use the
fast path for these arguments. Note we also set the precision
of inf to 0 here, which ensures we use consistent precision
on output where possible.
* tests/misc/seq-precision.sh: Add corresponding test cases.
Pádraig Brady [Sat, 11 Oct 2014 09:06:48 +0000 (10:06 +0100)]
seq: support inf last item more generally/efficiently
* src/seq.c (main): Call seq_fast for infinite last value.
This implicitly avoids format conversion on the
999999 -> 1000000 transition.
* src/seq.c (seq_fast): Generalize the buffer handling,
and adjust to handle the "inf" last value specifics.
* tests/misc/seq-precision.sh: A new test.
* tests/local.mk: Reference the new test.
Pádraig Brady [Mon, 22 Jun 2015 03:19:48 +0000 (04:19 +0100)]
numfmt: don't hardcode floating point limits
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
Pádraig Brady [Mon, 22 Jun 2015 01:12:32 +0000 (02:12 +0100)]
numfmt: handle leading zeros correctly
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits. Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
Pádraig Brady [Sun, 21 Jun 2015 03:04:29 +0000 (04:04 +0100)]
numfmt: avoid integer overflow when rounding
Due to existing limits this is usually triggered
with an increased precision. We also add further
restrictions to the output of increased precision numbers.
* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
Pádraig Brady [Fri, 19 Jun 2015 18:18:21 +0000 (19:18 +0100)]
numfmt: support user specified output precision
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers. Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion. If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules. numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
Pádraig Brady [Mon, 15 Jun 2015 02:55:46 +0000 (03:55 +0100)]
numfmt: handle suffixes consistently with --{from,to}-unit
* src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes
with the --from-unit and --to-unit options. Treat suffixes like
is done with --from=auto, which for example will change the meaning
of --to-unit=G to that of --to-unit=Gi. The suffix support was
previously undocumented and it's better to avoid the traditional
coreutils suffix handling in numfmt by default.
* doc/coreutils.texi: Document the new behavior. Also fix a typo
mentioning {from,to}=units=.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Pádraig Brady [Wed, 17 Jun 2015 11:58:53 +0000 (12:58 +0100)]
maint: remove stale online manual items at release
* gnulib: Update to get the new gnu-web-doc-update with --mirror option.
* README-release: Use the --mirror option in the instructions.
Also clarify and update various release steps.
Pádraig Brady [Tue, 9 Jun 2015 10:33:44 +0000 (11:33 +0100)]
tail: display consistent diagnostics upon file replacement
* src/tail.c (recheck): Display diagnostices for replaced files
even with reused inodes which is a common case.
* tests/tail-2/F-vs-missing.sh: Use correct diagnostic in comment.
* tests/tail-2/F-vs-rename.sh: Likewise.
Pádraig Brady [Mon, 8 Jun 2015 23:53:35 +0000 (00:53 +0100)]
tail: display file headers correctly with inotify
* src/tail.c (tail_forever_inotify): Use the fspec pointer to
distinguish previously output files, rather than a descriptor
from the inotify event. That event descriptor was that of
the parent directory when files were created or renamed etc.
(check_fspec): Adjust for the new comparison. Also show the
header when the file is truncated, since we show data
in this case also.
* tests/tail-2/F-headers.sh: A new test case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Pádraig Brady [Sun, 7 Jun 2015 16:18:39 +0000 (17:18 +0100)]
maint: sync .gitignore items with gnulib entries
* .gitignore: Add entries for potentially generated headers.
Also remove a couple of items already present in lib/.gitignore.
* cfg.mk (sc_gitignore_missing): A new syntax check rule to
identify missing .gitignore entries.
(sc_gitignore_redundant): A new syntax check rule to
identify redundant .gitignore entries.
Reported by Tomas Nordin.
Pádraig Brady [Sat, 6 Jun 2015 13:17:14 +0000 (14:17 +0100)]
tests: fix false failure in recent test adjustment
* configure.ac: Comment on why we link rather than run the test,
and remove the moot __ELF__ check since we never ran it anyway,
and the new CFLAGS and LDFLAGS are a more direct test of support.
* tests/misc/wc-parallel.sh: Fix a syntax error in the previous change.
* tests/misc/md5sum-parallel.sh: Use better error checking, consistent
with that used in wc-parallel.sh.
Paul Eggert [Sat, 6 Jun 2015 00:33:54 +0000 (17:33 -0700)]
build: port to AIX
Problems reported by Michael Felt, and and part of this fix taken
from code suggested by Pádraig Brady in:
http://bugs.gnu.org/20733#112
* configure.ac (stdbuf_supported): Check for warnings, and
for -fPIC and -shared, for AIX.
* src/stat.c (STRUCT_STATVFS): Define to struct statvfs64 if
STATFS is statvfs64.
* src/sync.c (sync_arg) [_AIX]: Open in write mode,
since AIX fsync doesn't work on read-only file descriptors.
* tests/misc/wc-parallel.sh: Skip test if xargs -P does not work.
Paul Eggert [Thu, 4 Jun 2015 19:15:35 +0000 (12:15 -0700)]
build: port single_binary_prog to POSIX shell
Problem reported privately by Michael Felt.
* Makefile.am (install-exec-hook):
* src/local.mk (src/coreutils_symlinks, src/coreutils_shebangs)
(clean-local):
Port to POSIX shell, which doesn't allow 'for i in ; do ...'.
Pádraig Brady [Wed, 3 Jun 2015 03:29:45 +0000 (04:29 +0100)]
build: add a dependency on the gnulib tempname module
* bootstrap.conf: Add "tempname" which is needed by mktemp(1).
The explicit dependency supports running gnulib-tool with
the --conditional-dependencies option, used to minimize built
modules. Note on a Fedora 22 system, that results in avoiding
redundant builds of: areadlinkat.o asnprintf.o fd-hook.o
fseterr.o printf-args.o printf-parse.o sockets.o vasnprintf.o.
However --conditional-dependencies is not enabled, since it
currently precludes the inclusion of gnulib tests.
Pádraig Brady [Mon, 1 Jun 2015 15:46:33 +0000 (16:46 +0100)]
tests: fix race in tail test without inotify
* tests/tail-2/wait.sh: Without inotify, skip a portion of the test
that checks that -F never outputs from a tailed descriptor
after the followed name is recreated, because tail_forever()
doesn't guarantee that.
Noticed at http://hydra.nixos.org/build/22766288
Pádraig Brady [Mon, 1 Jun 2015 14:38:23 +0000 (15:38 +0100)]
tests: fix false failure on loaded systems
* tests/misc/uniq-perf.sh: Use our standard 10s timeout,
which is sufficient to trigger the failure and also
avoids a false failure on slow/loaded systems.
Noticed at http://hydra.nixos.org/build/22766288
Pádraig Brady [Fri, 29 May 2015 14:33:59 +0000 (15:33 +0100)]
maint: avoid new coverity warnings
* src/sync.c (sync_arg): Initialise variable to avoid
unitialized access if assert is disabled.
* src/head.c (elide_tail_bytes_file): Support this function
with ---presume-input-pipe and larger files,
which regressed with commit v8.23-47-g2662702.
(elide_tail_lines_file): Likewise.
* src/dd.c (dd_copy): Explicitly don't try to ftruncate()
upon failure to lseek() (the existing check against
st_size was already protecting that).
* src/factor.c (factor_using_squfof): Assert (only when
linting due to performance) to avoid the implication of
divide by zero.
* src/od.c (read_block): Remove dead code.
* src/tac.c (tac_seekable): Likewise.
* src/ls.c (gobble_file): Likewise.
Pádraig Brady [Thu, 28 May 2015 01:25:29 +0000 (02:25 +0100)]
build: remove workarounds for unsupported gettext versions
Now that we depend on gettext >= 0.19.2 remove the workaround
for issues in autopoint 0.18.3. Note the scheme currently used in
newer gettext (autopoint) to avoid these issues requires
autoconf >= 2.69, therefore we update this requirement also.
Note the gettext version dependence from gnulib comes from
gnulib using gettext macros, and coreutils indirectly depends on
the gettext module due to:
http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=41dca647
bootstrap will then update m4/po.m4 and thus require a
supportng gettext version.
* bootstrap: Remove moot warning (resyncing with gnulib).
* configure.ac (AC_PREREQ): Change to 2.69 (now 3 years old).
Pádraig Brady [Thu, 28 May 2015 00:26:08 +0000 (01:26 +0100)]
copy: prefer our hardlink to symlink emulation on OS X 10.10
* src/copy.c (CAN_HARDLINK_SYMLINKS): Don't enable use of linkat()
on Darwin 14, as the gnulib fallback emulation there doesn't
preserve ownership and timestamps etc. This fixes a test failure
in tests/cp/link-symlink.sh
* tests/cp/link-deref.sh: Adjust accordingly.
Pádraig Brady [Thu, 21 May 2015 10:38:13 +0000 (11:38 +0100)]
mkdir: fix -pZ with existing parent directories
When the parent directory exists and has a different
default context to the final directory, the context
was incorrectly left as that of the parent directory.
* src/mkdir.c (process_dir): Because defaultcon() is called for
existing ancestors (as it must be to avoid races), then we must
unconditionally call restorecon() on the last component due to
the already documented caveat with make_dir_parents().
Alternatively you could temp disable o->set_security_context
around make_dir_parents(), but that would be subject to races.
* tests (tests/mkdir/restorecon.sh): Add a TODO for improvement.
Reference mknod and mkfifo with print_ver_.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20616
Pádraig Brady [Tue, 19 May 2015 11:36:29 +0000 (12:36 +0100)]
tests: fix non POSIX constructs causing failures with dash
* tests/cp/no-ctx.sh: Scope of `var=val func` is inconsistent
across shells, so avoid that construct with functions.
* tests/df/no-mtab-status.sh: Likewise.
* tests/tail-2/inotify-race.sh: `read` needs an argument.
* tests/tail-2/inotify-race2.sh: Likewise.
Pádraig Brady [Sun, 17 May 2015 14:52:00 +0000 (15:52 +0100)]
build: fix 'dist' and 'syntax-check' targets in VPATH build
* cfg.mk: Various syntax-check adjustments so that it's
not assumed the $builddir is the base distribution directory.
* Makefile.am: Likewise for the 'dist' target.
Pádraig Brady [Sun, 17 May 2015 09:51:15 +0000 (10:51 +0100)]
build: avoid issues with case insensitive file systems
* cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule.
* tests/tail-2/descriptor-vs-rename.sh: Rename from
tests/tail-2/f-vs-rename.sh
* tests/local.mk: Reference the renamed test.
Reported by Jim Meyering.
Pádraig Brady [Thu, 14 May 2015 10:43:54 +0000 (11:43 +0100)]
tests: fix async allocation race on BTRFS
* tests/dd/sparse.sh: Sync files before checking allocations,
which may be done asynchronously on NFS and BTRFS at least.
Also mark this test as very expensive on remote file systems.
* tests/du/2g.sh: Likewise, also use fallocate if available
to efficiently allocate the large file, otherwise skip
on remote file systems.
* tests/tail-2/inotify-rotate-resources.sh: Use the more
standard is_local_dir_() to check remoteness.
* tests/cp/fiemap-empty.sh: Comment on the sync issue
for this currerntly unused test.
Fixes http://bugs.gnu.org/20570
Pádraig Brady [Wed, 13 May 2015 12:41:42 +0000 (13:41 +0100)]
timeout: with --foreground don't send SIGCONT
* src/timeout.c (cleanup): Don't send SIGCONT to the monitored program
when --foreground is specified, as it's generally not needed for
foreground programs, and can cause intermittent signal delivery
issues with monitors like GDB for example.
* doc/coreutils.texi (timeout invocation): Mention that SIGCONT
is not sent with --foreground.
* NEWS: Mention the behavior change.
Pádraig Brady [Wed, 6 May 2015 00:48:40 +0000 (01:48 +0100)]
split: auto set suffix len for --numeric-suffixes=<N --number=N
Supporting `split --numeric-suffixes=1 -n100` for example.
* doc/coreutils.texi (split invocation): Mention the two
use cases for the FROM parameter, and the consequences on
the suffix length determination.
* src/split.c (set_suffix_length): Use the --numeric-suffixes
FROM parameter in the suffix width calculation, when it's
less than the number of files specified in --number.
* tests/split/suffix-auto-length.sh: Add test cases.
Fixes http://bugs.gnu.org/20511
Assaf Gordon [Wed, 13 May 2015 01:46:29 +0000 (02:46 +0100)]
doc: clarify the operation of wc -L
* src/wc.c (usage): State that it calculates display width.
* doc/coreutils.texi (wc invocation): Detail the distinct
items used to determine the display width.
Pádraig Brady [Mon, 11 May 2015 13:25:19 +0000 (14:25 +0100)]
tail: consistently output all data for truncated files
Generally if logs are truncated, they're truncated to 0 length,
so output all existing data when our heuristic determines truncation.
Note with inotify, truncate() and write() are often determined
independently and so all data would be written if that was the case.
* src/tail.c (check_fspec): Reset file offset to 0 upon truncation.
(tail_forever): Likewise.
(recheck): Add a FIXME for the related issue where tail may lose
data due to tail discounting older log files too early.
* tests/tail-2/truncate.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Pádraig Brady [Tue, 5 May 2015 21:11:24 +0000 (22:11 +0100)]
tail: fix inotify startup races
The previous fixes to races in the various tail tests,
identified actual races in the tail inotify implementation.
With --follow=descriptor, if the tailed file was replaced before
the inotify watch was added, then any subsequent changes were ignored.
Similarly in --follow=name mode, all changes to a new name were
effectively ignored if that name was created after the original open()
but before the inotify_add_watch().
* src/tail.c (tail_forever_inotify): Fix 3 cases.
1. With -f, don't stop tailing when file removed before watch.
2. With -f, watch right file when file replaced before watch.
3. With -F, inspect correct file when replaced before watch.
Existing tests identify these when tail compiled with TAIL_TEST_SLEEP.
* tests/tail-2/inotify-rotate-resources.sh:
This test also identifies the issue with --follow=name
when TAIL_TEST_SLEEP is used. Adjust so the test is immune
to such races, and also fail quicker on remote file systems.
* tests/tail-2/inotify-race2.sh: A new test using GDB,
based on inotify-race.sh, which tests the -F race
without needed recompilation with sleeps.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
Pádraig Brady [Fri, 1 May 2015 04:26:38 +0000 (05:26 +0100)]
tests: cleanup background processes upon interruption
Reap background processes so that:
- Stray processes aren't left on the system
- Files aren't held open causing deletion issues on NFS
- Partitions used to run the tests from can be unmounted
* tests/tail-2/F-vs-missing.sh: Add the `kill && wait` of the
background $pid(s) to cleanup_().
* tests/tail-2/F-vs-rename.sh: Likewise.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Likewise.
* tests/tail-2/assert-2.sh: Likewise.
* tests/tail-2/assert.sh: Likewise.
* tests/tail-2/flush-initial.sh: Likewise.
* tests/tail-2/inotify-hash-abuse.sh: Likewise.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Likewise.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/pid.sh: Likewise.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/retry.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
* tests/cp/existing-perm-race.sh: Likewise.
* tests/cp/file-perm-race.sh: Likewise.
* tests/cp/parent-perm-race.sh: Likewise.
* tests/cp/sparse-to-pipe.sh: Likewise.
* tests/dd/stats.sh: Likewise.
* tests/du/move-dir-while-traversing.sh: Likewise.
* tests/misc/cat-buf.sh: Likewise.
* tests/misc/help-version.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-spinlock-abuse.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/mv/i-3.sh: Likewise.
* tests/rm/dangling-symlink.sh: Likewise.
* tests/rm/isatty.sh: Likewise.
* cfg.mk (sc_prohibit_test_background_without_cleanup_):
A new syntax-check to ensure cleanup_() is defined
when background tasks are created in a test.
Pádraig Brady [Fri, 1 May 2015 17:03:37 +0000 (18:03 +0100)]
tests: avoid hung processes due to gdb SIGCONT handling
* tests/tail-2/inotify-race.sh: Add a `wait` to ensure that
we reap all background gdb and tail processes. That resulted
in the test hanging intermittently and upon investigation was
due to gdb intermittently failing to terminate the child process
due to receiving a SIGCONT signal. Therefore we avoid using
timeout(1) which sends that signal, and instead rely on tail's
inbuilt --pid monitoring on a background sleep process.
Given this new implementation, the VERY_EXPENSIVE guard was removed.
Related issues with this test hanging were previously discussed at:
https://lists.gnu.org/archive/html/bug-coreutils/2009-12/msg00025.html
Pádraig Brady [Sat, 2 May 2015 09:52:37 +0000 (10:52 +0100)]
tests: fix races in and standardize the tail tests
* tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options
(-s.1 --max-unchanged-stats=1) to speedup the non-inotify case.
Add the non-inotify case to the test. `wait` on the background
tail process to terminate which should avoid the need for the
non standard `retry_delay_ cleanup ...` on NFS.
* tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop,
to avoid a race in checking its contents. Also ensure 'a' & 'b'
files are present before the tail process starts. Use the standard
"fastpoll" options as above.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Use more standard variable names.
* tests/tail-2/flush-initial.sh: Use "fastpoll" options for
non-inotify platforms. Also `wait` on the background tail to avoid
stray processes and file cleanup issues on NFS.
* tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case.
Use "fastpoll" options. Use a more standard retry_delay_ instead
of a hardcoded sleep loop. Add a `wait` on the background tail.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Wait just on the
specific tail $pid needed.
* tests/tail-2/inotify-rotate.sh: Use "fastpoll" options.
* tests/tail-2/pid.sh: Use standard variable names.
Add a `wait` on the background tails.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/retry.sh: Use "fastpoll" options.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/wait.sh: Likewise. Speedup by using sub second
parameters to timeout(1). Improve the part ensuring that
-F never follows a renamed file.
* tests/tail-2/infloop-1.sh: Remove invalid test. tail(1) was not
being passed the --pid=$yes_pid option, retry_delay_ wasn't used
to avoid races, and yes could write huge files before being killed.
* tests/local.mk: Remove the invalid test reference.
* tests/tail-2/assert-2.sh: Rewrite using retry_delay_(). Since
no longer hardcoding large delays, remove the VERY_EXPENSIVE tag.
* tests/tail-2/assert.sh: Likewise.
doc: standardize messages about the '-' stdin FILE
* src/system.h (emit_stdin_note): A new function, refactoring
the usage note about the '-' FILE implying stdin.
* src/base64.c (usage): Use the new function to emit the
note in a standard location and with standard separation.
* src/cat.c (usage): Likewise.
* src/csplit.c (usage): Likewise.
* src/cut.c (usage): Likewise.
* src/expand.c (usage): Likewise.
* src/fmt.c (usage): Likewise.
* src/head.c (usage): Likewise.
* src/md5sum.c (usage): Likewise.
* src/nl.c (usage): Likewise.
* src/od.c (usage): Likewise.
* src/paste.c (usage): Likewise.
* src/pr.c (usage): Likewise.
* src/ptx.c (usage): Likewise.
* src/shred.c (usage): Likewise.
* src/shuf.c (usage): Likewise.
* src/sort.c (usage): Likewise.
* src/sum.c (usage): Likewise.
* src/tac.c (usage): Likewise.
* src/tail.c (usage): Likewise.
* src/tsort.c (usage): Likewise.
* src/unexpand.c (usage): Likewise.
* src/wc.c (usage): Likewise.
* src/join.c (usage): Adjust the separation used for
the message referring to FILE1 or FILE2 as stdin.
* src/comm.c (usage): Add a message using the same
wording (translation) as used in join.
* src/split.c (usage): Reword to using FILE rather than
INPUT, allowing use of emit_stdin_note(). Also remove
the mention of "fixed-size" pieces as this isn't now
always the case.
Fixes http://pad.lv/1450179
tests: don't skip df tests with /proc/self/mountinfo
* tests/df/no-mtab-status.sh: getmntent is no longer called
when /proc/self/mountinfo is present, thus causing the test
to be skipped. Therefore wrap fopen() to ignore mountinfo,
and use the test genmntent table instead.
* tests/df/skip-duplicates.sh: Likewise.
build: fix SINGLE_BINARY build when printf is a macro
* src/coreutils.c (usage): include coreutils.h outside
the printf call, because if it's a macro you will get the error:
embedding a #include directive within macro arguments is not supported
maint: avoid a new -Werror=return-type warning in yes.c
* src/yes.c (main): Simplify the logic so that the
compiler can see this function always returns a value.
This was seen with GCC 5.0 in SINGLE_BINARY mode.
tests: run ls capability coloring test irrespective of $LS_COLORS
* tests/ls/no-cap.sh: Ensure the test isn't skipped even if
capability coloring is disabled in the current $LS_COLORS.
Also just enable/disable capability coloring to avoid the
dircolors(1) overhead.
build: rely on gnulib to determine printf routines are safe
gnulib now only checks that the printf routines never crash,
which is all coreutils currrently requires, and so we revert
commit v8.23-81-gf57bfbb to let gnulib decide whether to replace
the system printf routines.
With GCC 5 and the newly added warnings from gnulib, ensure the
correct signed integer is passed for the printf format,
to avoid -Werror=format= failures.
* bootstrap.conf: 0.19.2 is available on openSUSE-13.2,
Debian-8.0, and Ubuntu-14.10. Given there were issues
with earlier 0.19 gettext releases, set this as the new minimum.
* configure.ac (AM_GNU_GETTEXT_VERSION): Likewise.
Reported by Bernhard Voelker
Michael Witten [Wed, 22 Apr 2015 15:07:02 +0000 (15:07 +0000)]
doc: fix grammar issue in truncate info
* doc/coreutils.texi (truncate invocation): The word 'their' is
incorrect; 'each file' is the antecedent, and is singular,
so 'its' is the correct pronoun.
build: fix potential factor build failure on arm and mips
* src/longlong.h: Sync with the latest longlong.h from libgmp to:
- Use __builtin_c[lt]zl on arm64.
- Fix sparc64 vis3 build failure due to missing __clz_tab.
- Avoid a clang build issue on mips.
- Support thumb2 arm 32 bit system.
* src/cfg.mk (sc_ensure_comma_after_id_est): Exclude longlong.h
to ease merges.
maint: avoid -Werror=strict-overflow warnings with GCC 5
All warnings were of the form: "assuming signed overflow does not occur
when simplifying conditional to constant [-Werror=strict-overflow]"
* src/dd.c (cache_round): Use an appropriately sized unsigned type,
to avoid possibility of undefined signed overflow.
* src/mknod.c (main): Likewise.
* src/pr.c (pad_down): Likewise.
* src/wc.c (main): Likewise.
* src/tail.c (main): Assert that argc >= 0 thus allowing the
compiler to assume without implication that argc - optind
is positive.
* src/dircolors.hin: Add the MISSING entry, to indicate
this as a possibility in new templates output from dircolors,
and also to ease comparison with existing databases that
generally do define a MISSING entry.
* doc/coreutils.texi (cp invocation): Mention that when copying files
without preserving permissions, the umask or a default ACL affect
the mode of new files.
* THANKS.in: Remove committer.
Related to http://bugs.gnu.org/8527
* doc/coreutils.texi: `mkfifo' and `mknod' use the optContext macro
which adds a description for the SELinux security context in addition to
the single option already described in each case. The result in both
cases is two options being introduced as `option' (singular). Fix this
by introducing them as `options' (plural).
df: fix use of uninitialized variable reported by valgrind
Conditional jump or move depends on uninitialised value(s)
at 0x40380C: get_field_values (df.c:840)
by 0x403E16: get_dev (df.c:994)
by 0x404D65: get_all_entries (df.c:1364)
by 0x405926: main (df.c:1714)
* src/df.c (get_dev): Initialize the fsu.fsu_bavail_top_bit_set
member, when adding placeholder entries.
(main): Avoid a "definitely lost" memory leak warning from valgrind,
reported by Bernhard Voelker.
* src/ls.c (usage): Add punctuation to avoid ambiguity in the
description of the --time option. Mention that both the -u
and --sort=time default order is newest first.
* src/tail.c (tail_forever_inotify): Only monitor write()s and
truncate()s to files in --follow=descriptor mode, thus avoiding
the bug where we removed the watch on renamed files.
Also adjust the inotify event processing code that is
now significant only in --follow=name mode.
* tests/tail-2/F-vs-rename.sh: Improve this existing test by running
in both polling and inotify modes.
* tests/tail-2/f-vs-rename.sh: A new test based on the existing one.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
Fixes http://bugs.gnu.org/19760
Assaf Gordon [Fri, 27 Mar 2015 09:38:48 +0000 (09:38 +0000)]
tests: fix false test failure with df on Debian/kFreeBSD
* tests/fs/skip-duplicates.sh: On this platform .mnt_opts is significant
so define to empty to avoid a NULL deref in read_file_system_list().
Fixes http://bugs.gnu.org/20210
Pádraig Brady [Thu, 26 Mar 2015 00:47:28 +0000 (00:47 +0000)]
doc: clarify the date standard output formats
* src/date.c (usage): Use FMT rather than TIMESPEC as the parameter,
since it's simpler to understand and can be better aligned.
Give an example for the --iso-8601 output format.
Adjust the example used for the 3 standard formats to be unambiguous
with respect to day/mon ordering and use of leading zeros in the time.
Reorder the options descriptions slightly, so that the
3 standards options are together.
Indent the multi-line descriptions so that grouping is obvious.
Remove a redundant description of the --rfc-3339 format,
which is obvious in the existing example.
Separate these 3 standards options to their own translatable string
to simplify translation.
Change 'date and time' to 'date/time' in the --iso-8601 description
to be consistent with --rfc-3339 and to help avoid the implication
that the time is always output or even output by default.
Fixes http://bugs.gnu.org/20203
Pádraig Brady [Mon, 23 Mar 2015 17:47:23 +0000 (17:47 +0000)]
maint: apply basic styling to online manual
* README-release: Reference http://www.gnu.org/s/coreutils/manual.css
to apply basic styling to the online coreutils manual, consistent
with the Emacs documentation.
Pádraig Brady [Mon, 23 Mar 2015 11:54:19 +0000 (11:54 +0000)]
wc: use a more adaptive wc -l implementation
* src/wc.c (wc): Allow any block to select the count implementation,
rather than just using the first 10 lines. This also simplifies
the code from 3 loops to 2.
projects / thirdparty / coreutils.git / log
