doc: document stat's output with the --terse option
* doc/coreutils.texi (stat invocation): Add a paragraph documenting
stat's output format when the --terse option is specified, both in
normal and in --file-system mode.
Reported by Dan Jacobson <jidanni@jidanni.org>
in http://bugs.gnu.org/18624
Nick Alcock [Thu, 2 Oct 2014 14:26:21 +0000 (15:26 +0100)]
tests: fix false failure for test referencing libdl
* init.cfg (gcc_shared_): -ldl has to be positioned after the object
files that may rely upon it. This fixes tests/cp/nfs-removal-race.sh
which references dlsym() from libdl.
A syntax-check recently added to gnulib would trigger a failure
(once gnulib gets updated here) for a statement introduced with
commit v8.23-43-gaf2a4ed:
dd: new status=progress level to print stats periodically
* src/dd.c: Report the transfer progress every second when the
new status=progress level is used. Adjust the handling and
description of the status= option so that they're treated as
mutually exclusive levels, rather than flags with implicit precedence.
* doc/coreutils.texi (dd invocation): Document the new progress
status level. Reference the new level in the description of SIGUSR1.
* tests/dd/stats.sh: Add new test for status=progress.
* tests/dd/misc.sh: Change so status=none only takes precedence
if it's the last level specified.
* NEWS: Mention the feature.
* src/dd.c (ifd_reopen): A new wrapper to ensure we
don't exit upon receiving a SIGUSR1 in a blocking open()
on a fifo for example.
(iftruncate): Likewise for ftruncate().
(iread): Process signals also after a short read.
(install_signal_handlers): Install SIGINFO/SIGUSR1 handler
even if set to SIG_IGN, as this is what the parent can easily
set from a shell script that can send SIGUSR1 without the
possiblity of inadvertently killing the dd process.
* doc/coreutils.texi (dd invocation): Improve the example to
show robust usage wrt signal races and short reads.
* tests/dd/stats.sh: A new test for various signal races.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
David Sterba [Wed, 24 Sep 2014 10:15:05 +0000 (11:15 +0100)]
mv: use reflink=auto mode by default
On some filesystems (BTRFS), moving a file within the filesystem may
cross subvolume boundaries and we can use a lightweight reflink copy,
similar to what cp(1) can do, which is faster than a full file copy.
This is enabled by default because it's only an optimization for
the fall back copy and does not break user expectations or usability.
* src/mv.c (cp_option_init): Set the reflink mode to AUTO.
* NEWS: Mention the improvement.
stty: only list supported options in --help and man pages
* src/stty.c (usage): Exclude unsupported options from --help,
which for example impacts the "dsusp" and "cdtrdsr" options on Linux.
Fixes http://bugs.gnu.org/18506
"E.g." stands for latin "exempli gratia" which is typically read
as "for example". "E.g." does not stand for the word "example".
As such, "for e.g." might be read as "for for example".
Fix this usage by simply replacing "e.g." with "example".
Bernhard Voelker [Mon, 22 Sep 2014 23:49:47 +0000 (01:49 +0200)]
build: pass program name to help2man after other options
* man/local.mk (.x.1): Move the program name argument down after
the last option argument when calling $(run_help2man).
While the other way would be accepted for the GNU help2man program,
it is not for the 'dummy-man' script (called as a fallback on
systems lacking perl).
The wrong order was introduced in commit v8.21-119-gb3578fc while
adding the --info-page option.
Bernhard Voelker [Mon, 22 Sep 2014 23:44:51 +0000 (01:44 +0200)]
build: fix argument count check in dummy-man again
* man/dummy-man: Fix argument count check, now only permitting
exactly 1 argument, the program name.
Reported by Andreas Schwab <schwab@linux-m68k.org>
build: fix dependency issues with man page generation
* .gitignore: Remove reference to no longer generated make file.
* configure.ac: Don't bother generating placeholder make file.
* man/local.mk: Hardcode the man page deps list for normal builds
to be compatible with all make implementations and configure options.
Note in SINGLE_BINARY mode, all man pages will be generated on
any change to the coreutils binary, but development will generally
not be done in this mode, so this shouldn't be an issue.
cp: fix handling of -H with multiply specified source dirs
Following on from commit v5.92-729-g130dd06, also avoid
the erroneous directory hardlink warning with -H.
* src/copy.c (copy_internal): Also handle the -H case
for command line arguments.
* tests/cp/duplicate-sources.sh: Augment the test case.
* NEWS: Augment the news entry.
cp: issue correct warning and ignore duplicate source dirs
* src/copy.c (copy_internal): Handle the case where we have the
same destination directory as already encountered, which can only
be due to the corresponding source directory being specified multiple
times.
* tests/cp/duplicate-sources.sh: Add a test for the new multiply
specified directory case, and the existing multiply specified file case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
doc: output correct --help references with --program-prefix
* src/system.h (emit_ancillary_info): Take the invariant PROGRAM_NAME
as a parameter, so that consistent references are made to online docs
and texinfo nodes, when a --program-prefix is in place. Note the
man pages don't need this fix as they're generated before the program
prefix is used.
* NEWS: Mention the improvements in references to online documentation.
doc: ensure the correct texinfo nodes are referenced in --help
* src/system.h (emit_ancillary_info): For commands that don't have
a 1:1 mapping with the texinfo node names, provide a mapping to
the correct node.
* doc/coreutils.texi: Add some extra cross references noticed while
checking this.
Fixes http://bugs.debian.org/762092
Bernhard Voelker [Tue, 16 Sep 2014 16:50:29 +0000 (18:50 +0200)]
tests: fix false du failure on newer XFS
On XFS, when creating the ~2G test file 'big' in a for-loop by
appending 20M each time, the file ends up using ~4G - visible in
'st_blocks'. The unused space would be reclaimed later.
This feature is called "speculative preallocation" which aims at
avoiding fragmentation.
According to the XFS FAQ [1], there are two particular aspects of
XFS speculative preallocation that are triggering this:
1. "Applications that repeatedly trigger preallocation and reclaim
cycles [after file close] can cause fragmentation.
Therefore, this pattern is detected and causes the preallocation
to persist beyond the lifecycle of the file descriptor."
2. "Preallocation sizes grow as files grow larger."
[1] http://xfs.org/index.php/XFS_FAQ
Avoid one of the above by only doing a single close (reclaim cycle).
* tests/du/2g.sh: Similar to the fix for a dd test (see commit v8.22-65-g7c03fe2), avoid speculative preallocation by creating
the 'big' file in one go instead of appending to it in the loop.
Remove debugging statements as the output with 'set -x' is
sufficient nowadays.
Paul Eggert [Thu, 11 Sep 2014 15:45:32 +0000 (08:45 -0700)]
cat: allow copying empty files to themselves
Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449
* src/cat.c (main): Allow copying an empty file to itself.
* tests/misc/cat-self.sh: New test.
* tests/local.mk (all_tests): Add it.
doc: reference online info pages directly from man pages
* src/system.h (emit_ancillary_info): Add a direct reference
to the corresponding online info documentation. Corresponding
redirects were put in place on www.gnu.org to allow for concise links.
* help2man: Adjust to add the "online help" link (and subsequent
translation bugs link) to a "REPORTING BUGS" section.
Also add the concise links for further information in --help
to the "SEE ALSO" section, and dispense with the more verbose
default for that.
old form: coreutils '$cmd invocation'
new form: '(coreutils) $cmd invocation'
The old form erroneously referenced the node for the 'coreutils'
multi-call program. Now that problematic node name was renamed
in commit v8.23-18-g72e470b, but the newer less ambiguous form
also has the advantage of working with the pinfo viewer for example.
Full discussion at http://bugs.gnu.org/18428
* man/local.mk: Adjust man page references to texinfo nodes.
* src/system.h: Adjust --help references to texinfo nodes.
build: adjust previous transformations on libstdbuf name
* src/local.mk (transform): commit v8.23-22-g6f9b018 discarded all
transformations on the libstdbuf.so name. Be more conservative and
only exclude the $(program_transform_name) portion for libstdbuf.
* src/local.mk (transform): Skip the transformation for libstdbuf
since that should not be subject to name clashes, and we need
to reference the name directly in LD_PRELOAD etc.
* configure.ac: Add a comment on the coupling of pkglibexec_PROGRAMS
to $(transform).
Issue reported at https://trac.macports.org/ticket/44922
Improved by Nick Bowler
Paul Eggert [Tue, 9 Sep 2014 03:41:44 +0000 (20:41 -0700)]
maint: avoid file-scope names of the form _[a-z]*
The C standard says this isn't portable, if you include
standard include files.
* build-aux/gen-single-binary.sh:
* src/coreutils-arch.c (single_binary_main_arch)
(single_binary_main_uname):
* src/coreutils-dir.c (single_binary_main_ls)
(_single_binary_main_dir):
* src/coreutils-vdir.c (single_binary_main_ls)
(_single_binary_main_vdir):
* src/coreutils.c (SINGLE_BINARY_PROGRAM):
Remove leading _ from single_binary prefix.
* src/numfmt.c (round_style): Rename from _round. All uses changed.
(inval_style): Rename from _invalid. All uses changed.
Paul Eggert [Mon, 8 Sep 2014 19:51:14 +0000 (12:51 -0700)]
doc: rename "coreutils invocation" to "Multi-call invocation"
This supports longstanding shell commands like
'info coreutils "touch invocation"'.
Problem reported by Vincent Lefevre via Bob Proulx in:
http://bugs.gnu.org/18428
* doc/coreutils.texi (Multi-call invocation):
Rename from "coreutils invocation".
Pádraig Brady [Sat, 23 Aug 2014 00:18:58 +0000 (01:18 +0100)]
maint: ensure fiemap extents flags are compared correctly
* src/extent-scan.c (extent_scan_read): Following on from the flags size
adjustment in commit v8.23-13-g1505b37, verify that the internal
representation of the flags is never truncated which could happen in the
unlikely case on 32 bit if the kernel flags ever expanded to 64 bits
which is theoretically possible given the reserved space.
Paul Eggert [Fri, 22 Aug 2014 19:07:11 +0000 (12:07 -0700)]
maint: avoid int64_t and similar types unless they're needed
C11 doesn't require them, even POSIX doesn't strictly require the
64-bit versions, and it makes the code a bit clearer if they're
used only when needed.
* src/copy.c (write_zeros, extent_copy):
* src/extent-scan.h (struct extent_info.ext_length):
Use off_t, not uint64_t, for a value derived from a file offset.
* src/extent-scan.h (struct extent_info.ext_flags)
Prefer plain unsigned int to uint32_t where either will do.
(struct extent_scan.ei_count):
Use size_t, not uint32_t, for a value bounded by SIZE_MAX.
* src/factor.c (MAGIC64, MAGIC63, MAGIC65):
Remove unnecessary casts to uint64_t.
Fridolin Pokorny [Thu, 21 Aug 2014 09:45:54 +0000 (11:45 +0200)]
doc: clarify that duplicate NFS mounts are skipped by df
* doc/coreutils.texi (df invocation): Add a sentence that eliding
duplicate entries for the same file system is not limited to bind
mounts, but also happens for remote file systems like NFS.
Pádraig Brady [Mon, 18 Aug 2014 16:59:26 +0000 (17:59 +0100)]
df: improve mount point selection with inaccurate mount list
v8.23 has a test failure on Fedora rawhide build servers
in tests/df/skip-duplicate.sh. This was due to no '/'
entry being output by df. That was due to an inaccurate
/proc/mounts on the build environment as stat(/mnt/point)
identified all these /proc/mounts entries as having the
same device id:
Since the device name on the right changes for a given id,
that causes the entries to be continually replaced, thus
resulting in no '/' entry. I'm guessing this is due to
the mock environment bind mounting unneeded or sensitive
items to a dummy file on the host / (/dev/md1) though
have not looked into those details.
So rather than relying on an accurate /proc/mounts,
the attached patch takes a more conservative replacement
approach and only swaps a new device entry when the
mount point matches. That should handle all practical
cases while also avoiding this situation.
* src/df.c (filter_mount_list): Only replace entries with
different device names when the mount point also matches.
If the hash structures grow sufficiently large so that
the system is actively swapping, then the deallocation
can take a significant amount of time. Details at:
http://lists.gnu.org/archive/html/coreutils/2014-08/msg00012.html
* src/cp.c (main): Only call hash deallocation routines
when in lint checking mode.
* THANKS.in: Remove as now in the git author list.
chroot: perform chdir("/") again unless new --skip-chdir is specified
Since commit v8.22-94-g99960ee, chroot(1) skips the chroot(2) syscall
for "/" arguments (and synonyms). The problem is that it also skips
the following chdir("/") call in that case. The latter breaks existing
scripts which expect "/" to be the working directory inside the chroot.
While the first part of the change - i.e., skipping chroot("/") - is
okay for consistency with systems where it might succeed for a non-root
user, the second part might be malicious, e.g.
cd /home/user && chroot '/' bin/foo
In the "best" case, chroot(1) could not execute 'bin/foo' with ENOENT,
but in the worst case, chroot(1) would execute '/home/user/bin/foo' in
the case that exists - instead of '/bin/foo'.
Revert that second part of the patch, i.e., perform the chdir("/)
in the common case again - unless the new --skip-chdir option is
specified. Restrict this new option to the case of "/" arguments.
* src/chroot.c (SKIP_CHDIR): Add enum.
(long_opts): Add entry for the new --skip-chdir option.
(usage): Add --skip-chdir option, and while at it, move the other
to options into alphabetical order.
(main): Accept the above new option, allowing it only in the case
when NEWROOT is the old "/".
Move down the chdir() call after the if-clause to ensure it is
run in any case - unless --skip-chdir is specified.
Add a 'newroot' variable for the new root directory as it is used
in a couple of places now.
* tests/misc/chroot-fail.sh: Invert the last tests which check the
working directory of the execvp()ed program when a "/"-like
argument was passed: now expect it to be "/" - unless --skip-chdir
is given.
* doc/coreutils.texi (chroot invocation): Document the new option.
Document that chroot(1) usually calls chdir("/") unless the new
--skip-chdir option is specified. Sort options.
* NEWS (Changes in behavior): Mention the fix.
(New features): Mention the new option.
* init.cfg (nonroot_has_perm_): Add chroot's new --skip-chdir option.
* tests/cp/preserve-gid.sh (t1): Likewise.
* tests/cp/special-bits.sh: Likewise.
* tests/id/setgid.sh: Likewise.
* tests/misc/truncate-owned-by-other.sh: Likewise.
* tests/mv/sticky-to-xpart.sh: Likewise.
* tests/rm/fail-2eperm.sh: Likewise.
* tests/rm/no-give-up.sh: Likewise.
* tests/touch/now-owned-by-other.sh: Likewise.
Reported by Andreas Schwab in http://bugs.gnu.org/18062
* configure.ac: Don't add stdbuf to the list of programs to build
if EXEEXT is set, as that is not handled in configure.ac for
libstdbuf.so yet (see bin_PRGRAMS handling in configure.ac).
Also the LD_PRELOAD mechanism will need to be adjusted to support
cygwin in any case, so avoid stdbuf completely in this case for now.
Problem reported by Eric Blake.
Paul Eggert [Sat, 19 Jul 2014 14:55:41 +0000 (07:55 -0700)]
maint: fix message translation glitches
Problem reported by Sebastian Rasmussen in: http://bugs.gnu.org/18054
* gl/lib/randread.c (randread_error): Don't put multiple string
literals inside _(...), as xgettext doesn't support that.
* src/chroot.c (main): In diagnostics, don't bother to distinguish
between setting the number of supplemental group IDs to a zero or
to a nonzero value, as the underlying system call is the same
either way. This also makes the string easier to translate correctly.
tests: fix false failure with spaces in the test directory
* tests/misc/env.sh: Skip if we can't execute the generated shebang,
which would be the case if there are spaces in the directory hierarchy.
This is triggered by `make distcheck`
This issue was identified by the manifest comparisons
done by `make distcheck`
* src/local.mk (noinst_HEADERS): Remove coreutils.h from this always
distributed list.
(nodist_src_coreutils_SOURCES): Add coreutils.h as its contents
are determined at configure time, so pointless to distribute.
(src_coreutils_SOURCES): Define explicitly so that the corresponding
nodist_ variable is honored.
(DISTCLEANFILES): Add coreutils.h to this rather than CLEANFILES,
as its contents are determined at configure time.
tests: fix false failure in cp --preserve=context test
With libselinux-2.2.1-6.fc20.x86_64, kernel-3.12.6-300.fc20.x86_64
`cp --preserve=context src dst` was seen to succeed when src and
dst where on the same fixed context file system, as lsetfilecon()
returned success in this case when the context wasn't being changed.
* tests/cp/cp-a-selinux.sh: Copy from a different file system to
most likely have a different context that will test context
setting logic correctly.
Paul Eggert [Thu, 17 Jul 2014 20:08:36 +0000 (13:08 -0700)]
build: port to non-GNU make
This fixes a problem with native Solaris 'make', which does not
grok '-include' lines (a GNU extension to POSIX 'make').
* configure.ac (man/dynamic-deps.mk): Create it, with an old
time stamp, if doing dynamic dependency tracking.
* man/local.mk (DISTCLEANFILES): Put man/dynamic-deps.mk here,
rather than in CLEANFILES.
(man/dynamic-deps.mk): Don't create it read-only, so that we
can easily touch it later.
Include it with '@AMDEP_TRUE@@am__include@', not with '-include',
as '-include' does not work with native Solaris 'make'.
Assaf Gordon [Tue, 15 Jul 2014 16:25:03 +0000 (12:25 -0400)]
numfmt: fix isblank() usage for some unibyte locales
* src/numfmt.c (simple_strtod_int): Replace isdigit() with c_isdigit()
to avoid locale concerns and -Wchar-subscripts warnings on cygwin.
Remove the now redundant locale guard.
(simple_strtod_human): Cast characters to unsigned so that the promoted
int value passed to isblank() is positive, allowing it to work correctly
for all characters in unibyte locales. Previously character 0xA0,
i.e. non-breaking space, would be misclassified for example.
(process_suffixed_number): Likewise.
(skip_fields): Likewise.
Both issues were triggered by the -Wchar-subscripts warning on GCC 4.8.3
on cygwin, due to the is*() implementations used there, but the issue
is present on all platforms defaulting to signed chars.
* NEWS: Mention the bug fix.
* tests/dd/ascii.sh: Quote so that '\\' is passed
to printf rather than a single '\', as that's not portable.
bash, dash, zsh and external solaris printf were seen to support
printf '\%03o' 1 2
while ksh, freebsh shell and external GNU printf need the more portable
printf '\\%03o' 1 2
Note we don't use env here to call the coreutils printf implementation,
as there are many printf calls, so relying on portable shell
implementations will be faster.
tests: avoid hang when files created with wrong permissions
* tests/split/b-chunk.sh: Never show the rm prompt which would
hang the test suite. This was seen when split erroneously
created files with no persmissions, which was triggered by
this compiler bug in clang 3.4:
http://llvm.org/bugs/show_bug.cgi?id=18346
* gnulib: Sync recent cleanups and the fix for
missing df entries in the presence of bind mounts:
http://bugs.gnu.org/17833
* NEWS: Detail the df bug fix.
sort: avoid undefined operation with destroying locked mutex
This didn't seem to cause any invalid operation on GNU/Linux at least,
but depending on the implementation, mutex deadlocks could occur.
For example this might be the cause of lockups seen on Solaris:
http://lists.gnu.org/archive/html/coreutils/2013-03/msg00048.html
This was identified with valgrind 3.9.0 with this setup:
With that, valgrind would _intermittently_ report the following:
Destroying locked mutex: mutex 0x5419548, recursion count 1, owner 2.
at 0x4C2E3F0: pthread_mutex_destroy(in vgpreload_drd-amd64-linux.so)
by 0x409FA2: sortlines (sort.c:3649)
by 0x409E26: sortlines (sort.c:3621)
by 0x40AA9E: sort (sort.c:3955)
by 0x40C5D9: main (sort.c:4739)
mutex 0x5419548 was first observed at:
at 0x4C2DE82: pthread_mutex_init(in vgpreload_drd-amd64-linux.so)
by 0x409266: init_node (sort.c:3276)
by 0x4092F4: init_node (sort.c:3286)
by 0x4090DD: merge_tree_init (sort.c:3234)
by 0x40AA5A: sort (sort.c:3951)
by 0x40C5D9: main (sort.c:4739)
Thread 2:
The object at address 0x5419548 is not a mutex.
at 0x4C2F4A4: pthread_mutex_unlock(in vgpreload_drd-amd64-linux.so)
by 0x4093CA: unlock_node (sort.c:3323)
by 0x409C85: merge_loop (sort.c:3531)
by 0x409F8F: sortlines (sort.c:3644)
by 0x409CE3: sortlines_thread (sort.c:3574)
by 0x4E44F32: start_thread (in /usr/lib64/libpthread-2.18.so)
by 0x514EEAC: clone (in /usr/lib64/libc-2.18.so)
* src/sort.c (sortlines): Move pthread_mutex_destroy() out to
merge_tree_destroy(), so that we don't overlap mutex destruction
with threads still operating on the nodes.
(sort): Call the destructors only with "lint" defined, as the
memory used will be deallocated implicitly at process end.
* NEWS: Mention the bug fix.
For tool usage and error details see:
http://valgrind.org/docs/manual/drd-manual.html
* src/sort.c (queue_insert): Unlock mutex _after_ signalling the
associated condition variable. Valgrind flags this with:
"Probably a race condition: condition variable 0xffeffffb0 has been
signaled but the associated mutex 0xffeffff88 is not locked by the
signalling thread."
The explanation at the above URL is:
"Sending a signal to a condition variable while no lock is held on
the mutex associated with the condition variable. This is a common
programming error which can cause subtle race conditions and
unpredictable behavior."
This should at least give more defined scheduling behavior.
(merge_tree_destroy): Make symmetrical with merge_tree_init() thus
destroying the correct mutex. Valgrind flags this with:
"The object at address 0x5476cf8 is not a mutex."
Jim Meyering [Sun, 13 Jul 2014 17:24:33 +0000 (10:24 -0700)]
build: adjust new rule not to depend on bash-4.x
* man/local.mk (man/dynamic-deps.mk): Use the same code to
derive FOO from man/FOO.1 as in the .x.1 rule below.
Using the more concise "name=$${man:4: -2}" is not portable enough.
Alex Deymo [Fri, 6 Jun 2014 02:50:32 +0000 (19:50 -0700)]
build: support building all tools in a single binary
Add the --enable-single-binary option to the configure file.
When enabled, this option builds a single binary file containing
the selected tools. Which tool gets executed depends on the value
of argv[0] which can be set implicitly through symlinks to the
single program.
This setup reduces significantly the size of a complete coreutils
install, since code from lib/libcoreutils.a is not duplicated in
every one of the more than 100 binaries. Runtime overhead is
increased due to more dynamic libraries being loaded, and extra
initialization being performed for all utils. Also initially
a larger binary is loaded from storage, though this is usually
alleviated due to caching and lazy mmaping of unused blocks,
and in fact the single binary should have better caching
characteristics.
Comparing the size of the individual versus single binary on x86_64:
$ cd src
$ size coreutils
$ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
text data bss dec hex filename 1097416 5388 88432 1191236 122d44 src/coreutils 4901010 124964 163768 5189742 4f306e (TOTALS)
Storage requirements are reduced similarly:
$ cd src
$ du -h coreutils
$ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
1.2M coreutils
5.3M total
When installing, the makefile will create either symlinks or
shebangs based on the --enable-single-binary setting, for
each configured tool. In this way, all the tools are still
callable individually, but they are all implemented by the same
"coreutils" binary installed on the same directory.
* .gitignore: Add new generated files.
* Makefile.am: New rules to generate build-aux/gen-single-binary.sh
and install symlinks.
* NEWS: Mention the new feature.
* README: Add "coreutils" to the list of utils.
* bootstrap.conf: Regenerate src/single-binary.mk
* build-aux/gen-lists-of-programs.sh: New --list-progs option.
* build-aux/gen-single-binary.sh: Regenerate
* configure.ac: New --enable-single-binary option and other variables.
Disallow --enable-single-binary=symlinks with --program-prefix et. al.
* man/coreutils.x: Manpage hook.
* man/local.mk: Add manpage hook and fix dependencies.
* src/coreutils.c: Multicall implementation.
* src/local.mk: New rules for the single binary option.
* tests/local.mk: Add $single_binary_progs to support
require_built_() from init.cfg
* tests/misc/env.sh: Avoid the use of symlink to echo.
* tests/misc/help-version.sh: Add exception for coreutils.
* tests/install/basic-1.sh: Really avoid using ginstall strip
functionality if there is an issue with the independent strip command.
* src/kill.c: Changes to call exit() in main.
* src/readlink.c: Likewise.
* src/shuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
maint: simplify parsing df's output in shell scripts
Avoid complicated and error-prone parsing of df's output via
sed(1), cut(1), etc., and instead use df's more modern --output
option.
* src/ioblksize.h (in a comment): Simplify the extraction of the
device name of the mounted file system from df's output.
* tests/dd/skip-seek-past-dev.sh: Likewise.
* tests/du/2g.sh: Likewise for the 'avail' column here.
Also avoid the deprecated use of "tail -NUM".
* tests/misc/stat-mount.sh: While at it, remove the determination
of the mount point of "." via df(1) plus sed(1) as it is unused
since commit v8.5-159-gf57cb37 anyway. Instead, improve this test
by verifying that the output of "stat -c%m ." at least starts with
a slash '/'.
Revert commit v8.22-131-g3e89d5b as even though POSIX
states that the default mode should be -L,
common practice for stand-alone pwd implementations
is to default to -P.
* src/pwd.c (usage): Retain mention of the default mode of operation.
tests: avoid false failure with spaces in mount point paths
* tests/ls/readdir-mountpoint-inode.sh: Quote appropriately
to process mount points with spaces in the path.
Previously items like these would usually be skipped,
though if the path also contained a '-' for example,
that would stat stdin, thus producing a wrong inode
and a false failure.
Fixes http://bugs.gnu.org/17863
Jim Meyering [Tue, 1 Jul 2014 16:40:50 +0000 (09:40 -0700)]
tests: use "env builtin" in place of "$abs_top_builddir/src/builtin"
This works for any program that might also be a shell built-in.
It is more concise. In addition, it makes output more reproducible:
some diagnostics include argv[0], which will now be just the program
name, rather than the full absolute name of the executable.
* tests/misc/sort-compress.sh: Use env kill, rather than absolute name.
* tests/install/trap.sh: Likewise.
* tests/misc/timeout.sh: Likewise.
* tests/touch/no-dereference.sh: Do the same for two uses of "test".
* tests/touch/no-create-missing.sh: Likewise.
Jim Meyering [Tue, 1 Jul 2014 15:32:57 +0000 (08:32 -0700)]
tests: adapt tests to change in pwd's default
* tests/misc/pwd-option.sh (base): Initialize with -P,
now that -L is the default, to accommodate an initial
working directory with a symlink component.
* tests/misc/readlink-fp-loop.sh: Use $(env pwd -P) to get the
absolute working directory. Using "env" ensures we do not invoke
any shell built-in, and PATH ensures we invoke the one from coreutils.
* tests/readlink/can-e.sh: Likewise.
* tests/readlink/can-f.sh: Likewise.
* tests/readlink/can-m.sh: Likewise.
maint: fix const correctness warnings with security_context_t
The security_context_t type was always an artificial separation
from a standard char* string, and various libselinux using code
assumed both were synonymous. In addition, prior to libselinux 2.3
function declarations were incorrect wrt constness of this type.
Here we replace security_context_t with char*, and also
provide a wrapper function to cater for the const issue on
older libselinux.
* src/system.h (se_const): A new function to avoid and identify
the const issue on older libselinux.
* src/copy.c: s/security_context_t/char */.
* src/cp.c: Likewise.
* src/id.c: Likewise.
* src/install.c: Likewise.
* src/ls.c: Likewise.
* src/mkdir.c: Likewise.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/runcon.c: Likewise.
* src/selinux.c: Likewise.
* tests/cp/no-ctx.sh: Likewise.
* src/chcon.c: Likesize.
Namhyung Kim [Mon, 30 Jun 2014 15:12:41 +0000 (00:12 +0900)]
chcon: use security_check_context() for context validation
context_new() and _free() are used for checking validity of a
specified context. libselinux provides security_check_context
for this purpose so use it.
Note that context_new() can fail for a valid context - e.g. ENOMEM.
* src/chcon.c (main): Use security_check_context().
Pádraig Brady [Mon, 30 Jun 2014 09:15:26 +0000 (10:15 +0100)]
pwd: assume -L mode by default, as per POSIX
* src/pwd.c (main): Adjust default mode to be "logical"
and independent of the POSIXLY_CORRECT env var.
(usage): Mention the default mode of operation.
* doc/coreutils.texi (pwd invocation): Adjust accordingly.
* tests/misc/pwd-option.sh: Likewise.
* NEWS: Mention the change in behavior.
Pádraig Brady [Thu, 26 Jun 2014 11:41:01 +0000 (12:41 +0100)]
tests: use predetermined NON_ROOT_GID
* HACKING: GID is more useful in tests than group name, so rename
input param from NON_ROOT_GROUP to NON_ROOT_GID to make it obvious
that only a group ID is now acceptable, thus allowing GID lookups
to be avoided throughout the tests.
* init.cfg (require_root_): Likewise.
* tests/misc/truncate-owned-by-other.sh: Avoid looking up the GID.
* tests/touch/now-owned-by-other.sh: Likewise.
* tests/misc/chroot-credentials.sh: Likewise. Also fix an instance
of comparison against NON_ROOT_GROUP which would have given a false
failure if a non numeric value was passed in.
* tests/id/setgid.sh: Use previously looked up gid as a more
accurate base for the subsequent adjustment, and move
the uid lookup within chroot, rather than having the overhead
of a separate `id` invocation.
Petr Stodůlka [Wed, 25 Jun 2014 17:26:23 +0000 (18:26 +0100)]
id: output the effective group for the process
* src/id.c (print_full_info): When no user is specified,
output the effective group for the _process_, rather than
the default group from the system database, which may be different.
* tests/id/setgid.sh: Add a case for `id` as well as `id -G`.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/7320
Reported at http://bugzilla.redhat.com/1016163
Pádraig Brady [Tue, 24 Jun 2014 14:34:39 +0000 (15:34 +0100)]
df: report correct device in presence of eclipsed mounts
* src/df.c (last_device_for_mount): A new function to identify
the last device mounted for a mount point.
(get_disk): Use the above to discard mount entries for a device,
where a later mount entry uses a different device name than
that of the user specified device.
* tests/df/over-mount-device.sh: A new root test.
* tests/local.mk: Reference the new test.
* NEWS: Reword for all these related recent fixes.
Discussed at: http://bugs.gnu.org/16539#69
Pádraig Brady [Wed, 18 Jun 2014 12:10:17 +0000 (13:10 +0100)]
df: look for accessible mount points for specified devices
* src/df.c (get_disk): Include whether we can access the mount dir,
in the mount entry selection criteria. This handles the case where
a device is (bind) mounted multiple times with the shortest mount path
not being accessible, while some of the other mount points are.
Discussed at: http://bugs.gnu.org/16539#63
Pádraig Brady [Tue, 3 Jun 2014 23:09:11 +0000 (00:09 +0100)]
df: output placeholder values for inaccessible mount points
A system provided mount entry may be unavailable due to TOCTOU race,
or if another device has been over-mounted at that position, or due to
access permissions. In all these cases output "-" placeholder values
rather than either producing an error, or in the over-mount case
outputting values for the wrong device.
* src/df.c (device_list): A new global list now updated by
filter_mount_list().
(filter_mount_list): Adjust to take a parameter as to whether
update the global mount list, or only the mount <-> device ID mapping.
(get_dev): Use the device ID mapping to ensure we're not outputting
stats for the wrong device. Also output placeholder values when we
can't access a system specified mount point.
(get_all_entries): Set the DEVICE_ONLY param for filter_mount_list().
(devname_for_dev): A new function to search the mount <-> dev mapping.
* test/df/skip-duplicates.sh: Adjust accordingly.
* NEWS: Mention the bug fixes.
Pádraig Brady [Thu, 19 Jun 2014 11:11:00 +0000 (12:11 +0100)]
du: ignore directory cycles due to bind mounts
* src/du.c (process_file): Treat cycles due to bind mounts
like cycles due to following symlinks.
* tests/du/bind-mount-dir-cycle.sh: Adjust accordingly.
* NEWS: Mention the change in behavior.
Reported at http://bugzilla.redhat.com/836557
Pádraig Brady [Thu, 19 Jun 2014 17:49:18 +0000 (18:49 +0100)]
doc: clarify chgrp restrictions
* doc/coreutils.texi (chown invocation): Mention the system dependent
restrictions on setting groups.
(chgrp invocation): Likewise. Reference the 'chown' superset.
* man/chgrp.x: Cross reference chown(1) which is the superset interface,
and also chown(2) which gives details of the platform restrictions.
Fixes http://bugs.gnu.org/17495
Pádraig Brady [Mon, 16 Jun 2014 23:18:47 +0000 (00:18 +0100)]
df: use all of the last device details provided
* src/df.c (filter_mount_list): Recent commit v8.22-108-g25a2c94
failed to copy file system type along with the updated device name.
Therefore simply replace the existing mount entry with the
current one with all the latest device details. Note the name,
even if not shorter in this entry, will be replaced with a shorter
name in a subsequent mount entry.
* tests/df/skip-duplicates.sh: Add a test case.
Pádraig Brady [Wed, 18 Jun 2014 13:30:57 +0000 (14:30 +0100)]
seq: fix incorrect output with start or end of -0
* src/seq.c (main): Avoid seq_fast() with a start or end of -0.
* tests/misc/seq.pl: Add test cases.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/17800
Pádraig Brady [Wed, 11 Jun 2014 23:35:14 +0000 (00:35 +0100)]
maint: suppress a theoretical buffer overflow warning
* src/numfmt.c (double_to_human): The printf format is built up in
a stack buffer which is big enough to hold any of the possible formats.
However the size parameter passed to snprintf was too big by 1
when GROUP was true. So decrease the buffer available to snprintf
to avoid this theoretical in practise but valid coverity warning.
Bernhard Voelker [Wed, 11 Jun 2014 09:28:03 +0000 (11:28 +0200)]
install: allow options -D and -t to be used together
* src/install.c (install_file_in_file_parents): Factor out the
creation of any parent directories into ...
(mkancesdirs_safe_wd): ... this new function.
(install_file_in_dir): Add the parameter 'mkdir_and_install', and
call the above new function if it evaluates to true.
(main): During parsing of the -t option, move the check whether
the target_directory exists down after the option parsing loop,
and do not complain about stat(optarg,...) failing if -D was given.
Pass 'mkdir_and_install' to install_file_in_dir().
* doc/coreutils.texi (install invocation): Remove the (false)
restriction that -D would be ignored together with -t. Instead,
clarify install's new bahavior.
Fix the node's reference in the top-level @direntry for consistency.
* src/install/basic-1.sh: Add tests for the now-allowed combination
of the -D and -t options.
* NEWS: Mention the improvement.
Pádraig Brady [Tue, 10 Jun 2014 23:39:45 +0000 (00:39 +0100)]
build: update GCC warnings
* configure.ac: Remove the -Wsuggest-attribute=pure
enablement on GCC >= 4.7, as that was moot since
gnulib was already enabling that warning in its default set.
The false positive was seen with 4.6.2, but confirmed
not present in 4.6.3, so that's sufficiently old to
just leave this enabled unconditionally.
Remove the -Wsuggest-attribute={const,noreturn}
enablement, as gnulib already has those in the default set.
Enable the -Wlogical-op warning for GCC >= 4.8.0
as that is confirmed OK with coreutils at least, due to
fixing: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=43772
Remove the -Wno-logical-op override since the main
-Wlogical-op flag is now sufficient to control this warning
as of GCC 4.6.3 at least.
Pádraig Brady [Tue, 3 Jun 2014 09:50:09 +0000 (10:50 +0100)]
tests: consolidate tests for true and false
* src/true.c (main): Add a comment about the possibility
of true returning EXIT_FAILURE due to write failure.
* tests/misc/false-status.sh: Fix so we're testing
the tool and not the shell builtin. Add a case for true(1).
* tests/misc/help-version.sh: Skip /dev/full test
for true as well as false since the exit status is tested separately.
Also remove the iterations for different LC_MESSAGES, as this was only
applied for false(1). Translations are not honored in the test dir
and so would need separate handling in any case.
Ben Walton [Mon, 2 Jun 2014 19:32:16 +0000 (20:32 +0100)]
maint: avoid compiler warnings with some assert() implementations
* src/df.c (get_dev): asssert() on Solaris 10 is not marked as
__noreturn__ and thus the compiler may think V is uninitialized
later on in the function.
* THANKS.in: Remove the now committer.
Ben Walton [Mon, 2 Jun 2014 08:54:56 +0000 (09:54 +0100)]
build: be more specific about .git repo before enabling warnings
* configure.ac: When looking for a .git checkout, exclude repos that
contain a .tarball-version file as these are probably releases
that are imported into git for patch management.
Pádraig Brady [Fri, 30 May 2014 16:44:32 +0000 (17:44 +0100)]
cut: restore special case handling of -f with -d$'\n'
commits v8.20-98-g51ce0bf and v8.20-99-gd302aed changed cut(1)
to process each line independently and thus promptly output
each line without buffering. As part of those changes we removed
the special handling of --delimiter=$'\n' --fields=... which
could be used to select arbitrary (ranges of) lines, so as to
simplify and optimize the implementation while also matching the
behavior of different cut(1) implementations.
However that GNU behavior was in place for a long time, and
could be useful in certain cases like making a separated list like
`seq 10 | cut -f1- -d$'\n' --output-delimiter=,` although other tools
like head(1) and paste(1) are more suited to this operation.
This patch reinstates that functionality but restricts the
"line behind" buffering behavior to only the -d$'\n' case.
We also fix the following related edge case to be more consistent:
* src/cut.c (cut_fields): Adjust as discussed above.
* tests/misc/cut.pl: Likewise.
* NEWS: Mention the change in behavior both for v8.21
and this effective revert.
* cfg.mk (old_NEWS_hash): Adjust for originally omitted v8.21 entry.
* src/paste.c: s/delimeter/delimiter/ comment typo fix.
Pádraig Brady [Thu, 29 May 2014 14:30:46 +0000 (15:30 +0100)]
df: use the last device name provided by the system
The device name reported for a particular mount entry
may no longer be valid if the mount point was subsequently
mounted on a different device. Therefore honor the order
of the mount list returned by the system and use the last
reported device name.
* src/df.c (filter_mount_list): When discarding the current
mount entry, ensure that a new device name is not also discarded.
* tests/df/skip-duplicates.sh: Add a test case. Also fix
a false failure in the edge case of a system with only a
single file system.
* NEWS: Mention the fix.
projects / thirdparty / coreutils.git / log
