Karel Zak [Thu, 29 Aug 2019 13:50:58 +0000 (15:50 +0200)]
libmount; fix and improve read+poll mountinfo
* fix read() buffer size (stupid bug...)
* split read() EINTR and poll() based attempts
* use 100 attempts
* wait 10000 usec between attempts, but first 10 attempts are without
this delay. It makes the function usable for usual use-cases as well
as on very busy systems (successfully tested with 300 concurrent
mount/umount processes)
Karel Zak [Wed, 28 Aug 2019 13:47:16 +0000 (15:47 +0200)]
libmount: improve mountinfo reliability
The standard way how we read mount table is not reliable because
during the read() syscalls the table may be modified by some another
process. The changes in the table is possible to detect by poll()
event, and in this case it seems better to lseek to the begin of the file
and read it again. It's expensive, but better than races...
This patch does not modify mountinfo parser, but it reads all file to
memory (by read()+poll()) and than it creates memory stream
from the buffer and use it rather than a regular file stream.
It means the parser is still possible to use for normal files
(e.g. fstab) as well as for mountinfo and it's also portable to
systems where for some reason is no fmemopen().
Note that re-read after poll() event is limited to 5 attempts (but
successful read() without event zeroize the counter). It's because we
do not want to wait for consistent mountinfo for ever. It seems better
to use old (less reliable) way than hang up in read()+poll()
loop.
Addresses: https://github.com/systemd/systemd/issues/10872 Reported-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> Signed-off-by: Karel Zak <kzak@redhat.com>
Triggered by commit f612c4c67 (tests: fix --unbuffered mode with
ASAN, 2019-08-27), which says:
Well, this patch sucks. It would be nice to have things in
the way how it has been original expected by Patrick's patch,
but ...
So this commit here effectively reverts it and instead tries to
improve the shortcomings of the original patch. First, it uses
env(1) to set ASAN_OPTIONS instead of directly adding it to the
args array to fix execution of "${args[@]}" "$@".
Second, it now supports both unbuffer(1) and stdbuf(1). The
latter uses LD_PRELOAD tricks, which doesn't play nicely with
ASAN, so it will not be used if ASAN has been requested. It's
still valuable to have support for both, as many more systems
will have stdbuf(1) from coreutils installed but not unbuffer(1)
from expect.
The test fdisk/oddinput hardcodes strings returned by strerror(3P) for
both the errors ENOENT and ENOTTY. As these strings are unportable,
convert the tests to use the test_strerror helper instead to convert
them with sed(1).
Karel Zak [Tue, 27 Aug 2019 11:02:38 +0000 (13:02 +0200)]
tests: fix --unbuffered mode with ASAN
Unfortunately, ASAN is pretty sensitive to LD_PRELOAD, but stdbuf from
coreutils is based on LD_PRELOAD. So, I have replaced stdbuf with
unbuffer (from expect pkg).
The another problem is "${args[@]}" "$@" which does not work as expected.
Well, this patch sucks. It would be nice to have things in the way
how it has been original expected by Patrick's patch, but ...
Karel Zak [Tue, 27 Aug 2019 08:11:02 +0000 (10:11 +0200)]
Merge branch '2019wk33' of https://github.com/kerolasa/util-linux
* '2019wk33' of https://github.com/kerolasa/util-linux:
docs: try to find broken man references and fix them
docs: correct su.1 runuser reference from section 8 to 1
po: remove possibility to translate static option arguments
The col/multibyte test has a hardcoded error string as part of its
expected output that is returned by glibc's strerror(3P) function. Even
though many of these strings are the same across libc implementations,
they are not standardiced and some are certainly different. One example
is the string for EILSEQ on musl libc.
To fix this, we introduce a new test helper "test_strerror". The helper
can be invoked with an error code like "EILSEQ", which will cause it to
print out the the respective error message for that code. Note that
"test_strerror" cannot act on the error's value (e.g. 84 for EILSEQ), as
these aren't standardized either. Instead, we thus need to have an array
of the error's string representation ("EILSEQ") to its respective error
code (the define EILSEQ). The array can trivially be extended as
required, thus it is only sparsely populated with EILSEQ right now.
To fix the col/multibyte test, we introduce a call to sed(1) to replace
the strerror(3P) message from EILSEQ with "EILSEQ". Furthermore, as
we're running tests with the POSIX locale by default which treats all
bytes as valid multibyte sequences, we have to change to the C.UTF-8
locale instead to actually get an error.
tests: (column) use actually invalid multibytes to test encoding
If reading an invalid multibyte sequence, column(1) will encode the byte
as "\x<hex>" instead. The tests try to verify that by piping "£" into
column(1). As the tests run with LC_ALL=POSIX by default, though, libc
implementations strictly adhering to the POSIX standard will treat all
characters as valid multibyte characters. As a consequence, no EILSEQ is
raised by mbtowc(3P) and the character is not encoded as hex, breaking
the test.
Fix this by setting LC_ALL=C.UTF-8. As "£" is a valid UTF-8 character,
we also change the test to use a proper illegal multibyte sequence.
tests: (colcrt) fix reliance on EILSEQ in POSIX locale
The input file "crash1" in the colcrt/regressions test contains the
illegal byte sequence "\x94\x7e". While "\x7e" is '~', "\x94" is not a
valid character. Thus, the test assumes that getwc(3P) will return
`WEOF` and set `errno=EILSEQ`, causing colcrt(1) to abort reading the
stream and thus not print the trailing '~'.
This assumption holds just fine for glibc as it will dutifully report
EILSEQ, but musl libc will happily read the complete stream without
complaining about the illegal character. But in fact, as tests run with
LC_ALL=POSIX by default, glibc's behaviour is wrong while musl is right.
Quoting mbrtowc(3P) from POSIX.1-2017:
[EILSEQ] An invalid character sequence is detected. In the POSIX locale an
[EILSEQ] error cannot occur since all byte values are valid
characters.
Fix the issue by running the colcrt tests with C.UTF8 locale.
tests: (libfdisk) remove reliance on buffer behaviour of standard streams
The tests in libfdisk/mkpart-full all rely on the buffering behaviour of
standard output and standard error streams, most importantly that stderr
is non-buffering and stdout is buffering. This doesn't hold on all libc
implementations when redirecting to a file, breaking the test suite on
such platforms.
Use `ts_run --unbuffered` to stop buffering of the standard output
stream to fix this.
tests: remove reliance on buffer behaviour of stderr/stdout streams
In the test cases "rename::exit_codes" and "rename::exit_codes", we rely
on the flushing behaviour of stderr and stdout streams relative to each
other. Streams in glibc will not flush on newlines if stdout is pointing
to a non-TTY file descriptor, but relying on this is fragile and may
break on systems with a different behaviour like musl libc.
Fix this by introducing a new parameter "--unbuffered" to `ts_run`. If
this parameter is passed and stdbuf(1) from coreutils is available, then
it will use it to disable buffering of standard output completely. Like
this, we can selectively run tests with this if ordering of messages
from stdout and stderr is being checked.
Sami Kerola [Sat, 24 Aug 2019 21:34:16 +0000 (22:34 +0100)]
docs: try to find broken man references and fix them
Unfortunately methods I used to find and fix were based on quite manual
process that cannot be easily repeated so I do not see how this fix could be
turned into a tools/checkmans.sh addition. Well, lets hope doing this
manually twice every decade is good enough.
Sami Kerola [Wed, 21 Aug 2019 19:25:31 +0000 (20:25 +0100)]
po: remove possibility to translate static option arguments
These strings are expected to be wrote exactly as they are parsed, so make
translating them impossible. Since mkfs.cramfs -N option arguments need
this treatment use opportunity to slice usage() output to multiple lines.
Addresses: https://bugs.debian.org/907568 Signed-off-by: Sami Kerola <kerolasa@iki.fi>
Karel Zak [Thu, 22 Aug 2019 12:00:32 +0000 (14:00 +0200)]
libfidk: (dos) fix tiny partitions calculation
The code uses last_sector -= 1 for tiny partitions. This does not make
sense. This stupidity has been introduced by (my) badly commented commit 09a4ca5e450c5e7cbe860453b76fd68c60fe591f ... sorry.
Fortunately, this issue is visible only for tiny partitions on large
devices (partitions where size < grain; usual grain is 1MiB) if the last
sector is specified by relative notation (+size{siffix}).
Note that "last -= 1" makes sense when the "last" is align to the
optimal I/O boundary; in this case we need to set the end of the
partition one sector before the boundary. For tiny devices it does not
makes sense as we do not align these partitions.
Addresses: https://github.com/karelzak/util-linux/issues/843 Signed-off-by: Karel Zak <kzak@redhat.com>
Sami Kerola [Sun, 18 Aug 2019 10:03:44 +0000 (11:03 +0100)]
include/xalloc: use multiline function declarations
There are two things happening in this commit. 1) Move each function
attribute to it's own line 2) move function return value after attributes so
that function inputs and outputs are on same line.
Karel Zak [Wed, 21 Aug 2019 08:51:18 +0000 (10:51 +0200)]
partx: don't report ENXIO as error on -d
The errno ENXIO should be ignored, unfortunately the current code uses
variable 'rc' for ioctl return code as well as for final del_parts()
return value. So, failed ioctl (which should be ignored) affects all
del_parts() status.
There's an obvious copy-paste error in `mnt_parse_table_line` when
referring to the 6th field of the fstab format, where it says "freq"
instead of "passno". This has probably been copied from the 5th field,
which is in fact correctly labelled as "freq".
Fix the mistake by changing the comment to say "passno" instead.
The reason is that the two same mnt entry "/mnt/.psplash" will cause
the dst->tab set to NULL when umount this mnt entry the second time.
This will trigger an endless loop in mnt_reset_table() because that
mnt entry is linked on the libmnt_table but its .tab is set to NULL.
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Karel Zak [Mon, 19 Aug 2019 12:28:50 +0000 (14:28 +0200)]
tests: Skip fdisk/mbr-nondos-mode on Sparc as unsupported
The test has been originally designed as usable on sparc, but now we
use it for many features which are MBR specific.
Reported-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Co-Author: Anatoly Pugachev <matorola@gmail.com> Signed-off-by: Karel Zak <kzak@redhat.com>
Eric Desrochers [Wed, 14 Aug 2019 16:58:39 +0000 (12:58 -0400)]
fstrim shouldn't run inside a container
Container type implies the following products:
openvz OpenVZ/Virtuozzo
lxc Linux container implementation by LXC
lxc-libvirt Linux container implementation by libvirt
systemd-nspawn systemd's minimal container implementation, see systemd-nspawn(1)
docker Docker container manager
podman Podman container manager
rkt rkt app container runtime
wsl Windows Subsystem for Linux
Sami Kerola [Fri, 2 Aug 2019 18:39:05 +0000 (19:39 +0100)]
mountpoint: add --nofollow option
The no follow option will allow user to distinct mount points from symbolic
links pointing to them. Arguably this is pretty pedantic option, mounting a
device or bind mount to a directory via symlink does not have or cause any
issues.
Addresses: https://github.com/karelzak/util-linux/issues/832 Signed-off-by: Sami Kerola <kerolasa@iki.fi>
Karel Zak [Wed, 31 Jul 2019 14:18:27 +0000 (16:18 +0200)]
libblkid: fix file descriptor leak in blkid_verify()
The function blkid_verify() uses private device file descriptor and
uses blkid_probe_set_device() to assign the descriptor to low-level
probing code. Unfortunately, close() in this case is not enough as the
prober can internally open whole-disk device too.
The library API has been extended so blkid_probe_set_device()
deallocates and close previously used prober for whole-disk. This new
functionality is used in blkid_verify() now.
Addresses: https://bugzilla.redhat.com/show_bug.cgi?id=1734545 Signed-off-by: Karel Zak <kzak@redhat.com>
column: fix outputing empty column at the end of line
The following commands manifests the problem. In old versions before
commit 4762ae9d60 ("column: use libsmartcols for --table"), both of them
should output with 2 "|"
Fixes: 4762ae9d60 ("column: use libsmartcols for --table") Signed-off-by: Yousong Zhou <zhouyousong@yunionyun.com> Reviewed-by: Sami Kerola <kerolasa@iki.fi>
Sami Kerola [Sun, 21 Jul 2019 20:36:28 +0000 (21:36 +0100)]
agetty: simplify code in dolog() preprocessor blocks
Aim is to make dolog() a lot more readable and understandable, with downside
of when (rarely?) USE_SYSLOG it not defined the function will use a bit more
space from stack. I think that is price well worth paying.
Note that we cannot make scols_table_enable_minout() default because
for example "column --table" is pretty commonly used with non-blank
columns separator and in this case all cells has to be filled.
$ echo -e "aa,b,ccc\na,,\naaa,bbb,ccc" | column --table --separator ',' --output-separator '|'
aa |b |ccc
a | |
aaa|bbb|ccc
Addresses: https://github.com/karelzak/util-linux/issues/826 Signed-off-by: Karel Zak <kzak@redhat.com>
Karel Zak [Wed, 17 Jul 2019 10:11:38 +0000 (12:11 +0200)]
libfdisk: don't use FAT as MBR
The current libfdisk MBR detection is too weak, the result is that it
reuses MBR from FAT. The correct behavior is to create a new MBR, wipe
first sector (on write) and warn about obsolete FAT superblock.
Reported-by: Mike Fleetwood <mike.fleetwood@googlemail.com> Signed-off-by: Karel Zak <kzak@redhat.com>
Carlos Santos [Wed, 3 Jul 2019 16:31:34 +0000 (13:31 -0300)]
hwclock: use CMOS clock only if available
- Add --disable-hwclock-cmos configuration argument
- Add USE_HWCLOCK_CMOS (enabled by default for i386/x86_64)
- Add define(USE_HWCLOCK_CMOS)
- Compile hwclock-cmos.c only if USE_HWCLOCK_CMOS is true
- Remove all unnecessary #ifdefs from hwclock-cmos.c
- Add #ifdef USE_HWCLOCK_CMOS around the determine_clock_access_method()
call in hwclock.c
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Karel Zak [Mon, 15 Jul 2019 10:07:42 +0000 (12:07 +0200)]
libblkid: (drbd) simplify padding
We do not need all the metadata and it seems the extra padding is
problematic in some cases. Let's keep is simple and use fixed offset
for the metadata rather than sizeof().
References: https://github.com/karelzak/util-linux/pull/820 Signed-off-by: Karel Zak <kzak@redhat.com>
build-sys: Include <stdlib.h> in ./configure wchar_t test
Without #include <stdlib.h>, this configure check fails for strict
C99/C11 compilers which do not support implicit function declarations
(which are a C90 feature removed from C99).
Sami Kerola [Sun, 14 Jul 2019 09:17:32 +0000 (10:17 +0100)]
libblkid: fix address sanitizer issues
With aligned attribute many blkid tests fail with following error. So
instead of aligning to 4K add padding that makes the struct same size
without causing asan trip over.
libblkid/src/superblocks/drbd.c:179:6: runtime error: member access
within misaligned address 0x55913d7e6958 for type 'struct
meta_data_on_disk_9', which requires 4096 byte alignment
In zfs structure it seems compiler is adding padding, that does not mix well
with be32_to_cpu() and other bit operations.
libblkid/src/superblocks/zfs.c:109:23: runtime error: load of misaligned
address 0x7ff6406540e4 for type 'uint32_t' (aka 'unsigned int'), which
requires 8 byte alignment
Karel Zak [Thu, 27 Jun 2019 07:22:18 +0000 (09:22 +0200)]
lsblk: force to print PKNAME for partition
PKNAME (parent kernel device name) is based on printed tree according
to parent -> child relationship. The tree is optional and not printed
if partition specified (.e.g "lsblk -o+PKNAME /dev/sda1"), but old
versions print the PKNAME also in this case.
Addresses: https://github.com/karelzak/util-linux/issues/813 Signed-off-by: Karel Zak <kzak@redhat.com>
Karel Zak [Thu, 6 Jun 2019 09:17:13 +0000 (11:17 +0200)]
wdctl; read from /sys if necessary
The device can be inaccessible for non-root user or busy (already used
by another process). In this case it seems better to read information
from /sys.
Note that /sys does not provide struct watchdog_info.options, so we
cannot print list of supported watchdog features.
Addresses: https://github.com/karelzak/util-linux/issues/804 Signed-off-by: Karel Zak <kzak@redhat.com>