Yu Watanabe [Tue, 6 Feb 2018 07:00:34 +0000 (16:00 +0900)]
core: make ExecRuntime be manager managed object
Before this, each ExecRuntime object is owned by a unit. However,
it may be shared with other units which enable JoinsNamespaceOf=.
Thus, by the serialization/deserialization process, its sharing
information, more specifically, reference counter is lost, and
causes issue #7790.
This makes ExecRuntime objects be managed by manager, and changes
the serialization/deserialization process.
Alan Jenkins [Mon, 5 Feb 2018 16:53:40 +0000 (16:53 +0000)]
journal: include kmsg lines from the systemd process which exec()d us (#8078)
Let the journal capture messages emitted by systemd, before it ran
exec("/usr/lib/systemd/systemd-journald"). Usually such messages will only
appear with `systemd.log_level=debug`. kmsg lines written after the exec()
will be ignored as before.
In other words, we are avoiding reading our own lines, which start
"systemd-journald[100]: " assuming we are PID 100. But now we will start
allowing ourself to read lines which start "systemd[100]: ", or any other
prefix which is not "systemd-journald[100]: ".
So this can't help you see messages when we fail to exec() journald :). But,
it makes it easier to see what the pre-exec() messages look like in
the successful case. Comparing messages like this can be useful when
debugging. Noticing weird omissions of messages, otoh, makes me anxious.
nss-systemd: add work-around to silence gcc warning
In file included from ../src/basic/fs-util.h:32,
from ../src/nss-systemd/nss-systemd.c:28:
../src/nss-systemd/nss-systemd.c: In function '_nss_systemd_getgrnam_r':
../src/nss-systemd/nss-systemd.c:416:32: warning: argument to 'sizeof' in 'memset' call is the same pointer type 'char *' as the destination; expected 'char' or an explicit length [-Wsizeof-pointer-memaccess]
memzero(buffer, sizeof(char*));
^~~~
../src/basic/util.h:118:39: note: in definition of macro 'memzero'
#define memzero(x,l) (memset((x), 0, (l)))
^
gcc is trying to be helpful, and it's not far from being right. It _looks_ like
sizeof(char*) is an error, but in this case we're really leaving a space empty
for a pointer, and our calculation is correct. Since this is a short file,
let's just use simplest option and turn off the warning above the two functions
that trigger it.
man: document meaning of age in tmpfiles.d (#8092)
This documents how the age of a file is determined, which previously was
only alluded to in other parts of the documentation. Fixes #8091.
The phrasings of “last modification timestamp” etc. are taken from
man:inode(7) (as of man-pages 4.14). The debug messages in tmpfiles.c
use different messages (“modify time”), which according to a code
comment follow man:stat(1); however, my copy of that manpage (from GNU
coreutils 8.29) documents %y as “time of last data modification”
instead.
test: sort imports and use "new" string formatting
Followed PEP8 and PEP3101 rules (#8079)
Imports re-ordered by Alphabetical Standarts for following PEP8
Old type string formattings (" example %s " % exampleVar ) re-writed as new type string
formattings ( " example {} ".format(exampleVar) ) for following PEP3101
Yu Watanabe [Thu, 1 Feb 2018 10:39:30 +0000 (19:39 +0900)]
systemctl: show: use EnvironmentFiles= instead of EnvironmentFile=
EnvironmentFile= is used in the unit file, but in the dbus,
the related field name is EnvironmentFiles=.
As the other variables, let's use the field name instead of the name
used in the unit file setting.
Alan Jenkins [Fri, 2 Feb 2018 16:06:32 +0000 (16:06 +0000)]
seccomp: allow x86-64 syscalls on x32, used by the VDSO (fix #8060)
The VDSO provided by the kernel for x32, uses x86-64 syscalls instead of
x32 ones.
I think we can safely allow this; the set of x86-64 syscalls should be
very similar to the x32 ones. The real point is not to allow *x86*
syscalls, because some of those are inconveniently multiplexed and we're
apparently not able to block the specific actions we want to.
Boucman [Fri, 2 Feb 2018 14:58:40 +0000 (15:58 +0100)]
do not report total time when kernel time is not provided (#8063)
the whole systemd-analyze time logic is based on the fact that monotonic
time 0 is the start of the kernel.
If the firmware does not provide a correct time, firmware_time degrades to
0, which is the start of the kernel. The diference between FinishTime and
firmware_time is thus correct.
That assumption is still true with containers, but the start time of the
kernel is not what the user expects : It's the time when the host booted.
The total is thus still correct, but highly misleading. Containers can be
easily detected (and, in fact, already are) by systemd not reporting any
kernel non-monotonic timestamp.
This patch simply avoids printing a misleading time when it can detect that
case
basic/hashmap: tweak code to avoid pointless gcc warning
gcc says:
[196/1142] Compiling C object 'src/basic/basic@sta/hashmap.c.o'.
../src/basic/hashmap.c: In function ‘cachemem_maintain’:
../src/basic/hashmap.c:1913:17: warning: suggest parentheses around assignment used as truth value [-Wparentheses]
mem->active = r = true;
^~~
which conflates two things: the first is transitive assignent a = b = c = d;
the second is assignment of the value of an expression, which happens to be a
an assignment expression here, and boolean. While the second _should_ be
parenthesized, the first should _not_, and it's more natural to understand
our code as the first, and gcc should treat this as an exception and not emit
the warning. But since it's a while until this will be fixed, let's update
our code too.
This is close to %sysusers_create_inline and %sysusers_create that we had
already, but expects a file name and uses --replace= to implement proper
priority.
This is used like:
%sysusers_create_package %{name} %SOURCE1
where %SOURCE1 is a file with called %{name}.conf that will be installed
into /usr/lib/sysusers.d/.
The tough part is that the file needs to be available before %prep,
i.e. outside of the source tarball. This is because the spec file is
parsed (and any macros expanded), before the sources are unpackaged.
v2:
- disallow the case case when --config-name= is given but there are no
positional args. Most likely this would be a user error, so at least for now
forbid it.
v3:
- replace --config-name= with --target=
- drop quotes around %1 and %2 — if necessary, the caller should add
those.
v4:
- replace --target with --replace
- add a big comment
sysusers: allow admin/runtime overrides to command-line config
When used in a package installation script, we want to invoke systemd-sysusers
before that package is installed (so it can contain files owned by the newly
created user), so the configuration to use is specified on the command
line. This should be a copy of the configuration that will be installed as
/usr/lib/sysusers.d/package.conf. We still want to obey any overrides in
/etc/sysusers.d or /run/sysusers.d in the usual fashion. Otherwise, we'd get a
different result when systemd-sysusers is run with a copy of the new config on
the command line and when systemd-sysusers is run at boot after package
instalation. In the second case any files in /etc or /run have higher priority,
so the same should happen when the configuration is given on the command line.
More generally, we want the behaviour in this special case to be as close to
the case where the file is finally on disk as possible, so we have to read all
configuration files, since they all might contain overrides and additional
configuration that matters. Even files that have lower priority might specify
additional groups for the user we are creating. Thus, we need to read all
configuration, but insert our new configuration somewhere with the right
priority.
If --target=/path/to/file.conf is given on the command line, we gather the list
of files, and pretend that the command-line config is read from
/path/to/file.conf (doesn't matter if the file on disk actually exists or
not). All package scripts should use this option to obtain consistent and
idempotent behaviour.
The corner case when --target= is specified and there are no positional
arguments is disallowed.
v1:
- version with --config-name=
v2:
- disallow --config-name= and no positional args
v3:
- remove --config-name=
v4:
- add --target= and rework the code completely
v5:
- fix argcounting bug and add example in man page
v6:
- rename --target to --replace
sysusers: take configuration as positional arguments
If the configuration is included in a script, this is more convient.
I thought it would be possible to use this for rpm scriptlets with
'%pre -p systemd-sysuser "..."', but apparently there is no way to pass
arguments to the executable ($1 is used for the package installation count).
But this functionality seems generally useful, e.g. for testing and one-off
scripts, so let's keep it.
There's a slight change in behaviour when files are given on the command line:
if we cannot parse them, error out instead of ignoring the failure. When trying
to parse all configuration files, we don't want to fail even if some config
files are broken, but when parsing a list of items specified explicitly, we
should.
sysusers: emit a bit more info at debug level when locking fails
This is the first error message when running unprivileged, and the message is
unspecific, so let's at least add some logging at debug level to make this less
confusing.
Yu Watanabe [Thu, 1 Feb 2018 09:11:02 +0000 (18:11 +0900)]
strv: drop strv_join_quoted() (#8057)
The function `strv_join_quoted()` is now not used, and has a bug
in the buffer size calculation when the strings needs to escaped,
as reported in #8056.
So, let's remove the function.
Michael Vogt [Thu, 1 Feb 2018 04:47:50 +0000 (05:47 +0100)]
sysusers: allow force reusing existing user/group IDs (#8037)
On Debian/Ubuntu systems the default passwd/group files use a
slightly strange mapping. E.g. in passwd:
```
man:x:6:12::/var/cache/man:/sbin/nologin
```
and in group:
```
disk:x:6:
man:x:12:
```
This is not supported in systemd-sysusers right now because
sysusers will not re-use an existing uid/gid in its normal
mode of operation. Unfortunately this reuse is needed to
replicate the default Debian/Ubuntu users/groups.
This commit enforces reuse when the "uid:gid" syntax is used
to fix this.
I also added a test that replicates the Debian base-passwd
passwd/group file to ensure things are ok.
Booting with `systemd.log_level=debug` and looking in `dmesg -u` showed
messages like this:
systemd[433]: Failed to add rule for system call n/a() / 156, ignoring:
Numerical argument out of domain
This commit fixes it to:
systemd[449]: Failed to add rule for system call _sysctl() / 156,
ignoring: Numerical argument out of domain
Some of the messages could be even more misleading, e.g. we were reporting
that utimensat() / 320 was skipped as non-existent on x86, when actually
the syscall number 320 is kexec_file_load() on x86 .
The problem was that syscall NRs are looked up (and correctly passed to
libseccomp) as native syscall NRs. But we forgot that when we tried to
go back from the syscall NR to the name.
I think the natural way to write this would be
seccomp_syscall_resolve_num(nr), however there is no such function.
I couldn't work out a short comment that would make this clearer. FWIW
I wrote it up as a ticket for libseccomp instead.
https://github.com/seccomp/libseccomp/issues/104
Technically, `data` is a sequence of bytes without a trailing zero,
so the use of `memcmp` seems to be logical here. Besides, this helps get
around a bug that makes `asan` report the false positive mentioned in
#8052.
journalctl: add highlighting for matched substring
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.
Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
journalctl: make matching optionally case sensitive
Case sensitive or case insensitive matching can be requested using
--case-sensitive[=yes|no].
Unless specified, matching is case sensitive if the pattern contains any
uppercase letters, and case insensitive otherwise. This matches what
forward-search does in emacs, and recently also --ignore-case in less. This
works surprisingly well, because usually when one is wants to do case-sensitive
matching, the pattern is usually camel-cased. In the less frequent case when
case-sensitive matching is required with an all-lowercase pattern,
--case-sensitive can be used to override the automatic logic.
Vito Caputo [Sat, 27 Jan 2018 00:38:01 +0000 (16:38 -0800)]
basic: implement the IteratedCache
Adds the basics of the IteratedCache and constructor support for the
Hashmap and OrderedHashmap types.
iterated_cache_get() is responsible for synchronizing the cache with
the associated Hashmap and making it available to the caller at the
supplied result pointers. Since iterated_cache_get() may need to
allocate memory, it may fail, so callers must check the return value.
On success, pointer arrays containing pointers to the associated
Hashmap's keys and values, in as-iterated order, are returned in
res_keys and res_values, respectively. Either may be supplied as NULL
to inhibit caching of the keys or values, respectively.
Note that if the cached Hashmap hasn't changed since the previous call
to iterated_cache_get(), and it's not a call activating caching of the
values or keys, the cost is effectively zero as the resulting pointers
will simply refer to the previously returned arrays as-is.
A cleanup function has also been added, iterated_cache_free().
This only frees the IteratedCache container and related arrays. The
associated Hashmap, its keys, and values are not affected. Also note
that the associated Hashmap does not automatically free its associated
IteratedCache when freed.
One could, in theory, safely access the arrays returned by a
successful iterated_cache_get() call after its associated Hashmap has
been freed, including the referenced values and keys. Provided the
iterated_cache_get() was performed prior to the hashmap free, and that
the type of hashmap free performed didn't free keys and/or values as
well.
Alan Jenkins [Sat, 27 Jan 2018 18:02:06 +0000 (18:02 +0000)]
systemd-shutdown: use log_set_prohibit_ipc(true)
Now we have log_set_prohibit_ipc(), let's use it to clarify that
systemd-shutdown is not expected to try and log via journald (which it is
about to kill). We avoided ever asking systemd-shutdown to do this, but
it's more convenient for the reader if they don't have to think about that.
In that sense, it's similar to using assert() to validate a function's
arguments.
Alan Jenkins [Fri, 26 Jan 2018 13:42:53 +0000 (13:42 +0000)]
rationalize interface for opening/closing logging
log_open_console() did not switch from stderr to /dev/console, when
"always_reopen_console" was set. It was necessary to call
log_close_console() first.
By contrast, log_open() did switch between e.g. journald and kmsg according
to the value of "prohibit_ipc".
Let's fix log_open() to respect the values of all the log options, and we
can make log_close_*() private.
Also log_close_console() is changed. There was some precaution, avoiding
closing the console fd if we are not PID 1. I think commit 48a601fe made
a little mistake in leaving this in, and it only served to confuse
readers :).
Also I changed systemd-shutdown. Now we have log_set_prohibit_ipc(), let's
use it to clarify that systemd-shutdown is not expected to try and log via
journald (which it is about to kill). We avoided ever asking it to, but
it's more convenient for the reader if they don't have to think about that.
In that sense, it's similar to using assert() to validate a function's
arguments.
journal: losen restrictions on journal file suffix (#8013)
Previously, we'd refuse open journal files with suffixes that aren't
either .journal or .journal~. With this change we only care when we are
creating the journal file.
I looked over the sources to see whether we ever pass files discovered
by directory enumeration to journal_file_open() without first checking
the suffix (in which case the old check made sense), but I couldn't find
any. hence I am pretty sure removing this check is safe.
meson: use env object instead of string in tags targets
I used 'tags' before because this way we avoided a unnecessary
line about 'env' detection. But we cannot use 'env' in test(), so
previous commit added 'env' detection. We might just as well use
it in custom_target().
This is a bit painful because a separate build of systemd is necessary. The
tests are guarded by tests!=false and slow-tests==true. Running them is not
slow, but compilation certainly is. If this proves unwieldy, we can add a
separate option controlling those builds later.
The build for each sanitizer has its own directory, and we build all fuzzer
tests there, and then pull them out one-by-one by linking into the target
position as necessary. It would be nicer to just build the desired fuzzer, but
we need to build the whole nested build as one unit.
[I also tried making systemd and nested meson subproject. This would work
nicely, but meson does not allow that because the nested target names are the
same as the outer project names. If that is ever fixed, that would be the way
to go.]
v2:
- make sure things still work if memory sanitizer is not available
v3:
- switch to syntax which works with meson 0.42.1 found in Ubuntu
Alan Jenkins [Fri, 26 Jan 2018 22:47:16 +0000 (22:47 +0000)]
pid1: when we can't log to journal, remember our fallback log target
If we have to force the logging to close the journal fd, then we can open
any fallback log target. E.g. kmsg, if the target was the default
JOURNAL_OR_KMSG.
This is the behaviour I would expect from the documentation. I couldn't
find any justification in the code, for why we would want to start dropping
log messages instead of sending them to the fallback target.
This means we will match the behaviour of processes which we fork and which
set `open_when_needed`, and with generators - which use
log_set_prohibit_ipc(true) - which we fork+exec during a reload.
IMO this illustrates that the log_open/log_close interface is too clunky.
So with the behaviour settled, I will refactor the interface in the next
commit :).
networkd: assume no link local addresses for where it isn't used
It turns out that link local doesn't make much sense in its context.
Since link local is disabled by the kernel driver, it's important that
networkd assumes it's off too, so that the link can reach the
"configured" stage, without waiting indefinitely for link local
addresses which will never come.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
projects / thirdparty / systemd.git / log
