Jan Janssen [Fri, 1 May 2015 13:15:16 +0000 (15:15 +0200)]
journalctl: Improve boot ID lookup
This method should greatly improve offset based lookup, by simply jumping
from one boot to the next boot. It starts at the journal head to get the
a boot ID, makes a _BOOT_ID match and then comes from the opposite
journal direction (tail) to get to the end that boot. After flushing the matches
and advancing the journal from that exact position, we arrive at the start
of next boot. Rinse and repeat.
This is faster than the old method of aggregating the full boot listing just
so we can jump to a specific boot, which can be a real pain on big journals
just for a mere "-b -1" case.
As an additional benefit --list-boots should improve slightly too, because
it does less seeking.
Note that there can be a change in boot order with this lookup method
because it will use the order of boots in the journal, not the realtime stamp
stored in them. That's arguably better, though.
Another deficiency is that it will get confused with boots interleaving in the
journal, therefore, it will refuse operation in --merge, --file and --directory mode.
Tom Gundersen [Mon, 18 May 2015 15:07:04 +0000 (17:07 +0200)]
udevd: process all SIGCHLD events every time the handler is invoked
We were returning rather than continuing in some cases. The intention
was always to fully process all pending events before returning
from the SIGCHLD handler. Restore this behaviour.
Alban Crequy [Mon, 18 May 2015 14:45:30 +0000 (16:45 +0200)]
nspawn: close extra fds before execing init
When systemd-nspawn gets exec*()ed, it inherits the followings file
descriptors:
- 0, 1, 2: stdin, stdout, stderr
- SD_LISTEN_FDS_START, ... SD_LISTEN_FDS_START+LISTEN_FDS: file
descriptors passed by the system manager (useful for socket
activation). They are passed to the child process (process leader).
- extra lock fd: rkt passes a locked directory as an extra fd, so the
directory remains locked as long as the container is alive.
systemd-nspawn used to close all open fds except 0, 1, 2 and the
SD_LISTEN_FDS_START..SD_LISTEN_FDS_START+LISTEN_FDS. This patch delays
the close just before the exec so the nspawn process (parent) keeps the
extra fds open.
This patch supersedes the previous attempt ("cloexec extraneous fds"):
http://lists.freedesktop.org/archives/systemd-devel/2015-May/031608.html
Alban Crequy [Mon, 18 May 2015 10:20:28 +0000 (12:20 +0200)]
core: Private*/Protect* options with RootDirectory
When a service is chrooted with the option RootDirectory=/opt/..., then
the options PrivateDevices, PrivateTmp, ProtectHome, ProtectSystem must
mount the directories under $RootDirectory/{dev,tmp,home,usr,boot}.
The test-ns tool can test setup_namespace() with and without chroot:
$ sudo TEST_NS_PROJECTS=/home/lennart/projects ./test-ns
$ sudo TEST_NS_CHROOT=/home/alban/debian-tree TEST_NS_PROJECTS=/home/alban/debian-tree/home/alban/Documents ./test-ns
Karel Zak [Mon, 18 May 2015 10:30:37 +0000 (12:30 +0200)]
fstab-generator: add x-systemd.requires and x-systemd.requires-mounts-for
Currently we have no way how to specify dependencies between fstab
entries (or another units) in the /etc/fstab. It means that users are
forced to bypass fstab and write .mount units manually.
The patch introduces new systemd fstab options:
x-systemd.requires=<PATH>
- to specify dependence an another mount (PATH is translated to unit name)
x-systemd.requires=<UNIT>
- to specify dependence on arbitrary UNIT
x-systemd.requires-mounts-for=<PATH ...>
- to specify dependence on another paths, implemented by
RequiresMountsFor=. The option may be specified more than once.
Eric Cook [Mon, 18 May 2015 05:02:39 +0000 (01:02 -0400)]
zsh-completion: actually complete template names for subcommands enable, reenable and disable.
compadd's -a option treats non-option arguments as arrays. So
$(_systemctl_get_template_names) expands to some words that aren't
legal array names. Even if there were, they would be empty; thus adding
nothing.
Tom Gundersen [Sat, 16 May 2015 08:14:20 +0000 (10:14 +0200)]
udevd: queue - update queue state when events are queued/freed
This way it is more obvious that the queue flag file is always
up-to-date. Moreover, we only have to touch/unlink it when the
first/last event is allocated/freed.
Only if both keep_free and max_use are actually 0 we can shortcut things
and avoid vacuuming. If either are positive or -1 we need to execute the
vacuuming.
socket-util: socket_address_parse() should not log errors on its own
Given that socket_address_parse() is mostly a "library" call it
shouldn't log on its own, but leave that to its caller.
This patch removes logging from the call in case IPv6 is not available
but and IPv6 address shall be parsed. Instead a new call
socket_address_parse_and_warn() is introduced which first invokes
socket_address_parse() and then logs if necessary.
This should fix "make check" on ipv6-less kernels:
Direct leak of 928 byte(s) in 1 object(s) allocated from:
#0 0x7f782f788db1 in __interceptor_calloc (/usr/lib64/libasan.so.2+0x96db1)
#1 0x562a83ae60cf in bus_message_from_header src/libsystemd/sd-bus/bus-message.c:480
#2 0x562a83ae6f5a in bus_message_from_malloc src/libsystemd/sd-bus/bus-message.c:576
#3 0x562a83ad3cad in bus_socket_make_message src/libsystemd/sd-bus/bus-socket.c:915
#4 0x562a83ad4cfc in bus_socket_read_message src/libsystemd/sd-bus/bus-socket.c:1051
#5 0x562a83ab733f in bus_read_message src/libsystemd/sd-bus/sd-bus.c:1647
#6 0x562a83ab98ea in sd_bus_call src/libsystemd/sd-bus/sd-bus.c:2038
#7 0x562a83b1f46d in sd_bus_call_method src/libsystemd/sd-bus/bus-convenience.c:94
#8 0x562a83aab3e1 in context_read_ntp src/timedate/timedated.c:192
#9 0x562a83aae1af in main src/timedate/timedated.c:730
#10 0x7f782eb238c4 in __libc_start_main (/lib64/libc.so.6+0x208c4)
Indirect leak of 77 byte(s) in 1 object(s) allocated from:
#0 0x7f782f788f6a in realloc (/usr/lib64/libasan.so.2+0x96f6a)
#1 0x562a83ad418a in bus_socket_read_message src/libsystemd/sd-bus/bus-socket.c:963
#2 0x562a83ab733f in bus_read_message src/libsystemd/sd-bus/sd-bus.c:1647
#3 0x562a83ab98ea in sd_bus_call src/libsystemd/sd-bus/sd-bus.c:2038
#4 0x562a83b1f46d in sd_bus_call_method src/libsystemd/sd-bus/bus-convenience.c:94
#5 0x562a83aab3e1 in context_read_ntp src/timedate/timedated.c:192
#6 0x562a83aae1af in main src/timedate/timedated.c:730
#7 0x7f782eb238c4 in __libc_start_main (/lib64/libc.so.6+0x208c4)
Indirect leak of 2 byte(s) in 1 object(s) allocated from:
#0 0x7f782f75493f in strdup (/usr/lib64/libasan.so.2+0x6293f)
#1 0x562a83b0229b in bus_message_parse_fields src/libsystemd/sd-bus/bus-message.c:5382
#2 0x562a83ae7290 in bus_message_from_malloc src/libsystemd/sd-bus/bus-message.c:601
#3 0x562a83ad3cad in bus_socket_make_message src/libsystemd/sd-bus/bus-socket.c:915
#4 0x562a83ad4cfc in bus_socket_read_message src/libsystemd/sd-bus/bus-socket.c:1051
#5 0x562a83ab733f in bus_read_message src/libsystemd/sd-bus/sd-bus.c:1647
#6 0x562a83ab98ea in sd_bus_call src/libsystemd/sd-bus/sd-bus.c:2038
#7 0x562a83b1f46d in sd_bus_call_method src/libsystemd/sd-bus/bus-convenience.c:94
#8 0x562a83aab3e1 in context_read_ntp src/timedate/timedated.c:192
#9 0x562a83aae1af in main src/timedate/timedated.c:730
#10 0x7f782eb238c4 in __libc_start_main (/lib64/libc.so.6+0x208c4)
SUMMARY: AddressSanitizer: 1007 byte(s) leaked in 3 allocation(s).
This is due to missing _cleanup_bus_message_unref_ in context_read_ntp()
core: Execute first boot presets in an enable-only preset-mode.
This means any existing enabled units well be preserved and no
pre-created symlinks will be removed. This is done on first boot, when
the assumption is that /etc is not populated at all (no machine-id
setup). For minimal containers that gives a significant first boot
speed up, approximately ~20ms / ~16% in my trials.
It turns out we don't actually need to set the global ip_forward setting.
The only relevant setting is the one on each interface.
What the global toggle actually does is switch forwarding on/off for all
currently present interfaces and change the default for new ones.
That means that by setting the global ip_forward we
- Introduce a race condition, because if the interface with IPForward=yes
is brought up after one with IPForward=no, both will have forwarding
enabled, because the global switch turns it on for all interfaces.
If the other interface comes up first networkd correctly sets forward=0
and it doesn't get overridden.
- Change the forwarding setting for interfaces that networkd is not
configured to touch, even if the user disabled forwarding via sysctl,
either globally or per-interface
As forwarding works fine without this, as long as all relevant interfacest
individually set IPForward=yes: just drop it
This means that non-networkd interfaces use the global default while
networkd interfaces default to off if IPForward isn't given.
Building with address sanitizer enabled on GCC 5.1.x a memory leak
is reported because we never close the bus, fix it by using
cleanup variable attribute.
GNU memmem() requires a nonnull first parameter. Let's introduce
memmem_safe() that removes this restriction for zero-length parameters,
and make use of it where appropriate.
util: add generic calls for prefixing a root directory to a path
So far a number of utilities implemented their own calls for this, unify
them in prefix_root() and prefix_roota(). The former uses heap memory,
the latter allocates from the stack via alloca().
If systemd is built with GCC address sanitizer or leak sanitizer
the following memory leak ocurs:
May 12 02:02:46 linux.site systemd[326]: =================================================================
May 12 02:02:46 linux.site systemd[326]: ==326==ERROR: LeakSanitizer: detected memory leaks
May 12 02:02:46 linux.site systemd[326]: Direct leak of 101 byte(s) in 3 object(s) allocated from:
May 12 02:02:46 linux.site systemd[326]: #0 0x7fd1f504993f in strdup (/usr/lib64/libasan.so.2+0x6293f)
May 12 02:02:46 linux.site systemd[326]: #1 0x55d6ffac5336 in strv_new_ap src/shared/strv.c:163
May 12 02:02:46 linux.site systemd[326]: #2 0x55d6ffac56a9 in strv_new src/shared/strv.c:185
May 12 02:02:46 linux.site systemd[326]: #3 0x55d6ffa80272 in generator_paths src/shared/path-lookup.c:223
May 12 02:02:46 linux.site systemd[326]: #4 0x55d6ff9bdb0f in manager_run_generators src/core/manager.c:2828
May 12 02:02:46 linux.site systemd[326]: #5 0x55d6ff9b1a10 in manager_startup src/core/manager.c:1121
May 12 02:02:46 linux.site systemd[326]: #6 0x55d6ff9a78e3 in main src/core/main.c:1667
May 12 02:02:46 linux.site systemd[326]: #7 0x7fd1f394e8c4 in __libc_start_main (/lib64/libc.so.6+0x208c4)
May 12 02:02:46 linux.site systemd[326]: Direct leak of 29 byte(s) in 1 object(s) allocated from:
May 12 02:02:46 linux.site systemd[326]: #0 0x7fd1f504993f in strdup (/usr/lib64/libasan.so.2+0x6293f)
May 12 02:02:46 linux.site systemd[326]: #1 0x55d6ffac5288 in strv_new_ap src/shared/strv.c:152
May 12 02:02:46 linux.site systemd[326]: #2 0x55d6ffac56a9 in strv_new src/shared/strv.c:185
May 12 02:02:46 linux.site systemd[326]: #3 0x55d6ffa80272 in generator_paths src/shared/path-lookup.c:223
May 12 02:02:46 linux.site systemd[326]: #4 0x55d6ff9bdb0f in manager_run_generators src/core/manager.c:2828
May 12 02:02:46 linux.site systemd[326]: #5 0x55d6ff9b1a10 in manager_startup src/core/manager.c:1121
May 12 02:02:46 linux.site systemd[326]: #6 0x55d6ff9a78e3 in main src/core/main.c:1667
May 12 02:02:46 linux.site systemd[326]: #7 0x7fd1f394e8c4 in __libc_start_main (/lib64/libc.so.6+0x208c4)
May 12 02:02:46 linux.site systemd[326]: SUMMARY: AddressSanitizer: 130 byte(s) leaked in 4 allocation(s).
There is a leak due to the the use of cleanup_free instead _cleanup_strv_free_
nspawn: skip symlink to a combined cgroup hierarchy if it already exists
If a symlink to a combined cgroup hierarchy already exists and points to
the right path, skip it. This avoids an error when the cgroups are set
manually before calling nspawn.
nspawn: rework custom mount point order, and add support for overlayfs
Previously all bind mount mounts were applied in the order specified,
followed by all tmpfs mounts in the order specified. This is
problematic, if bind mounts shall be placed within tmpfs mounts.
This patch hence reworks the custom mount point logic, and alwas applies
them in strict prefix-first order. This means the order of mounts
specified on the command line becomes irrelevant, the right operation
will always be executed.
While we are at it this commit also adds native support for overlayfs
mounts, as supported by recent kernels.
Direct leak of 32 byte(s) in 1 object(s) allocated from:
#0 0x7f623c961c4a in malloc (/usr/lib64/libasan.so.2+0x96c4a)
#1 0x5651f79ad34e in malloc_multiply (/home/crrodriguez/scm/systemd/systemd-modules-load+0x2134e)
#2 0x5651f79b02d6 in strjoin (/home/crrodriguez/scm/systemd/systemd-modules-load+0x242d6)
#3 0x5651f79be1f5 in files_add (/home/crrodriguez/scm/systemd/systemd-modules-load+0x321f5)
#4 0x5651f79be6a3 in conf_files_list_strv_internal (/home/crrodriguez/scm/systemd/systemd-modules-load+0x326a3)
#5 0x5651f79bea24 in conf_files_list_nulstr (/home/crrodriguez/scm/systemd/systemd-modules-load+0x32a24)
#6 0x5651f79ad01a in main (/home/crrodriguez/scm/systemd/systemd-modules-load+0x2101a)
#7 0x7f623c11586f in __libc_start_main (/lib64/libc.so.6+0x2086f)
SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s).
This happens due to the wrong cleanup attribute is used (free vs strv_free)
Tom Gundersen [Tue, 12 May 2015 14:55:29 +0000 (16:55 +0200)]
udevd: explicitly update queue file before answering to ping
This avoids updating the flag files twice for every loop, and also removes another dependency
in the main-loop, so we are freer to reshufle it as we want.
Tom Gundersen [Tue, 12 May 2015 14:51:31 +0000 (16:51 +0200)]
udevd: explicitly read out uevents we create ourselves
Rather than skippling ctrl handling whenever we have handlede inotify events
(and hence may have synthesized a 'change' event), just call the uevent
handling explicitly from on_inotify() so that the event queue is up-to-date.
It's primarily just a property of the Manager object after all, and we
try to refer to PID 1 as "manager" instead of "systemd", hence let's to
stick to this here too.
unit: move unit_warn_if_dir_nonempty() and friend to unit.c
The call is only used by the mount and automount unit types, but that's
already enough to consider it generic unit functionality, hence move it
out of mount.c and into unit.c.
This changes log_unit_info() (and friends) to take a real Unit* object
insted of just a unit name as parameter. The call will now prefix all
logged messages with the unit name, thus allowing the unit name to be
dropped from the various passed romat strings, simplifying invocations
drastically, and unifying log output across messages. Also, UNIT= vs.
USER_UNIT= is now derived from the Manager object attached to the Unit
object, instead of getpid(). This has the benefit of correcting the
field for --test runs.
Also contains a couple of other logging improvements:
- Drops a couple of strerror() invocations in favour of using %m.
- Not only .mount units now warn if a symlinks exist for the mount
point already, .automount units do that too, now.
- A few invocations of log_struct() that didn't actually pass any
additional structured data have been replaced by simpler invocations
of log_unit_info() and friends.
- For structured data a new LOG_UNIT_MESSAGE() macro has been added,
that works like LOG_MESSAGE() but prefixes the message with the unit
name. Similar, there's now LOG_LINK_MESSAGE() and
LOG_NETDEV_MESSAGE().
- For structured data new LOG_UNIT_ID(), LOG_LINK_INTERFACE(),
LOG_NETDEV_INTERFACE() macros have been added that generate the
necessary per object fields. The old log_unit_struct() call has been
removed in favour of these new macros used in raw log_struct()
invocations. In addition to removing one more function call this
allows generated structured log messages that contain two object
fields, as necessary for example for network interfaces that are
joined into another network interface, and whose messages shall be
indexed by both.
- The LOG_ERRNO() macro has been removed, in favour of
log_struct_errno(). The latter has the benefit of ensuring that %m in
format strings is properly resolved to the specified error number.
- A number of logging messages have been converted to use
log_unit_info() instead of log_info()
- The client code in sysv-generator no longer #includes core code from
src/core/.
- log_unit_full_errno() has been removed, log_unit_full() instead takes
an errno now, too.
- log_unit_info(), log_link_info(), log_netdev_info() and friends, now
avoid double evaluation of their parameters