systemd.journal-fields(7) documents CODE_FUNC=. Internally, we were
inconsistent: sd_journal_print uses CODE_FUNC=, log.h has CODE_FUNCTION=,
python-systemd and bootchart also used CODE_FUNC=, when they were internal.
Most external projects use sd_journal_* functions, so CODE_FUNC=,
python-systemd still uses CODE_FUNC=, as does systemd-bootchart, and
independent reimplementations in golang-github-coreos-go-systemd, qtbase,
network manager, glib, pulseaudio. Hence, I don't think there's much
choice.
share/log: change log_syntax from "[a:b] " to "a:b: "
Those square brackets don't fit how our other messages look like; we use colons
everywhere else. The "[a:b]" format was originally added in ed5bcfbe3c3b68e59242c03649eea03a9707d318, and remained unchanged for 7 years,
but in the meantime other conventions evolved.
The new version is also one character shorter.
[/etc/systemd/system/systemd-networkd.service.d/override.conf:2] Failed to parse sec value, ignoring: ...
↓
/etc/systemd/system/systemd-networkd.service.d/override.conf:2: Failed to parse sec value, ignoring: ...
tools/catalog-report.py: a script to scour the journal for bad catalog entries
I think it can be a useful tool to find such issues.
SD_MESSAGE_UNIT_STARTING 7d4958e842da4a758f6c1cdc7b36dcc5: no field UNIT
../src/core/unit.c:1239 unit_status_log_starting_stopping_reloading
Starting Paths.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
USER_UNIT=paths.target
SD_MESSAGE_UNIT_STARTED 39f53479d3a045ac8e11786248231fbf: no field UNIT
../src/core/job.c:721 job_log_status_message
Reached target Paths.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
RESULT=done
USER_UNIT=paths.target
SD_MESSAGE_STARTUP_FINISHED b07a249cd024414a82dd00cd181378ff: no field KERNEL_USEC
../src/core/manager.c:2532 manager_check_finished
Startup finished in 19ms.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
USERSPACE_USEC=19670
SD_MESSAGE_STARTUP_FINISHED b07a249cd024414a82dd00cd181378ff: no field INITRD_USEC
../src/core/manager.c:2532 manager_check_finished
Startup finished in 19ms.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
USERSPACE_USEC=19670
unknown 0ce153587afa4095832d233c17a88001: no catalog entry
gsm-manager.c:1366 start_phase
Entering running state
SYSLOG_IDENTIFIER=gnome-session
PRIORITY=5
SD_MESSAGE_UNIT_STOPPING de5b426a63be47a7b6ac3eaac82e2f6f: no field UNIT
../src/core/unit.c:1239 unit_status_log_starting_stopping_reloading
Stopping Default.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
USER_UNIT=default.target
SD_MESSAGE_UNIT_STOPPED 9d1aaa27d60140bd96365438aad20286: no field UNIT
../src/core/job.c:729 job_log_status_message
Stopped target Default.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
RESULT=done
USER_UNIT=default.target
SD_MESSAGE_TIME_CHANGE c7a787079b354eaaa9e77b371893cd27: no field REALTIME
src/core/manager.c:2049 manager_dispatch_time_change_fd
Time has been changed
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
PRIORITY=6
unknown f3ea493c22934e26811cd62abe8e203a: no catalog entry
shell-global.c:1375 shell_global_log_structured
GNOME Shell started at Sat Jun 11 2016 12:37:46 GMT-0400 (EDT)
SYSLOG_IDENTIFIER=gnome-shell
SD_MESSAGE_UNIT_FAILED be02cf6855d2428ba40df7e9d022f03d: no field UNIT
src/core/job.c:803 job_log_status_message
Failed to start GNOME Terminal Server.
SYSLOG_FACILITY=3
SYSLOG_IDENTIFIER=systemd
RESULT=failed
PRIORITY=3
USER_UNIT=gnome-terminal-server.service
SD_MESSAGE_LID_CLOSED b72ea4a2881545a0b50e200e55b9b070: no catalog entry
src/login/logind-button.c:198 button_dispatch
Lid closed.
SYSLOG_FACILITY=4
SYSLOG_IDENTIFIER=systemd-logind
PRIORITY=6
SD_MESSAGE_LID_OPENED b72ea4a2881545a0b50e200e55b9b06f: no catalog entry
src/login/logind-button.c:219 button_dispatch
Lid opened.
SYSLOG_FACILITY=4
SYSLOG_IDENTIFIER=systemd-logind
PRIORITY=6
unknown fef1cc509d5047268b83a3a553f54b43: no catalog entry
/usr/lib/python3.5/site-packages/dnf-plugins/system_upgrade.py:422 log_status
Rebooting to perform upgrade.
SYSLOG_IDENTIFIER=python3
DNF_VERSION=1.1.10
TARGET_RELEASEVER=25
SYSTEM_RELEASEVER=24
PRIORITY=5
unknown 3e0a5636d16b4ca4bbe5321d06c6aa62: no catalog entry
/usr/lib/python3.5/site-packages/dnf-plugins/system_upgrade.py:422 log_status
Starting system upgrade. This will take a while.
SYSLOG_IDENTIFIER=python3
DNF_VERSION=1.1.10
SYSTEM_RELEASEVER=24
PRIORITY=5
TARGET_RELEASEVER=25
unknown 0123456789abcdef0123456789abcdef: no catalog entry
<doctest systemd.journal.JournalHandler[9]>:1 <module>
Message with ID
SYSLOG_IDENTIFIER=/usr/lib/python2.7/site-packages/py/test.py
LOGGER=custom_logger_name
PRIORITY=4
THREAD_NAME=MainThread
pid1,catalog: use a different MESSAGE_ID for user manager startup
This add a new message id for the end of user instance startup.
User manager startup is a different beast then the system startup.
Their descriptions are completely different too. Let's just separate
them.
Partially fixes #3351.
Also remove "successful" from the description, since we don't know if
the startup was successful or not.
hwdb_parser: make sure that our patterns match the full property
We would catch stuff like:
ACCEL_MOUNT_MATRIX=0, -1, 0; -1, 0, 0; 0, 0.0., 0
but not
ACCEL_MOUNT_MATRIX=0, -1, 0; -1, 0, 0; 0, 0, 0.0.
because the match would stop at the next-to-last char. Fix that
by requiring a line end.
Bastien Nocera [Tue, 6 Dec 2016 16:16:43 +0000 (17:16 +0100)]
udev: Add rules for accelerometer orientation quirks
This commit adds a rules file to extract the properties from hwdb
to set on i2c IIO devices. This is used to set the ACCEL_MOUNT_MATRIX
property on IIO devices, to be consumed by iio-sensor-proxy or
equivalent daemon.
The hwdb file contains documentation on how to write quirks. Note
however that mount information is usually exported in:
- the device-tree for ARM devices
- the ACPI DSDT for Intel-compatible devices
but currently not extracted by the kernel.
Also note that some devices have the framebuffer rotation that changes
between the bootloader and the main system, which might mean that the
accelerometer is then wrongly oriented. This is a missing feature in the
i915 kernel driver: https://bugs.freedesktop.org/show_bug.cgi?id=94894
which needs to be fixed, and won't require quirks.
%c and %r rely on settings made in the unit files themselves and hence resolve
to different values depending on whether they are used before or after Slice=.
Let's simply deprecate them and drop them from the documentation, as that's not
really possible to fix. Moreover they are actually redundant, as the same
information may always be queried from /proc/self/cgroup and /proc/1/cgroup.
(Accurately speaking, %R is actually not broken like this as it is constant.
However, let's remove all cgroup-related specifiers at once, as it is also
redundant, and doesn't really make much sense alone.)
core: turn on specifier expansion for more unit file settings
Let's permit specifier expansion at a numbre of additional fields, where
arbitrary strings might be passed where this might be useful one day. (Or at
least where there's no clear reason where it wouldn't make sense to have.)
core: use unit_full_printf() at a couple of locations we used unit_name_printf() before
For settings that are not taking unit names there's no reason to use
unit_name_printf(). Use unit_full_printf() instead, as the names are validated
anyway in one form or another after expansion.
core: resolve more specifiers in unit_name_printf()
unit_name_printf() is usually what we use when the resulting string shall
qualify as unit name, and it hence avoids resolving specifiers that almost
certainly won't result in valid unit names.
Add a couple of more specifiers that unit_full_printf() resolves also to the
list unit_name_printf() resolves, as they are likely to be useful in valid unit
names too. (Note that there might be cases where this doesn't hold, but we
should still permit this, as more often than not they are safe, and if people
want to use them that way, they should be able to.)
core: move specifier expansion out of service.c/socket.c
This monopolizes unit file specifier expansion in load-fragment.c, and removes
it from socket.c + service.c. This way expansion becomes an operation done exclusively at time of loading unit files.
Previously specifiers were resolved for all settings during loading of unit
files with the exception of ExecStart= and friends which were resolved in
socket.c and service.c. With this change the latter is also moved to the
loading of unit files.
This adds support for discovering and making use of properly tagged dm-verity
data integrity partitions. This extends both systemd-nspawn and systemd-dissect
with a new --root-hash= switch that takes the root hash to use for the root
partition, and is otherwise fully automatic.
Verity partitions are discovered automatically by GPT table type UUIDs, as
listed in
https://www.freedesktop.org/wiki/Specifications/DiscoverablePartitionsSpec/
(which I updated prior to this change, to include new UUIDs for this purpose.
mkosi with https://github.com/systemd/mkosi/pull/39 applied may generate images
that carry the necessary integrity data. With that PR and this commit, the
following simply lines suffice to boot up an integrity-protected container image:
```
# mkdir test
# cd test
# mkosi --verity
# systemd-nspawn -i ./image.raw -bn
```
Note that mkosi writes the image file to "image.raw" next to a a file
"image.roothash" that contains the root hash. systemd-nspawn will look for that
file and use it if it exists, in case --root-hash= is not specified explicitly.
This adds support to the image dissector to deal with encrypted images (only
LUKS). Given that we now have a neatly isolated image dissector codebase, let's
add a new feature to it: support for automatically dealing with encrypted
images. This is then exposed in systemd-dissect and nspawn.
It's pretty basic: only support for passphrase-based encryption.
In order to ensure that "systemd-dissect --mount" results in mount points whose
backing LUKS DM devices are cleaned up automatically we use the DM_DEV_REMOVE
ioctl() directly on the device (in DM_DEFERRED_REMOVE mode). libgcryptsetup at
the moment doesn't provide a proper API for this. Thankfully, the ioctl() API
is pretty easy to use.
dissect: add small "systemd-dissect" tool as wrapper around dissect-image.c
This adds a small tool that may be used to look into OS images, and mount them
to any place. This is mostly a friendlier version of test-dissect-image.c. I am
not sure this should really become a proper command of systemd, hence for now
do not install it into bindir, but simply libexecdir.
This tool is already pretty useful since you can mount image files with it,
honouring the various partitions correctly. I figure this is going to become
more interesting if the dissctor learns luks and verity support.
machined: add API for querying the OS release of a machine image
This adds a bus call GetImageOSRelease() to the Manager interface that
retrieves the /etc/os-release file of a machine image. It matches the existing
GetMachineOSRelease() call, however operates on a disk image rather than a
running container.
The backend for this call on .raw images is implemented via the generalized
image dissector, which makes this scheme relatively easy to implement.
util-lib: add easy helpers for temporary directories that rmdir()ed via _cleanup_
This adds mkdtemp_malloc() that is a combination of mkdtemp() plus strdup(). It
initializes its return paremeter only if the temporary directory could be
created successfully, so that the parameter is exactly non-NULL when the
directory exists.
rmdir_and_free() and rmdir_and_freep() are also added, and the latter may be
used inside of _cleanup_ for such a directory string variable, to automatically
rmdir() the directory if it is non-NULL when the scope exits.
rmdir_and_free() is similar to the existing rm_rf_and_free() however, is only
removes a single directory and does not operate recursively.
nspawn: port nspawn to new generalized image dissection code
Let's make use of the new internal API. This mostly doesn't change anything for
the caller, however, "systemd-nspawn --image=/dev/sda7" works now as the new
code can handle disk images with no partition tables, and make any detected
images directly the root.
util-lib: split out image dissecting code and loopback code from nspawn
This adds two new APIs to systemd:
- loop-util.h is a simple internal API for allocating, setting up and releasing
loopback block devices.
- dissect-image.h is an internal API for taking apart disk images and figuring
out what the purpose of each partition is.
Both APIs are basically refactored versions of similar code in nspawn. This
rework should permit us to reuse this in other places than just nspawn in the
future. Specifically: to implement RootImage= in the service image, similar to
RootDirectory=, but operating on a disk image; to unify the gpt-auto-discovery
generator code with the discovery logic in nspawn; to add new API to machined
for determining the OS version of a disk image (i.e. not just running
containers). This PR does not make any such changes however, it just provides
the new reworked API.
The reworked code is also slightly more powerful than the nspawn original one.
When pointing it to an image or block device with a naked file system (i.e. no
partition table) it will simply make it the root device.
time-util: accept "µs" as time unit, in addition to "us" (#4836)
Let's accept "µs" as alternative time unit for microseconds. We already accept
"us" and "usec" for them, lets extend on this and accept the proper scientific
unit specification too.
We will never output this as time unit, but it's fine to accept it, after all
we are pretty permissive with time units already.
When we are in link_enter_configured we assume that the
link->state should be LINK_STATE_SETTING_ROUTES but in some
situation it's LINK_STATlE_SETTING_ADDRESSES.
Just ignore the wrong state.
Also since the return value not used any where
make link_enter_configured return type void.
Franck Bui [Fri, 2 Dec 2016 17:40:10 +0000 (18:40 +0100)]
journal: make sure to initially populate the space info cache (#4807)
Make sure to populate the cache in cache_space_refresh() at least once
otherwise it's possible that the system boots fast enough (and the journal
flush service is finished) before the invalidate cache timeout (30 us) has
expired.
Evidently this code path was never hit, because we'd crash with NameError.
The exception message also seems bogus. So just replace the whole thing
with the standard exception for invalid input.
Spot inconsistent quoting (just one single quote) (#4732)
It is possible to specify only one quote in udev rules, which is not
detected as an invalid quoting (" instead of "" for empty string).
Technically this doesn't lead to a bug, because the string ends in two
terminating nul characters at this position, but a user should still be
reminded that his configuration is invalid.
nspawn: optionally, automatically allocated --bind=/--overlay source from /var/tmp
This extends the --bind= and --overlay= syntax so that an empty string as source/upper
directory is taken as request to automatically allocate a temporary directory
below /var/tmp, whose lifetime is bound to the nspawn runtime. In combination
with the "+" path extension this permits a switch "--overlay=+/var::/var" in
order to use the container's shipped /var, combine it with a writable temporary
directory and mount it to the runtime /var of the container.
nspawn: permit prefixing of source paths in --bind= and --overlay= with "+"
If a source path is prefixed with "+" it is taken relative to the container's
root directory instead of the host. This permits easily establishing bind and
overlay mounts based on data from the container rather than the host.
This also reworks custom_mounts_prepare(), and turns it into two functions: one
custom_mount_check_all() that remains in nspawn.c but purely verifies the
validity of the custom mounts configured. And one called
custom_mount_prepare_all() that actually does the preparation step, sorts the
custom mounts, resolves relative paths, and allocates temporary directories as
necessary.
Peter Hutterer [Tue, 22 Nov 2016 05:21:24 +0000 (15:21 +1000)]
hwdb: add XKB_FIXED_LAYOUT/VARIANT to the keyboard hwdb
Yubikeys and other pseudo keyboards require that they are in the US layout,
otherwise the data they send is invalid. Add two new keys to signal this to
processes that handles (XKB) layouts.
nspawn: make use of CHASE_NON_EXISTING when locking image
If --template= is used on an image, then the image might not exist initially.
We can use CHASE_NON_EXISTING to properly lock the image already before it
exists. Let's do so.
fs-util: add new CHASE_NON_EXISTING flag to chase_symlinks()
This new flag controls whether to consider a problem if the referenced path
doesn't actually exist. If specified it's OK if the final file doesn't exist.
Note that this permits one or more final components of the path not to exist,
but these must not contain "../" for safety reasons (or, to be extra safe,
neither "./" and a couple of others, i.e. what path_is_safe() permits).
This new flag is useful when resolving paths before issuing an mkdir() or
open(O_CREAT) on a path, as it permits that the file or directory is created
later.
The return code of chase_symlinks() is changed to return 1 if the file exists,
and 0 if it doesn't. The latter is only returned in case CHASE_NON_EXISTING is
set.
Let's remove chase_symlinks_prefix() and instead introduce a flags parameter to
chase_symlinks(), with a flag CHASE_PREFIX_ROOT that exposes the behaviour of
chase_symlinks_prefix().
fs-util: change chase_symlinks() behaviour in regards to escaping the root dir
Previously, we'd generate an EINVAL error if it is attempted to escape a root
directory with relative ".." symlinks. With this commit this is changed so that
".." from the root directory is a NOP, following the kernel's own behaviour
where /.. is equivalent to /.
fs-util: add chase_symlinks_prefix() and extend comments
chase_symlinks() currently expects a fully qualified, absolute path, relative
to the host's root as first argument. Which is useful in many ways, and similar
to the paths unlink(), rename(), open(), … expect. Sometimes it's however
useful to first prefix the specified path with the specified root directory.
Add a new call chase_symlinks_prefix() for this, that is a simple wrapper.
nspawn: accept --ephemeral --template= as alternative for --ephemeral --directory=
As suggested in PR #3667.
This PR simply ensures that --template= can be used as alternative to
--directory= when --ephemeral is used, following the logic that for ephemeral
options the source directory is actually a template.
This does not deprecate usage of --directory= with --ephemeral, as I am not
convinced the old logic wouldn't make sense.
nspawn: properly handle image/directory paths that are symlinks
This resolves any paths specified on --directory=, --template=, and --image=
before using them. This makes sure nspawn can be used correctly on symlinked
images and directory trees.
tree-wide: stop using canonicalize_file_name(), use chase_symlinks() instead
Let's use chase_symlinks() everywhere, and stop using GNU
canonicalize_file_name() everywhere. For most cases this should not change
behaviour, however increase exposure of our function to get better tested. Most
importantly in a few cases (most notably nspawn) it can take the correct root
directory into account when chasing symlinks.
projects / thirdparty / systemd.git / log
