mkosi: Stop passing package environment variables to tools image
The tools image is not guaranteed to be the same distribution as the
target distribution and so might have different package environment
variables than the main image yet we currently unconditionally use the
same package environment variables for both of them.
Let's fix this by not passing the package environment variables to the
tools image and subimages anymore, and instead having the main, tools and
build images separately include a config file with the required environment
variables.
mkosi: Use mkosi.tools.conf for tools tree configuration
This allows us to use the regular settings instead of having to bother
with ToolsTreeXXX variants. It'll also allow us to share configuration
between the regular images and the tools tree image, which we'll make
use of in the next commit.
mkosi: Drop number prefixes from configuration files
We already removed these in some places, let's migrate the others as
well. There's no ordering required at all between these configuration
files so let's not bother with any numbered prefixes.
test: use 'exit 0' instead of 'return' in test scripts
14385s [ 66.896852] TEST-87-AUX-UTILS-VM.sh[3744]: + test -x /usr/lib/systemd/systemd-validatefs
14385s [ 66.898544] TEST-87-AUX-UTILS-VM.sh[3744]: + echo 'no systemd-validatefs'
14385s [ 66.899115] TEST-87-AUX-UTILS-VM.sh[3744]: no systemd-validatefs
14385s [ 66.899699] TEST-87-AUX-UTILS-VM.sh[3744]: + return
14385s [ 66.900189] TEST-87-AUX-UTILS-VM.sh[3744]: .//usr/lib/systemd/tests/testdata/units/TEST-87-AUX-UTILS-VM.validatefs.sh: line 13: return: can only `return' from a function or sourced script
introduce notify_socket_prepare() and use it where applicable (#36911)
This introduces notify_socket_prepare(), which creates an autobind
notify socket and IO event source for the socket. Then, use it where we
send notification messages from worker processes to their manager
process.
- drop 'Options' sections,
- drop underlining for link,
- fix indentation.
Prompted by https://github.com/systemd/systemd/pull/36850#discussion_r2020594171
> the underline stuff we only use for long --help texts that have sections,
> for the section headers. systemctl --help does that for example. This one
> here is not that long, hence doesn't really need section headers, and
> hence no underlining. The clickable links don't need to be explicitly
> underlined, the terminal emulators that supper hyperlinks will underline
> them on their own (for example gnome-terminal uses a dotted line).
Also, let's not get too tangled up in the style of defining variables
in between. The functions are short enough, and vars involved are still
effectively at the beginning... Put differently, the separation from
'int r' is too deliberate and brings no actual value in my eyes.
Yu Watanabe [Mon, 31 Mar 2025 16:14:33 +0000 (01:14 +0900)]
introduce systemd-validatefs@.service that ensures file systems can only be used in the way they were intended (#36714)
If we have multiple trusted fs (i.e. luks or dm-verity) we generate via
repart at boot, we must make sure they cannot be "misappropriated", i.e.
used for a different mount they were intended for.
Hence, let's introduce "mount constraint" data (encoded in xattrs on the
root inode of the fs) that tells us where a file system has to be
mounted, and what the gpt partition metadata has to be for the fs to be
valid.
Inspired by this thread:
https://lists.freedesktop.org/archives/systemd-devel/2025-March/051244.html
If the target dir is tmpfs and we run on old kernels we cannot extract
xattrs and the extracting will fail if there are any. hence add
-no-xattrs to the two remaining unsquashfs invocations that don't have
it.
(Also all other invocations across our test tree spell "-dest" instead
of "-d", hence do so here too.)
Let's automatically generate validatefs xattrs by default, that encode
the intended use of partitions.
This defaults to on, since the structure of repart definition files
tells us enough on use for this to be safe. There's an option however,
to turn this off.
validatefs: add new tool that enforces mount constraints
This new tool looks for a three xattr on the root inode of a file system
that encode mount constraints of the file system. The tool is supposed
to be hooke into the mount logic and is supposed to protect against
misappropriating trusted file systems in unintended ways.
Consider the following scenario: we boot up on first boot and create a
tpm-locked pair of /var/ and /srv/ partitions via systemd-repart. An
attacker then offline modifies the partition table, exchanging the
metadata of the /var/ and /srv/ partition. So far we'd happily accept
that, honour the modified metadata and boot up. This could be used to
revert changes to /var/ or similar. And all that even though both
partitions are encrypted and locked to TPM!
With this new mechanism we can encode in the protected contents of the
file systems the ways it can be used: the partition type uuid, the
partition label and the intended mount point can be stored in xattrs,
and we can check them automatically on mount, and take action on
mismatch. (action would typically be immediate reboot).
pcrextend: whenever we fail to extend PCRs, reboot immediately
PCR extensions are supposed to be useful for "destroying" the ability to
access TPM bound secrets. Hence, if for some reason we fail to extend a
PCR, it's safer to just reboot, instead of going on without the
extension, leaving secrets potentially accessible which should not be
accessible.
Note that the services exit gracefully if no TPM is found, hence this
should not be triggered on TPM-less systems. However, this enforces that
if there is a TPM that is accessible to Linux and that works properly,
the PCR measurement must complete too.
Let's always prefer quotactl_fd() when it's available and use quotactl()
only as as a fallback on old kernels.
This way we can operate on the fds we typically already have open, or if
needed we can open a new one, and use for multiple fs operation.
In the long run we should really focus on operating exclusively by fd
instead of by path, by device nor or otherwise. This gets us a step
closer to that.
Luca Boccassi [Fri, 28 Mar 2025 13:31:28 +0000 (13:31 +0000)]
mkosi: Use build image prepare scripts for tools tree as well (#36861)
Instead of listing dependencies manually for the default tools tree,
let's reuse the prepare scripts from the build image. To make this work,
the sync script has to be configured for the tools tree as well so that
it's invoked both when building the tools tree and for the regular
image,
otherwise, when doing the first build in a fresh checkout, the sync
script
won't have executed yet as sync scripts for the regular images are
executed
after building the default tools tree.
Daan De Meyer [Fri, 28 Mar 2025 11:35:54 +0000 (12:35 +0100)]
test: Drop --no-rebuild from meson test invocations
With meson 1.7.0, meson won't rebuild the world anymore when running
meson test so since meson 1.7.0 is in tumbleweed, debian testing, fedora
rawhide and Arch Linux, drop --no-rebuild from the docs since all the default
tools trees built by mkosi in the systemd repo will have meson 1.7.0 or newer
installed.
We keep --no-rebuild in CI because in CI we run with the same tools tree
distribution as the target distribution we're building and in those it's not
guaranteed for meson 1.7.0 to be available yet.
The advantage of dropping --no-rebuild is that meson will now reconfigure itself
if needed, which is required if we change or move around meson files, as otherwise
meson tends to explode spectacularly in hard to debug ways.
Daan De Meyer [Tue, 25 Mar 2025 20:02:24 +0000 (21:02 +0100)]
mkosi: Use build image prepare scripts for tools tree as well
Instead of listing dependencies manually for the default tools tree,
let's reuse the prepare scripts from the build image. To make this work,
the sync script has to be configured for the tools tree as well so that
it's invoked both when building the tools tree and for the regular image,
otherwise, when doing the first build in a fresh checkout, the sync script
won't have executed yet as sync scripts for the regular images are executed
after building the default tools tree.
Daan De Meyer [Thu, 27 Mar 2025 14:53:28 +0000 (15:53 +0100)]
test: Mark tests as skipped if requirements were not satisfied
Instead of not creating the test at all, let's always create test but
simply mark them as skipped as this is more observable than simply not
creating the test at all.
Daan De Meyer [Thu, 27 Mar 2025 14:49:06 +0000 (15:49 +0100)]
TEST-06-SELINUX: Only enable if meson was invoked as root
This test only works if the image was built as root. Since that's
impossible to check as meson generally runs before we build the image,
let's use whether meson is run as root as a proxy.
Daan De Meyer [Thu, 27 Mar 2025 13:24:28 +0000 (14:24 +0100)]
test: Boot integration test VMs without an initrd if possible
If we don't need an initrd, let's not waste time in one. We have to
gate this by distributions that have the necessary kernel modules to
make this work as builtin modules, hopefully we can expand the list in
the future.
This is a required first step before testing and it's not specifically
mentioned in the doc. This tripped me up for a while, so let's save the
trouble for the next person.
Daan De Meyer [Thu, 27 Mar 2025 20:38:00 +0000 (21:38 +0100)]
test: Make it possible to run the integration tests standalone (#36868)
Currently, to run the integration tests, it's still necessary to
install various other build tools besides meson: A compiler, gperf,
libcap, ... which we want to avoid in CI systems where we receive
prebuilt systemd packages and only want to test them. Examples are
Debian's autopkgtest CI and Fedora CI. Let's make it possible for
these systems to run the integration tests without having to install
any other build dependency besides meson by extracting the logic
required to run the integration tests with meson into a separate
subdirectory and adding a standalone top-level meson.build file which
can be used to configure a meson tree with as its only purpose running
the integration tests.
Practically, we do the following:
- all the integration test directories and integration-test-wrapper.py
are moved from test/ to test/integration-tests/.
- All the installation logic is kept out of test/integration-tests/ or
any of its subdirectories and moved into test/meson.build instead.
- We add test/integration-tests/standalone/meson.build to run the
integration tests standalone. This meson file includes
test/integration-tests via a cute symlink hack to trick meson into
including a parent directory with subdir().
- Documentation is included on how to use the new standalone mode.
Daan De Meyer [Wed, 26 Mar 2025 13:30:20 +0000 (14:30 +0100)]
test: Make it possible to run the integration tests standalone
Currently, to run the integration tests, it's still necessary to
install various other build tools besides meson: A compiler, gperf,
libcap, ... which we want to avoid in CI systems where we receive
prebuilt systemd packages and only want to test them. Examples are
Debian's autopkgtest CI and Fedora CI. Let's make it possible for
these systems to run the integration tests without having to install
any other build dependency besides meson by extracting the logic
required to run the integration tests with meson into a separate
subdirectory and adding a standalone top-level meson.build file which
can be used to configure a meson tree with as its only purpose running
the integration tests.
Practically, we do the following:
- all the integration test directories and integration-test-wrapper.py
are moved from test/ to test/integration-test/.
- All the installation logic is kept out of test/integration-test/ or
any of its subdirectories and moved into test/meson.build instead.
- We add test/integration-test/standalone/meson.build to run the
integration tests standalone. This meson file includes
test/integration-test via a cute symlink hack to trick meson into
including a parent directory with subdir().
- Documentation is included on how to use the new standalone mode.
- TEST-64-UDEV-STORAGE and TEST-85-NETWORK are changed to generate separate
units for each testcase to make them behave more like the other integration
tests.
shared/exec-util: fix logging of the args of an executed program
The debug logs has lots of "About to execute /some/path (null)". This
occurs when the args array is empty. Instead, only print "(null)" if
we failed with oom.
Having strv_skip() return NULL makes this pleasant to write without repeating
strv_isempty() a few times.
strv_skip was written to carefully return the original array, but this turns
out to be an unnecessary complication. After the previous patch, no caller
cares about the distinction between NULL and { NULL }, but various callers need
to wrap the process the returned value with strv_isempty(), sometimes more than
once. Let's always return NULL for an empty result to allow callers to be
simplified.
Yu Watanabe [Wed, 26 Mar 2025 21:10:53 +0000 (06:10 +0900)]
core: drop manager_get_unit_by_pid() and friends (#36872)
Except for one place, they are only used by test-watch-pid. Let's also
use manager_get_unit_by_pidref() and friends in the test, and drop the
_pid() variants.
Yu Watanabe [Wed, 26 Mar 2025 19:27:01 +0000 (04:27 +0900)]
coredump: make sure pid1/journal coredumps are not lost when Storage=journal is selected (#36870)
It confused the hell of me, that if pid1 crashes in an mkosi system run
from the build tree there's no coredump kept. Because mkosi configures
journal storage for coredumps, but that's not going to work for pid1 or
journald. Hence use external storage for these two even if everything
else is stored in the journal.
projects / thirdparty / systemd.git / log
