We do the image build and run the tests in a btrfs loopback so we
can make use of btrfs subvolumes and COW to keep the disk space
requirements to a minimum and speed up the ephemeral copies we make
of the image to run the tests.
We also switch to building debug packages and publishing the built
packages as artifacts.
- Stop using logging module since the default output formatting is
pretty bad. Prefer print() for now.
- Log less, logging the full mkosi command line is rather verbose,
especially when it contains multi-line dropins.
- Streamline the journalctl command we output for debugging failed
tests.
- Don't force usage of the disk image format.
- Don't force running without unit tests.
- Don't force disabling RuntimeBuildSources.
- Update documentation to streamline the command for running a single
test and remove sudo as it's not required anymore.
- Improve the console output by having the test unit's output logged
to both the journal and the console.
- Disable journal console log forwarding as we have journal forwarding
as a better alternative.
- Delete existing journal file before running test.
- Delete journal files of succeeded tests to reduce disk usage.
- Rename system_mkosi target to just mkosi
- Pass in mkosi source directory explicitly to accomodate arbitrary
build directory locations.
- Add test interactive debugging if stdout is connected to a tty
- Stop explicitly using the 'system' image since it'll likely be
dropped soon.
- Only forward journal if we're not running in debugging mode.
- Stop using testsuite.target and instead just add the necessary
extras to the main testsuite unit via the credential dropin.
- Override type to idle so test output is not interleaved with
status output.
- Don't build mkosi target by default
- Always add the mkosi target if mkosi is found
- Remove dependency of the integration tests on the mkosi target
as otherwise the image is always built, even though we configure
it to not be built by default.
- Move mkosi output, cache and build directory into build/ so that
invocations from meson and regular invocations share the same
directories.
- Various aesthetic cleanups.
Building debug packages on ubuntu requires the "debug" option to be
specified explicitly. Debug packages on Ubuntu have the .ddeb extension,
so let's make sure we handle that by copying the .ddeb packages in the
build script as well.
isc-dhcp-server does not ship units, only sysv scripts, so the mkosi
presets that disable it have no effect. The generated unit is started on
each boot and fails, causing delays and noise.
Mask it so that the generated unit is overridden. It is installed only
to bring in binaries used by the networkd tests anyway.
Mike Yuan [Mon, 22 Apr 2024 16:06:11 +0000 (00:06 +0800)]
kernel-install/60-ukify: also search in {/run,/usr/local/lib}/kernel/
After db26d8025e5bbc188f93b645124126bbc550caa5,
kernel-install takes config files under /run/
and /usr/local/lib/ into account too. The same
should apply to ukify.
Note that ukify when invoked alone searches under
/run/systemd/, /etc/systemd/, /usr/local/lib/systemd/,
and /usr/lib/systemd/, respectively, which is against
our usual order...
Currently, A large amount of unit test output is logged directly
to the console instead of to the per test log file as any subprocesses
executed by a test manager will detect that stderr is not connected
to the journal and log directly to /dev/console instead.
To solve this issue, let's make sure all tests are connected directly
to the journal by running them with systemd-run. We also simplify the
entire test script by getting rid of the custom queue and replicating
it with xargs instead. By using bash's function export feature, we can
make our run_test() function available to the bash subprocess spawned
by xargs.
Once a test is finished, we read its logs from the journal and put them
in the appropriate file if needed.
Luca Boccassi [Thu, 14 Mar 2024 23:44:20 +0000 (23:44 +0000)]
nspawn: ensure single-process container running as --user can access credentials
When starting a container with --user, the new uid will be resolved and switched to
only in the inner child, at the end of the setup, by spawning getent. But the
credentials are set up in the outer child, long before the user is resolvable,
and the directories/files are made only readable by root and read-only, which
means they cannot be changed later and made visible to the user.
When this particular combination is specified, it is obvious the caller wants
the single-process container to be able to use credentials, so make them world
readable only in that specific case.
resolved: rename DnssdService "name" field to "id"
"name" is a bit confusing since this field is *not* the DNS-SD service
identifier, bust just some string derived from the .dnssd filename that
is used as handle for the service. Let's hence give it a better name:
"id".
While we are at it, switch from basename() to path_extract_filename().
Enable the exec_fd logic for Type=notify* services too, and change it
to send a timestamp instead of a '1' byte. Record the timestamp in a
new ExecMainHandoverTimestamp property so that users can track accurately
when control is handed over from systemd to the service payload, so
that latency and startup performance can be trivially and accurately
tracked and attributed.
Right now systemd-tpm2-setup-early and systemd-pcrphase-initrd.service
are not ordered against each other. However, they require the same slow
resource to operate: the TPM2. If we allow them to access the device
simultaneously, the kernel resource manager like has to save/restore TPM
state while they operate, slowing things down further.
hence, let's avoid all this mess, and just order them against each other
so that the shared resource is first used in full by one and then by the
other.
I opted to order systemd-pcrphase-initrd before
systemd-tpm2-setup-early, since there's value in having the former as
early as possible in userspace, to be a good marker for the transition
from kernel to first userspace. I can see no benefit in the opposite
order however.
Mike Yuan [Mon, 22 Apr 2024 09:40:53 +0000 (17:40 +0800)]
shared/verbs: show list of verbs when missing
Replaces #32062
As discussed in #32062, making 'help' the default verb
is not very appealing for two reasons:
1) If the verb is missing, showing a help which is pages long
isn't really helpful to locate the problem.
(https://github.com/systemd/systemd/pull/32062#issuecomment-2064997158)
2) We want to reserve the right to set default verbs to be
more useful ones, instead of help. E.g. 'busctl' lists all
bus peers by default.
So, when there are more than 2 verbs, let's instead add
the list of available verbs to the "Command verb required"
message, that serves as a hint. That way we try to be friendlier
to users, but still make the problem obvious.
We shouldn't try to use any ANSI escape sequences if TERM=dumb.
Also, the "\r\n" we output can get interpreted as a double newline
(for example by Github Actions), so let's output just "\n" when
TERM=dumb to clean up the CI logs.
tpm2-util: add generic wrapper tpm2_context_new_or_warn() that wrpas tpm2_context_new and logs about errors
We so far just print a short log message that is not very useful, let's
add some recognizable error codes, and output better log messages if we
can't get TPM stuff to work.
sd-event: fix fd leak when fd is owned by IO event source
When an IO event source owns relevant fd, replacing with a new fd leaks
the previously assigned fd.
===
sd_event_add_io(event, &s, fd, ...);
sd_event_source_set_io_fd_own(s, true);
sd_event_source_set_io_fd(s, new_fd); <-- The previous fd is not closed.
sd_event_source_unref(s); <-- new_fd is closed as expected.
===
Without the change, valgrind reports the leak:
==998589==
==998589== FILE DESCRIPTORS: 4 open (3 std) at exit.
==998589== Open file descriptor 4:
==998589== at 0x4F119AB: pipe2 (in /usr/lib64/libc.so.6)
==998589== by 0x408830: test_sd_event_source_set_io_fd (test-event.c:862)
==998589== by 0x403302: run_test_table (tests.h:171)
==998589== by 0x408E31: main (test-event.c:935)
==998589==
==998589==
==998589== HEAP SUMMARY:
==998589== in use at exit: 0 bytes in 0 blocks
==998589== total heap usage: 33,305 allocs, 33,305 frees, 1,283,581 bytes allocated
==998589==
==998589== All heap blocks were freed -- no leaks are possible
==998589==
==998589== For lists of detected and suppressed errors, rerun with: -s
==998589== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Try path without sbin even if compiled with split-bin=true
I'm working on the transition to merged sbin in Fedora. While the transition is
happening (and probably for a while after), we need to compile systemd with
split-bin=true to support systems upgraded from previous versions. But when the
system has been upgraded and already has /usr/sbin that is a symlink, be nice
and give $PATH without sbin.
We check for both /usr/sbin and /usr/local/sbin. If either exists and is not a
symlink to ./bin, we retain previous behaviour. This means that if both are
converted, we get the same behaviour as split-bin=false, and otherwise we
get the same behaviour as before.
sd-path uses the same logic. This is not a hot path, so I got rid of the nulstr
macros that duplicated the logic.
test: Don't allocate scope if already running in unit with delegated cgroup
If we're already running in a unit with delegation turned on, let's
skip allocation of a scope unit and cgroup subroot. This allows journald
to correctly attribute the logs of all subprocesses spawned by tests such
as test-execute to the test-execute service when the test is running in a service.
Specifically, this will make tests log to the journal if stderr is
connected to the journal. This means we get proper log priorities
and such compared to if we just logged to stderr.
mkosi: Install various system users/groups in opensuse images
These don't get pulled in automatically and there doesn't seem to
be a "system-users" package so install the ones we need for the
integration tests manually.
mkosi: Backport Fedora basic users/groups sysusers files to CentOS
CentOS does not ship these sysusers dropins which set up basic system
users and groups. Until we can move to CentOS Stream 10, let's add the
dropins ourselves to make sure the base system users/groups are available
on CentOS.
The Arch Linux PKGBUILD does not (yet) have versioned dependencies
between the systemd packages, causing systemd-libs to not get updated
to 256-devel if systemd 256-devel is installed. Let's explicitly install
the newer version of systemd-libs as well for now until this problem is
fixed.
This introduces dependencies on various environment variables set
by rpmbuild which will not be set when rerunning meson inside the
VM/container so let's disable package notes to avoid these dependencies,
as the package notes aren't terribly useful in this scenario anyway.
projects / thirdparty / systemd.git / log
