Daan De Meyer [Fri, 3 May 2024 21:33:58 +0000 (23:33 +0200)]
qemu: Use systemd-run to allocate scopes
This doesn't drastically change behavior, but will open the way for
adding a RuntimeProperties= setting to allow configuring various
properties of the scope unit.
Since allocating a scope with systemd-run involves communicating with
a daemon running on the host, there's no point in running it from the
sandbox so we run it from the host instead.
Because systemd-run needs to run as the uid that started mkosi, we can't
use preexec_fn anymore to allocate the user namespace for virtiofsd.
Instead, we reimplement what become_root() does on top of unshare and
chain execute into that which then itself chain executes virtiofsd.
Daan De Meyer [Fri, 3 May 2024 14:24:50 +0000 (16:24 +0200)]
qemu: Configure caching properly
For the scratch device, we can use unsafe caching as it's thrown
away immediately after the machine shuts down. For the root disk,
we can use unsafe caching only if --ephemeral is enabled.
The default home directory for system users created with sysusers.d is /.
Which will automatically be mounted inside sync scripts, causing a lot of
headaches, since we're essentially trying to run on the host then.
E.g. /proxy.cacert cannot be created.
Don't look in --extra-search-paths for qemu and virt-fw-vars
These take as input a file that might be in or out of the tools tree.
Because find_ovmf_firmware() will return a file in the tools tree
regardless of whether virt-fw-vars and qemu will be executed on the
host or not, let's disable the --extra-search-paths= integration for
these two for now.
Tobias Fleig [Tue, 26 Mar 2024 12:46:00 +0000 (05:46 -0700)]
Call ukify with --ucode if supported
This is the companion change to https://github.com/systemd/systemd/pull/31872.
If both systemd-stub and ukify are version `256~devel` or greater, mkosi
will put the microcode initrd in the new `.ucode` UKI section by using
the `--ucode` flag in ukify. Otherwise it will put the microcode initrd
as first regular initrd (same as before).
In systemd we want to (optionally) make sure all mkosi output is
written to the meson build directory. To make this work, we want to
write a mkosi.conf to the meson build directory so that developers
can do "mkosi -C build". Currently this doesn't work because "build"
is interpreted as a verb. Let's make sure that doesn't happen.
We need a better solution for this hack but for now this is the best
I can come up with.
Don't build the image automatically for boot/qemu verbs without --force
Let's insist on --force to build the image if boot/qemu are used to
give users an easy way to insist that an image has already been built
when booting it.
mkosi might be invoked from a terminal but redirected to a file, in
that case we should ignore $TERM from the environment, so let's make
sure we handle that properly.
Let's make sure the limits are configured so we can always write at
least 4G of logs. We also enable compact mode again in all cases to
reduce the size used by journal files as for example Github Actions
machines aren't exactly swimming in free space.
(We pick 4G because that's the max journal file size when the compact
mode is used)
(We'll probably have to revisit this again at some point but for now
this should do the trick)
We want to support users writing to these directories from the host
so --cache=always is not an option as that assumes virtiofsd has
exclusive write access.
If we find a binary in any path configured with ExtraSearchPaths=,
we prefer using it over the tools tree.
This commit also gets rid of all the find_binary() for "btrfs" in
tree.py and stops passing in the tools tree to those functions.
Instead, we rely on the new behavior of run() to not fail hard
when check=False and the btrfs binary is not found.
Make check argument apply to whether the command exists
Let's not fail hard when check=False and the command is not found.
This will allow us to avoid having to pass the tools tree path around
in various places.
Let's drop this compat kludge for OpenMandriva. No other distro
does this and we should just assume gnu tar/cpio as the official
API of the tar/cpio binaries.
Switch to reprepro for generating local apt repositories
We don't install dpkg-dev in tools trees anymore to avoid pulling
in perl, which means we don't have access to dpkg-scanpackages in
tools trees anymore.
Instead of adding back dpkg-dev, let's instead switch to reprepro
for generating our local apt repository. It's written in C, packaged
everywhere and has hardly any dependencies.
Same as systemd. While in development we suffix with ~devel. Also
update the do-a-release script to add a second commit to switch back
to ~devel when we do a release.
We keep the old version in pyproject.toml because suffixing with
~devel conflicts with PEP 440 and python tools complain about that.
Add VolatilePackages= and InitrdVolatilePackages= settings
Let's allow configuring packages which should be installed after
running build scripts and which are not cached. This is useful for
installing packages which are built in a build script or which change
often and shouldn't invalidate the cache.
Every part of the default tools tree is cached. Thus, we can check
if the cache is out of date on incremental builds and if it isn't,
just reuse the previous tools tree instead of doing unnecessary work.
Only force repository metadata refresh if -ff was specified
Let's make sure we use the default metadata expiration checks unless
-ff was specified. Apt doesn't support anything like this so we don't
change anything there.
Let's run this logic for all distributions, and also run it after
running postinst scripts, to deal with kernel packages that are
installed in a postinst script.
When building an image using internal repositories, often various
internal certificates are required to be able to access these. These
are of course not going to be available in any default tools tree, so
let's add an option to use certificates and keys from the host with a
tools tree.
makepkg is always invoked in the image itself to build a package,
so there's no point in mounting over the makepkg.conf from the tools
tree into the sandbox.
bubblewrap does not support forwarding signals yet,
see https://github.com/containers/bubblewrap/pull/586. As a workaround,
we need to make sure we send our signals to the inner process. To
make this work, we create a pipe, pass it through to the subprocess,
and prefix with a bash command that writes its pid to the pipe before
exec-ing the actual command.
The other thing we get from this is that we can register the inner pid
as a scope which makes the systemctl status output for the scopes we
create a lot more useful.
This allows us to get rid of the shell hack to create and clean up
a subdirectory of /var/tmp. To avoid having to change every callsite
to use with(), we pass in a context manager directly into run() and
spawn().
Because we don't return a list anymore from sandbox_cmd(), we add an
extra "extra" argument to allow appending extra commands to the sandbox.
Stop installing dpkg-dev explicitly in tools trees
apt will pull in dpkg as a dependency. dpkg-dev specifically only
includes stuff required to build packages, which you generally only
want to install in the image itself as the dpkg build tooling does
not support operating on a chroot.
This gets rid of perl in tools trees as dpkg-dev depends on a bunch
of perl scripts but dpkg doesn't.
In spawn(), if an exception is raised by the context manager block,
we'll let it take priority over any exception we raise in spawn()
itself. Let's make use of this to make sure the qemu/VM exit code
always takes priority by raising the CalledProcessError from within
the qemu spawn() block.
We also let any qemu error take precedence over any EXIT_STATUS error.
projects / thirdparty / mkosi.git / log
