Daan De Meyer [Tue, 21 Oct 2025 11:33:10 +0000 (13:33 +0200)]
Drop support for CDROM=
We want to align on systemd-vmspawn in the future, and it's very
unlikely it'll ever support this option, so let's drop support for
it since it's very niche and not useful except for testing systemd-repart
ISO stuff.
Daan De Meyer [Tue, 21 Oct 2025 11:27:29 +0000 (13:27 +0200)]
Remove support for RuntimeScratch=
systemd-vmspawn solves the problem solved by RuntimeScratch= by
resizing the entire image instead. Let's align with vmspawn and drop
RuntimeScratch= and require users to use RuntimeSize= instead.
Martin Hundebøll [Mon, 20 Oct 2025 07:24:23 +0000 (09:24 +0200)]
Match compressed pacman packages too
Arch packages can be compressed using various codecs, in which case the
package filename is suffixed with the compression method (e.g. .zst for
zstandard compression). Make sure to find such compressed packages in
the volatile package directory exposed to the build script as
$PACKAGEDIR, so they are installable using VolatilePackages=.
Fixes: 71ffced0 ("Rework PACKAGE_GLOBS to be a PackageManager classmethod")
Daan De Meyer [Thu, 16 Oct 2025 12:52:37 +0000 (14:52 +0200)]
Make sure apt sources are installed when BaseTrees= is in the mix
Let's make sure to install apt sources every time we install the apt
package, even if BaseTrees= or volatile packages or such are in the mix.
To make this work we unfortunately have to reintroduce install_packages().
While we're at it, make sure the installed apt sources don't use the
configured mirror or snapshot. We only install a default set of sources
and for anything custom users should install their own sources.
Daan De Meyer [Fri, 17 Oct 2025 07:36:46 +0000 (09:36 +0200)]
Simplify implementation of distribution installers
- Use __init_subclass__() to automatically have distribution
installers register themselves with the DistributionInstaller class
on import
- Remove duplicated methods from Distribution that only forward to the
corresponding DistributionInstaller and instead have callers access
the Installer class directly. Make it a property to make this less
verbose
Daan De Meyer [Wed, 15 Oct 2025 19:39:02 +0000 (21:39 +0200)]
Look for .sdmagic before we consider a PE binary a UKI/addon
In ubuntu devel they started using the stubble bootloader for their
kernel images which includes a .linux and a .osrel section which means
it identifies as a UKI so we skip it. Let's additionally look for
.sdmagic as a workaround to fix this for now.
To implement this we can't make use of bootctl kernel-identify anymore
so we reimplement the logic ourselves with pefile. While we're at it,
we move KernelType to bootloader.py as it makes more sense to live there
than in qemu.py.
Let's introduce some profiles and remove unused packages to reduce
the size of the default image.
Both building the rpm and running mkosi inside the default image are
only rarely useful and shouldn't slow down image builds by default,
so we move both of these operations into their own profiles.
We drop the fish,zsh shells from the default image and also drop
qemu-user-static.
We also clean up a little inside mkosi-vm, getting rid of duplicated
packages across the main and per distro configs.
Daan De Meyer [Wed, 15 Oct 2025 10:32:13 +0000 (12:32 +0200)]
Add support for assert sections
Often you only want to support a single distribution. Adding a
[Match] section to mkosi.conf will be very confusing for users as
they will end up with an empty image. By using [Assert], they'll get
a clear error that what they're doing is not supported.
Daan De Meyer [Wed, 15 Oct 2025 10:58:40 +0000 (12:58 +0200)]
ubuntu: Switch to devel as the default release
Same reasoning as for the other distros, devel will always point to
something recent, whereas hardcoding stable release names will get out
of date very soon.
Daan De Meyer [Thu, 9 Oct 2025 06:50:44 +0000 (08:50 +0200)]
Don't unconditionally sync when PackageCacheDirectory=/var
If PackageCacheDirectory=/var and a cache directory is not configured
(like in mkosi-initrd), then we'd always sync repository metadata,
regardless of the value of CacheOnly=. Let's fix that by disallowing
CacheOnly=metadata|always if a cache directory is not configured and
only syncing if CacheOnly=never|auto, regardless if a cache directory
is configured or not.
config: serialize dataclass instances in our JSONEncoder
During config parsing we have partial dictionaries of our config, that are not
our Config object, but that we need to serialize as well. These may contain
dataclass instances such as ConfigTree objects on which the default encoder
chokes.
There are many many kernel packages in pmOS+Alpine and the pkg names
don't follow any specific pattern that sets them apart from some other
non-kernel packages, so the implementation tries to exclude known
pkgs that are not kernels and assume anything that doesn't match these
package names/patterns are kernel packages.
Currently, because we build the default initrd as a substep of building
a regular image, we have lots of special cased logic for it and we still
propagate settings manually from the regular image to its default initrd.
Let's streamline this by treating the default initrd as a regular image.
The only complication about making this change is that we used to build
the default initrd on demand only if a kernel was actually installed into
the image. Because we have to make the decision of whether to build the
default initrd or not way earlier now, we can't check if kernels were
installed into the image or not. Instead, we check if any known kernel
packages are listed to be installed which should be a decent enough
heuristic.
Another regression is that the default initrd won't have access to any
packages built as part of the main image build anymore. We used to rely
on this in systemd but now we build the systemd packages in a separate
build subimage and those will still be available to the default initrd
image build.
We have to stop using Bootable=yes in a few tests as using it now means
the resources folder has to be available and we don't propagate it
during tests.
apk does not use any subdirectories under /var/cache/apk to store
packages, which means that our usual tricks to mount package cache
directories from the package cache directory and repository metadata
from the metadata cache directory into the expected locations don't work.
There might be a way to get this to work with overlayfs but this would
be a very complex change. Instead, let's just disable repository metadata
caching for apk and always use the package cache directory for everything.
Let's simplify things by always caching repository metadata per image
instead of sharing repository metadata for some scenarios. We already
stopped sharing repository metadata for pacman and zypper due to these
package managers not being able to handle this use case. For dnf and apt,
while they can handle the use case, the repository metadata isn't so big
that sharing it across multiple mkosi projects saves a lot.
On the other hand, we can drastically reduce complexity by not sharing
repository metadata and reduce the number of copies as well. Instead of
always copying repository metadata to a temporary directory, we now have
package managers read it directly from the metadata cache directory if one
is configured and a temporary directory otherwise. This avoids having to copy
the repository metadata around completely which means we can remove
copy_repository_metadata() completely.
To avoid introducing a requirement on Incremental=yes to have cached repository
metadata, we simply always create a repository metadata (and keyring) cache if
a cache directory is configured. The usage of repository metadata and keyring
caching is now fully independent of Incremental=. Only CacheOnly= will affect
whether we sync repository metadata or use the already cached repository metadata.
When cleaning, we stop cleaning repository metadata and keyring caches with -ff
and move this to -fff. Additionally, we stop making -fff clean up the package cache
as I doubt users will ever want to clean up the shared package cache with mkosi clean
and are mostly interested in cleaning up their project directory of mkosi files.
Currently we only read uppercase variables starting with a letter, but shell
variables only restriction is not starting with a number, so lowercase names
and names starting with an underscore are allowed.
Add ASCII flag to regexes using backslash character classes
By default Python regexes like strings are unicode meaning that they match
anything unicode consides, e.g. a number in the case of \d, which is more than
the usually expected [0-9]. Tighten this in the places where these classes are
used for better readability.
Also reorder the character classes for KERNEL_VERSION_PATTERN and the
systemd-stub version to be the same order for clarity and escape the dash in
the latter, since the need to escape a dash in a character range is position
dependent.
postmarketos: Set up usrmerge in install() instead of setup()
We try to not touch the rootfs directory in setup() at all, so set
up merged /usr in install() instead like we do for debian. Additionally,
we also make sync() does not touch the rootfs either by having it operate
on a temporary directory instead of the real rootfs.
Clayton Craft [Wed, 4 Jun 2025 22:44:50 +0000 (15:44 -0700)]
Detect kernel ver from modules sub dir if unable to extract from filename
postmarketOS doesn't include the kernel version in the filename of the
kernel image. It would be cleaner to update kernel packages to make a
symlink for the kernel under /usr/lib/modules/<ver>, but this would be
difficult to implement in postmarketOS where we ship something like >400
kernel packages for a huge array of different devices / kernel versions.
Currently Alpine/pmOS only support having 1 kernel / version installed
at one time, so this fails if, for some reason, multiple kernel dirs
are found since there's no way to determine which one maps to the kernel
binary mkosi is using.
dnf install will not upgrade packages unless --best is specified.
While this might also break downgrades, not upgrading is worse than
not downgrading, so let's specify --best unconditionally again so we
make sure we always upgrade if a newer version is available.
Now that libffi marks the trampoline file descriptor opened by ctypes
as O_CLOEXEC, we can move file descriptor packing to the very end of
setting up the sandbox and close all file descriptors marked with
O_CLOEXEC already which will also close the libffi trampoline file
descriptor.
DaanDeMeyer [Mon, 25 Aug 2025 09:27:44 +0000 (11:27 +0200)]
Replace RuntimeHome= with BindUser=
Let's replace our home grown RuntimeHome= option with BindUser= to
mimick systemd-nspawn's --bind-user= option. systemd-vmspawn will soon
learn to support --bind-user= as well and we'll enable the option for
it when that happens as well.
Daan De Meyer [Sun, 23 Mar 2025 18:36:49 +0000 (19:36 +0100)]
Pass credentials as files where applicable
- Credentials can be rather large, which leads to huge command lines
if we pass them as a (encoded) string
- Credentials can be security sensitive, and passing them as a string
makes it very easy to discover them via `ps` or similar.
Let's solve both issues by always passing credentials as files.
sandbox: Make all relative paths absolute during argparsing
We change the current working directory during processing in
mkosi-sandbox so let's make sure that we don't have any relative paths
leftover by that point that would become invalid.
dnf: Fix /var package cache directory check in package_subdirs()
package_subdirs() receives the full cache directory path, not the
configured package cache directory (something like /var/cache/libdnf5),
so we fix the check to check if the parent directory of the given
cache directory is /var/cache so we don't have to know whether the
last component is dnf or libdnf5.
Drop BuildSourcesEphemeral=yes from default image config
I'm making sure the mkosi rpm spec build cleans up after itself in
https://src.fedoraproject.org/rpms/mkosi/pull-request/24# so we don't
need this anymore
Clayton Craft [Tue, 26 Aug 2025 18:21:50 +0000 (11:21 -0700)]
Add devicetree-auto support for UKI
This renames Devicetree --> Devicetrees, and adds support for listing
globs and files as currently supported for KernelModule* options.
When multiple devicetrees are found, they are added to dtbauto sections
when building a UKI.
Multiple dtbs are not supported for type 1 booting.
In systemd CI, we often run into issues caused by updates to third-party
components like the kernel package in rolling release distributions
like Arch Linux or Fedora Rawhide. When these happen, the corresponding
CI job starts failing on every PR and bisecting the distribution to figure
out when the breakage was introduced is rather tedious.
To mitigate this problem, we need to be able to pin the rolling release
distributions to a specific snapshot which we control. This allows us to
update the pinned snapshot in a PR created by a bot, so that any failures
introduced by moving to a newer snapshot will be limited to the PR that bumps
the snapshot. Any regressions can then be debugged and fixed before merging
the PR that switches us to the new snapshot.
To make this possible, let's introduce a new Snapshot= setting and implement
it for every distribution that has a snapshot concept or something that maps
to it. Per distribution:
- Debian => snapshot.debian.org (unlimited)
- Ubuntu => snapshot.ubuntu.com (unlimited)
- Arch => archive.archlinux.org (unlimited)
- OpenSUSE => download.opensuse.org/history (limited to a month of snapshots)
- CentOS => composes.stream.centos.org (limited to 3 weeks of snapshots)
- Fedora => https://kojipkgs.fedoraproject.org (limited to 2 weeks of snapshots)
Additionally, for CentOS, we also support using composes from mirror.facebook.net
which keeps them around forever so we get unlimited snapshots there as well for
CentOS Stream.
We also add a latest-snapshot verb to be able to easily figure out the latest
snapshot so it can be bumped regularly via a CI workflow. Because we do not track
sufficient information from config files to be able to insert the updated snapshot
into the right config file ourselves, we output it on stdout instead and leave it to
users to insert it into the right config file.
projects / thirdparty / mkosi.git / log
