config: serialize dataclass instances in our JSONEncoder
During config parsing we have partial dictionaries of our config, that are not
our Config object, but that we need to serialize as well. These may contain
dataclass instances such as ConfigTree objects on which the default encoder
chokes.
There are many many kernel packages in pmOS+Alpine and the pkg names
don't follow any specific pattern that sets them apart from some other
non-kernel packages, so the implementation tries to exclude known
pkgs that are not kernels and assume anything that doesn't match these
package names/patterns are kernel packages.
Currently, because we build the default initrd as a substep of building
a regular image, we have lots of special cased logic for it and we still
propagate settings manually from the regular image to its default initrd.
Let's streamline this by treating the default initrd as a regular image.
The only complication about making this change is that we used to build
the default initrd on demand only if a kernel was actually installed into
the image. Because we have to make the decision of whether to build the
default initrd or not way earlier now, we can't check if kernels were
installed into the image or not. Instead, we check if any known kernel
packages are listed to be installed which should be a decent enough
heuristic.
Another regression is that the default initrd won't have access to any
packages built as part of the main image build anymore. We used to rely
on this in systemd but now we build the systemd packages in a separate
build subimage and those will still be available to the default initrd
image build.
We have to stop using Bootable=yes in a few tests as using it now means
the resources folder has to be available and we don't propagate it
during tests.
apk does not use any subdirectories under /var/cache/apk to store
packages, which means that our usual tricks to mount package cache
directories from the package cache directory and repository metadata
from the metadata cache directory into the expected locations don't work.
There might be a way to get this to work with overlayfs but this would
be a very complex change. Instead, let's just disable repository metadata
caching for apk and always use the package cache directory for everything.
Let's simplify things by always caching repository metadata per image
instead of sharing repository metadata for some scenarios. We already
stopped sharing repository metadata for pacman and zypper due to these
package managers not being able to handle this use case. For dnf and apt,
while they can handle the use case, the repository metadata isn't so big
that sharing it across multiple mkosi projects saves a lot.
On the other hand, we can drastically reduce complexity by not sharing
repository metadata and reduce the number of copies as well. Instead of
always copying repository metadata to a temporary directory, we now have
package managers read it directly from the metadata cache directory if one
is configured and a temporary directory otherwise. This avoids having to copy
the repository metadata around completely which means we can remove
copy_repository_metadata() completely.
To avoid introducing a requirement on Incremental=yes to have cached repository
metadata, we simply always create a repository metadata (and keyring) cache if
a cache directory is configured. The usage of repository metadata and keyring
caching is now fully independent of Incremental=. Only CacheOnly= will affect
whether we sync repository metadata or use the already cached repository metadata.
When cleaning, we stop cleaning repository metadata and keyring caches with -ff
and move this to -fff. Additionally, we stop making -fff clean up the package cache
as I doubt users will ever want to clean up the shared package cache with mkosi clean
and are mostly interested in cleaning up their project directory of mkosi files.
Currently we only read uppercase variables starting with a letter, but shell
variables only restriction is not starting with a number, so lowercase names
and names starting with an underscore are allowed.
Add ASCII flag to regexes using backslash character classes
By default Python regexes like strings are unicode meaning that they match
anything unicode consides, e.g. a number in the case of \d, which is more than
the usually expected [0-9]. Tighten this in the places where these classes are
used for better readability.
Also reorder the character classes for KERNEL_VERSION_PATTERN and the
systemd-stub version to be the same order for clarity and escape the dash in
the latter, since the need to escape a dash in a character range is position
dependent.
postmarketos: Set up usrmerge in install() instead of setup()
We try to not touch the rootfs directory in setup() at all, so set
up merged /usr in install() instead like we do for debian. Additionally,
we also make sync() does not touch the rootfs either by having it operate
on a temporary directory instead of the real rootfs.
Clayton Craft [Wed, 4 Jun 2025 22:44:50 +0000 (15:44 -0700)]
Detect kernel ver from modules sub dir if unable to extract from filename
postmarketOS doesn't include the kernel version in the filename of the
kernel image. It would be cleaner to update kernel packages to make a
symlink for the kernel under /usr/lib/modules/<ver>, but this would be
difficult to implement in postmarketOS where we ship something like >400
kernel packages for a huge array of different devices / kernel versions.
Currently Alpine/pmOS only support having 1 kernel / version installed
at one time, so this fails if, for some reason, multiple kernel dirs
are found since there's no way to determine which one maps to the kernel
binary mkosi is using.
dnf install will not upgrade packages unless --best is specified.
While this might also break downgrades, not upgrading is worse than
not downgrading, so let's specify --best unconditionally again so we
make sure we always upgrade if a newer version is available.
Now that libffi marks the trampoline file descriptor opened by ctypes
as O_CLOEXEC, we can move file descriptor packing to the very end of
setting up the sandbox and close all file descriptors marked with
O_CLOEXEC already which will also close the libffi trampoline file
descriptor.
DaanDeMeyer [Mon, 25 Aug 2025 09:27:44 +0000 (11:27 +0200)]
Replace RuntimeHome= with BindUser=
Let's replace our home grown RuntimeHome= option with BindUser= to
mimick systemd-nspawn's --bind-user= option. systemd-vmspawn will soon
learn to support --bind-user= as well and we'll enable the option for
it when that happens as well.
Daan De Meyer [Sun, 23 Mar 2025 18:36:49 +0000 (19:36 +0100)]
Pass credentials as files where applicable
- Credentials can be rather large, which leads to huge command lines
if we pass them as a (encoded) string
- Credentials can be security sensitive, and passing them as a string
makes it very easy to discover them via `ps` or similar.
Let's solve both issues by always passing credentials as files.
sandbox: Make all relative paths absolute during argparsing
We change the current working directory during processing in
mkosi-sandbox so let's make sure that we don't have any relative paths
leftover by that point that would become invalid.
dnf: Fix /var package cache directory check in package_subdirs()
package_subdirs() receives the full cache directory path, not the
configured package cache directory (something like /var/cache/libdnf5),
so we fix the check to check if the parent directory of the given
cache directory is /var/cache so we don't have to know whether the
last component is dnf or libdnf5.
Drop BuildSourcesEphemeral=yes from default image config
I'm making sure the mkosi rpm spec build cleans up after itself in
https://src.fedoraproject.org/rpms/mkosi/pull-request/24# so we don't
need this anymore
Clayton Craft [Tue, 26 Aug 2025 18:21:50 +0000 (11:21 -0700)]
Add devicetree-auto support for UKI
This renames Devicetree --> Devicetrees, and adds support for listing
globs and files as currently supported for KernelModule* options.
When multiple devicetrees are found, they are added to dtbauto sections
when building a UKI.
Multiple dtbs are not supported for type 1 booting.
In systemd CI, we often run into issues caused by updates to third-party
components like the kernel package in rolling release distributions
like Arch Linux or Fedora Rawhide. When these happen, the corresponding
CI job starts failing on every PR and bisecting the distribution to figure
out when the breakage was introduced is rather tedious.
To mitigate this problem, we need to be able to pin the rolling release
distributions to a specific snapshot which we control. This allows us to
update the pinned snapshot in a PR created by a bot, so that any failures
introduced by moving to a newer snapshot will be limited to the PR that bumps
the snapshot. Any regressions can then be debugged and fixed before merging
the PR that switches us to the new snapshot.
To make this possible, let's introduce a new Snapshot= setting and implement
it for every distribution that has a snapshot concept or something that maps
to it. Per distribution:
- Debian => snapshot.debian.org (unlimited)
- Ubuntu => snapshot.ubuntu.com (unlimited)
- Arch => archive.archlinux.org (unlimited)
- OpenSUSE => download.opensuse.org/history (limited to a month of snapshots)
- CentOS => composes.stream.centos.org (limited to 3 weeks of snapshots)
- Fedora => https://kojipkgs.fedoraproject.org (limited to 2 weeks of snapshots)
Additionally, for CentOS, we also support using composes from mirror.facebook.net
which keeps them around forever so we get unlimited snapshots there as well for
CentOS Stream.
We also add a latest-snapshot verb to be able to easily figure out the latest
snapshot so it can be bumped regularly via a CI workflow. Because we do not track
sufficient information from config files to be able to insert the updated snapshot
into the right config file ourselves, we output it on stdout instead and leave it to
users to insert it into the right config file.
Something in the PPC runners is broken causing all CI jobs to get
cancelled when trying to rerun CI jobs. Let's disable the PPC jobs
temporarily until the issues with the underlying runners are fixed
so the rest of CI can run again and we can merge PRs again.
DaanDeMeyer [Fri, 22 Aug 2025 14:29:19 +0000 (16:29 +0200)]
Return false from want_efi() for UKI outputs
This may seem counter intuitive, but we do not want to make UKIs
bootable on EFI firmware. Making stuff bootable on EFI firmware involves
installing systemd-boot and shim and other things which aren't required
when building a UKI, so let's return false for UKIs from want_efi() unless
explicitly requested.
DaanDeMeyer [Mon, 25 Aug 2025 12:23:51 +0000 (14:23 +0200)]
kmod: Beef up firmware symlink chasing
- It turns out we need to handle absolutely symlinks after all as
/usr/lib/firmware/regulatory.db is a symlink to /etc/alternatives
in Debian and derivatives.
- Our previous implementation didn't handle cases where a symlink target
consisted out of two parts e.g. if C -> A/B then we wouldn't try to resolve
A.
Fix both issues by switching to a minimal implementation of the chase()
function in systemd.
To avoid having to include /etc stuff in the kernel modules initrd, we resolve
get rid of any intermediate symlinks to /etc/alternatives that we encounter.
DaanDeMeyer [Mon, 25 Aug 2025 19:55:38 +0000 (21:55 +0200)]
kmod: Narrow glob used to select firmware
We want to select the firmware regardless of which compression extension
it has, but let's insist on having at least a '.' which indicates what
follows is an extension in the first place.
DaanDeMeyer [Sun, 24 Aug 2025 15:48:39 +0000 (17:48 +0200)]
Make Bootable= determine whether we build a UKI for esp images
Currently, we build a UKI for ESP images only if a kernel is installed.
Let's make this a bit more flexible by hooking it up to the Bootable=
setting. If Bootable=no, then we won't add a UKI to the esp regardless
if a kernel is installed in the image or not.
tree-wide: Remove numbered prefixes from config files
With the following changes, there's no more need for numbered prefixes
for ordering:
- Assume EPEL is available for CentOS Stream 9/10
- Stop enabling epel-next repository for CentOS Stream 9
- Remove orphan_file hack for ubuntu jammy since we do it internally now
So we make these changes and remove the numbered prefixes throughout the
tree.
projects / thirdparty / mkosi.git / log
