apk does not use any subdirectories under /var/cache/apk to store
packages, which means that our usual tricks to mount package cache
directories from the package cache directory and repository metadata
from the metadata cache directory into the expected locations don't work.
There might be a way to get this to work with overlayfs but this would
be a very complex change. Instead, let's just disable repository metadata
caching for apk and always use the package cache directory for everything.
Let's simplify things by always caching repository metadata per image
instead of sharing repository metadata for some scenarios. We already
stopped sharing repository metadata for pacman and zypper due to these
package managers not being able to handle this use case. For dnf and apt,
while they can handle the use case, the repository metadata isn't so big
that sharing it across multiple mkosi projects saves a lot.
On the other hand, we can drastically reduce complexity by not sharing
repository metadata and reduce the number of copies as well. Instead of
always copying repository metadata to a temporary directory, we now have
package managers read it directly from the metadata cache directory if one
is configured and a temporary directory otherwise. This avoids having to copy
the repository metadata around completely which means we can remove
copy_repository_metadata() completely.
To avoid introducing a requirement on Incremental=yes to have cached repository
metadata, we simply always create a repository metadata (and keyring) cache if
a cache directory is configured. The usage of repository metadata and keyring
caching is now fully independent of Incremental=. Only CacheOnly= will affect
whether we sync repository metadata or use the already cached repository metadata.
When cleaning, we stop cleaning repository metadata and keyring caches with -ff
and move this to -fff. Additionally, we stop making -fff clean up the package cache
as I doubt users will ever want to clean up the shared package cache with mkosi clean
and are mostly interested in cleaning up their project directory of mkosi files.
Currently we only read uppercase variables starting with a letter, but shell
variables only restriction is not starting with a number, so lowercase names
and names starting with an underscore are allowed.
Add ASCII flag to regexes using backslash character classes
By default Python regexes like strings are unicode meaning that they match
anything unicode consides, e.g. a number in the case of \d, which is more than
the usually expected [0-9]. Tighten this in the places where these classes are
used for better readability.
Also reorder the character classes for KERNEL_VERSION_PATTERN and the
systemd-stub version to be the same order for clarity and escape the dash in
the latter, since the need to escape a dash in a character range is position
dependent.
postmarketos: Set up usrmerge in install() instead of setup()
We try to not touch the rootfs directory in setup() at all, so set
up merged /usr in install() instead like we do for debian. Additionally,
we also make sync() does not touch the rootfs either by having it operate
on a temporary directory instead of the real rootfs.
Clayton Craft [Wed, 4 Jun 2025 22:44:50 +0000 (15:44 -0700)]
Detect kernel ver from modules sub dir if unable to extract from filename
postmarketOS doesn't include the kernel version in the filename of the
kernel image. It would be cleaner to update kernel packages to make a
symlink for the kernel under /usr/lib/modules/<ver>, but this would be
difficult to implement in postmarketOS where we ship something like >400
kernel packages for a huge array of different devices / kernel versions.
Currently Alpine/pmOS only support having 1 kernel / version installed
at one time, so this fails if, for some reason, multiple kernel dirs
are found since there's no way to determine which one maps to the kernel
binary mkosi is using.
dnf install will not upgrade packages unless --best is specified.
While this might also break downgrades, not upgrading is worse than
not downgrading, so let's specify --best unconditionally again so we
make sure we always upgrade if a newer version is available.
Now that libffi marks the trampoline file descriptor opened by ctypes
as O_CLOEXEC, we can move file descriptor packing to the very end of
setting up the sandbox and close all file descriptors marked with
O_CLOEXEC already which will also close the libffi trampoline file
descriptor.
DaanDeMeyer [Mon, 25 Aug 2025 09:27:44 +0000 (11:27 +0200)]
Replace RuntimeHome= with BindUser=
Let's replace our home grown RuntimeHome= option with BindUser= to
mimick systemd-nspawn's --bind-user= option. systemd-vmspawn will soon
learn to support --bind-user= as well and we'll enable the option for
it when that happens as well.
Daan De Meyer [Sun, 23 Mar 2025 18:36:49 +0000 (19:36 +0100)]
Pass credentials as files where applicable
- Credentials can be rather large, which leads to huge command lines
if we pass them as a (encoded) string
- Credentials can be security sensitive, and passing them as a string
makes it very easy to discover them via `ps` or similar.
Let's solve both issues by always passing credentials as files.
sandbox: Make all relative paths absolute during argparsing
We change the current working directory during processing in
mkosi-sandbox so let's make sure that we don't have any relative paths
leftover by that point that would become invalid.
dnf: Fix /var package cache directory check in package_subdirs()
package_subdirs() receives the full cache directory path, not the
configured package cache directory (something like /var/cache/libdnf5),
so we fix the check to check if the parent directory of the given
cache directory is /var/cache so we don't have to know whether the
last component is dnf or libdnf5.
Drop BuildSourcesEphemeral=yes from default image config
I'm making sure the mkosi rpm spec build cleans up after itself in
https://src.fedoraproject.org/rpms/mkosi/pull-request/24# so we don't
need this anymore
Clayton Craft [Tue, 26 Aug 2025 18:21:50 +0000 (11:21 -0700)]
Add devicetree-auto support for UKI
This renames Devicetree --> Devicetrees, and adds support for listing
globs and files as currently supported for KernelModule* options.
When multiple devicetrees are found, they are added to dtbauto sections
when building a UKI.
Multiple dtbs are not supported for type 1 booting.
In systemd CI, we often run into issues caused by updates to third-party
components like the kernel package in rolling release distributions
like Arch Linux or Fedora Rawhide. When these happen, the corresponding
CI job starts failing on every PR and bisecting the distribution to figure
out when the breakage was introduced is rather tedious.
To mitigate this problem, we need to be able to pin the rolling release
distributions to a specific snapshot which we control. This allows us to
update the pinned snapshot in a PR created by a bot, so that any failures
introduced by moving to a newer snapshot will be limited to the PR that bumps
the snapshot. Any regressions can then be debugged and fixed before merging
the PR that switches us to the new snapshot.
To make this possible, let's introduce a new Snapshot= setting and implement
it for every distribution that has a snapshot concept or something that maps
to it. Per distribution:
- Debian => snapshot.debian.org (unlimited)
- Ubuntu => snapshot.ubuntu.com (unlimited)
- Arch => archive.archlinux.org (unlimited)
- OpenSUSE => download.opensuse.org/history (limited to a month of snapshots)
- CentOS => composes.stream.centos.org (limited to 3 weeks of snapshots)
- Fedora => https://kojipkgs.fedoraproject.org (limited to 2 weeks of snapshots)
Additionally, for CentOS, we also support using composes from mirror.facebook.net
which keeps them around forever so we get unlimited snapshots there as well for
CentOS Stream.
We also add a latest-snapshot verb to be able to easily figure out the latest
snapshot so it can be bumped regularly via a CI workflow. Because we do not track
sufficient information from config files to be able to insert the updated snapshot
into the right config file ourselves, we output it on stdout instead and leave it to
users to insert it into the right config file.
Something in the PPC runners is broken causing all CI jobs to get
cancelled when trying to rerun CI jobs. Let's disable the PPC jobs
temporarily until the issues with the underlying runners are fixed
so the rest of CI can run again and we can merge PRs again.
DaanDeMeyer [Fri, 22 Aug 2025 14:29:19 +0000 (16:29 +0200)]
Return false from want_efi() for UKI outputs
This may seem counter intuitive, but we do not want to make UKIs
bootable on EFI firmware. Making stuff bootable on EFI firmware involves
installing systemd-boot and shim and other things which aren't required
when building a UKI, so let's return false for UKIs from want_efi() unless
explicitly requested.
DaanDeMeyer [Mon, 25 Aug 2025 12:23:51 +0000 (14:23 +0200)]
kmod: Beef up firmware symlink chasing
- It turns out we need to handle absolutely symlinks after all as
/usr/lib/firmware/regulatory.db is a symlink to /etc/alternatives
in Debian and derivatives.
- Our previous implementation didn't handle cases where a symlink target
consisted out of two parts e.g. if C -> A/B then we wouldn't try to resolve
A.
Fix both issues by switching to a minimal implementation of the chase()
function in systemd.
To avoid having to include /etc stuff in the kernel modules initrd, we resolve
get rid of any intermediate symlinks to /etc/alternatives that we encounter.
DaanDeMeyer [Mon, 25 Aug 2025 19:55:38 +0000 (21:55 +0200)]
kmod: Narrow glob used to select firmware
We want to select the firmware regardless of which compression extension
it has, but let's insist on having at least a '.' which indicates what
follows is an extension in the first place.
DaanDeMeyer [Sun, 24 Aug 2025 15:48:39 +0000 (17:48 +0200)]
Make Bootable= determine whether we build a UKI for esp images
Currently, we build a UKI for ESP images only if a kernel is installed.
Let's make this a bit more flexible by hooking it up to the Bootable=
setting. If Bootable=no, then we won't add a UKI to the esp regardless
if a kernel is installed in the image or not.
tree-wide: Remove numbered prefixes from config files
With the following changes, there's no more need for numbered prefixes
for ordering:
- Assume EPEL is available for CentOS Stream 9/10
- Stop enabling epel-next repository for CentOS Stream 9
- Remove orphan_file hack for ubuntu jammy since we do it internally now
So we make these changes and remove the numbered prefixes throughout the
tree.
Remove mirror from default package cache directory cache key again
We added this initially to deal with pacman not having the mirror
in its cache key of repository metadata. The downside of this approach
is that we cannot cache packages across different mirrors. As an
aternative, let's simply not cache repository metadata for pacman in
the package cache directory.
While we're at it, remove the hack we did for zypper to ensure it had
the mirror in its cache key and also don't store its repository metadata
in the package cache directory. The reasoning here is while we can make
sure our own generated repository ids have the hashed mirror in them, we
cannot do so for any repositories added by users, which might end up causing
conflicts.
dnf: Share package cache between repositories with different baseurl=
Currently, the dnf5 package is not shared between repositories with the
same id but different baseurl=. For building images this is not ideal,
we do not want to have to redownload all packages when switching the
baseurl= or similar for a repository, so let's fix this by having
package_subdirs() return a tuple of source and destination path, and
in dnf's implementation of it, use the same package cache directory
for all repositories with the same id, regardless of the baseurl= used.
Note that this only applies to the package cache directory, the repository
metadata is still cached in the cache directory that is keyed by the baseurl=
or equivalent setting.
DaanDeMeyer [Fri, 22 Aug 2025 13:58:18 +0000 (15:58 +0200)]
Stop passing --workspace-dir= in mkosi-initrd and mkosi-addon
The default value when running as root is /var/tmp now so there's no
need to specify --workspace-dir= explicitly anymore. This allows the
workspace directory to be changed in the configuration file in
/etc/mkosi-initrd and /etc/mkosi-addon.
https://github.com/systemd/mkosi/issues/3852 is better fixed
by not configuring --workspace-dir at all within mkosi-initrd.
This allows it to be changed via the config file as the CLI argument
won't override it anymore and the default value used when running as
root is /var/tmp anyway.
DaanDeMeyer [Thu, 21 Aug 2025 11:47:03 +0000 (13:47 +0200)]
fedora: Rework rawhide GPG key logic
- Drop secondary key logic as looking at https://github.com/rpm-software-management/distribution-gpg-keys/tree/main/keys/fedora,
this hasn't been used for a long time.
- If repository key fetching is enabled, always look up the key remotely
as e.g. on CentOS 9 or so the rawhide symlink might be horribly outdated.
- If not using repository key fetching, Use all local keys newer than the
rawhide key as well to maximize the chances of including the current rawhide
key.
- Resolve symlinks within the sandbox in find_rpm_gpgkey() as we might not be
able to resolve the symlinks outside of the sandbox.
Luca Boccassi [Mon, 18 Aug 2025 13:04:59 +0000 (14:04 +0100)]
mkosi-tools: virtiofsd is only available on a subset of architectures on debian/ubuntu
Package virtiofsd is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'virtiofsd' has no installation candidate
‣ "/work/prepare final" returned non-zero exit code 123.
Afaict, this should work quite similar to the previous dpkg-scanpackages but
also doesn't use perl and is part of apt. Certain distros don't have reprepro
packaged, e.g. nixpkgs. It's also a simpler to work with and smaller compared
to reprepro.
projects / thirdparty / mkosi.git / log
