nfs-utils-2.8.4 will provide its own nfsroot-generator [1] to allow mounting the
real rootfs via NFSv4, so this initrd profile will enable this feature.
systemd-boot-efi is only available for EFI architectures, but
we use mkosi to do the integration tests on all architectures,
so this commit breaks them:
5044s E: Package 'systemd-boot-efi' has no installation candidate
Xavier Moffett [Fri, 30 May 2025 00:34:06 +0000 (20:34 -0400)]
fix: Remove `erofs-utils` from initrd
No longer necessary, since fsck is no longer invoked by systemd for
read-only filesystems. This fixes a bug causing the initrd stage
of Ubuntu images to fail.
Daan De Meyer [Wed, 4 Jun 2025 14:04:33 +0000 (16:04 +0200)]
ci: Drop differential shellcheck
We already run shellcheck on all scripts as part of the main CI job,
so we don't really get any benefit of a differential shellcheck as all
our code is already checked by shellcheck so let's just drop it.
Jörg Behrmann [Wed, 4 Jun 2025 09:04:26 +0000 (11:04 +0200)]
resources: add systemd-boot-efi
Since
https://salsa.debian.org/systemd-team/systemd/-/commit/a4bdf87f03e06fe687f0bf7596af31d0f2ac80b1
systemd-boot is not pulled in automatically by systemd-boot anymore
Daan De Meyer [Thu, 29 May 2025 08:36:29 +0000 (10:36 +0200)]
mkosi-tools: Introduce debug profile and move perf to it
perf pulls in a ton of dependencies (> 100 extra packages on Fedora).
Let's introduce a separate debug profile and move perf to it to avoid
everyone having to pull in all these packages for a tool they most likely
won't use.
Daan De Meyer [Thu, 29 May 2025 18:01:16 +0000 (20:01 +0200)]
ci: Use ext4 temporarily
The commit causing https://lore.kernel.org/linux-fsdevel/20250115185608.GA2223535@zen.localdomain/T/#u
got backported to the ubuntu stable kernel. While it has been reverted
upstream, the revert still hasn't reached the ubuntu stable kernel,
so let's use ext4 temporarily until that's the case.
Daan De Meyer [Wed, 28 May 2025 11:27:10 +0000 (13:27 +0200)]
Check that commit exists in config_parse_minimum_version()
merge-base will blow up when given a commit that does not exist in
the checked out repository, so let's check that first before checking
whether the currently checked out commit has the minimum version one as
its ancestor.
Daan De Meyer [Tue, 27 May 2025 12:11:13 +0000 (14:11 +0200)]
mkosi-initrd: Optionally match t64 suffix for tss2 libraries
On debian, these libraries have a t64 suffix on arm for "reasons".
Let's take that into account by optionally matching a t64 suffix for
these libraries.
Daan De Meyer [Mon, 26 May 2025 21:02:10 +0000 (23:02 +0200)]
sandbox: Support using mkosi-sandbox as a library
Let's allow using mkosi-sandbox to create an in process sandbox by
not taking a command line to execute if mkosi-sandbox was itself
imported and not executed.
If we were imported, also don't do any print()'s to behave like a
proper library.
Daan De Meyer [Tue, 27 May 2025 09:00:07 +0000 (11:00 +0200)]
Enforce C.UTF-8 locale for all commands we run
So this is a shitty situation either way. We generally have no idea
which locales will be available in the tools tree. Most likely, it'll
just be the C.UTF-8 one. In that case, using the locale environment
variables from the host is pointless and leads to annoying errors from
lots of tools.
We could make sure all locales are installed but glibc-all-langpacks is
> 200M installed size on Fedora which feels like too much. So let's
instead always use C.UTF-8, it's not like mkosi itself is locale aware
anyway.
Daan De Meyer [Mon, 26 May 2025 15:01:25 +0000 (17:01 +0200)]
Serialize pid in state and check if still exists on load
It happens often enough that mkosi is SIGKILLed and doesn't get to
remove the state file, causing it to leak and cause issues. Let's
serialize the pid of the mkosi process to the state file and check
if it is still running on load.
Daan De Meyer [Mon, 26 May 2025 13:09:43 +0000 (15:09 +0200)]
Only pick up /etc/pki/tls and /etc/pki/ca-trust as certificate dirs
Extra rpm GPG keys can be configured in /etc/pki/rpm-gpg. Let's not
assume /etc/pki consists solely of certificates but instead, let's pick
out the two directories out of there that do concern themselves with
certificates.
This allows copying extra rpm gpg keys to /etc/pki/rpm-gpg with
SandboxTrees= without these getting overridden by the /etc/pki bind
mount we used to have before.
Daan De Meyer [Mon, 26 May 2025 13:06:19 +0000 (15:06 +0200)]
Look for rpm gpg keys from inside the sandbox
The current approach does not take sandbox trees into account. Let's
switch things around and look for keys from inside the sandbox to make
sure we always take all sources of files that go into the sandbox into
account.
Daan De Meyer [Mon, 26 May 2025 13:44:42 +0000 (15:44 +0200)]
Fix version bump check if image version was passed on CLI
We only want to generate a new version if no version was specified on
the CLI. To do that we need to check that the version on the CLI is None,
not the opposite.
Alberto Planas [Thu, 24 Apr 2025 13:57:56 +0000 (15:57 +0200)]
repart: use --append-fstab=auto if available
When generating images there is a chance that a pre-existent /etc/fstab
is present in the mkosi buildroot. systemd-repart can merge the already
existing fstab with the one generated, separating the sections in a way
that the systemd-repart can be reused several times.
Clayton Craft [Wed, 21 May 2025 22:59:36 +0000 (15:59 -0700)]
sandbox: handle case where dev node for tty doesn't exist
This fixes this crash, which happens even when isatty(2) is True:
Traceback (most recent call last):
File "/sandbox.py", line 1095, in <module>
main()
File "/sandbox.py", line 891, in main
ttyname = os.ttyname(2) if os.isatty(2) else ""
^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory
IIUC this is because isatty(2) is a valid file descriptor (inherited
from the parent process?), but ttyname() is unable find a device node
for it because it's not mounted in the sandbox.
This doesn't quite work as on the second run, mkosi.tools will be
picked up as the default value of ToolsTree= instead of "default"
so we lose cache invalidation.
Let's revert for now until we find a better solution.
Don't use default value if optional settings are set to none
Just like it's possible to disable usage of default values for collection
based settings, we should also support it for optional settings.
This can be used to disable usage of the tools tree even if mkosi.tools.conf
exists by setting ToolsTree= in mkosi.local.conf.
These symlinks to a different kernel directory make mkosi to fail:
```
‣ Applying kernel module filters
‣ Running modinfo to fetch kernel module dependencies
modinfo: ERROR: could not get modinfo from 'nvidia': No such file or directory
libkmod: ERROR: kmod_module_parse_depline: ctx=0x55cda95602a0 path=/usr/lib/modules/6.14.1-1-default/weak-updates/updates/thinkpad_ec.ko error=No such file or directory
modinfo: ERROR: could not get modinfo from 'hdaps': No such file or directory
libkmod: ERROR: kmod_module_parse_depline: ctx=0x55cda95602a0 path=/usr/lib/modules/6.14.1-1-default/weak-updates/updates/nvidia-open-driver-G06-signed-570.133.07/nvidia-modeset.ko.zst error=No such file or directory
modinfo: ERROR: could not get modinfo from 'nvidia_drm': No such file or directory
libkmod: ERROR: kmod_module_parse_depline: ctx=0x55cda95602a0 path=/usr/lib/modules/6.14.1-1-default/weak-updates/updates/nvidia-open-driver-G06-signed-570.133.07/nvidia.ko.zst error=No such file or directory
modinfo: ERROR: could not get modinfo from 'nvidia_uvm': No such file or directory
modinfo: ERROR: could not get modinfo from 'thinkpad_ec': No such file or directory
libkmod: ERROR: kmod_module_parse_depline: ctx=0x55cda95602a0 path=/usr/lib/modules/6.14.1-1-default/weak-updates/updates/nvidia-open-driver-G06-signed-570.133.07/nvidia.ko.zst error=No such file or directory
modinfo: ERROR: could not get modinfo from 'nvidia_modeset': No such file or directory
libkmod: ERROR: kmod_module_parse_depline: ctx=0x55cda95602a0 path=/usr/lib/modules/6.14.1-1-default/weak-updates/updates/thinkpad_ec.ko error=No such file or directory
modinfo: ERROR: could not get modinfo from 'tp_smapi': No such file or directory
```
Also, do not run `depmod` for this old kernel directory that contains only the
`update` directory.
These KMPs install their kernel modules under the updates folder of a certain
kernel directory, even with that kernel directory without any other content:
```
$ mkosi-initrd -k 6.14.0-1-default
‣ Validating certificates and keys
‣ Copying repository metadata
‣ Building main image
‣ Copying in sandbox trees…
‣ Installing openSUSE
...
‣ Applying kernel module filters
‣ Running modinfo to fetch kernel module dependencies
modinfo: ERROR: Module nvidia not found.
modinfo: ERROR: Module thinkpad-ec not found.
modinfo: ERROR: Module nvidia-modeset not found.
modinfo: ERROR: Module tp-smapi not found.
modinfo: ERROR: Module hdaps not found.
modinfo: ERROR: Module nvidia-uvm not found.
modinfo: ERROR: Module nvidia-drm not found.
‣ "modinfo --set-version 6.14.0-1-default --null nvidia thinkpad-ec nvidia-modeset tp-smapi hdaps nvidia-uvm nvidia-drm" returned non-zero exit code 1.
```
But, there is no need fail so far (after calling the package manager within
mkosi), some basic checks can be performed before spawning other processes.
Ryan Wilson [Mon, 21 Apr 2025 17:00:53 +0000 (10:00 -0700)]
mkosi: Override misconfigured gitconfig HTTP/HTTPS proxy with ProxyUrl
When running mkosi on a server with an outgoing HTTP/HTTPS proxy,
Build.ProxyUrl must be set. However, some repositories (e.g. systemd)
call git in their mkosi scripts (e.g. cloning package repositories for
distros) and .gitconfig can also contain different http.proxy and https.proxy
settings. Thus, if .gitconfig is misconfigured with the wrong proxy,
the user will get confusing errors related to git fetching repositories.
To ensure we use a consistent proxy across git and mkosi, we override
git http proxy configuration via GIT_CONFIG_COUNT, GIT_CONFIG_KEY_{n},
GIT_CONFIG_VALUE{n} environment variables. Most of the code complexity
is dealing with the case when these variables are set by the user via
Environment / EnvironmentFiles.
We disabled plugins by default a long time ago because the
versionlock plugin's configuration from the host was interfering
with image builds. Now that we have completely ownership over /etc,
there shouldn't be a need to disable plugins anymore if we assume
they won't do anything nasty by default without configuration.
Dan Streetman [Tue, 15 Apr 2025 23:28:49 +0000 (19:28 -0400)]
mkosi-obs: simplify generation of signed UEFI auth files
We can use systemd-keyutil to generate the PKCS#7 signature from the
OBS-provided PKCS#1 signature and OBS-provided x509 certificate, instead of the
'kernel-sign-file' script.
Also we can use the ESL files directly, instead of the perl statement that
removes the auth header, as mkosi.postoutput leaves them for us to use.
Currently, --auto-bump= bumps the version after finishing a build.
Effectively this means the version in mkosi.version doesn't reflect
the actual version of the image which seems weird. Instead, let's
generate the new version upfront so that it's used by the build but
only write it to mkosi.version when we do a successful build.
Additionally, look for a mkosi.bump script and use it if it exists
to generate the new version so we can generate a timestamp as the
version in particleos.
Only write CLI arguments to history instead of full config
The huge downside of writing the full config to the history is that
any modifications to config files are ignored until the image is
rebuilt.
If we make the argument that changes to config files are usually
supposed to be permanent rather than ephemeral, it doesn't make sense
to not take them into account when reusing the history. Especially since
the primary purpose of the history is to avoid specifying the same CLI
arguments over and over again without encoding those CLI arguments into
a config file.
So instead of saving the full config to the history and not reparsing
the configuration when reusing the history, let's only save the CLI
arguments to the history and always reparse the configuration, even when
we end up reusing the history.
This keeps the crucial benefit of the history intact (not having to repeat
yourself endlessly) while still making sure any new settings added to the
configuration files are taken into account immediately.
It also simplifies the implementation quite a bit, as we can get away
again with maintaining a single history file instead of two.
Additionally, we opt to completely ignore the history when using the sandbox
verb. The idea here is that the history only applies to the final image (and
subimages). This was already the case as any --tools-tree-xxx CLI options aren't
saved in the history as they are not fields of the Config object.
Disallow using --rerun-build-scripts with --force again
In the next commit, we're going to move history handling purely into
parse_config(), which means we can't check anymore if we built the
image or not, so let's disallow using --force with --rerun-build-scripts
again as the only way that works is if we can check whether we built the
image or not before writing the history.
installer: Make sure package manager state is preserved in the image
Package manager state should be preserved in the image as various
local features of the package manager may depend on this data to be
available. For pacman and zypper we already store local state inside
the image. For apt, we achieve this by storing all state in the image
by default except lists which is the repository metadata which we still
don't include in the image. For dnf/dnf5, we make sure the persistdir
points to inside the image.
Note that this state is still subject to removal by CleanPackageMetadata=
if the package manager is not installed or the option is explicitly enabled.
Only copy repository metadata from specific subdirs from /var
/var usually contains state specific to the local system so it's more
prudent to specify an allowlist of what should be copied rather than a
denylist of what shouldn't be copied.
This doesn't matter so much when using mkosi's own package cache directory
since that is only used when syncing repository metadata and not when actually
installing stuff, but it does matter when PackageCacheDirectory=/var is used since
then we're copying from a state directory in /var that is also used when installing
packages and as such will contain a lot of stuff that we don't want.
mkosi-obs: append certs from mkosi.uefi.db/ to 'db'
Allow appending an arbitrary number of certs to the autoenrolled UEFI
db. Expects list of PEM formatted files. Only implemented for OBS
builds for now.
Fail if --rerun-build-scripts is used and tools is out of date
If --rerun-build-scripts is used, we shouldn't rebuild the tools
tree unless --force is specified as well, so make sure we check for
that specific case.
Until now, we read the tools tree configuration both inside and
outside the sandbox. This is problematic for a few reasons:
- For a number of reasons, the tools tree configuration inside and
outside the sandbox may be different. This is especially true when
the history is used, as when invoking mkosi sandbox and mkosi is
invoked again within the sandbox, we don't know when invoking mkosi
sandbox whether the nested invocation will reuse the history or not.
This can lead to situations where mkosi sandbox thinks the tools tree
is up-to-date whereas mkosi inside the sandbox thinks the tools tree
is out of date, which is a situation that is very hard to resolve for
users.
- Even if we read the tools tree configuration inside the sandbox, there's
no real point to it except maybe for showing the summary. Since the
tools tree has already been built and mounted at that point, we can't
build or use a new tools tree, or clean up an old one.
Let's fix these issues by *not* reading the tools tree configuration
inside the sandbox, neither from configuration not from history. The
one piece of tools tree configuration we do need to pass into the sandbox
is the default tools tree path as that gets included in the history of
the other images so we pass it in via an environment variable
$MKOSI_DEFAULT_TOOLS_TREE_PATH.
As a consequence of this approach, we cannot write the tools tree history
from within the sandbox anymore, so we write it to a separate file from
outside the sandbox instead.
Additionally, we make more commands use the history. Up until now summary and
other verbs did not make use of the history. This doesn't really make sense,
so let's make sure they make use of the history as well. summary in particular
is important as it may be used to query information about the built image after
building it which needs to take the history into account.
We still allow summary to ignore the history by passing --force. Also we
still ignore the history for clean as there are use cases both for having it
use the history and not yet the --force flag already has meaning there.
Allow PCR signing settings to be overridden in sub-images
A sub-image is usually built from the same base image. That base image may be
used for a rescue image and a primary workstation image. If the rescue image
has PCR signing enabled then it could allow the disk to be automatically
unlocked. This change allows specifying separate PCR signing keys which will
support only the workstation sub-image performing an automatic unlock.
zypper: do not fail if a package configured to be removed is not found
`dnf` does not fail in this case:
```
‣ Removing 1 packages…
No matches were found for the following plugin name patterns while enabling libdnf plugins: builddep, reflink, versionlock
No matches were found for the following plugin name patterns while disabling libdnf plugins: *
No packages to remove for argument: shadow-utils
Nothing to do.
‣ Removing files…
```
But `zypper` fails by default, breaking the build. Found with mkosi-initrd in
openSUSE, the `shadow` package is not installed when using `WithRecommends=yes`.
```
‣ Removing 2 packages…
Warning: Enforced setting: $releasever=20241001
Reading installed packages...
'shadow' not found in package names. Trying capabilities.
No provider of 'shadow' found.
Resolving package dependencies...
The following package is going to be REMOVED:
sysuser-shadow
1 package to remove.
Package install size change:
| 0 B required by packages that will be installed
-3.0 KiB | - 3.0 KiB released by packages that will be removed
Backend: classic_rpmtrans
Continue? [y/n/v/...? shows all options] (y): y
(1/1) Removing: sysuser-shadow-3.3-8.2.noarch ..........................................................................................[done]
Installation has completed with error.
‣ "zypper --installroot=/buildroot --cache-dir=/var/cache/zypp --non-interactive --no-refresh --releasever=20241001 remove --clean-deps shadow sysuser-shadow" returned non-zero exit code 104.
```
This behavior can be changed using the `--ignore-unknown` global option (which
goes before the command).
action: Use inputs to work around github actions empty variables
Even using env doesn't seem to work, so let's try the other workaround
from the issue, using inputs default values to capture the right values
before they become empty.
projects / thirdparty / mkosi.git / log
