Daan De Meyer [Thu, 16 Jan 2025 13:53:16 +0000 (14:53 +0100)]
centos: Enable EPEL for c10s tools tree as well
Now that EPEL exists for c10s, let's enable it for the c10s tools
tree as EPEL has distribution-gpg-keys which is crucial to be able
to use c10s as a tools tree.
Daan De Meyer [Wed, 15 Jan 2025 22:07:07 +0000 (23:07 +0100)]
Add a keyring cache and use it for Arch Linux
Currently, the Arch keyring from the host or tools tree is always
used directly without any option to modify it (except by modifying
it directly on the host or in the tools tree). We also don't support
RepositoryKeyFetch= for the Arch Linux keyring and it's one of the
sources of annoying mounts outside of /usr which complicates
"mkosi sandbox".
Let's improve the situation by switching to our own pacman keyring
instead of using the one from the tools tree or host. Instead, we'll
only use /usr/share/pacman/keyrings from the host or tools tree to
populate the keyring we maintain ourselves. Users can extend the
keyring created by mkosi with their own keys via sandbox trees. If
RepositoryKeyFetch= is enabled, we'll download the archlinux-keyring
package and extract the keyring from there into the sandbox.
The keyring cache is maintained together with the repository metadata
cache. The only difference is that sync_repository_metadata() will
update the global package cache whereas the keyring directory is always
maintained per cache directory instead of globally.
We take the opportunity to stop using Michel's kernel-utils ppa with a
more recent archlinux-keyring package in favor of enabling
RepositoryKeyFetch= by default for Arch builds on Ubuntu.
Bjorn Andersson [Thu, 29 Feb 2024 15:00:13 +0000 (16:00 +0100)]
Introduce support for overriding system's DeviceTree
While experimenting with OS images on development boards where either
the bootloader doesn't load a Flattened DeviceTree Blob, or if one wants
to replace the one provided by the bootloader with a specific/new one
for development/experimentation/testing purposes, it's convenient to use
the OS-loaders ability to load a specific DeviceTree blob.
Add a new option to mkosi, to cause a specified DeviceTree blob to be
baked into the UKI, or copied into /boot and added to the systemd loader
entry.
As different distributions store the dtb files in different locations
the requested file is searched for in the locations spotted in Debian,
Fedora, Arch Linux packages (both from distro and kernel build system -
as these differs as well).
Resolves #2439
Based on initial effort by Manuel Traut.
Daan De Meyer [Wed, 15 Jan 2025 08:48:21 +0000 (09:48 +0100)]
Use MKOSI_HOST_DISTRIBUTION in config_default_tools_tree_distribution()
It doesn't really matter since we won't use the tools tree in the
sandbox anyway but this reduces the diff between mkosi summary outside
and inside the sandbox.
Daan De Meyer [Tue, 14 Jan 2025 14:32:20 +0000 (15:32 +0100)]
Make sure we use the same host distribution/release in sandbox
If the distribution is not explicitly specified in the config file
and the host distribution does not match the tools tree distribution,
then currently mkosi -f sandbox mkosi -f and mkosi -f will build images
for different distributions.
Let's make sure these commands use the same default distribution by
communicating the host distribution and release to the sandbox via
environment variables and using them to determine the default values to
use if available.
Martin Hundebøll [Tue, 14 Jan 2025 07:09:22 +0000 (08:09 +0100)]
Bind mount /dev/fuse in sandbox
/dev/fuse is needed to support building container images from within the
mkosi.build scripts. Make it avaiable in the sandbox just like /dev/null
and its friends.
Since not all mkosi builds runs in host contexts where /dev/fuse is
actually available, update the logic to skip the /dev/fuse node if it is
not present.
Daan De Meyer [Mon, 13 Jan 2025 11:10:29 +0000 (12:10 +0100)]
Only use extra search paths when running in sandbox
Since ExtraSearchPaths= might be built within the sandbox, we have
to make sure that we're in the sandbox when potentially executing
binaries from ExtraSearchPaths=. This was almost always the case,
except when setup= was used when constructing the sandbox, as the
command in setup= is executed outside of the sandbox.
To fix this we also had the option to remove the setup= argument
from sandbox_cmd() and execute the command from setup= within the
sandbox as well, but this would mean the synchronization mechanism
we added in https://github.com/systemd/mkosi/pull/3298 to make sure
we don't register vms with machined before the scope is started would
stop working, so for now we keep setup= and instead make sure we don't
use paths from ExtraSearchPaths= when executing the commands in setup=.
Implementation wise, we get rid of modifying mkosi's own $PATH to append
the extra search paths, and instead construct the proper $PATH in
sandbox_cmd().
Because we get rid of prepend_to_environ_path(), we also get rid of
the logic to construct a separate directory for any files listed in
ExtraSearchPaths=, but this logic turned out to be undocumented and
broken because the separate directory we add to $PATH wasn't mounted
into the sandbox so since this logic was always broken and nobody ever
complained we opt to remove it here to keep the implementation simple.
Luca Boccassi [Sat, 11 Jan 2025 01:48:47 +0000 (01:48 +0000)]
apt: avoid ?exact-name for systemd packages
There is a strange interaction between ?exact-name, apt repositories
generated by OBS and multiple architectures, that breaks installing
packages:
$ apt install --dry-run '?exact-name(systemd-cryptsetup)'
NOTE: This is only a simulation!
apt needs root privileges for real execution.
Keep also in mind that locking is deactivated,
so don't depend on the relevance to the real current situation!
systemd-cryptsetup is already the newest version (257.999+731+g8c5b35957-0).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
Unsatisfied dependencies:
systemd-cryptsetup : Conflicts: systemd-cryptsetup:i386 but 257.999+731+g8c5b35957-0 is to be installed
Conflicts: systemd-cryptsetup:arm64 but 257.999+731+g8c5b35957-0 is to be installed
systemd-cryptsetup:i386 : Conflicts: systemd-cryptsetup but 257.999+731+g8c5b35957-0 is to be installed
Conflicts: systemd-cryptsetup:arm64 but 257.999+731+g8c5b35957-0 is to be installed
systemd-cryptsetup:arm64 : Depends: libc6:arm64 (>= 2.38) but it is not installable
Depends: libcryptsetup12:arm64 (>= 2:2.7) but it is not installable
Depends: libssl3t64:arm64 (>= 3.0.0) but it is not installable
Depends: libsystemd-shared:arm64 (= 257.999+731+g8c5b35957-0) but it is not going to be installed
Conflicts: systemd-cryptsetup but 257.999+731+g8c5b35957-0 is to be installed
Conflicts: systemd-cryptsetup:i386 but 257.999+731+g8c5b35957-0 is to be installed
Error: Unable to correct problems, you have held broken packages.
It seems ?exact-name selects _all_ available packages, unless an architecture
is specified:
$ apt install --dry-run '?exact-name(systemd-cryptsetup:amd64)'
NOTE: This is only a simulation!
apt needs root privileges for real execution.
Keep also in mind that locking is deactivated,
so don't depend on the relevance to the real current situation!
Summary:
Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
But this only happens with the repository metadata format generated
on OBS. There are multiple formats, apparently.
So either we set the specific architecture in the common mkosi.conf,
or we need a separate mkosi.conf with a trigger to filter out releases
where the packages are not available.
I chose the latter as it the filters are fixed, while adding by architecture
means every time a new arch is added, it needs to be duplicated.
This is only necessary for packages that are hosted on OBS, ie: the
systemd ones.
Daan De Meyer [Sun, 12 Jan 2025 11:29:57 +0000 (12:29 +0100)]
Fix passing repositories and sandbox trees to default initrd/tools
Currently if none are specified we end up passing the empty string
which overrides any repositories or sandbox trees specified in the
default initrd/tools configs themselves.
Daan De Meyer [Mon, 13 Jan 2025 10:27:04 +0000 (11:27 +0100)]
ci: Disable Azure
They're so flaky that I don't even look at them anymore, so let's
disable them until they do a new systemd stable release which
includes the required vsock fix.
Jörg Behrmann [Tue, 7 Jan 2025 14:54:18 +0000 (15:54 +0100)]
man: Implement various stye fixes
- Highlight command-names in text in bold instead of code style
- Consistent highlighting of command names
- Consistent hyphenation
- Removal of superfluous periods
- Addition of missing periods and colons
- Consistent capitalisation of RPM when not a command
Some modules reference a firmware file that is just a link to a "real"
file. Make sure those files are also included when needed by resolving
those referenced symlinks and including their target files in the set of
required firmware files.
Be less explicit about module matching sub directories
The kernel module include/exclude logic matches against paths below any
folder in /usr/lib/modules/$(uname -r)/*/, and not just the `kernel`
folder. Replace the explicitly mentioned `kernel` folder from the
example path with an asterisk to make it clear the folders like `extra`
and `updates` are searched too.
Daan De Meyer [Fri, 3 Jan 2025 19:59:09 +0000 (20:59 +0100)]
Keep more entries from $PATH in the sandbox
In the relaxed sandbox, we want to run executables frome everywhere
that isn't /usr. We make the implicit assumption that anything from
outside /usr doesn't depend (heavily) on what's in /usr and if it does
and a tools tree is used it should match the host sufficiently.
Daan De Meyer [Mon, 30 Dec 2024 16:10:53 +0000 (17:10 +0100)]
Parse profiles in subimages as well
Let's parse from mkosi.profiles in subimages as well. We'll still insist
that the profiles used are set in the main image though. To make this work
we have to remove the check to see if a profile exists or not, since it might
not exist in every mkosi.profiles/ directory that we parse. At the same time
we remove the restriction that profiles are set before we parse mkosi.conf.d/
since this isn't really useful anymore now that we parse profiles last and not
first anymore. This allows us to get rid of the concept of immutable settings.
Daan De Meyer [Fri, 27 Dec 2024 11:07:56 +0000 (12:07 +0100)]
Rename various [Runtime] options and rename mkosi qemu to mkosi vm
- Let's get rid of the "Qemu" prefix for runtime options as they apply
both to the qemu and vmspawn verbs.
- Let's rename QemuGui= to Console= and make it take all the options
that are accepted by vmspawn's --console= option.
- Let's rename the various runtime options to the corresponding vmspawn
option where applicable.
- Let's rename mkosi qemu to mkosi vm as well while we're at it.
Luca Boccassi [Thu, 26 Dec 2024 15:05:45 +0000 (15:05 +0000)]
Add option to control whether to register with machined
On an ephemeral test runner it is useless churn to register the
guest, and can trigger nasty bugs like the one where machined
kills the user session. Add a config knob, enabled by default,
so that it can be disabled where it's not useful.
This also allows running nspawn where machined is not available,
as currently it just fails since the default is that it tries to
register and fails if it can't.
Daan De Meyer [Mon, 23 Dec 2024 09:19:56 +0000 (10:19 +0100)]
Drop support for pesign secure boot signtool
sbsigntools is packaged in EPEL now and in the future we're looking
to standardize solely on systemd-sbsign so let's already drop support
for pesign which was only really useful on CentOS Stream because it
didn't have sbsigntools which it does have now.
Daan De Meyer [Sun, 22 Dec 2024 23:02:00 +0000 (00:02 +0100)]
Always check tools for build at the same place
Let's only check non-build verb tools early and check build verb
tools at the same place regardless of whether the verb is build or
not to keep things more consistent.
As a side effect this allows building tools from source to do the
build in sync scripts which is done in
https://github.com/davide125/arcadeos/blob/main/mkosi.sync.
Daan De Meyer [Fri, 20 Dec 2024 14:12:06 +0000 (15:12 +0100)]
burn: Fail if no arguments are provided
If no arguments are provided systemd-repart will operate on the
device backing the rootfs which in the case of burn we definitely
don't want to do most of the time so let's fail unless a device is
explicitly provided.
Daan De Meyer [Sun, 22 Dec 2024 11:47:06 +0000 (12:47 +0100)]
Wait until sandbox exec()'s specified command before continuing
Let's wait in spawn() until all setup logic has completed before
continuing if a sandbox is used. When using spawn() without run(),
this helps us fail early if the setup logic fails instead of later
on if we're not checking the result of the spawn() command immediately
afterwards.
As a side effect, this also acts as a synchronization point when using
systemd-run --scope where when spawn() returns, we can be 100% sure that
the scope has been created, which is important when calling RegisterMachine
from systemd-machined which does the wrong thing if the scope for the specified
machine does not yet exist and ends up killing the parent unit (often the user
session) instead of just the virtual machine.
Daan De Meyer [Fri, 20 Dec 2024 10:20:54 +0000 (11:20 +0100)]
Don't use ExtraSearchPaths= when building default tools tree
Since eba43f034c5c19a478249ba50fc1b97faffda75c, we always run binaries
from ExtraSearchPaths= within the tools tree if one is used. This generally
implies the binaries have to be built against the tools tree to be able to
run within it. This means that the binaries won't necessarily work when
executed on the host, which is the case when building the default tools tree.
To avoid failures when building the default tools tree and using ExtraSearchPaths=,
don't use binaries from ExtraSearchPaths= when building the default tools tree.
Daan De Meyer [Fri, 20 Dec 2024 08:56:48 +0000 (09:56 +0100)]
Use "tools" as the cache key for the default tools tree
Currently we still use "distribution~release~architecture", which
breaks checking if the cache is out of date, since we'll always check
against the cache of the currently configured tools tree distribution,
even if we used a different tools tree distribution to build the tools
tree. Let's fix this by using the same cache key for all default tools
trees.
This shouldn't introduce any performance regression as users are not
expected to change the default tools tree distribution all the time
unlike the target image distribution.
Daan De Meyer [Thu, 19 Dec 2024 11:35:47 +0000 (12:35 +0100)]
Run binaries from ExtraSearchPaths= within tools tree
Until now, there was always the implicit assumption that any paths
configured with ExtraSearchPaths= contained binaries built against
the host's /usr. With that assumption, it made sense to execute binaries
found in these paths outside of the tools tree as otherwise you might
end up with missing or out of date libraries if these are not available
within the tools tree.
However, with the introduction of "mkosi sandbox", what I want to do
in systemd is to have contributors build systemd within the sandbox and
then use those binaries in mkosi to build the image. This means that the
build directory configured with ExtrasSearchPaths= suddenly contains
binaries built against the tools tree's /usr (if one is configured) instead
of the host's /usr.
Given this new use case, let's get rid of the logic to not use the tools
tree for binaries in ExtraSearchPaths=, instead, users of ExtraSearchPaths=
using a tools tree will have to make sure to use a tools tree that mostly
matches their host's /usr.
Daan De Meyer [Wed, 18 Dec 2024 13:00:49 +0000 (14:00 +0100)]
Drop PEAddons=
Putting these inside the image isn't very useful except for the very
specific use case of being able to skip copying the addon when copying
the ESP to another disk to implement a "live" installer mode. For all
other use cases, PE addons are either supposed to be generated locally
or optionally downloaded alongside the image. For these use cases it
makes much more sense to output the addons alongside the image, but not
in it.
But if it makes more sense to output the addons alongside the image, then
it becomes unclear whether this is something that should be implemented inside
mkosi in the first place. Until we figure whether this makes sense and how
to implement it, let's remove the functionality.
Addons can still be added to the image or generated alongside it by running
ukify in a postinst script. The secure boot signing options will have to be
passed manually but this isn't an unsolvable problem.
projects / thirdparty / mkosi.git / log
