Very niche repo for a very specific use case. Let's not enable it
by default for Alma Linux images. It can always be added back via
--repository-directory if needed.
The configured repos don't end up in the actual image, so we don't
need to bother with a fancy name for the repos, let's just use the
repo ID instead and get rid of a bunch of cruft.
Use distro~release subdirectories under mkosi.output/mkosi.builddir
Currently, when using incremental mode, building for a different
release or distribution means throwing away the cached images for
the previous distribution or release used unless each distro/release
combo is configured with an explicit output directory. Let's try to
be smarter here, by using the same logic as used for the cache path.
We create distro~release subdirectory under mkosi.output/ and use that
as the output directory. This makes sure cached images stay intact
even if we build for a different distribution. The same reasoning
applies to mkosi.builddir/.
This will end up using slightly more disk space when building for many
different distros when using mkosi.output/, but this should be a good
tradeoff to make regardless. If looking to regain disk space, a user
simply has to remove the output or build subdirectory for the distros
they're not interested in keeping.
Booting a systemd-nspawn container with a Photon image produced by
mkosi without --netdev waits for 2 minutes until "wait for network
to come online" times out. After logging in there's 3 failed services
in systemctl status. This is without trying to use any of mkosi's
more advanced features.
Bootable images are not supported at all on Photon. Given this and
the broken container support, it's clear that Photon support isn't
really usable at the moment and probably hasn't been for quite a
while (see #664).
Given that Photon has been broken for multiple releases and that
aside form #664, no one has bothered to report concrete issues or
make an attempt to fix the issues aside from a mirror update, this
gives us a pretty clear indication that no one is using or trying
to use mkosi to build Photon images.
Given that none of the existing maintainers are familiar with Photon
(and are likely not interested in doing the effort needed to support
it), let's drop the Photon support from mkosi.
Force initializing the partition table every time create_image is called
When creating the build and final images `args.partition_table` is not
re-initialized which results in a mismatch between the on disk partition
layout and the records in memory.
action: Install dependencies in action repo checkout
Since we symlink systemd-nspawn from the build directory to /usr/bin,
if the directory that nspawn was built in is cleaned up (e.g. via
git clean), systemd-nspawn is gone as well. To make it harder to
accidentally delete systemd-nspawn, let's clone and build all the
dependencies we compile from source in the repo checkout of the
action instead of in the repo of the project that's using the mkosi
action.
Fixed regression when using UsrOnly during initial install
Several distributions call `run_workspace_command()` during installation and
this now requires the root directory that gets bind mounted in to exist. The
created directory will be removed anyway since it exists under a temporary
workspace.
As an example, the documentation removal when installing a Debian derivative
will fail without this change.
We shouldn't special case updates Arch if we don't do so on other
distros either. We don't guarantee availability of a package
manager in generated images, and the only reason Arch images ship
with pacman is because it's part of the base group. This might
change in the future and as such, there's no guarantee that Arch
images will always have pacman. Hence, let's remove the pacman hooks
we install for Arch to bring it more in line with the other distros.
opensuse: do not attempt to overwrite pam.d/common-auth
Tumbleweed builds started to fail:
File /usr/local/lib/python3.10/dist-packages/mkosi/__init__.py, line 3225, in install_opensuse
shutil.copy2(root / usr/etc/pam.d/common-auth, root / etc/pam.d/common-auth)
File /usr/lib/python3.10/shutil.py, line 434, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File /usr/lib/python3.10/shutil.py, line 254, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/mkosi-vb08b_gb/root/usr/etc/pam.d/common-auth'
/etc/pam.d/common-auth is already installed now. Do not attempt to
overwrite it if it already exists.
Due to lack of a maintainer, and deviating from established practices
almost universally (custom bootloader, custom package manager, ...),
it's not productive to keep support for Clear Linux in mkosi. We have
no idea if it works at all, no idea if something we do is going to
break it, and haven't received any feedback from Clear Linux users
for multiple years. Hence let's drop support for Clear Linux from
mkosi to reduce maintenance costs.
Building initrds is slow, let's add an option that allows building
the initrd as part of a cached image so that's it's only built once
and reused in any final images.
Daan De Meyer [Fri, 24 Jun 2022 10:04:35 +0000 (12:04 +0200)]
action: Pin systemd-container version
A faulty metadata update in Github Actions azure mirror for Ubuntu
is causing issues with unmet systemd-container versions. Let's pin
the systemd-container version as a workaround until the issue is
fixed.
Daan De Meyer [Mon, 7 Mar 2022 16:10:41 +0000 (16:10 +0000)]
Add support for qemu's direct Linux boot
We add a "linux" boot protocol that, when enabled, instructs mkosi
to extract the kernel image, initrd and kernel cmdline out of the
image when building it.
On top, we add a --qemu-boot option that takes one of the possible
values from --boot-protocols and instructs qemu to use that boot
protocol. In case of the new "linux" boot protocol, we use qemu's
-kernel, -initrd and -append options to do a direct Linux kernel
boot.
Being able to do direct Linux boots with qemu is a prerequisite for
booting images of architectures that don't support UEFI in qemu. It's
also faster than booting in UEFI mode which is useful when iterating
and not working on the bootloader.
mkosi: link /var/lib/rpm to /usr/lib/sysimage/rpm for compat with old rpm
This (partially) fixes #993. There are two aspects of compatiblity: rpm changed
the db backend from bdb to sqlite, and the location was changed. This patch
resolves the second issue. It does using the same logic that e.g. Fedora uses
after the move. But it doesn't do anything for the first part. But luckily, rpm
in Rocky/Centos 8 is linked with support for sqlite, so things work.
We need to unlink /var/lib/rpm because something creates it during
installation. I wanted to just create the symlink if there's nothing there,
but that doesn't work.
The same logic was used for dnf/tdnf/rpm/dpkg/apt, so let's split
it out to a helper.
I opted to use full paths with "/". Such paths are easier to read
and we avoid the risk of removing something from the host if somebody
uses an absolute path by mistake.
Daan De Meyer [Mon, 20 Jun 2022 12:51:10 +0000 (14:51 +0200)]
ci: Show output by default
Let's have pytest show output even for successful tests. This helps
with debugging issues that aren't related to a failing test (e.g.
why is gentoo slow to build).
Daan De Meyer [Wed, 16 Feb 2022 10:46:42 +0000 (10:46 +0000)]
Default to -cpu max when running VMs using QEMU
Currently, we don't specify the -cpu option when running under the TCG
accelerator. This leads QEMU to choose a very conservative default that
doesn't emulate all the instructions that modern distros are compiled with.
To avoid such issues, let's default to having QEMU emulate as many CPU
instructions as possible to avoid illegal opcode errors when running
virtual machines.
Daan De Meyer [Wed, 16 Feb 2022 10:44:41 +0000 (10:44 +0000)]
Change --qemu-smp default from 2 to 1
When trying to a boot a centos epel VM using QEMU, it will hang during
boot if the qemu smp option is set to a number higher than 1. To avoid
this and similar issues, let's default to 1 core per VM. If users need
more they can always configure the option explicitly.
Daan De Meyer [Mon, 21 Feb 2022 14:54:31 +0000 (14:54 +0000)]
Don't override stat of destination root when using copy_path()
Fixes unexpected scenarios where we modify the permissions of / when
using mkosi.extra/. See https://github.com/systemd/systemd/pull/22569#issuecomment-1045992142
for more information.
Daan De Meyer [Mon, 16 May 2022 14:57:02 +0000 (16:57 +0200)]
ci: Update to Ubuntu 22.04 LTS
Note: The LTS image is still in beta (https://github.com/actions/virtual-environments/issues/5490).
In Jammy, we have recent versions of zypper and dnf packaged so we
don't have to build them from source anymore. Also, sq is packaged
so we don't have to build sq from source anymore either.
The setup-github-actions.sh script is renamed to setup-pacman.sh
and it's reduced to only install the dependencies necessary to
build pacman and archlinux-keyring.
All other dependencies are moved to action.yaml and the action
mkosi.default script.
Daan De Meyer [Thu, 16 Jun 2022 21:43:45 +0000 (17:43 -0400)]
Communicate the associated dir in the ESP via /etc/kernel/entry-token
/etc/kernel/entry-token is the new way introduced in systemd v251 to
identify the directory that kernel-install and bootctl should install
their stuff to. If it exists, bootctl and kernel-install will read it
and use the directory inside to install things to.
Currently, in mkosi, we generate a random machine ID during the build
and use that as the directory under the ESP to install things to. Until
all distros we support get support for /etc/kernel/entry-token, we're
limited to using the machine ID as the directory to install stuff under
in the ESP.
Since the machine ID used during the build is scrubbed from the image,
users don't a way to figure out the directory in the ESP associated
with the rootfs after the build is finished. To fix this, let's write
the machine ID to /etc/kernel/entry-token before it is scrubbed so
that users can read the file to figure out which directory in the ESP
that they should look under to find stuff associated with the
corresponding rootfs.
Daan De Meyer [Mon, 13 Jun 2022 15:09:00 +0000 (11:09 -0400)]
Rework initramfs generation (again)
Currently, when building without unified kernel images, the generated
initramfs is generated as part of running the package manager for the
first time. Because of this, none of the changes made to the rootfs
after running the package manager (build script, extra-trees, postinst
script, ...) are taken into account when generating the initramfs. Also,
when building with unified kernel images, the generated initramfs does
take all changes made to the image into account.
To solve this inconsistency, let's generate the initrd manually instead
of relying on the package manager to do it for us. With the addition
of KERNEL_INSTALL_BYPASS, we can skip initramfs generation when
kernel-install is called by a post-installation script to generate the
initramfs. Similarly, we can use the INITRD environment variable to
do the same on Debian/Ubuntu systems.
To generate the initramfs manually, we simply call kernel-install at
a later point in the image build process. Because kernel-install doesn't
actually regenerate the initramfs on Debian/Ubuntu, we call
"dpkg-reconfigure dracut" there to make sure the initramfs is regenerated
before calling kernel-install.
Finally, because we now make sure the initramfs always includes all changes
made to the image, we modify install_unified_kernel() to call objcopy
again instead of dracut as we used to do but had to revert because calling
objcopy meant the initramfs wasn't regenerated.
Daan De Meyer [Fri, 17 Jun 2022 14:16:18 +0000 (10:16 -0400)]
machine: Add retries for ssh
We've been seeing quite a bit of "connection refused" errors in CI.
These are likely happening because sshd hasn't finished starting
yet.
The proper fix for this is to add notify socket support for systemd
running qemu VMs via virtio sockets, but even if that's added, it
will be a very long time before we can rely on it.
For now, let's add a retry mechanism for SSH connections to make
our CI setup more reliable.
Daan De Meyer [Fri, 17 Jun 2022 14:13:49 +0000 (10:13 -0400)]
Refactor command running in integration tests
Let's move run_command_image() into Machine.run(), introduce
run_systemd_cmdline() to get the systemd-run command line, and
remove all arguments from run_ssh() that aren't required anymore
now.
mkosi: optimize/fix patching of root part-type uuid
The bug was that the part-type write we did would get overwritten when the
partition table was subsequently rewritten when we were adding the verity and
verity-sig paritions. We don't need to write out the part-type manually, it's
enough to store the right value in our partition list. This makes things a bit
faster too.
We know that if we calculated the verity info, we'll insert a partition soon
after and then it'll get written correctly.
Daan De Meyer [Wed, 15 Jun 2022 20:24:41 +0000 (16:24 -0400)]
arch: Use gpgdir from host system
Instead of setting up the keyring in the image, let's reuse the
keyring from the host. If users want to use pacman in the image,
they just have to run pacman-key themselves in a postinst script
or such.
This speeds up building of images and hopefully also gets rid of
our CI issues with Arch where there's something keeping files open
in the root mount (which I expect is gpg-agent).
Daan De Meyer [Fri, 20 May 2022 13:05:09 +0000 (15:05 +0200)]
Fix losetup race condition with initializing partition devices
This fixes the same issue we've seen in the systemd repo where
using PARTSCAN introduces a race condition with trying to use
the partition device since the kernel initializes the partition
devices asynchronously. To avoid the issue, we initialize partition
devices manually using the BLKPG ioctl(). We also avoid the same
problem on detaching loop devices by removing partition devices
explicitly using the BLKPG ioctl().
See https://github.com/systemd/systemd/pull/22992,
https://github.com/systemd/systemd/pull/23427,
https://github.com/systemd/systemd/issues/23174 and
https://github.com/systemd/systemd/issues/17469 for more context.
projects / thirdparty / mkosi.git / log
