Various fixes for grub packages in default tools trees
- Install grub2-tools on all architectures on CentOS/Fedora instead
of just x86-64.
- Only install grub-pc-bin on x86-64 on Debian/Ubuntu as it is not
available on other architectures
- Install grub-common instead of grub2 as grub-common contains all
the binaries we need and grub2 configures the system to use grub
as the bootloader which we don't want when these packages might be
installed on user's systems via mkosi dependencies.
- Do not install grub2 on OpenSUSE due to the same problem, it
configures the system to use grub as the bootloader which we don't
want. Unfortunately there is no subpackage with only the grub binaries
on OpenSUSE. Reported as https://bugzilla.opensuse.org/show_bug.cgi?id=1227464.
The tests are changed to not try to build or boot opensuse images
with bios.
Debian has a bespoke setup where if only grub-pc-bin is installed,
grub-bios-setup is installed in /usr/lib/i386-pc instead of in
/usr/bin. Let's take that into account and look for binaries in
/usr/lib/grub/i386-pc as well.
This requires another rework for the repositories() method for
OpenSUSE. Whereas before we picked up all the repositories from
https://download.opensuse.org/tumbleweed/repo/, it turns out that
the debug and source repositories can also be found at
https://download.opensuse.org/debug/tumbleweed/repo/ and
https://download.opensuse.org/source/tumbleweed/repo/ respectively.
Furthermore, the latter locations are the only ones that are available
for ports under https://download.opensuse.org/ports/ so we switch to
those instead.
Additionally, openSUSE-current is not available for ports so we disallow
using current, stable and release with architectures other than x86-64.
While there are more ports than just aarch64, for now let's just add
aarch64 and wait for user demand before we add any others.
run: Don't try to communicate inner pid for debug shell
The pipe write end has already been closed by then so bash will
fail with "bad file descriptor". Since there's no reason in having
the inner pid in the debug shell let's make sure we don't try to
communicate it there.
Always execute ukify with the configured python interpreter
We extend python_binary() with a binary argument so we can pass in
the binary we want to execute with python so python_binary() can
figure out whether we're going to use the python binary from the
tools tree or not.
Only use cache.direct if image size is aligned to page size
Trying to use cache.direct with an image whose size is not a multiple
of the page size fails so let's make sure we only use cache.direct if
that's the case and make sure the image size is a multiple of the page
size if we can.
Let's reduce duplication by using mkosi dependencies to install
all required dependencies to build the default image. To make this
work we add support for tools trees on jammy/bookworm. We also drop
support for a focal default image as it's too much of a burden to
maintain.
We also remove the EPEL only packages from the CentOS tools tree so
that mkosi dependencies works on a CentOS Stream system without EPEL
enabled.
Finally, curl-minimal is replaced by curl because trying to install
curl-minimal on a system with curl leads to a dnf failure so we opt
for the larger curl in order to make mkosi dependencies useful in
more situations.
Add %F to access the default filesystem of a distribution
One annoyance about using mkosi.repart has always been that to keep
using the default filesystem per distribution you have to write a
lot of matches. Now that systemd-repart supports
$SYSTEMD_REPART_OVERRIDE_FSTYPE_ROOT, let's add a specifier to access
the default filesystem so that it can be combined with the environment
variable to get the same result.
edk2 now has a virtio serial driver so let's switch to virtconsole
for the serial console as it's significantly faster compared to the
old ISA serial console.
All the latest releases of distributions now have a systemd stub
that knows how to read extra kernel command line arguments from
SMBIOS (It was backported to CentOS Stream 9), so let's drop the
default kernel command line and rely completely on passing the
console to use via SMBIOS.
This makes sure that users trying to deploy our images on bare metal
have a working console. Users that want to access the system via the
serial console will have to add the required console= argument
themselves.
Call rmtree() on context.root after finalize_staging()
After we've finalized the staging directory, there is no more need
for the root directory in the workspace. It used to get cleaned by
the setup_workspace() context manager but this is rather slow (can
take more than a second). By calling rmtree() explicitly, if we're
on a btrfs filesystem, we'll call btrfs subvolume delete which is
much faster than rm.
This allows using these to mount e.g. kernel modules as well
before switch-root. If there is no initramfs, the modules will still
be loaded in the rootfs.
Always rerun build if Format=none and don't remove previous outputs
Let's always rerun the build if Format=none. Also, since we know
Format=none won't touch any of the outputs we know about, let's keep
the existing outputs intact. This allows using Format=none to rerun
the build script without removing the existing output (e.g. directory
or disk image).
We'll be able to make use of this in mkosi-kernel to rebuild the kernel
modules without removing the disk image produced in a previous step.
We also simplify check_outputs() to only check the main output and not
the auxiliary outputs.
Add mirror to package cache key if one is specified
On Arch Linux, pacman seems to only check if the sync db on the
mirror is newer than the one available locally, which breaks when
using e.g. the archive where the sync db is older by definition.
Let's add any specified mirror to the default package cache dir key
to make sure that a different cache is used for explicitly specified
mirrors.
If not set in Environment=, we default to the user's home directory.
Let's instead use GNUPGHOME from the user's environment if it is set
before falling back to the user's home directory.
Michael Ferrari [Mon, 1 Jul 2024 09:10:36 +0000 (11:10 +0200)]
Support decompressing tar archives
This switches `extract_tar` to read the file itself instead of from
stdin to be able to auto-detect the compression method used.
BSD tar can auto-detect and decompress from stdin, but GNU tar only
auto-detects the compression method and notifies the user, but refuses
to run when not explicitly specifying the format on the command line.
Daan De Meyer [Sat, 29 Jun 2024 09:16:47 +0000 (11:16 +0200)]
Fix mirror paths for centos/fedora family of distributions.
We assumed that fedora would always be in the fedora subdirectory
of the mirror and that centos would always be in the centos-stream
subdirectory of the mirror but this is not guaranteed to be true
(see https://pagure.io/centos-infra/issue/1430).
Let's fix this by not always appending centos-stream/ or fedora/ to
the given mirror so the user can determine whether to include this
or not.
This is a breaking change but making sure all mirrors can be used
regardless of layout seems more important in this case.
For EPEL we do make a bigger assumption on the mirror layout since
we're forced to as we don't have a mechanism to specify more than one
mirror. We also allow users to override the epel mirror via an
environment variable.
mkosi-initrd: provide a separate mkosi-initrd script
Extract the basic functionality implemented on the kernel-install plugin into a
separate script. This allows to simplify the initrd/UKI build auto-filling
arguments of the `mkosi` call, and also auto-include user/distro configuration
from `/{etc,usr/lib}/mkosi-initrd`.
NamedTuple is an outdated solution… Make a normal dataclass to
match other places. Dataclasses also have nicer string representation.
Also let's not make the class nested. This doesn't matter terribly here,
but it makes the code unusual. Rename the type to "AptRepository"
to match RpmRepository.
Dataclasses are the more modern approach, with better string representation and
less hacks. A NamedTuple is also a tuple, which is rather surprising when we
don't use that.
Daan De Meyer [Wed, 26 Jun 2024 10:04:19 +0000 (12:04 +0200)]
Deal with systemd split packages added in Debian testing
systemd-repart and systemd-cryptsetup were moved to subpackages in
Debian testing. Let's make sure we account for that in mkosi-tools
and mkosi-initrd.
Daan De Meyer [Mon, 24 Jun 2024 11:37:29 +0000 (13:37 +0200)]
Fix sdmagic check on CentOS
systemd-stub on CentOS has misaligned PE sections causing us to read
a bunch of null bytes from the sdmagic section. Let's treat this case
as not finding a stub version to fix the CI builds.
Luca Boccassi [Sun, 16 Jun 2024 22:34:02 +0000 (23:34 +0100)]
qemu/vmspawn: check for an uncompressed kernel too
On some architecture/distributions (eg: ppc64el and riscv64 on Debian/Ubuntu)
the kernel is shipped uncompressed, as vmlinux. If vmlinuz cannot be found,
try looking for vmlinux.
Daan De Meyer [Thu, 13 Jun 2024 20:46:42 +0000 (22:46 +0200)]
centos: Fix GPG key search logic
Let's only look for the release key of the release we're building for
and add support for searching for the SHA256 key which is used by centos
stream 10.
We can't use the symlinked names because those don't exist on centos.org/keys.
Daan De Meyer [Thu, 13 Jun 2024 19:38:39 +0000 (21:38 +0200)]
centos: Change local rpm GPG key location
Let's look for the one suffixed with the release number. This doesn't
change behavior for c9s as this is a symlink to the file we looked for
previously but will fix our behavior for c10s which uses a different key.
Michael Ferrari [Sun, 9 Jun 2024 10:34:38 +0000 (12:34 +0200)]
Set default output name for `mkosi-tools`
This should allow one to invoke `mkosi --directory "" --include mkosi-tools`
in the project root to create a tools tree that is automatically
detected and used in subsequent invocations.
projects / thirdparty / mkosi.git / log
