Daan De Meyer [Tue, 29 Mar 2022 14:30:16 +0000 (16:30 +0200)]
centos: Rebuild the rpm db in bdb format on older centos releases
When building centos stream 8 images from Fedora, the resulting rpmdb
in the image is written in the sqlite format. The rpm version available
in centos stream 8 only supports reading the sqlite format but not writing
to it, which effectively makes installing rpm packages in the resulting
image impossible.
As a temporary workaround, we run rpm --rebuilddb inside the image to rebuild
the db in the older bdb format. Ideally we'd use the rpm from the host for this
but rpm has removed bdb support in newer versions so this isn't possible.
Luckily, we can remove this hack once we remove support for centos stream 8
and lower.
Daan De Meyer [Sat, 5 Mar 2022 15:51:21 +0000 (15:51 +0000)]
Replace dracut unified kernel image kernel-install script with objcopy
When using objcopy, we can generate the initrd in the install phase
which is cached which speeds up builds of incremental images. We use
objcopy from the host system to avoid having to install binutils in
all generated images.
We also call kernel-install manually in opensuse, debian and ubuntu
builds to make sure we can find an initrd image to pass to objcopy.
By calling kernel-install, we also make sure that
--without-unified-kernel-images now works for these distros as well.
gsegatti [Thu, 10 Mar 2022 17:38:10 +0000 (09:38 -0800)]
Merging CI tests with MkosiMachineTest class
- Removing the usage of the file at /tests/pexpect/boot.py
- Replacing the build/boot process ownership to the mkosi/machine.py file.
- All tests will attempt to build a MkosiMachineTestCase class and run the test functions.
- We leverage the load_args() function from __init__.py in order to replace the if's from the ci.yml file.
- If a set of options is invalid, we skip such test.
- We also define a specific exception for a invalid combination of arguments to be parsed, named "MkosiNotSupportedException".
- MkosiNotSupportedException inherits MkosiException.
gsegatti [Mon, 7 Mar 2022 15:29:06 +0000 (07:29 -0800)]
Adding "shell" verb to Machine class.
Summary: Adding support for running Mkosi images from within the Machine class using "shell".
- What we do here is skip booting an image whenever using such verb. Then, for running commands, we utilise run_shell_cmdline()'s to leverage systemd-nspawn.
- We also add the shell run of the current set of tests to the CI.
- We add the "--pipe" option to the cmdline utilised by run_shell_cmdline() in case future scripts use Mkosi from outside a terminal.
Daan De Meyer [Fri, 4 Mar 2022 20:57:24 +0000 (20:57 +0000)]
debian: Switch default release from unstable to testing
Unstable is a bit too bleading edge. Let's switch to testing which
is roughly 5 days behind unstable so we don't get affected by
temporary issues that make it to unstable.
Daan De Meyer [Fri, 18 Feb 2022 10:46:26 +0000 (10:46 +0000)]
Install a more specific match when --network-veth is used (#906)
* Install a more specific match when --network-veth is used
When doing more complicated network stuff in a QEMU VM, the network
file we install starts matching stuff it shouldn't match. Let's narrow
the match down by driver so we don't accidentally match against stuff
we don't want to match against.
Daan De Meyer [Mon, 14 Feb 2022 14:24:14 +0000 (14:24 +0000)]
Fix needs_build() logic
We should only check the -f option if the verb is one of the verbs
that needs a build. With the current check, we start rebuilding an
image even if the verb is "summary".
Daan De Meyer [Fri, 11 Feb 2022 12:12:58 +0000 (12:12 +0000)]
Error when trying to build an image without --bootable and run it with qemu
We can only check this when we're building the image, since when we're not
building the image it's not required to pass the --bootable option in order
to be able to use the qemu verb to run it in qemu.
This also fixes needs_build() to take the args.force option into account.
This fixes usage of needs_build() before unlink_output() is called.
Use Gentoo.EMERGE_UPDATE_OPTS for invoke_emerge call to install pkgs_boot
Using Gentoo.EMERGE_UPDATE_OPTS allows for greater maintainability than directly
specifying `--update` and `--deep`.
The configured `opts` for `invoke_emerge` enable the package manager to replace
blocking packages. This is required to successfully replace
`sys-kernel/installkernel-gentoo` (which is already installed in the gentoo
stage3 tar ball) with the requested `sys-kernel/installkernel-systemd-boot`.
Daan De Meyer [Thu, 3 Feb 2022 11:38:18 +0000 (11:38 +0000)]
ci: Add python integration tests job
To run the actual integration tests, we need to have the mkosi
action and dependencies installed. Instead of running the integration
tests as part of the unit tests job, let's add a new job for these
integration tests where we install the necessary dependencies (including
mkosi itself) and then run all tests marked with "integration" in pytest.
Add `--update` and `--deep` to invoke_emerge call for pkgs_boot
When generating bootable images for gentoo, the emerge call to install
pkgs_boot fails for `sys-kernel/installkernel-systemd-boot`:
```
[…]
‣ Invoking emerge(1) inside stage3
!!! /etc/portage/binrepos.conf is missing (or PORTAGE_BINHOST is unset), but use is requested.
* IMPORTANT: 9 news items need reading for repository 'gentoo'.
* Use eselect news read to view new items.
!!! /etc/portage/binrepos.conf is missing (or PORTAGE_BINHOST is unset), but use is requested.
[binary N ] app-arch/cpio-2.12-r1-1
[binary N ] sys-firmware/edk2-ovmf-202105-r2-1
[binary N ] dev-libs/elfutils-0.186-1
[binary N ] sys-kernel/installkernel-systemd-boot-1-1
[binary N ] sys-kernel/dracut-055-r4-2
[binary N ] virtual/libelf-3-1
[binary N ] sys-kernel/gentoo-kernel-bin-5.15.19-1
[binary N ] virtual/dist-kernel-5.15.19-1
[blocks B ] sys-kernel/installkernel-gentoo ("sys-kernel/installkernel-gentoo" is soft blocking sys-kernel/installkernel-systemd-boot-1)
[blocks B ] sys-kernel/installkernel-systemd-boot ("sys-kernel/installkernel-systemd-boot" is soft blocking sys-kernel/installkernel-gentoo-3)
* Error: The above package list contains packages which cannot be
* installed at the same time on the same system.
(sys-kernel/installkernel-gentoo-3-1:0/0::gentoo, installed) pulled in by
sys-kernel/installkernel-gentoo required by (sys-kernel/gentoo-kernel-bin-5.15.19-1:5.15.19/5.15.19::gentoo, binary scheduled for merge) USE="initramfs -test"
(sys-kernel/installkernel-systemd-boot-1-1:0/0::gentoo, binary scheduled for merge) pulled inby
sys-kernel/installkernel-systemd-boot
gsegatti [Mon, 24 Jan 2022 12:25:00 +0000 (04:25 -0800)]
Removal of Gentoo check
The parse_args() method has a check for whether Gentoo's modules are currently installed.
While this allows the system to fail early, it makes unit testing dependable on installing such packages.
By removing this if() we're able to test argument parsing without on any distro with no package dependance.
Additionally, system's correctness ir preserved since this same check is done later on gentoo.py
Luca Boccassi [Tue, 18 Jan 2022 00:08:54 +0000 (00:08 +0000)]
build: check for BUILD_ID/VERSION_ID before adding systemd-boot workaround
Instead of hard-coding the debian release, which breaks booting on
testing/bookworm, check os-release to see if BUILD_ID or VERSION_ID
are actually there.
Daan De Meyer [Mon, 3 Jan 2022 16:29:12 +0000 (08:29 -0800)]
Support older versions of sfdisk without "grain" support
If "grain" is not available, let's fall back to the default grain
size of 1 MiB. To check if "grain" is available, let's run sfdisk
against /dev/full and try to configure the "grain" which will fail
if "grain" is not supported.
Daan De Meyer [Mon, 3 Jan 2022 10:09:09 +0000 (11:09 +0100)]
action: Fix libsolv in combination with rpm 4.17.0
This applies the patch from https://github.com/systemd/mkosi/issues/861#issuecomment-1003624115
until the issue is fixed upstream and the fix is released.
Daan De Meyer [Thu, 23 Dec 2021 14:32:43 +0000 (15:32 +0100)]
docs: Move supported output formats section down a little
The verbs and execution flow are likely more important info to new
users than all the supported output formats so let's reorder the
sections accordingly.
Daan De Meyer [Thu, 23 Dec 2021 14:29:45 +0000 (15:29 +0100)]
docs: Remove Other Features section
This just lists some very specific features that are already
documented in their respective config options. Let's remove it
so we get rid of one more section that should be updated when we
add new features.
Daan De Meyer [Wed, 8 Dec 2021 11:23:12 +0000 (12:23 +0100)]
typing: Fix typing error when calling add_signer()
The add_signer() method of PKCS7SignatureBuilder only supports a
subset of the key types returned by load_pem_private_key() so let's
make sure the loaded key is of one of the supported types to fix
the error.
Paymon MARANDI [Wed, 24 Nov 2021 17:36:59 +0000 (12:36 -0500)]
gentoo: unmerge baselayout early on.
fixes ``Couldn't find '=sys-apps/baselayout-2.7' to unmerge``
intro:
there are 2 problems that require messing with baselayout package.
both are being worked on upstream [bug1][bug2].
1. `/etc/os-release` missing `VERSION_ID` which causes KUI generated by
dracut/installkernel et al not being accepted by the firmware/bootloader.
2. `/usr` is not merged in stage3. but `baselayout` with `"USE=build
-split-usr"` has a usrmerge layout.
[bug1] is fixed but that fix hasn't made it into the release.
[bug2] has 6 blocking bugs as of this writing.
we merge baselayout before copying stage3 tree over into `root` so the
usrmerge layout is in place.
source of the problem:
hard-coded baselayout version, "-2.7", was the source of the problem
which was inevitable to cause problem once upstream bumped the version
of baselayout they ship with stage3.
in the old logic we removed baselayout by hand during fetch_fix_stage3()
by removing members of baselaouy_qlist array from stage3 before copying
stage3 tree into `root`. this would cause the root to have two "slots"
of baselayout installed which the call to `emerge(1)` in `update_stage3`
would try to fix by removing the *bad* slot (=sys-apps/baselayout-2.7).
the new logic simply unmerges stage3's baselayout during
fetch_fix_stage3() using emerge(1).
Daan De Meyer [Wed, 27 Oct 2021 21:49:32 +0000 (22:49 +0100)]
Set new TTYRows and TTYColumns properties when overriding getty units
Sometimes, it's not sufficient to set the LINES and COLUMNS environment
variables to configure the serial terminal size. To properly configure
the serial terminal size when the env variables are not sufficient, we
need to configure the tty size in the kernel as well. To accomplish this,
two new properties TTYRows and TTYCols were added to systemd. Let's set
these properties when we override the getty units so systemd properly
configures the kernel tty size for us when systemd's version is recent
enough.
Additionally, this commit updates the getty unit resources to include
the recent systemd change to how the TTY is passed to agetty. Without
this change, the tty size configured with TTYRows and TTYCols is reset
when systemd invokes agetty. See https://github.com/systemd/systemd/pull/21171
for more information.
Luca Boccassi [Mon, 1 Nov 2021 23:38:37 +0000 (23:38 +0000)]
ssh: optionally connect to ssh-agent
If a path to the ssh-agent socket (typically /run/user/1000/gnupg/S.gpg-agent.ssh) is passed,
connect to it to fetch the public key(s) with ssh-add -L and avoid passing
the key(s) manually to ssh.
Daan De Meyer [Sun, 31 Oct 2021 21:01:28 +0000 (21:01 +0000)]
Call flush() in a few more places
Since we never explicitly close these files, sometimes the stuff
we write to them doesn't seem to get flushed which causes the
files to be empty in the output directory. Add calls to flush()
to make sure the files aren't empty when mkosi finishes.
Paymon MARANDI [Thu, 21 Oct 2021 09:51:13 +0000 (05:51 -0400)]
gentoo: support gentoo (#717)
Flow is the following:
we fetch stage3, untar it into the image's root and run emerge(1) from
within that root tree.
we _append_ our configurations to /etc/portage/env/mkosi.conf users
should not use that file since it, to some extend, documents what was
done during the image creation. users could instead drop-in their conf
file under mkosi.skeleton/etc/portage/env/ (say user.conf) and a pointer
under package.env/ (say user) with:
*/* user.conf
profile:
for now we only support 17.1/no-multilib; arm64 support is on the
roadmap.
projects / thirdparty / mkosi.git / log
