Daan De Meyer [Sun, 1 Oct 2023 18:20:12 +0000 (20:20 +0200)]
Add RuntimeSize= setting
Currently we unconditionally grow disk images to 8G before booting
them in systemd-nspawn or qemu. Let's do better here by making the
size configurable and not growing the disk images by default.
We also move format_bytes() to config.py as most other formatting
functions are located there.
Instead, let's show individual tools tree packages in the summary.
This makes things a bit more consistent as we don't show default
initrd presets in the summary either (and making that happen is no
trivial task so we opt to do the reverse and not show default tools
trees either).
We also add a table to the documentation showing which packages are
in the default tools tree for which distributions.
Daan De Meyer [Sun, 1 Oct 2023 08:26:12 +0000 (10:26 +0200)]
Don't reuse unprivileged cache when building as root
When using --incremental and first building an image unprivileged,
followed by building it as root, we can't reuse the cache as the
ownership will be wrong. So let's make sure we don't reuse the
cache when that's the case.
This makes it easier to print a message from the installer referring
to the distro by the proper name.
I looked at the home pages of the distros and took the spelling used
there. In particular Fedora is making a concious effort to use "Fedora"
for the whole project, and "Fedora Linux" for the distro. OTOH, Ubuntu
is known for not using "linux" anywhere where users could see it.
distributions: use a generator to reduce list boilerplate
When the list of repos to return is conditionalized, we can use yield instead
of concatenating lists. Iterable[Repo] can be satisfied by both the functions
which are generators and the functions which just return a list as before.
It is an optional component. The user would be unhappy when we fail at the end
of a lengthy installation process. So let's check early if we would need it and
refuse to continue.
(When config.bootable == ConfigFeature.auto, we cannot to do the check before
the installation is performed, so let's opt to be strict and do the check.)
Inspired by https://bugzilla.redhat.com/show_bug.cgi?id=2240598.
distributions: rename all installers to "Installer"
We were doing "double namespacing": "centos.CentosInstaller",
"ubuntu.UbuntuInstaller", etc. Let's simplify things by dropping the second
name. This avoids the awkward issue how capitalize the names of distributions
that already have capitals in their name, e.g. CentoOS, or dashes in the name,
e.g. RHEL-UBI. The one call site where we import the right installer becomes
simpler.
In Distribution.installer(), try..except is replaced by assert. If the
installer class for a distro is missing, then it's a programming error and not
not something we should suppress.
Make sure we append the setpgid command after all the options. Let's
also use -- to avoid any potential confusion about whether the following
arguments are options or not.
Georges Discry [Fri, 15 Sep 2023 20:19:41 +0000 (22:19 +0200)]
Convert script settings to lists
With `mkosi.conf.d`, `mkosi.presets` and/or `[Match]`, it's become
really easy to split the configuration of an image in logical parts. It
might be done to isolate some settings that go together or to share the
same configuration across several images.
However, for each `mkosi.prepare`, `mkosi.build`, `mkosi.postinst` and
`mkosi.finalize` step, only one script could be executed as the one with
highest priority overrides the others.
Each script is now managed as a list of executables so that it's also
possible to split them if desired.
Backward compatibility is kind of maintained by keeping the same options
on the command line and by introducing aliases for the renamed INI
settings (with a deprecation warning). These aliases also parse the
settings as a list, so compatibility is still broken when a script was
expected to override a previously defined one, as they are now both
executed.
Georges Discry [Tue, 19 Sep 2023 11:44:18 +0000 (13:44 +0200)]
Backward-compatibility for renamed INI settings
A setting can now declare aliases used when parsing INI files. These
aliases are useful when a setting is renamed as they maintain
backward-compatibility while emitting a deprecation warning.
Even if we don't have a bootloader or aren't configured to be
bootable, make sure we copy a kernel out of the image so we can
linux direct boot images in qemu.
It turns out this can be useful after all, for example to specify
whether to build in debug mode or release mode, or which config to
use when building kernels, so let's pass arguments to the build
script again.
Mount over /etc/passwd when building an image and running build script
Various tools parse /etc/passwd, so let's make sure it contains the
expected lines when building an image. Also, when running the build
script, mount over /etc/passwd in the image as various build tools
will also query the current user in /etc/passwd (e.g. the kernel's
build script runs "whoami"). Also add a root entry to the /etc/passwd
we generate so both the user running mkosi and root are covered.
Currently we only clean up /work if the build script succeeds.
However, we have to always clean up /work since we don't make a
copy of the build overlay so let's make sure we do that by using
a trap instead of only cleaning up if a script succeeds.
Let's also make sure that we don't fail if /work already exists and
make sure that if gets the right permissions, even if it already
exists.
The next release of util-linux will include setpgid which has a
--foreground option that does exactly the same as our foreground()
function. This new utility allows us to make the processes executed
by bubblewrap the foreground process, which makes sure that SIGINT
isn't intercepted by bwrap.
Georges Discry [Fri, 22 Sep 2023 13:35:09 +0000 (15:35 +0200)]
Directly ignore deprecated command-line options
A new argparse action is used for the deprecated command-line options.
There is no more need to post-process the parsed arguments as the action
directly takes care of displaying a deprecation warning while leaving
the namespace untouched.
A deprecated option is now obvious at its declaration (at the
`parser.add_argument()` call) and it can be dropped by simply deleting
that declaration.
Georges Discry [Fri, 22 Sep 2023 13:10:58 +0000 (15:10 +0200)]
Immediately change directory on -C/--directory
Common utilities (e.g., make and tar) that have a `-C`/`--directory`
option immediately change the working directory when that option is
parsed, with the following properties:
- the option is order sensitive and only applies to the following
options
- the option can be used multiple times and each is interpreted relative
to the previous one
In addition, mkosi uses an empty path for `-C`/`--directory` to indicate
that the configuration files in the current directory should be ignored.
At first, mkosi would call `os.chdir()` after parsing the command-line
arguments, so the following options with relative paths would resolve
relative to the initial working directory and not the new one. This
issue was reported in #1879.
The fix in #1880 reversed that approach. A first (and lighter) pass on
the command-line arguments would get the last `-C`/`--directory` and
call `os.chdir()`. Afterwards, all the command-line arguments were fully
parsed so that the relative paths would resolve to that directory.
Neither approach implements the properties above. Only the last value is
used and applies to none/all of the command-line arguments.
By calling `os.chdir()` as the options are parsed, both properties are
respected and the arguments can be parsed in a single-pass. The
resulting directory is still tracked in `args.directory` but as a `Path`
object defaulting to `Path.cwd()`, with `None` indicating that the
configuration files should be ignored.
Furthermore, it's now possible to call `mkosi -C <dir> -C ''` to work in
a given directory (for the output and workspace) while also ignoring the
configuration files present in that directory.
We want to allow all users to access /work, so let's create it with
relaxed permissions. Let's also make sure we clean up the /work
directory after bwrap exits, so that whatever we do in bwrap doesn't
affect the cleanup of /work.
Don't skip UKI generation when building a directory image
When building a cpio/UKI image, it doesn't generally make sense to
generate a UKI to put inside the cpio/UKI. This isn't entirely true
for directory images though, it could be that the directory image
is intended to be packed up as some kind of disk image later, so
let's not skip UKI generation automatically when building a directory
image.
Pass split initrd as -initrd if it exists with directory image
When we're booting a directory image, direct boot only works if the
kernel has the virtiofs driver builtin. Since this is not the case
for distribution kernels, let's make sure we pass the initrd to qemu
as well if one exists so that booting straight into a virtiofs can
also work with distro kernels provided an initrd is available.
Using virtiofsd, we can boot straight into a virtiofs instance of
a directory image. This does require the virtiofsd instance to run
as (fake) root so we can't switch to the user running mkosi anymore
when running unprivileged but that shouldn't be a problem.
This also only works with kernels that have the virtiofs driver
builtin which I don't think is the case in any major distros yet.
- ip= is also parsed by the kernel and can cause significant boot
delays. We can avoid these delays by passing ip=none as the final
value which doesn't affect network-generator but does disable the
kernel auto-probing.
- Pull in network.target instead of network-online.target. This
makes sure that systemd-networkd is started without introducing
boot delays because we're waiting for an IP address to appear.
Let's allow mounting various directories into containers/VMs that
we run using a new RuntimeTrees= option. For containers, we use
nspawn's --bind option. For VMs, we use virtiofs along with the
virtiofsd project.
When using a packaged mkosi with its resources in /usr, using a
tools tree breaks setting up autologin as the resources can't be loaded
anymore after mounting the tools tree. Let's fix the problem by inlining
the resources so they're immediately loaded. This isn't a problem
for the mkosi.md resource as it is never loaded after mounting a
tools tree.
ruff [1] is a new very fast linter that combines rules from many commonly used
linters like pyflakes, isort or pyupgrade.
This commit uses the rules from pycodestyle (enabeled by default), pyflakes,
isort and pyupgrade and reintroduces the 119 character line length limit that
was used in the past with black and flake8, but was lost when black was disabled.
Georges Discry [Fri, 15 Sep 2023 20:01:26 +0000 (22:01 +0200)]
Warn when a setting is in the wrong section
Except for the `[Match]` section, a setting can be set in any of the
valid sections because the actual section is ignored. This fuzziness
works because each setting is used in only one section so there is not
conflict.
Instead of silently parsing such a file, a warning is now displayed with
a hint on how to fix it. In the future, this could become an error.
Make sure the root directory is a regular directory if Overlay= is enabled
overlayfs combined with a btrfs subvolume as the upperdir results in
EXDEV when trying to access files from a lowerdir, so let's make sure
we use a regular directory when Overlay= is enabled.
If we replace /tmp, we'll also make any directories located in /tmp
inaccessible, e.g. the workspace directory. Work around the problem
by reusing the host's /tmp.
207af68047ce22777b06878b7efe77491a7c4108 broke backwards-compat. mkosi-initrd
used a call like this:
mkosi --default /usr/lib/mkosi-initrd/fedora.mkosi --finalize-script=/usr/lib/mkosi-initrd/mkosi.finalize --image-version=6.5.0-0.rc7.20230821gitf7757129e3de.50.fc39.x86_64 --environment=KERNEL_VERSION=6.5.0-0.rc7.20230821gitf7757129e3de.50.fc39.x86_64 -o /tmp/kernel-install.staging.JEzApD/initrd
which is now refused with:
/tmp/initrd is not a valid filename
which is not very clear, because "/tmp/initrd" is an OK filename.
Let's make the error message very clear about what is wrong.
Since we are defaulting to testing for Debian tools trees, we will always fight
testing being occasionally broken, e.g. when packages disappear because their
newer versions can't transition from unstable.
ToolsTreePackages= adds the ability to keep using a default tools tree, but add
custom packages on top.
projects / thirdparty / mkosi.git / log
