Daan De Meyer [Wed, 4 Oct 2023 11:30:19 +0000 (13:30 +0200)]
Parse setting paths before parsing main config file
Currently, paths either configure default values or append to a
list (When path_default is False, it's always a list based setting).
When paths are configuring default values, it makes more sense for
default values set in the mkosi.conf file to override path based
default values.
When appending to a list, (e.g. ExtraTrees=), it makes more sense
for the trees configured in the mkosi.conf to come after the tree
from the path (mkosi.extra).
Both these goals are achieved by parsing the path based values before
parsing the main mkosi.conf file.
Daan De Meyer [Wed, 4 Oct 2023 08:46:39 +0000 (10:46 +0200)]
Stop explicitly setting distribution in tests
We now default to "custom" when we can't figure out the host
distribution instead of failing so let's stop explicitly setting
the distribution in tests.
Joerg Behrmann [Wed, 4 Oct 2023 07:52:24 +0000 (09:52 +0200)]
config: factor out settings_lookup_by* and match_lookup from parse_config
All three (settings_lookup_by_name, settings_lookup_by_dest, and match_lookup)
only repackage global variables. Moving them outside makes them usable in other
places and only calculates them once instead of on every invocation of
parse_config.
json: add alternative default constructors to MkosiArgs and MkosiConfig
These can be used to get a default MkosiArgs/MkosiConfig just have if mkosi had
been called in an empty directory with no cmdline without adding defaults to
the attributes and allowing instances of MkosiArgs/MkosiConfig to be made with
missing keys.
- reformat overly long lines
- remove typing.Type in favour of type
- import Iterablefrom collections.abc instead of typing
- compare singletons with is
- don't use "ambiguous variable name: l"
Daan De Meyer [Tue, 3 Oct 2023 08:08:41 +0000 (10:08 +0200)]
Always set the ttyS0 credentials
Even when booting with GUI, we might still have a serial terminal,
so let's always set the serial terminal specific size credentials
as these don't affect the GUI terminals at all.
Daan De Meyer [Mon, 2 Oct 2023 08:24:19 +0000 (10:24 +0200)]
Only add console=ttyS0 and tty sizes when QemuGui= is disabled
With console=ttyS0, we don't get any boot logs when running qemu
in GUI mode. The tty sizes don't matter either when booting in GUI
mode as Linux can figure it out itself in that scenario.
Daan De Meyer [Sun, 1 Oct 2023 18:20:12 +0000 (20:20 +0200)]
Add RuntimeSize= setting
Currently we unconditionally grow disk images to 8G before booting
them in systemd-nspawn or qemu. Let's do better here by making the
size configurable and not growing the disk images by default.
We also move format_bytes() to config.py as most other formatting
functions are located there.
Instead, let's show individual tools tree packages in the summary.
This makes things a bit more consistent as we don't show default
initrd presets in the summary either (and making that happen is no
trivial task so we opt to do the reverse and not show default tools
trees either).
We also add a table to the documentation showing which packages are
in the default tools tree for which distributions.
Daan De Meyer [Sun, 1 Oct 2023 08:26:12 +0000 (10:26 +0200)]
Don't reuse unprivileged cache when building as root
When using --incremental and first building an image unprivileged,
followed by building it as root, we can't reuse the cache as the
ownership will be wrong. So let's make sure we don't reuse the
cache when that's the case.
This makes it easier to print a message from the installer referring
to the distro by the proper name.
I looked at the home pages of the distros and took the spelling used
there. In particular Fedora is making a concious effort to use "Fedora"
for the whole project, and "Fedora Linux" for the distro. OTOH, Ubuntu
is known for not using "linux" anywhere where users could see it.
distributions: use a generator to reduce list boilerplate
When the list of repos to return is conditionalized, we can use yield instead
of concatenating lists. Iterable[Repo] can be satisfied by both the functions
which are generators and the functions which just return a list as before.
It is an optional component. The user would be unhappy when we fail at the end
of a lengthy installation process. So let's check early if we would need it and
refuse to continue.
(When config.bootable == ConfigFeature.auto, we cannot to do the check before
the installation is performed, so let's opt to be strict and do the check.)
Inspired by https://bugzilla.redhat.com/show_bug.cgi?id=2240598.
distributions: rename all installers to "Installer"
We were doing "double namespacing": "centos.CentosInstaller",
"ubuntu.UbuntuInstaller", etc. Let's simplify things by dropping the second
name. This avoids the awkward issue how capitalize the names of distributions
that already have capitals in their name, e.g. CentoOS, or dashes in the name,
e.g. RHEL-UBI. The one call site where we import the right installer becomes
simpler.
In Distribution.installer(), try..except is replaced by assert. If the
installer class for a distro is missing, then it's a programming error and not
not something we should suppress.
Make sure we append the setpgid command after all the options. Let's
also use -- to avoid any potential confusion about whether the following
arguments are options or not.
Georges Discry [Fri, 15 Sep 2023 20:19:41 +0000 (22:19 +0200)]
Convert script settings to lists
With `mkosi.conf.d`, `mkosi.presets` and/or `[Match]`, it's become
really easy to split the configuration of an image in logical parts. It
might be done to isolate some settings that go together or to share the
same configuration across several images.
However, for each `mkosi.prepare`, `mkosi.build`, `mkosi.postinst` and
`mkosi.finalize` step, only one script could be executed as the one with
highest priority overrides the others.
Each script is now managed as a list of executables so that it's also
possible to split them if desired.
Backward compatibility is kind of maintained by keeping the same options
on the command line and by introducing aliases for the renamed INI
settings (with a deprecation warning). These aliases also parse the
settings as a list, so compatibility is still broken when a script was
expected to override a previously defined one, as they are now both
executed.
Georges Discry [Tue, 19 Sep 2023 11:44:18 +0000 (13:44 +0200)]
Backward-compatibility for renamed INI settings
A setting can now declare aliases used when parsing INI files. These
aliases are useful when a setting is renamed as they maintain
backward-compatibility while emitting a deprecation warning.
Even if we don't have a bootloader or aren't configured to be
bootable, make sure we copy a kernel out of the image so we can
linux direct boot images in qemu.
It turns out this can be useful after all, for example to specify
whether to build in debug mode or release mode, or which config to
use when building kernels, so let's pass arguments to the build
script again.
Mount over /etc/passwd when building an image and running build script
Various tools parse /etc/passwd, so let's make sure it contains the
expected lines when building an image. Also, when running the build
script, mount over /etc/passwd in the image as various build tools
will also query the current user in /etc/passwd (e.g. the kernel's
build script runs "whoami"). Also add a root entry to the /etc/passwd
we generate so both the user running mkosi and root are covered.
Currently we only clean up /work if the build script succeeds.
However, we have to always clean up /work since we don't make a
copy of the build overlay so let's make sure we do that by using
a trap instead of only cleaning up if a script succeeds.
Let's also make sure that we don't fail if /work already exists and
make sure that if gets the right permissions, even if it already
exists.
The next release of util-linux will include setpgid which has a
--foreground option that does exactly the same as our foreground()
function. This new utility allows us to make the processes executed
by bubblewrap the foreground process, which makes sure that SIGINT
isn't intercepted by bwrap.
Georges Discry [Fri, 22 Sep 2023 13:35:09 +0000 (15:35 +0200)]
Directly ignore deprecated command-line options
A new argparse action is used for the deprecated command-line options.
There is no more need to post-process the parsed arguments as the action
directly takes care of displaying a deprecation warning while leaving
the namespace untouched.
A deprecated option is now obvious at its declaration (at the
`parser.add_argument()` call) and it can be dropped by simply deleting
that declaration.
Georges Discry [Fri, 22 Sep 2023 13:10:58 +0000 (15:10 +0200)]
Immediately change directory on -C/--directory
Common utilities (e.g., make and tar) that have a `-C`/`--directory`
option immediately change the working directory when that option is
parsed, with the following properties:
- the option is order sensitive and only applies to the following
options
- the option can be used multiple times and each is interpreted relative
to the previous one
In addition, mkosi uses an empty path for `-C`/`--directory` to indicate
that the configuration files in the current directory should be ignored.
At first, mkosi would call `os.chdir()` after parsing the command-line
arguments, so the following options with relative paths would resolve
relative to the initial working directory and not the new one. This
issue was reported in #1879.
The fix in #1880 reversed that approach. A first (and lighter) pass on
the command-line arguments would get the last `-C`/`--directory` and
call `os.chdir()`. Afterwards, all the command-line arguments were fully
parsed so that the relative paths would resolve to that directory.
Neither approach implements the properties above. Only the last value is
used and applies to none/all of the command-line arguments.
By calling `os.chdir()` as the options are parsed, both properties are
respected and the arguments can be parsed in a single-pass. The
resulting directory is still tracked in `args.directory` but as a `Path`
object defaulting to `Path.cwd()`, with `None` indicating that the
configuration files should be ignored.
Furthermore, it's now possible to call `mkosi -C <dir> -C ''` to work in
a given directory (for the output and workspace) while also ignoring the
configuration files present in that directory.
We want to allow all users to access /work, so let's create it with
relaxed permissions. Let's also make sure we clean up the /work
directory after bwrap exits, so that whatever we do in bwrap doesn't
affect the cleanup of /work.
Don't skip UKI generation when building a directory image
When building a cpio/UKI image, it doesn't generally make sense to
generate a UKI to put inside the cpio/UKI. This isn't entirely true
for directory images though, it could be that the directory image
is intended to be packed up as some kind of disk image later, so
let's not skip UKI generation automatically when building a directory
image.
Pass split initrd as -initrd if it exists with directory image
When we're booting a directory image, direct boot only works if the
kernel has the virtiofs driver builtin. Since this is not the case
for distribution kernels, let's make sure we pass the initrd to qemu
as well if one exists so that booting straight into a virtiofs can
also work with distro kernels provided an initrd is available.
Using virtiofsd, we can boot straight into a virtiofs instance of
a directory image. This does require the virtiofsd instance to run
as (fake) root so we can't switch to the user running mkosi anymore
when running unprivileged but that shouldn't be a problem.
This also only works with kernels that have the virtiofs driver
builtin which I don't think is the case in any major distros yet.
- ip= is also parsed by the kernel and can cause significant boot
delays. We can avoid these delays by passing ip=none as the final
value which doesn't affect network-generator but does disable the
kernel auto-probing.
- Pull in network.target instead of network-online.target. This
makes sure that systemd-networkd is started without introducing
boot delays because we're waiting for an IP address to appear.
Let's allow mounting various directories into containers/VMs that
we run using a new RuntimeTrees= option. For containers, we use
nspawn's --bind option. For VMs, we use virtiofs along with the
virtiofsd project.
projects / thirdparty / mkosi.git / log
