Let's add back some form of direct linux boot by allowing to boot
cpio images. To make this work, either the user has to pass -kernel
or a kernel has to be installed inside the cpio, which we'll copy
out and use as the -kernel argument.
Let's introduce an enum and a helper to refer to the compressed output
path.
We also go back to always reading the password in find_password() if
set in a mkosi.rootpw file. This was changed to only be done during
a build so that the file could be owned by root, but this doesn't apply
anymore because mkosi doesn't need to be root anymore to do a build.
The current compression level takes a really long time to complete
when the output is large. Let's use the default compression level
instead so compression is a bit faster while still giving good results.
Let's just add an argument log to run that's true by default which
allows us to disable logging if the command failed. We also disable
logging in run_ssh(), run_qemu() and run_shell() since we expect
those to fail and shouldn't log if they do (like we did before).
While this achieved the desired effect of dpkg being looked up in
the PATH, it also modified the PATH variable for everything executed
by dpkg, which we don't want. Let's just lookup dpkg in PATH before
running apt and put the result in Dir::Bin::dpkg which looks up
dpkg in the PATH without modifying the PATH used by dpkg itself.
Let's always use regular exceptions and stop outputting stack traces
on exceptions by default. To get a stacktrace, users can run mkosi with
--debug run and we will output the usual stacktrace.
This also makes all outut that was conditional on some other settings
unconditional, since it was a bit mixed what we always showed and for which we
defaulted to some fallback string anyway.
with_docs was conditioned on which distributions supported it, but distribution
support for other features has been mixed as well and accompanying settings have
been shown regardless.
There are valid scenarios where one might want to install a kernel
without wanting a bootable image, so let's add back the --bootable=
option as a feature option. It defaults to "auto" which is the current
behavior, but can also be explicitly enabled or disabled. If enabled,
we'll fail if we can't produce a bootable image. If disabled, we won't
generate a bootable image even if all the necessary components are
available.
With the new Fedora website the location of the gpg keys has changed.
Also, all the gpg keys are now in one file instead of having a separate
file for each release.
Make sure we resolve relative paths in config files correctly
Relative paths in config files should always be interpreted relative
to their config file. We could pass the directory to each parse method,
but that's rather verbose, so let's use chdir() so that relative paths
are resolved correctly instead.
Make sure the /efi mountpoint exists for every distro
https://0pointer.net/blog/linux-boot-partitions.html recommends
using the /efi mountpoint universally so let's accomodate that
and do it for every distro.
Do not fail when writing root password and /etc/shadow is missing
I had this failure when testing https://github.com/systemd/mkosi/pull/1442. But
it would happen whenever the installed system has no /etc/shadow and we want to
set the password, so it's worth fixing regardless.
Let's get rid of our dependency on debootstrap by replicating its
core functionality ourselves. To make sure the necessary tools for
maintainer scripts to run are available in the chroot, we have to
extract the essential debs manually first before installing all the
essential debs.
This also allows us to get rid of our skeleton tree hack we added for
debootstrap.
Only write repositories to image on Debian/Ubuntu if apt was installed
For other distributions, we don't install the repositories at all but
because Debian and Ubuntu do not have an easy way to install the
repositories via a package we give them some leeway and install the
repositories if apt was installed.
We also rework repository handling to accomodate this. When running
apt ourselves on the host, we now use the apt directory in the
workspace as the config directory for apt instead of using etc/apt
in the image.
Additionally, we also use the keyring from the host instead of the
one from the image when running apt.
Finally, we switch to the http mirror for debian security updates
to avoid having to install ca-certificates in the image for apt
to work properly.
This makes 'summary' behave similarly to various systemd and git commands that
start a pager whenever lengthy output is expected. The interesting part of the
summary is often near the top, paging makes it easier to immediately see the
interesting lines.
The pager is enabled by default (when on tty, etc.), but can be disabled with
--pager=no.
This copies the implementation I wrote for rpmautospec [1].
Same as in rpmautospec, we don't want to page normal output, but only stuff
like 'summary' and '--help', where we can generate the whole string upfront and
pass it to the built-in pager. The only tweak that is made is that $LESS is set
in the environment so that the output looks nice. $MKOSI_LESS can be used to
override the options if desired.
When 'timedatectl' was ran with stdin=tty, stdout=PIPE, mkosi would get stopped
by SIGTTY. I don't understand why this happens — 'timedatectl' shouldn't touch
its stdin at all. Redirecting it from /dev/null solves the issue, and seems the
right thing to do anyway.
Fedora plans to switch to dnf5 for F40. For now, the new binary is called
'dnf5', and the old 'dnf'. Let's try to call 'dnf5' if available. It is
generally faster. E.g.
'mkosi build' in the mkosi directory after all packages have been downloaded:
950 ms with dnf5
1350 ms with dnf
Dnf5 also doesn't download file metadata by default, so it'll be faster if
downloads are necessary too.
Let's try the new shiny to save some time on installations.
For some reason, dnf5 doesn't like it if command options are before the
command. Old dnf is fine with either order, so let's just move the command
earlier.
Package managers have the annoying quirk that they use the user
and group information from the host system instead of the user and
group information from the chroot. As a workaround, in
run_with_apivfs(), let's overmount the files from the host with the
ones from the root if they exist.
Let's also always install a distribution in 2 steps. First, we install
the package that provides passwd. Then, we install the rest of the
packages, during which passwd and related files will be overmounted.
Let's move most of the logic related to installing packages into
install_packages() for each distribution, and only keep the first
time installation stuff in install(). This allows us to move most
of the base_image checks out of the distribution specific files into
a single check in install_distribution() where if a base image is
provided, we call install_packages(). Otherwise, we call install().
Drop centos logic to rebuild database to bdb format
This image might just be a base image that's intended to be added
onto with sysexts and such. If so, we need to keep the database in
sqlite format so that it can still be written to by rpm on the host.
So let's leave the rebuilding to be done in a postinst script by the
user.
Let's not muck around with distro defaults after all. This is trivial
to fix in a postinst script and should be done there instead of by
default in mkosi.
We install minbase with debootstrap which should already be very
minimal, so if there's docs in there let's assume they're critical.
We already specify --no-install-recommends for apt which should
exclude docs packages as they're optional so there's no need for
dpkg specific overrides to exclude docs on top of that.
Instead of disabling all services on Debian, let's adopt the new
preset directive "ignore" which tells preset to ignore any matching
services (neither enable nor disable). This makes sure we don't
override any defaults set by individual Debian services. On older
systemd, preset will simply ignore any line it can't parse so this
doesn't break those systems but will only result in a harmless
warning message.
bwrap seems to run into permission errors with relative paths in
some cases so let's follow systemd best practices and convert all
config paths to absolute paths as we're parsing them.
Instead of doing everything via zypper, let's just write the repo
definition files ourselves directly. Also, to mimick what we do
with dnf, let's write the repo files outside of the root instead of
inside the root and leave installing repos inside the root up to
the user.
Instead, we just do the necessary setup for a bootable image if all
the required components are available in the image. In practice, this
means that if a user installs systemd-boot, the kernel, and dracut,
they'll get a bootable image.
Let's trim down on the default packages we install, and only install
the most basic filesystem package by default. We'll leave everything
else up to the user.
This change is important when considering mkosi's use to build system
extensions and similar, where even the default packages we have now
are too much.
rpm blocks most signals when doing a transaction. dnf doesn't block
anything. The end result is that when we do ctrl+c when dnf is running
the rpm transaction, dnf gets interrupted and exits, which means mkosi
exits and tries to remove the workspace directory on which rpm is still
operating, causing all kinds of nastiness.
Because we don't care about transaction safety since we're operating on
a chroot, let's add --die-with-parent when running bubblewrap. This makes
bubblewrap ensure cleanup of all child processes underneath it, and
because rpm can't ignore SIGKILL, it also cleans up rpm properly, fixing
the issue.
It's also useful to be able to match against stuff that isn't the
current configuration. Let's add a PathExists= match that is satisfied
when the given path exists.
This will be useful in systemd where we conditionally build a kernel if
mkosi.kernel/ exists in the top level repo directory. With this setting,
we can only install the necessary packages to build the kernel if the
mkosi.kernel/ path actually exists.
Rework configuration parsing ordering and overrides
Currently, aside from list based settings, if a setting is used
multiple times across different configuration files, the later
assignments override earlier assignments. On top of that, the CLI
arguments are parsed last and override everything from config files.
This has worked very well until now, but breaks now that we have
[Match] sections. If we override a setting using the CLI, we want
any configured [Match] sections to compare against the value specified
via the CLI. Because the CLI values are applied last, this currently
isn't the case.
Similarly, if we add 90-local.conf to override the distribution release
used for Debian, all the config files that are processed earlier will
still compare against the default release configured for Debian in
50-debian.conf. If we rename 90-local.conf to 00-local.conf, the default
release in 50-debian.conf will still override that value. Only by putting
the override after the config file that assigns the default release but
before the first config file that matches on that release can we make this
work, but depending on the configuration this might require a lot of
different override files which isn't ideal.
Also, because later values can override earlier ones, it's possible to have
[Match] sections that match against different values of the same setting
both apply, if the setting happens to be reassigned inbetween, which is not
intuitive and error prone. Ideally, [Match] settings only apply to the final
value of a setting.
To fix these problems, this commit reworks the way we order and override
settings. Instead of having later assignments override earlier ones, for
all settings aside from list settings, the first assignment is the one that
is used. All later assignments of the same setting are ignored. Along with, we
switch to parsing CLI arguments first, which means that any settings assigned
in CLI args take precedence over those configured in config files.
Because the first value is immediately the final value, any [Match] sections
will only ever see the final value. One caveat is default values for settings.
We only want to assign these at the end of parsing, if no value has been
explicitly configured, but we also want [Match] sections to apply to default
values if no value is explicitly configured. The solution to this is that if
we encounter a setting in a [Match] section and it has not been explicitly
assigned a value yet, it is assigned its default value.
For list based settings, ! now configures an ignore glob, which means that if any
later assignments try to assign values that match an ignore glob, those values
are ignored. We also prepend list values instead of appending so that list
values that are configured in a preceeding config file appear later in the final
list value than values configured in a later assignment.
Implementation wise, this commit reworks config parser functions to return the
new value that should be assigned instead of assigning it themselves. This makes
the config parsing functions slightly more generic.
Instead of messing with the internals of argparse, let's implement
proper configuration file parsing.
- Parsing classes and functions are located in a new file config.py
- Configuration settings are split up from CLI settings in a list of
the new ConfigSetting dataclass
- The CLI options for configuration settings now share the same
argparse setting which simply delegates parsing to the corresponding
configuration setting parser.
- We add support for [Match] sections to enable conditionally including
configuration files. Currently Match support is implemented for
Distribution= and Release=.
- Configuration file searching is reworked. In mkosi.conf.d/, we parse
all files ending with ".conf" and directories. If a directory is parsed,
we parse mkosi.conf, mkosi.conf.d/ and all mkosi specific paths in it.
All paths are interpreted relative to the directory that we're parsing.
projects / thirdparty / mkosi.git / log
