This is like --with-network but also tells "dnf" not to use networking.
This adds a minimal level of reproducability: if the package cache in
mkosi.cache/ is populated once, every rebuild should produce very
similar images.
mkosi: incremental builds are not supported for squashfs or minimized file systems
This was never supported for squashfs, but failed badly. Let's generate
a nice error instead.
(The thing with these builds are that while we build these images we
don't mount the final file systems inside the image file. Instead we
mount a temporary directory in /var/tmp that is packed up only much
later. This makes caching harder, since it's not just one dir or file we
need to cache but the image *and* this directory. We can add support for
this eventually, but for now let's just friendly decline operation. It's
not too limiting, as in particular minimization is mostly an
optimization that is only useful on final builds, not on test builds so
much, while incremental mode is useful for test builds but not so much
for final builds.)
Let's add some reproducability. The inode size has effects on the mtime
granularity, fscrypt support and more. mkfs.ext4's internal heuristic
for determining the inode size hence has some weird effects where mtime
problems suddenly arise when you build smaller images. Let's fix that,
and stick to a fixed size, most powerful inode size.
Clayton Craft [Sat, 9 Nov 2019 19:50:12 +0000 (11:50 -0800)]
Use console=pipe in install_grub() when calling nspawn
The mount bind to /dev on the host conflicts with this patch in systemd,
which doesn't expect /dev/console to exist:
https://github.com/systemd/systemd/commit/3acc84ebd9aebe8cf1771b42644ebbfbecdfaa37
The result is that nspawn dies because it is unable to create a symlink
at /dev/console. This adds a new parameter when calling nspawn to
prevent it from attempting to create/use /dev/console since the default
behavior is to create/use /dev/console and install_grub() doesn't seem to
require an interactive console in order to install grub.
Joerg Behrmann [Fri, 8 Nov 2019 09:41:02 +0000 (10:41 +0100)]
arch: use base package instead of base group
Arch has changed its base package group to a meta package that pulls in several
other packages as dependencies. It is slimmer than the package group was before
and necessary to request support, so we can (and should) install it in full.
James Shubin [Wed, 18 Sep 2019 07:17:04 +0000 (03:17 -0400)]
Optionally include the .git directory with the files it contains
When copying files in from git, using copy-git-cached ignores the files
in the .git/ directory. This introduces a copy-git-more option that also
pulls those files in. This is useful for builds that use `git describe`
or another git command for their version number or tests. With this
feature they can now query it.
James Shubin [Wed, 18 Sep 2019 07:09:31 +0000 (03:09 -0400)]
Add flag to skip (second) final image build phase
This adds a new flag to cause the (second) final stage to be skipped.
This is useful for when you're more interested in the build artifacts
being built in the correct environment, and less interested in the final
image.
Jörg Behrmann [Fri, 11 Oct 2019 09:36:20 +0000 (11:36 +0200)]
Fix type error on args variable (#368)
The CommandLineArguments are in a instead of args, so the hasattr check will
always fail. Skip using hasattr since a will have set directory from parse_args
James Shubin [Fri, 20 Sep 2019 19:38:04 +0000 (15:38 -0400)]
Add env to build that specifies which "default" we're using
It's not possible to determine which "mkosi.default" we're using, when
using a single "mkosi.build" script with `mkosi --default <file> build`
so this adds a variable that allows the build script to know which one
is being used.
Lucas De Marchi [Fri, 24 May 2019 06:58:07 +0000 (23:58 -0700)]
Fix pytest
It was failing due to not being able to access the cache directories
created by mkosi. Since we are running mkosi as root, let's do the same
for pytest.
Something went wrong on my machine and /dev/loop8p2 was not present,
even though loop8 and loop8p1 were. (I think the loopback device was
mounted somewhere and the kernel wouldn't reread the partition table.)
Since we are running as root, we can easily create a new file in /dev.
Let's avoid this.
Lénaïc Huard [Mon, 18 Mar 2019 22:02:16 +0000 (23:02 +0100)]
Disable COW (for btrfs) on qcow2 files
According to qemu-img(1):
Btrfs has low performance when hosting a VM image file,
even more when the guest on the VM also using btrfs as file
system.
Turning off COW is a way to mitigate this bad performance.
This is the reason of the disable_cow() function, but this function
applies only to the raw file. So, let’s disable COW also on the qcow2
one.
Adrian Freihofer [Sun, 24 Feb 2019 15:49:24 +0000 (16:49 +0100)]
parse_args: major refactoring and improvements
The man page describes the powerful configuration approach
based on a mkosi.default file and several file in a
mkosi.default.d folder as well as command line arguments.
This does not work for many parameters. The problem is that
argparse is called before config files are loaded.
For some values the implementation basically looks like:
a = False # argparse lines
...
if a is None: # config file loading, lower priority
a = True
This new implementation delegates loading of mkosi.default files
to python's argparse.ArgumentParser. According to docummentation
ArgumentParser supports loading of settings from files by
overloading the convert_arg_line_to_args function. The function
gets one line from the config file and returns the corresponding
command line counterpart. For example WithTest=yes found in a
defaults file would be converted to ['--with-tests', 'yes'].
Unfortunately this line by line conversion is not really compatible
with python's configparser module which works file wise. To stay
with ArgumentParser and ConfigParser the new ArgumentParserMkosi
overloads the undocumented _read_args_from_files function of the
ArgumentParser. This function is called for each @ prefixed
command line argument.
The new implementation supports unlimited number of mkosi.default
files. The files are loaded according their alphabetical priority.
Command line arguments are processed with highest priority.
The second problem solved by this patch is data type conversion
for settings loaded from mkosi.default files. Since ArgumentParser
handles all settings, data type conversion is always done by
ArgumentParser.
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
Adrian Freihofer [Sun, 13 Jan 2019 11:55:21 +0000 (12:55 +0100)]
parse_args: Improve handling of boolean arguments
First step to handle boolean configuration parameters from
mkosi.default files as well as from command line arguments. The
implementation supports multiple definitions of one argument. The
last value gets priority. Examples:
--foo=yes --foo=no --> args.foo=False
--foo --> args.foo=True
--without-foo --> args.foo=False
--foo --without-foo --> args.foo=False
Purpose:
Let argparse import mkosi.default files in a future commit.
Make parse_boolean function case-insensitive.
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
Adrian Freihofer [Thu, 24 Jan 2019 23:26:16 +0000 (00:26 +0100)]
parse_args: Support removing list entries with ! prefix
Entries added handled by the ListAction class may be prefixed with
! to remove an entry if it is already in the list.
!* removes all entries from the list.
Examples:
- mkosi -p httpd -p !httpd
has no effect
- mkosi -p vi -p emacs -p nano -p "!*,httpd"
Only the httpd package will be added to the image.
Note: This function becomes even more useful if the configuration
files are also processed by argparse. This will be the case
with a subsequent commit.
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
The package was renamed some time ago (around March 2016)[1]. The
package btrfs-progs is available from Stretch[2] but it's also in Jessie
backports[3]; in Ubuntu it was introduced in Bionic[4], so it's probably
more appropriate to use the new name.
Adrian Freihofer [Sun, 13 Jan 2019 15:35:31 +0000 (16:35 +0100)]
Do not override run_build_script function
Does not change anything. It's a renaming of a function parameter
from run_build_script to do_run_build_script to resolve the name
clashing with function run_build_script.
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
Lucas De Marchi [Tue, 19 Feb 2019 21:54:56 +0000 (13:54 -0800)]
arch: kill dirmngr only
Using fuser to identify processes using a path doesn't work as I wanted:
it relies on the filesystem mounted there. The outcome is that when
using "-t directory" it will try to kill all processes using that
filesystem rather than processes using files under that path.
Even by bind-mounting root so it's always a mountpoint, it doesn't work.
So instead of killing everything, pinpoint what process pacman/pacstrap
left running and kill only that one.
Lucas De Marchi [Tue, 19 Feb 2019 21:53:06 +0000 (13:53 -0800)]
arch: trust new signatures while downloading base packages
Otherwise it aborts with:
error: libxml2: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
:: File /var/tmp/mkosi-__fdh4oy/root/var/cache/pacman/pkg/libxml2-2.9.9-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]