Daan De Meyer [Sun, 6 Aug 2023 08:55:25 +0000 (10:55 +0200)]
Documentation and summary improvements
- Reorder config items in the summary to put more related stuff together
- Add missing config items to the summary
- Move Ssh= option to Content as it modifies the image
- Move CacheOnly= option to Distribution
- Document Passphrase=
- Reorder documentation to match summary
Daan De Meyer [Sat, 5 Aug 2023 15:14:08 +0000 (17:14 +0200)]
Use umask to control new file/directory permissions
The primary reason to use umask is that python's mkdir() functions
and methods only apply the given mode to the final component of the
path, and not to its parent paths if parents=True is specified. Aside
from that, it's also just nicer to make sure the file/directory has
the right mode from the start instead of having to modify it later with
chmod().
We also clean up permissions in general, making sure we set umask
explicitly whenever we create a file or directory in state.root and
remove explicit permissions when we're not writing files in state.root.
Daan De Meyer [Sat, 5 Aug 2023 16:55:10 +0000 (18:55 +0200)]
Get rid of scandir_recursive() and find_files()
- Use rglob() from pathlib instead of scandir_recursive()
- Use absolute paths when dealing with kernel modules instead of
relative paths. Only make the paths relative in make_cpio()
Daan De Meyer [Sat, 5 Aug 2023 14:12:43 +0000 (16:12 +0200)]
Rework tar archive handling
- Instead of relying on shutil.unpack_archive(), let's always use tar
- Introduce archive_tree() and extract_tree() to abstract tar archives
- Make sure tar always uses the user/group information from the root dir
- Enable all features
- Make sure tar doesn't overwrite directory permissions
Daan De Meyer [Sat, 5 Aug 2023 13:49:54 +0000 (15:49 +0200)]
Remove TarStripSelinuxContext=
We now generate images with correct selinux labels. If selinux labels
are not needed, they can simply be excluded when extracting the archive,
but let's always include them when generating the archive and then users
can choose what to do with the labels when extracting.
Daan De Meyer [Fri, 4 Aug 2023 11:27:25 +0000 (13:27 +0200)]
Install boot loader after running postinst script
The boot loader package might be installed by the postinst script so let's
account for that and only do the boot loader stuff after running the
postinst script.
Daan De Meyer [Fri, 4 Aug 2023 10:26:30 +0000 (12:26 +0200)]
Add some extra environment variables for scripts
When running a command with mkosi-chroot, one often wants to refer
to paths under $SRCDIR, $OUTPUTDIR, ... inside the chroot. The issue
is that variable expansion happens before mkosi-chroot is called and
has a chance to modify them to point to the locations inside the chroot.
To allow easily referring to the locations inside the chroot, let's
add variants of the directory environment variables prefixed with
"CHROOT_" containing the location of the paths inside the chroot.
Daan De Meyer [Thu, 3 Aug 2023 15:50:23 +0000 (17:50 +0200)]
Make sure package managers are always configured
Currently, when using cached images, setup_dnf() and friends won't
be called when a cached image is used. Yet it's now entirely possible
that dnf might be called to install extra packages in the postinstall
script, so we always need the package managers to be configured
properly. Let's make sure we do that by introducing a new setup()
method that is always called.
Scripts currently run chrooted in the image. This is not great because
it means the tool you want to run from a script has to be installed in
the image, even if it has --root support to run from outside the image.
Specifically, to install extra packages from a script, you currently
have to install the package manager inside the image itself. Even then,
it might be completely different than the package manager on the host
(both in version and in build options), possibly leading to all kinds
of weird issues.
To allow users to install extra packages from scripts, we now default
to running scripts on the host. Additionally, to make life easy for
users, we provide a set of scripts in the PATH for the package managers
we support that call them with the necessary options to install packages
in the same way mkosi installs packages into the root directory. This
means all that users have to do is "dnf install xxx" from the script to
install a new package into the root directory.
Because some tools do not have a --root option and to provide an easy
migration for users that depend on scripts running in the image, we
also put a script "mkosi-chroot" in the PATH which uses bubblewrap to chroot
into the image as we did before. Users can keep their old scripts working
by simply adding the following to the top of their script:
```
if [ "$container" != "mkosi" ]; then
exec mkosi-chroot $SCRIPT "$@"
fi
```
When running scripts on the host, no APIVFS directories are mounted into
the image. When using the "mkosi-chroot" and package manager scripts, the APIVFS
directories are automatically mounted before executing the corresponding
command. The apivfs_cmd() function is introduced to make implementing this
easier.
Additionally, we now always consider the current working directory a
BuildSources= directory. BuildSources= is now used to declare additional
source directories. The current working directory can always be overmounted
with another directory by simply specifying a source directory in BuildSources=
without a target directory.
To allow scripts running on the host to find the image, we set the
BUILDROOT variable for all scripts.
To prevent scripts on the host from messing with the host system when mkosi
is running as root, we extend the sandboxing to cover many more directories
which are all mounted read-only while a script is executing.
This change also allows scripts to be written in python or other scripting
languages without having to install python into the image itself.
Daan De Meyer [Wed, 2 Aug 2023 11:20:38 +0000 (13:20 +0200)]
Make sure outputs are always prefixed with the output name
Our output unlinking logic currently works based off the name of
the output. Everything that is prefixed with the output name is
removed. We need to fix that properly, but for now, let's make sure
that all outputs that we generate are prefixed with the name of the
output.
Let's make the following changes
- Instead of having an instance of the installer object stored in
MkosiState, let's have an installer() method on the Distribution
enum that returns the specific type implementing the distribution
- Let's move Distribution to mkosi.distributions
- Let's add a method to DistributionInstaller to return the package
type instead of storing it directly in the Distribution enum
- Let's add methods to the Distribution enum that delegate to its
installer type
- Use enum.auto() instead of specifying values directly.
Let's leave it to bwrap() to set up any sandboxing that we need.
Let's also add a bit more sandboxing to bwrap(), to avoid details
from the host accidentally leaking into the image builds.
This was removed when we had a brief experiment with running everything
with bwrap(), but since we don't do that anymore, let's add this back
to make chroot detection work again.
I initially added this because no shell is pulled into the initrd
by default on gentoo when just installing systemd and util-linux.
Since this seems to override other distro's default shell, let's
limit this to just gentoo and rely on other distros pulling in a
shell via dependencies.
- Drop --update, we never want to update packages, only install new
ones
- Drop --changed-use and --new-use, these apply to updating an existing
system, we're always installing a new one
- Drop --deep and --complete-graph-if-new-use, these don't seem to
have any noticeable effect
- Move getbinpkg from FEATURE to emerge CLI option --getbinpkg
- Do not disable spinner and candy option
- Explicitly disable all sandboxing features
This function is intended to be a more efficient way of moving trees
than shutil.move(). Specifically, it will use reflinks and btrfs
snapshots if possible to make copying more efficient.
Rename btrfs.py to tree.py and move copy_path() into it
Let's hide btrfs subvolumes as an implementation detail in a new
tree module. We have two functions, make_tree() and copy_tree()
(renamed from copy_path()), which will both take advantage of btrfs
subvolumes if configured to do so.
We also extend copy_tree() to not try to snapshot files.
gentoo: Set FEATURES by appending to /etc/portage/make.conf
Setting via the environment variable doesn't work in all cases, so
let's append to /etc/portage/make.conf instead. This allows us to
get rid of the custom devtpms for portage as it doesn't try to chown
tty's anymore now that userpriv is actually disabled.
Drop filesystem_options() and make xfs default for centos again
Requires https://github.com/systemd/systemd/pull/26541 but after that
building XFS root filesystems will work reliably again and we can use
xfs as the default filesystem for centos which removes the need for
filesystem_options().
gentoo: Use bwrap() instead of run_workspace_command() to run emerge
Similar to the other distros, run the package manager with bwrap()
instead of run_workspace_command(). Also disable the sandboxing of
emerge as we do it in mkosi already.
projects / thirdparty / mkosi.git / log
