docs/BUILDING_IMAGES.md

   1 ---
   2 title: Safely Building Images
   3 category: Concepts
   4 layout: default
   5 SPDX-License-Identifier: LGPL-2.1-or-later
   6 ---
   7
   8 # Safely Building Images
   9
  10 In many scenarios OS installations are shipped as pre-built images, that
  11 require no further installation process beyond simple `dd`-ing the image to
  12 disk and booting it up. When building such "golden" OS images for
  13 `systemd`-based OSes a few points should be taken into account.
  14
  15 Most of the points described here are implemented by the
  16 [`mkosi`](https://github.com/systemd/mkosi) OS image builder developed and
  17 maintained by the systemd project. If you are using or working on another image
  18 builder it's recommended to keep the following concepts and recommendations in
  19 mind.
  20
  21 ## Resources to Reset
  22
  23 Typically the same OS image shall be deployable in multiple instances, and each
  24 instance should automatically acquire its own identifying credentials on first
  25 boot. For that it's essential to:
  26
  27 1. Remove the
  28    [`/etc/machine-id`](https://www.freedesktop.org/software/systemd/man/machine-id.html)
  29    file or write the string `uninitialized\n` into it. This file is supposed to
  30    carry a 128bit identifier unique to the system. Only when it is reset it
  31    will be auto-generated on first boot and thus be truly unique. If this file
  32    is not reset, and carries a valid ID every instance of the system will come
  33    up with the same ID and that will likely lead to problems sooner or later,
  34    as many network-visible identifiers are commonly derived from the machine
  35    ID, for example, IPv6 addresses or transient MAC addresses.
  36
  37 2. Remove the `/var/lib/systemd/random-seed` file (see
  38    [`systemd-random-seed(8)`](https://www.freedesktop.org/software/systemd/man/systemd-random-seed.service.html)),
  39    which is used to seed the kernel's random pool on boot. If this file is
  40    shipped pre-initialized, every instance will seed its random pool with the
  41    same random data that is included in the image, and thus possibly generate
  42    random data that is more similar to other instances booted off the same
  43    image than advisable.
  44
  45 3. Remove the `/loader/random-seed` file (see
  46    [`systemd-boot(7)`](https://www.freedesktop.org/software/systemd/man/systemd-boot.html))
  47    from the UEFI System Partition (ESP), in case the `systemd-boot` boot loader
  48    is used in the image.
  49
  50 4. It might also make sense to remove
  51    [`/etc/hostname`](https://www.freedesktop.org/software/systemd/man/hostname.html)
  52    and
  53    [`/etc/machine-info`](https://www.freedesktop.org/software/systemd/man/machine-info.html)
  54    which carry additional identifying information about the OS image.
  55
  56 5. Remove `/var/lib/systemd/credential.secret` which is used for protecting
  57    service credentials, see
  58    [`systemd.exec(5)`](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Credentials)
  59    and
  60    [`systemd-creds(1)`](https://www.freedesktop.org/software/systemd/man/systemd-creds.html)
  61    for details. Note that by removing this file access to previously encrypted
  62    credentials from this image is lost. The file is automatically generated if
  63    a new credential is encrypted and the file does not exist yet.
  64
  65 ## Boot Menu Entry Identifiers
  66
  67 The
  68 [`kernel-install(8)`](https://www.freedesktop.org/software/systemd/man/kernel-install.html)
  69 logic used to generate
  70 [Boot Loader Specification Type #1](https://uapi-group.org/specifications/specs/boot_loader_specification/#type-1-boot-loader-specification-entries)
  71 entries by default uses the machine ID as stored in `/etc/machine-id` for
  72 naming boot menu entries and the directories in the ESP to place kernel images
  73 in. This is done in order to allow multiple installations of the same OS on the
  74 same system without conflicts. However, this is problematic if the machine ID
  75 shall be generated automatically on first boot: if the ID is not known before
  76 the first boot it cannot be used to name the most basic resources required for
  77 the boot process to complete.
  78
  79 Thus, for images that shall acquire their identity on first boot only, it is
  80 required to use a different identifier for naming boot menu entries. To allow
  81 this the `kernel-install` logic knows the generalized *entry* *token* concept,
  82 which can be a freely chosen string to use for identifying the boot menu
  83 resources of the OS. If not configured explicitly it defaults to the machine
  84 ID. The file `/etc/kernel/entry-token` may be used to configure this string
  85 explicitly. Thus, golden image builders should write a suitable identifier into
  86 this file, for example, the `IMAGE_ID=` or `ID=` field from
  87 [`/etc/os-release`](https://www.freedesktop.org/software/systemd/man/os-release.html)
  88 (also see below). It is recommended to do this before the `kernel-install`
  89 functionality is invoked (i.e. before the package manager is used to install
  90 packages into the OS tree being prepared), so that the selected string is
  91 automatically used for all entries to be generated.
  92
  93 ## Booting with Empty `/var/` and/or Empty Root File System
  94
  95 `systemd` is designed to be able to come up safely and robustly if the `/var/`
  96 file system or even the entire root file system (with exception of `/usr/`,
  97 i.e. the vendor OS resources) is empty (i.e. "unpopulated"). With this in mind
  98 it's relatively easy to build images that only ship a `/usr/` tree, and
  99 otherwise carry no other data, populating the rest of the directory hierarchy
 100 on first boot as needed.
 101
 102 Specifically, the following mechanisms are in place:
 103
 104 1. The `switch-root` logic in systemd, that is used to switch from the initrd
 105    phase to the host will create the basic OS hierarchy skeleton if missing. It
 106    will create a couple of directories strictly necessary to boot up
 107    successfully, plus essential symlinks (such as those necessary for the
 108    dynamic loader `ld.so` to function).
 109
 110 2. PID 1 will initialize `/etc/machine-id` automatically if not initialized yet
 111    (see above).
 112
 113 3. The
 114    [`nss-systemd(8)`](https://www.freedesktop.org/software/systemd/man/nss-systemd.html)
 115    glibc NSS module ensures the `root` and `nobody` users and groups remain
 116    resolvable, even without `/etc/passwd` and `/etc/group` around.
 117
 118 4. The
 119    [`systemd-sysusers(8)`](https://www.freedesktop.org/software/systemd/man/systemd-sysusers.service.html)
 120    component will automatically populate `/etc/passwd` and `/etc/group` on
 121    first boot with further necessary system users.
 122
 123 5. The
 124    [`systemd-tmpfiles(8)`](https://www.freedesktop.org/software/systemd/man/systemd-tmpfiles-setup.service.html)
 125    component ensures that various files and directories below `/etc/`, `/var/`
 126    and other places are created automatically at boot if missing. Unlike the
 127    directories/symlinks created by the `switch-root` logic above this logic is
 128    extensible by packages, and can adjust access modes, file ownership and
 129    more. Among others this will also link `/etc/os-release` →
 130    `/usr/lib/os-release`, ensuring that the OS release information is
 131    unconditionally accessible through `/etc/os-release`.
 132
 133 6. The
 134    [`nss-myhostname(8)`](https://www.freedesktop.org/software/systemd/man/nss-myhostname.html)
 135    glibc NSS module will ensure the local host name as well as `localhost`
 136    remains resolvable, even without `/etc/hosts` around.
 137
 138 With these mechanisms the hierarchies below `/var/` and `/etc/` can be safely
 139 and robustly populated on first boot, so that the OS can safely boot up. Note
 140 that some auxiliary package are not prepared to operate correctly if their
 141 configuration data in `/etc/` or their state directories in `/var/` are
 142 missing. This can typically be addressed via `systemd-tmpfiles` lines that
 143 ensure the missing files and directories are created if missing. In particular,
 144 configuration files that are necessary for operation can be automatically
 145 copied or symlinked from the `/usr/share/factory/etc/` tree via the `C` or `L`
 146 line types. That said, we recommend that all packages safely fall back to
 147 internal defaults if their configuration is missing, making such additional
 148 steps unnecessary.
 149
 150 Note that while `systemd` itself explicitly supports booting up with entirely
 151 unpopulated images (`/usr/` being the only required directory to be populated)
 152 distributions might not be there yet: depending on your distribution further,
 153 manual work might be required to make this scenario work.
 154
 155 ## Adapting OS Images to Storage
 156
 157 Typically, if an image is `dd`-ed onto a target disk it will be minimal:
 158 i.e. only consist of necessary vendor data, and lack "payload" data, that shall
 159 be individual to the system, and dependent on host parameters. On first boot,
 160 the OS should take possession of the backing storage as necessary, dynamically
 161 using available space. Specifically:
 162
 163 1. Additional partitions should be created, that make no sense to ship
 164    pre-built in the image. For example `/tmp/` or `/home/` partitions, or even
 165    `/var/` or the root file system (see above).
 166
 167 2. Additional partitions should be created that shall function as A/B
 168    secondaries for partitions shipped in the original image. In other words: if
 169    the `/usr/` file system shall be updated in an A/B fashion it typically
 170    makes sense to ship the original A file system in the deployed image, but
 171    create the B partition on first boot.
 172
 173 3. Partitions covering only a part of the disk should be grown to the full
 174    extent of the disk.
 175
 176 4. File systems in uninitialized partitions should be formatted with a file
 177    system of choice.
 178
 179 5. File systems covering only a part of a partition should be grown to the full
 180    extent of the partition.
 181
 182 6. Partitions should be encrypted with cryptographic keys generated locally on
 183    the machine the system is first booted on, ensuring these keys remain local
 184    and are not shared with any other instance of the OS image.
 185
 186 Or any combination of the above: i.e. first create a partition, then encrypt
 187 it, then format it.
 188
 189 `systemd` provides multiple tools to implement the above logic:
 190
 191 1. The
 192    [`systemd-repart(8)`](https://www.freedesktop.org/software/systemd/man/systemd-repart.service.html)
 193    component may manipulate GPT partition tables automatically on boot, growing
 194    partitions or adding in partitions taking the backing storage size into
 195    account. It can also encrypt partitions automatically it creates (even bind
 196    to TPM2, automatically) and populate partitions from various sources. It
 197    does this all in a robust fashion so that aborted invocations will not leave
 198    incompletely set up partitions around.
 199
 200 2. The
 201    [`systemd-growfs@(8).service`](https://www.freedesktop.org/software/systemd/man/systemd-growfs.html)
 202    tool can automatically grow a file system to the partition it is contained
 203    in. The `x-systemd.growfs` mount option in `/etc/fstab` is sufficient to
 204    enable this logic for specific mounts. Alternatively appropriately set up
 205    partitions can set GPT partition flag 59 to request this behaviour, see the
 206    [Discoverable Partitions Specification](https://uapi-group.org/specifications/specs/discoverable_partitions_specification)
 207    for details. If the file system is already grown it executes no operation.
 208
 209 3. Similar, the `systemd-makefs@.service` and `systemd-makeswap@.service`
 210    services can format file systems and swap spaces before first use, if they
 211    carry no file system signature yet. The `x-systemd.makefs` mount option in
 212    `/etc/fstab` may be used to request this functionality.
 213
 214 ## Provisioning Image Settings
 215
 216 While a lot of work has gone into ensuring `systemd` systems can safely boot
 217 with unpopulated `/etc/` trees, it sometimes is desirable to set a couple of
 218 basic settings *after* `dd`-ing the image to disk, but *before* first boot. For
 219 this the tool
 220 [`systemd-firstboot(1)`](https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html)
 221 can be useful, with its `--image=` switch. It may be used to set very basic
 222 settings, such as the root password or hostname on an OS disk image or
 223 installed block device.
 224
 225 ## Distinguishing First Boot
 226
 227 For various purposes it's useful to be able to distinguish the first boot-up of
 228 the system from later boot-ups (for example, to set up TPM hardware
 229 specifically, or register a system somewhere). `systemd` provides mechanisms to
 230 implement that. Specifically, the `ConditionFirstBoot=` and `AssertFirstBoot=`
 231 settings may be used to conditionalize units to only run on first boot. See
 232 [`systemd.unit(5)`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#ConditionFirstBoot=)
 233 for details.
 234
 235 A special target unit `first-boot-complete.target` may be used as milestone to
 236 safely handle first boots where the system is powered off too early: if the
 237 first boot process is aborted before this target is reached, the following boot
 238 process will be considered a first boot, too. Once the target is reached,
 239 subsequent boots will not be considered first boots anymore, even if the boot
 240 process is aborted immediately after. Thus, services that must complete fully
 241 before a system shall be considered fully past the first boot should be ordered
 242 before this target unit.
 243
 244 Whether a system will come up in first boot state or not is derived from the
 245 initialization status of `/etc/machine-id`: if the file already carries a valid
 246 ID the system is already past the first boot. If it is not initialized yet it
 247 is still considered in the first boot state. For details see
 248 [`machine-id(5)`](https://www.freedesktop.org/software/systemd/man/machine-id.html).
 249
 250 ## Image Metadata
 251
 252 Typically, when operating with golden disk images it is useful to be able to
 253 identify them and their version. For this the two fields `IMAGE_ID=` and
 254 `IMAGE_VERSION=` have been defined in
 255 [`os-release(5)`](https://www.freedesktop.org/software/systemd/man/os-release.html). These
 256 fields may be accessed from unit files and similar via the `%M` and `%A`
 257 specifiers.
 258
 259 Depending on how the images are put together it might make sense to leave the
 260 OS distribution's `os-release` file as is in `/usr/lib/os-release` but to
 261 replace the usual `/etc/os-release` symlink with a regular file that extends
 262 the distribution's file with one augmented with these two additional
 263 fields.
 264
 265 ## Links
 266
 267 [`machine-id(5)`](https://www.freedesktop.org/software/systemd/man/machine-id.html)<br>
 268 [`systemd-random-seed(8)`](https://www.freedesktop.org/software/systemd/man/systemd-random-seed.service.html)<br>
 269 [`os-release(5)`](https://www.freedesktop.org/software/systemd/man/os-release.html)<br>
 270 [Boot Loader Specification](https://uapi-group.org/specifications/specs/boot_loader_specification)<br>
 271 [Discoverable Partitions Specification](https://uapi-group.org/specifications/specs/discoverable_partitions_specification)<br>
 272 [`mkosi`](https://github.com/systemd/mkosi)<br>
 273 [`systemd-boot(7)`](https://www.freedesktop.org/software/systemd/man/systemd-boot.html)<br>
 274 [`systemd-repart(8)`](https://www.freedesktop.org/software/systemd/man/systemd-repart.service.html)<br>
 275 [`systemd-growfs@(8).service`](https://www.freedesktop.org/software/systemd/man/systemd-growfs.html)<br>