docs/ARCHITECTURE.md

   1 ---
   2 title: systemd Repository Architecture
   3 category: Contributing
   4 layout: default
   5 SPDX-License-Identifier: LGPL-2.1-or-later
   6 ---
   7
   8 # The systemd Repository Architecture
   9
  10 ## Code Map
  11
  12 This document provides a high-level overview of the various components of the
  13 systemd repository.
  14
  15 ## Source Code
  16
  17 Directories in `src/` provide the implementation of all daemons, libraries and
  18 command-line tools shipped by the project. There are many, and more are
  19 constantly added, so we will not enumerate them all here — the directory
  20 names are self-explanatory.
  21
  22 ### Shared Code
  23
  24 The code that is shared between components is split into a few directories,
  25 each with a different purpose:
  26
  27 - `src/basic/` and `src/fundamental/` — those directories contain code
  28   primitives that are used by all other code. `src/fundamental/` is stricter,
  29   because it used for EFI and user-space code, while `src/basic/` is only used
  30   for user-space code. The code in `src/fundamental/` cannot depend on any
  31   other code in the tree, and `src/basic/` can depend only on itself and
  32   `src/fundamental/`. For user-space, a static library is built from this code
  33   and linked statically in various places.
  34
  35 - `src/libsystemd/` implements the `libsystemd.so` shared library (also
  36   available as static `libsystemd.a`). This code may use anything in
  37   `src/basic/` or `src/fundamental/`.
  38
  39 - `src/shared/` provides various utilities and code shared between other
  40   components that is exposed as the `libsystemd-shared-<nnn>.so` shared library.
  41
  42 The other subdirectories implement individual components. They may depend only
  43 on `src/fundamental/` + `src/basic/`, or also on `src/libsystemd/`, or also on
  44 `src/shared/`.
  45
  46 You might wonder what kind of code belongs where. In general, the rule is that
  47 code should be linked as few times as possible, ideally only once. Thus code that
  48 is used by "higher-level" components (e.g. our binaries which are linked to
  49 `libsystemd-shared-<nnn>.so`), would go to a subdirectory specific to that
  50 component if it is only used there. If the code is to be shared between
  51 components, it'd go to `src/shared/`. Shared code that is used by multiple
  52 components that do not link to `libsystemd-shared-<nnn>.so` may live either in
  53 `src/libsystemd/`, `src/basic/`, or `src/fundamental/`. Any code that is used
  54 only for EFI goes under `src/boot/efi/`, and `src/fundamental/` if is shared
  55 with non-EFI compoenents.
  56
  57 To summarize:
  58
  59 `src/fundamental/`
  60 - may be used by all code in the tree
  61 - may not use any code outside of `src/fundamental/`
  62
  63 `src/basic/`
  64 - may be used by all code in the tree
  65 - may not use any code outside of `src/fundamental/` and `src/basic/`
  66
  67 `src/libsystemd/`
  68 - may be used by all code in the tree that links to `libsystem.so`
  69 - may not use any code outside of `src/fundamental/`, `src/basic/`, and
  70   `src/libsystemd/`
  71
  72 `src/shared/`
  73 - may be used by all code in the tree, except for code in `src/basic/`,
  74   `src/libsystemd/`, `src/nss-*`, `src/login/pam_systemd.*`, and files under
  75   `src/journal/` that end up in `libjournal-client.a` convenience library.
  76 - may not use any code outside of `src/fundamental/`, `src/basic/`,
  77   `src/libsystemd/`, `src/shared/`
  78
  79 ### PID 1
  80
  81 Code located in `src/core/` implements the main logic of the systemd system (and user)
  82 service manager.
  83
  84 BPF helpers written in C and used by PID 1 can be found under `src/core/bpf/`.
  85
  86 #### Implementing Unit Settings
  87
  88 The system and session manager supports a large number of unit settings. These can generally
  89 be configured in three ways:
  90
  91 1. Via textual, INI-style configuration files called *unit* *files*
  92 2. Via D-Bus messages to the manager
  93 3. Via the `systemd-run` and `systemctl set-property` commands
  94
  95 From a user's perspective, the third is a wrapper for the second. To implement a new unit
  96 setting, it is necessary to support all three input methods:
  97
  98 1. *unit* *files* are parsed in `src/core/load-fragment.c`, with many simple and fixed-type
  99 unit settings being parsed by common helpers, with the definition in the generator file
 100 `src/core/load-fragment-gperf.gperf.in`
 101 2. D-Bus messages are defined and parsed in `src/core/dbus-*.c`
 102 3. `systemd-run` and `systemctl set-property` do client-side parsing and translation into
 103 D-Bus messages in `src/shared/bus-unit-util.c`
 104
 105 So that they are exercised by the fuzzing CI, new unit settings should also be listed in the
 106 text files under `test/fuzz/fuzz-unit-file/`.
 107
 108 ### systemd-udev
 109
 110 Sources for the udev daemon and command-line tool (single binary) can be found under
 111 `src/udev/`.
 112
 113 ### Unit Tests
 114
 115 Source files found under `src/test/` implement unit-level testing, mostly for
 116 modules found in `src/basic/` and `src/shared/`, but not exclusively. Each test
 117 file is compiled in a standalone binary that can be run to exercise the
 118 corresponding module. While most of the tests can be run by any user, some
 119 require privileges, and will attempt to clearly log about what they need
 120 (mostly in the form of effective capabilities). These tests are self-contained,
 121 and generally safe to run on the host without side effects.
 122
 123 Ideally, every module in `src/basic/` and `src/shared/` should have a
 124 corresponding unit test under `src/test/`, exercising every helper function.
 125
 126 ### Fuzzing
 127
 128 Fuzzers are a type of unit tests that execute code on an externally-supplied
 129 input sample. Fuzzers are called `fuzz-*`. Fuzzers for `src/basic/` and
 130 `src/shared` live under `src/fuzz/`, and those for other parts of the codebase
 131 should be located next to the code they test.
 132
 133 Files under `test/fuzz/` contain input data for fuzzers, one subdirectory for
 134 each fuzzer. Some of the files are "seed corpora", i.e. files that contain
 135 lists of settings and input values intended to generate initial coverage, and
 136 other files are samples saved by the fuzzing engines when they find an issue.
 137
 138 When adding new input samples under `test/fuzz/*/`, please use some
 139 short-but-meaningful names. Names of meson tests include the input file name
 140 and output looks awkward if they are too long.
 141
 142 Fuzzers are invoked primarily in three ways: firstly, each fuzzer is compiled
 143 as a normal executable and executed for each of the input samples under
 144 `test/fuzz/` as part of the test suite. Secondly, fuzzers may be instrumented
 145 with sanitizers and invoked as part of the test suite (if `-Dfuzz-tests=true`
 146 is configured). Thirdly, fuzzers are executed through fuzzing engines that try
 147 to find new "interesting" inputs through coverage feedback and massive
 148 parallelization; see the links for oss-fuzz in [Code quality](CODE_QUALITY).
 149 For testing and debugging, fuzzers can be executed as any other program,
 150 including under `valgrind` or `gdb`.
 151
 152 ## Integration Tests
 153
 154 Sources in `test/TEST-*` implement system-level testing for executables,
 155 libraries and daemons that are shipped by the project. They require privileges
 156 to run, and are not safe to execute directly on a host. By default they will
 157 build an image and run the test under it via `qemu` or `systemd-nspawn`.
 158
 159 Most of those tests should be able to run via `systemd-nspawn`, which is
 160 orders-of-magnitude faster than `qemu`, but some tests require privileged
 161 operations like using `dm-crypt` or `loopdev`. They are clearly marked if that
 162 is the case.
 163
 164 See `test/README.testsuite` for more specific details.
 165
 166 ## hwdb
 167
 168 Rules built in the static hardware database shipped by the project can be found
 169 under `hwdb.d/`. Some of these files are updated automatically, some are filled
 170 by contributors.
 171
 172 ## Documentation
 173
 174 ### systemd.io
 175
 176 Markdown files found under `docs/` are automatically published on the
 177 [systemd.io](https://systemd.io) website using Github Pages. A minimal unit test
 178 to ensure the formatting doesn't have errors is included in the
 179 `meson test -C build/ github-pages` run as part of the CI.
 180
 181 ### Man pages
 182
 183 Manpages for binaries and libraries, and the DBUS interfaces, can be found under
 184 `man/` and should ideally be kept in sync with changes to the corresponding
 185 binaries and libraries.
 186
 187 ### Translations
 188
 189 Translations files for binaries and daemons, provided by volunteers, can be found
 190 under `po/` in the usual format. They are kept up to date by contributors and by
 191 automated tools.
 192
 193 ## System Configuration files and presets
 194
 195 Presets (or templates from which they are generated) for various daemons and tools
 196 can be found under various directories such as `factory/`, `modprobe.d/`, `network/`,
 197 `presets/`, `rules.d/`, `shell-completion/`, `sysctl.d/`, `sysusers.d/`, `tmpfiles.d/`.
 198
 199 ## Utilities for Developers
 200
 201 `tools/`, `coccinelle/`, `.github/`, `.semaphore/`, `.mkosi/` host various
 202 utilities and scripts that are used by maintainers and developers. They are not
 203 shipped or installed.
 204
 205 # Service Manager Overview
 206
 207 The Service Manager takes configuration in the form of unit files, credentials,
 208 kernel command line options and D-Bus commands, and based on those manages the
 209 system and spawns other processes. It runs in system mode as PID1, and in user
 210 mode with one instance per user session.
 211
 212 When starting a unit requires forking a new process, configuration for the new
 213 process will be serialized and passed over to the new process, created via a
 214 posix_spawn() call. This is done in order to avoid excessive processing after
 215 a fork() but before an exec(), which is against glibc's best practices and can
 216 also result in a copy-on-write trap. The new process will start as the
 217 `systemd-executor` binary, which will deserialize the configuration and apply
 218 all the options (sandboxing, namespacing, cgroup, etc.) before exec'ing the
 219 configured executable.
 220
 221 ```
 222  ┌──────┐posix_spawn() ┌───────────┐execve() ┌────────┐
 223  │ PID1 ├─────────────►│sd-executor├────────►│program │
 224  └──────┘  (memfd)     └───────────┘         └────────┘
 225 ```