]>
Commit | Line | Data |
---|---|---|
c3e270f4 FB |
1 | --- |
2 | title: Hacking on systemd | |
4cdca0af | 3 | category: Contributing |
b41a3f66 | 4 | layout: default |
0aff7b75 | 5 | SPDX-License-Identifier: LGPL-2.1-or-later |
c3e270f4 FB |
6 | --- |
7 | ||
5a8a9dee FA |
8 | # Hacking on systemd |
9 | ||
10 | We welcome all contributions to systemd. If you notice a bug or a missing | |
54080482 DDM |
11 | feature, please feel invited to fix it, and submit your work as a |
12 | [GitHub Pull Request (PR)](https://github.com/systemd/systemd/pull/new). | |
5a8a9dee | 13 | |
1b4dc2ea ZJS |
14 | Please make sure to follow our [Coding Style](CODING_STYLE) when submitting |
15 | patches. Also have a look at our [Contribution Guidelines](CONTRIBUTING). | |
5a8a9dee | 16 | |
c7354249 | 17 | To start, run the following commands in the systemd git repository to set up |
6233a769 | 18 | git correctly (running `meson` will run these commands for you automatically): |
c7354249 DDM |
19 | |
20 | ```shell | |
21 | $ git config submodule.recurse true | |
22 | $ git config fetch.recurseSubmodules on-demand | |
e33d43b0 | 23 | $ git config push.recurseSubmodules no |
c7354249 DDM |
24 | ``` |
25 | ||
5a8a9dee FA |
26 | When adding new functionality, tests should be added. For shared functionality |
27 | (in `src/basic/` and `src/shared/`) unit tests should be sufficient. The general | |
28 | policy is to keep tests in matching files underneath `src/test/`, | |
29 | e.g. `src/test/test-path-util.c` contains tests for any functions in | |
30 | `src/basic/path-util.c`. If adding a new source file, consider adding a matching | |
31 | test executable. For features at a higher level, tests in `src/test/` are very | |
1e268f42 | 32 | strongly recommended. If that is not possible, integration tests in `test/` are |
5a8a9dee FA |
33 | encouraged. |
34 | ||
1b4dc2ea | 35 | Please also have a look at our list of [code quality tools](CODE_QUALITY) we |
75e09908 | 36 | have setup for systemd, to ensure our codebase stays in good shape. |
5a8a9dee FA |
37 | |
38 | Please always test your work before submitting a PR. For many of the components | |
4bb37359 | 39 | of systemd testing is straightforward as you can simply compile systemd and |
5a8a9dee FA |
40 | run the relevant tool from the build directory. |
41 | ||
8f1a581e | 42 | For some components (most importantly, systemd/PID 1 itself) this is not |
5a8a9dee | 43 | possible, however. In order to simplify testing for cases like this we provide |
4d0f1451 | 44 | a set of `mkosi` config files directly in the source tree. |
54080482 DDM |
45 | [mkosi](https://github.com/systemd/mkosi) is a tool for building clean OS images |
46 | from an upstream distribution in combination with a fresh build of the project | |
4d0f1451 DDM |
47 | in the local working directory. To make use of this, please install the latest |
48 | version of mkosi from the [GitHub repository](https://github.com/systemd/mkosi). | |
49 | `mkosi` will build an image for the host distro by default. First, run | |
50 | `mkosi genkey` to generate a key and certificate to be used for secure boot and | |
51 | verity signing. After that is done, it is sufficient to type `mkosi` in the | |
52 | systemd project directory to generate a disk image you can boot either in | |
53 | `systemd-nspawn` or in a UEFI-capable VM: | |
5a8a9dee | 54 | |
b2c9da05 | 55 | ```sh |
2edcf8e7 | 56 | $ sudo mkosi boot # nspawn still needs sudo for now |
5a8a9dee FA |
57 | ``` |
58 | ||
59 | or: | |
60 | ||
b2c9da05 DDM |
61 | ```sh |
62 | $ mkosi qemu | |
5a8a9dee FA |
63 | ``` |
64 | ||
65 | Every time you rerun the `mkosi` command a fresh image is built, incorporating | |
2fe24ccc | 66 | all current changes you made to the project tree. |
c38667f7 | 67 | |
52842bb2 DDM |
68 | By default a directory image is built. This requires `virtiofsd` to be installed |
69 | on the host. To build a disk image instead which does not require `virtiofsd`, | |
70 | add the following to `mkosi.local.conf`: | |
71 | ||
72 | ```conf | |
73 | [Output] | |
74 | Format=disk | |
75 | ``` | |
76 | ||
77 | To boot in UEFI mode instead of using QEMU's direct kernel boot, add the following | |
78 | to `mkosi.local.conf`: | |
79 | ||
80 | ```conf | |
81 | [Host] | |
82 | QemuFirmware=uefi | |
83 | ``` | |
84 | ||
f478b6e9 DDM |
85 | Putting this all together, here's a series of commands for preparing a patch |
86 | for systemd: | |
87 | ||
88 | ```sh | |
bcb335ac DDM |
89 | $ git clone https://github.com/systemd/mkosi.git # If mkosi v19 or newer is not packaged by your distribution |
90 | $ ln -s $PWD/mkosi/bin/mkosi /usr/local/bin/mkosi # If mkosi v19 or newer is not packaged by your distribution | |
f478b6e9 DDM |
91 | $ git clone https://github.com/systemd/systemd.git |
92 | $ cd systemd | |
93 | $ git checkout -b <BRANCH> # where BRANCH is the name of the branch | |
94 | $ vim src/core/main.c # or wherever you'd like to make your changes | |
95 | $ mkosi -f qemu # (re-)build and boot up the test image in qemu | |
96 | $ git add -p # interactively put together your patch | |
97 | $ git commit # commit it | |
98 | $ git push -u <REMOTE> # where REMOTE is your "fork" on GitHub | |
99 | ``` | |
100 | ||
101 | And after that, head over to your repo on GitHub and click "Compare & pull request" | |
102 | ||
c38667f7 | 103 | If you want to do a local build without mkosi, most distributions also provide |
f478b6e9 | 104 | very simple and convenient ways to install most development packages necessary |
b43ed972 | 105 | to build systemd: |
5a8a9dee | 106 | |
b43ed972 DDM |
107 | ```sh |
108 | # Fedora | |
109 | $ sudo dnf builddep systemd | |
110 | # Debian/Ubuntu | |
4df5799f | 111 | $ sudo apt-get build-dep systemd |
b43ed972 | 112 | # Arch |
ba96ba04 MY |
113 | $ sudo pacman -S devtools |
114 | $ pkgctl repo clone --protocol=https systemd | |
115 | $ cd systemd | |
b43ed972 | 116 | $ makepkg -seoc |
5a8a9dee FA |
117 | ``` |
118 | ||
f478b6e9 | 119 | After installing the development packages, systemd can be built from source as follows: |
5a8a9dee FA |
120 | |
121 | ```sh | |
f478b6e9 DDM |
122 | $ meson setup build <options> |
123 | $ ninja -C build | |
124 | $ meson test -C build | |
5a8a9dee FA |
125 | ``` |
126 | ||
5a8a9dee FA |
127 | Happy hacking! |
128 | ||
89f52a78 ZJS |
129 | ## Templating engines in .in files |
130 | ||
131 | Some source files are generated during build. We use two templating engines: | |
132 | * meson's `configure_file()` directive uses syntax with `@VARIABLE@`. | |
133 | ||
ba777d01 ZJS |
134 | See the |
135 | [Meson docs for `configure_file()`](https://mesonbuild.com/Reference-manual.html#configure_file) | |
136 | for details. | |
89f52a78 | 137 | |
c9d311c7 | 138 | {% raw %} |
89f52a78 ZJS |
139 | * most files are rendered using jinja2, with `{{VARIABLE}}` and `{% if … %}`, |
140 | `{% elif … %}`, `{% else … %}`, `{% endif … %}` blocks. `{# … #}` is a | |
141 | jinja2 comment, i.e. that block will not be visible in the rendered | |
c9d311c7 ZJS |
142 | output. `{% raw %} … `{% endraw %}`{{ '{' }}{{ '% endraw %' }}}` creates a block |
143 | where jinja2 syntax is not interpreted. | |
89f52a78 | 144 | |
ba777d01 ZJS |
145 | See the |
146 | [Jinja Template Designer Documentation](https://jinja2docs.readthedocs.io/en/stable/templates.html#synopsis) | |
89f52a78 ZJS |
147 | for details. |
148 | ||
149 | Please note that files for both template engines use the `.in` extension. | |
5a8a9dee | 150 | |
4c8e5f44 ZJS |
151 | ## Developer and release modes |
152 | ||
153 | In the default meson configuration (`-Dmode=developer`), certain checks are | |
154 | enabled that are suitable when hacking on systemd (such as internal | |
89f52a78 ZJS |
155 | documentation consistency checks). Those are not useful when compiling for |
156 | distribution and can be disabled by setting `-Dmode=release`. | |
4c8e5f44 | 157 | |
69d638e6 DDM |
158 | ## Sanitizers in mkosi |
159 | ||
1b4dc2ea | 160 | See [Testing systemd using sanitizers](TESTING_WITH_SANITIZERS) for more information |
69d638e6 DDM |
161 | on how to build with sanitizers enabled in mkosi. |
162 | ||
5a8a9dee FA |
163 | ## Fuzzers |
164 | ||
165 | systemd includes fuzzers in `src/fuzz/` that use libFuzzer and are automatically | |
135a1add | 166 | run by [OSS-Fuzz](https://github.com/google/oss-fuzz) with sanitizers. |
53a42e62 | 167 | To add a fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput` |
5a8a9dee FA |
168 | function and add it to the list in `src/fuzz/meson.build`. |
169 | ||
170 | Whenever possible, a seed corpus and a dictionary should also be added with new | |
171 | fuzz targets. The dictionary should be named `src/fuzz/fuzz-foo.dict` and the seed | |
172 | corpus should be built and exported as `$OUT/fuzz-foo_seed_corpus.zip` in | |
173 | `tools/oss-fuzz.sh`. | |
174 | ||
175 | The fuzzers can be built locally if you have libFuzzer installed by running | |
d04af6aa FS |
176 | `tools/oss-fuzz.sh`, or by running: |
177 | ||
178 | ``` | |
179 | CC=clang CXX=clang++ \ | |
180 | meson setup build-libfuzz -Dllvm-fuzz=true -Db_sanitize=address,undefined -Db_lundef=false \ | |
181 | -Dc_args='-fno-omit-frame-pointer -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION' | |
182 | ninja -C build-libfuzz fuzzers | |
183 | ``` | |
184 | ||
185 | Each fuzzer then can be then run manually together with a directory containing | |
186 | the initial corpus: | |
187 | ||
188 | ``` | |
189 | export UBSAN_OPTIONS=print_stacktrace=1:print_summary=1:halt_on_error=1 | |
190 | build-libfuzz/fuzz-varlink-idl test/fuzz/fuzz-varlink-idl/ | |
191 | ``` | |
192 | ||
193 | Note: the `halt_on_error=1` UBSan option is especially important, otherwise | |
194 | the fuzzer won't crash when undefined behavior is triggered. | |
195 | ||
196 | You should also confirm that the fuzzers can be built and run using | |
39e63b44 | 197 | [the OSS-Fuzz toolchain](https://google.github.io/oss-fuzz/advanced-topics/reproducing/#building-using-docker): |
5a8a9dee FA |
198 | |
199 | ``` | |
39e63b44 EV |
200 | path_to_systemd=... |
201 | ||
202 | git clone --depth=1 https://github.com/google/oss-fuzz | |
203 | cd oss-fuzz | |
204 | ||
205 | for sanitizer in address undefined memory; do | |
206 | for engine in libfuzzer afl honggfuzz; do | |
207 | ./infra/helper.py build_fuzzers --sanitizer "$sanitizer" --engine "$engine" \ | |
208 | --clean systemd "$path_to_systemd" | |
209 | ||
210 | ./infra/helper.py check_build --sanitizer "$sanitizer" --engine "$engine" \ | |
211 | -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 systemd | |
212 | done | |
213 | done | |
214 | ||
61ad4f25 EV |
215 | ./infra/helper.py build_fuzzers --clean --architecture i386 systemd "$path_to_systemd" |
216 | ./infra/helper.py check_build --architecture i386 -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 systemd | |
217 | ||
39e63b44 EV |
218 | ./infra/helper.py build_fuzzers --clean --sanitizer coverage systemd "$path_to_systemd" |
219 | ./infra/helper.py coverage --no-corpus-download systemd | |
5a8a9dee FA |
220 | ``` |
221 | ||
222 | If you find a bug that impacts the security of systemd, please follow the | |
1b4dc2ea | 223 | guidance in [CONTRIBUTING.md](CONTRIBUTING) on how to report a security vulnerability. |
5a8a9dee FA |
224 | |
225 | For more details on building fuzzers and integrating with OSS-Fuzz, visit: | |
226 | ||
6cec69fc LK |
227 | - [Setting up a new project - OSS-Fuzz](https://google.github.io/oss-fuzz/getting-started/new-project-guide/) |
228 | - [Tutorials - OSS-Fuzz](https://google.github.io/oss-fuzz/reference/useful-links/#tutorials) | |
4cc06b80 | 229 | |
2d92c35b DDM |
230 | ## Debugging binaries that need to run as root in vscode |
231 | ||
232 | When trying to debug binaries that need to run as root, we need to do some custom configuration in vscode to | |
233 | have it try to run the applications as root and to ask the user for the root password when trying to start | |
234 | the binary. To achieve this, we'll use a custom debugger path which points to a script that starts `gdb` as | |
235 | root using `pkexec`. pkexec will prompt the user for their root password via a graphical interface. This | |
236 | guide assumes the C/C++ extension is used for debugging. | |
237 | ||
238 | First, create a file `sgdb` in the root of the systemd repository with the following contents and make it | |
239 | executable: | |
240 | ||
241 | ``` | |
242 | #!/bin/sh | |
243 | exec pkexec gdb "$@" | |
244 | ``` | |
245 | ||
246 | Then, open launch.json in vscode, and set `miDebuggerPath` to `${workspaceFolder}/sgdb` for the corresponding | |
247 | debug configuration. Now, whenever you try to debug the application, vscode will try to start gdb as root via | |
248 | pkexec which will prompt you for your password via a graphical interface. After entering your password, | |
249 | vscode should be able to start debugging the application. | |
250 | ||
251 | For more information on how to set up a debug configuration for C binaries, please refer to the official | |
252 | vscode documentation [here](https://code.visualstudio.com/docs/cpp/launch-json-reference) | |
253 | ||
66dc9b46 DDM |
254 | ## Debugging systemd with mkosi + vscode |
255 | ||
256 | To simplify debugging systemd when testing changes using mkosi, we're going to show how to attach | |
c84d14c5 DDM |
257 | [VSCode](https://code.visualstudio.com/)'s debugger to an instance of systemd running in a mkosi image using |
258 | QEMU. | |
66dc9b46 DDM |
259 | |
260 | To allow VSCode's debugger to attach to systemd running in a mkosi image, we have to make sure it can access | |
c84d14c5 DDM |
261 | the virtual machine spawned by mkosi where systemd is running. mkosi makes this possible via a handy SSH |
262 | option that makes the generated image accessible via SSH when booted. Thus you must build the image with | |
bcb335ac | 263 | `mkosi --ssh`. The easiest way to set the option is to create a file `mkosi.local.conf` in the root of the |
ee7304df | 264 | repository and add the following contents: |
66dc9b46 DDM |
265 | |
266 | ``` | |
267 | [Host] | |
268 | Ssh=yes | |
ee7304df | 269 | RuntimeTrees=. |
66dc9b46 DDM |
270 | ``` |
271 | ||
c84d14c5 | 272 | Also make sure that the SSH agent is running on your system and that you've added your SSH key to it with |
ee7304df | 273 | `ssh-add`. Also make sure that `virtiofsd` is installed. |
c84d14c5 DDM |
274 | |
275 | After rebuilding the image and booting it with `mkosi qemu`, you should now be able to connect to it by | |
276 | running `mkosi ssh` from the same directory in another terminal window. | |
66dc9b46 DDM |
277 | |
278 | Now we need to configure VSCode. First, make sure the C/C++ extension is installed. If you're already using | |
279 | a different extension for code completion and other IDE features for C in VSCode, make sure to disable the | |
280 | corresponding parts of the C/C++ extension in your VSCode user settings by adding the following entries: | |
281 | ||
282 | ```json | |
283 | "C_Cpp.formatting": "Disabled", | |
284 | "C_Cpp.intelliSenseEngine": "Disabled", | |
285 | "C_Cpp.enhancedColorization": "Disabled", | |
286 | "C_Cpp.suggestSnippets": false, | |
287 | ``` | |
288 | ||
289 | With the extension set up, we can create the launch.json file in the .vscode/ directory to tell the VSCode | |
026d2499 WR |
290 | debugger how to attach to the systemd instance running in our mkosi container/VM. Create the file, and possibly |
291 | the directory, and add the following contents: | |
66dc9b46 DDM |
292 | |
293 | ```json | |
294 | { | |
295 | "version": "0.2.0", | |
296 | "configurations": [ | |
297 | { | |
298 | "type": "cppdbg", | |
299 | "program": "/usr/lib/systemd/systemd", | |
3f3bc1f2 | 300 | "processId": "${command:pickRemoteProcess}", |
66dc9b46 DDM |
301 | "request": "attach", |
302 | "name": "systemd", | |
303 | "pipeTransport": { | |
304 | "pipeProgram": "mkosi", | |
305 | "pipeArgs": [ | |
306 | "-C", | |
307 | "/path/to/systemd/repo/directory/on/host/system/", | |
308 | "ssh" | |
309 | ], | |
310 | "debuggerPath": "/usr/bin/gdb" | |
311 | }, | |
312 | "MIMode": "gdb", | |
313 | "sourceFileMap": { | |
ee7304df | 314 | "/root/src/systemd": { |
66dc9b46 DDM |
315 | "editorPath": "${workspaceFolder}", |
316 | "useForBreakpoints": false | |
317 | }, | |
66dc9b46 DDM |
318 | } |
319 | } | |
320 | ] | |
321 | } | |
322 | ``` | |
323 | ||
324 | Now that the debugger knows how to connect to our process in the container/VM and we've set up the necessary | |
325 | source mappings, go to the "Run and Debug" window and run the "systemd" debug configuration. If everything | |
326 | goes well, the debugger should now be attached to the systemd instance running in the container/VM. You can | |
327 | attach breakpoints from the editor and enjoy all the other features of VSCode's debugger. | |
328 | ||
329 | To debug systemd components other than PID 1, set "program" to the full path of the component you want to | |
330 | debug and set "processId" to "${command:pickProcess}". Now, when starting the debugger, VSCode will ask you | |
331 | the PID of the process you want to debug. Run `systemctl show --property MainPID --value <component>` in the | |
332 | container to figure out the PID and enter it when asked and VSCode will attach to that process instead. | |
948d085e | 333 | |
818e46ae | 334 | ## Debugging systemd-boot |
948d085e | 335 | |
badea0d6 ZJS |
336 | During boot, systemd-boot and the stub loader will output messages like |
337 | `systemd-boot@0x0A` and `systemd-stub@0x0B`, providing the base of the loaded | |
338 | code. This location can then be used to attach to a QEMU session (provided it | |
339 | was run with `-s`). See `debug-sd-boot.sh` script in the tools folder which | |
340 | automates this processes. | |
341 | ||
342 | If the debugger is too slow to attach to examine an early boot code passage, | |
343 | the call to `DEFINE_EFI_MAIN_FUNCTION()` can be modified to enable waiting. As | |
344 | soon as the debugger has control, we can then run `set variable wait = 0` or | |
345 | `return` to continue. Once the debugger has attached, setting breakpoints will | |
346 | work like usual. | |
948d085e JJ |
347 | |
348 | To debug systemd-boot in an IDE such as VSCode we can use a launch configuration like this: | |
349 | ```json | |
350 | { | |
351 | "name": "systemd-boot", | |
352 | "type": "cppdbg", | |
353 | "request": "launch", | |
354 | "program": "${workspaceFolder}/build/src/boot/efi/systemd-bootx64.efi", | |
355 | "cwd": "${workspaceFolder}", | |
356 | "MIMode": "gdb", | |
357 | "miDebuggerServerAddress": ":1234", | |
358 | "setupCommands": [ | |
359 | { "text": "shell mkfifo /tmp/sdboot.{in,out}" }, | |
360 | { "text": "shell qemu-system-x86_64 [...] -s -serial pipe:/tmp/sdboot" }, | |
361 | { "text": "shell ${workspaceFolder}/tools/debug-sd-boot.sh ${workspaceFolder}/build/src/boot/efi/systemd-bootx64.efi /tmp/sdboot.out systemd-boot.gdb" }, | |
362 | { "text": "source /tmp/systemd-boot.gdb" }, | |
363 | ] | |
364 | } | |
365 | ``` |