]>
Commit | Line | Data |
---|---|---|
c3e270f4 FB |
1 | --- |
2 | title: Hacking on systemd | |
4cdca0af | 3 | category: Contributing |
b41a3f66 | 4 | layout: default |
c3e270f4 FB |
5 | --- |
6 | ||
5a8a9dee FA |
7 | # Hacking on systemd |
8 | ||
9 | We welcome all contributions to systemd. If you notice a bug or a missing | |
10 | feature, please feel invited to fix it, and submit your work as a GitHub Pull | |
11 | Request (PR) at https://github.com/systemd/systemd/pull/new. | |
12 | ||
75e09908 ZJS |
13 | Please make sure to follow our [Coding Style](CODING_STYLE.md) when submitting |
14 | patches. Also have a look at our [Contribution Guidelines](CONTRIBUTING.md). | |
5a8a9dee FA |
15 | |
16 | When adding new functionality, tests should be added. For shared functionality | |
17 | (in `src/basic/` and `src/shared/`) unit tests should be sufficient. The general | |
18 | policy is to keep tests in matching files underneath `src/test/`, | |
19 | e.g. `src/test/test-path-util.c` contains tests for any functions in | |
20 | `src/basic/path-util.c`. If adding a new source file, consider adding a matching | |
21 | test executable. For features at a higher level, tests in `src/test/` are very | |
1e268f42 | 22 | strongly recommended. If that is not possible, integration tests in `test/` are |
5a8a9dee FA |
23 | encouraged. |
24 | ||
75e09908 ZJS |
25 | Please also have a look at our list of [code quality tools](CODE_QUALITY.md) we |
26 | have setup for systemd, to ensure our codebase stays in good shape. | |
5a8a9dee FA |
27 | |
28 | Please always test your work before submitting a PR. For many of the components | |
29 | of systemd testing is straight-forward as you can simply compile systemd and | |
30 | run the relevant tool from the build directory. | |
31 | ||
32 | For some components (most importantly, systemd/PID1 itself) this is not | |
33 | possible, however. In order to simplify testing for cases like this we provide | |
34 | a set of `mkosi` build files directly in the source tree. `mkosi` is a tool for | |
35 | building clean OS images from an upstream distribution in combination with a | |
36 | fresh build of the project in the local working directory. To make use of this, | |
d55ad7fe ZJS |
37 | please install the `mkosi` package (if not packaged for your distro, it can be |
38 | downloaded from https://github.com/systemd/mkosi). `mkosi` will build an image | |
39 | for the host distro by default. It is sufficient to type `mkosi` in the systemd | |
40 | project directory to generate a disk image `image.raw` you can boot either in | |
75e09908 | 41 | `systemd-nspawn` or in an UEFI-capable VM: |
5a8a9dee FA |
42 | |
43 | ``` | |
c38667f7 | 44 | # mkosi boot |
5a8a9dee FA |
45 | ``` |
46 | ||
47 | or: | |
48 | ||
49 | ``` | |
c38667f7 | 50 | # mkosi qemu |
5a8a9dee FA |
51 | ``` |
52 | ||
53 | Every time you rerun the `mkosi` command a fresh image is built, incorporating | |
c38667f7 DDM |
54 | all current changes you made to the project tree. To save time when rebuilding, |
55 | you can use mkosi's incremental mode (`-i`). This instructs mkosi to build a set | |
56 | of cache images that make future builds a lot faster. Note that the `-i` flag | |
57 | both instructs mkosi to build cached images if they don't exist yet and to use | |
58 | cached images if they already exist so make sure to always specify `-i` if you | |
59 | want mkosi to use the cached images. | |
60 | ||
61 | If you're going to build mkosi images that use the same distribution and release | |
62 | that you're currently using, you can speed up the initial mkosi run by having it | |
63 | reuse the host's package cache. To do this, create a mkosi override file in | |
64 | mkosi.default.d/ (e.g 20-local.conf) and add the following contents: | |
5a8a9dee | 65 | |
c38667f7 DDM |
66 | ``` |
67 | [Packages] | |
68 | Cache=<full-path-to-package-manager-cache> # (e.g. /var/cache/dnf) | |
69 | ``` | |
5a8a9dee | 70 | |
c38667f7 DDM |
71 | If you want to do a local build without mkosi, most distributions also provide |
72 | very simple and convenient ways to install all development packages necessary | |
73 | to build systemd. For example, on Fedora the following command line should be | |
74 | sufficient to install all of systemd's build dependencies: | |
5a8a9dee FA |
75 | |
76 | ``` | |
77 | # dnf builddep systemd | |
78 | ``` | |
79 | ||
80 | Putting this all together, here's a series of commands for preparing a patch | |
81 | for systemd (this example is for Fedora): | |
82 | ||
83 | ```sh | |
172ad053 DDM |
84 | $ sudo dnf builddep systemd # install build dependencies |
85 | $ sudo dnf install mkosi # install tool to quickly build images | |
5a8a9dee FA |
86 | $ git clone https://github.com/systemd/systemd.git |
87 | $ cd systemd | |
172ad053 DDM |
88 | $ vim src/core/main.c # or wherever you'd like to make your changes |
89 | $ meson build # configure the build | |
8b08be40 LP |
90 | $ meson compile -C build # build it locally, see if everything compiles fine |
91 | $ meson test -C build # run some simple regression tests | |
172ad053 | 92 | $ ln -s .mkosi/mkosi.fedora mkosi.default # Configure mkosi to build a fedora image |
172ad053 | 93 | $ sudo mkosi # build a test image |
c38667f7 | 94 | $ sudo mkosi boot # boot up the test image |
172ad053 DDM |
95 | $ git add -p # interactively put together your patch |
96 | $ git commit # commit it | |
5a8a9dee | 97 | $ git push REMOTE HEAD:refs/heads/BRANCH |
172ad053 DDM |
98 | # where REMOTE is your "fork" on GitHub |
99 | # and BRANCH is a branch name. | |
5a8a9dee FA |
100 | ``` |
101 | ||
102 | And after that, head over to your repo on GitHub and click "Compare & pull request" | |
103 | ||
104 | Happy hacking! | |
105 | ||
89f52a78 ZJS |
106 | ## Templating engines in .in files |
107 | ||
108 | Some source files are generated during build. We use two templating engines: | |
109 | * meson's `configure_file()` directive uses syntax with `@VARIABLE@`. | |
110 | ||
ba777d01 ZJS |
111 | See the |
112 | [Meson docs for `configure_file()`](https://mesonbuild.com/Reference-manual.html#configure_file) | |
113 | for details. | |
89f52a78 | 114 | |
c9d311c7 | 115 | {% raw %} |
89f52a78 ZJS |
116 | * most files are rendered using jinja2, with `{{VARIABLE}}` and `{% if … %}`, |
117 | `{% elif … %}`, `{% else … %}`, `{% endif … %}` blocks. `{# … #}` is a | |
118 | jinja2 comment, i.e. that block will not be visible in the rendered | |
c9d311c7 ZJS |
119 | output. `{% raw %} … `{% endraw %}`{{ '{' }}{{ '% endraw %' }}}` creates a block |
120 | where jinja2 syntax is not interpreted. | |
89f52a78 | 121 | |
ba777d01 ZJS |
122 | See the |
123 | [Jinja Template Designer Documentation](https://jinja2docs.readthedocs.io/en/stable/templates.html#synopsis) | |
89f52a78 ZJS |
124 | for details. |
125 | ||
126 | Please note that files for both template engines use the `.in` extension. | |
5a8a9dee | 127 | |
4c8e5f44 ZJS |
128 | ## Developer and release modes |
129 | ||
130 | In the default meson configuration (`-Dmode=developer`), certain checks are | |
131 | enabled that are suitable when hacking on systemd (such as internal | |
89f52a78 ZJS |
132 | documentation consistency checks). Those are not useful when compiling for |
133 | distribution and can be disabled by setting `-Dmode=release`. | |
4c8e5f44 | 134 | |
5a8a9dee FA |
135 | ## Fuzzers |
136 | ||
137 | systemd includes fuzzers in `src/fuzz/` that use libFuzzer and are automatically | |
135a1add | 138 | run by [OSS-Fuzz](https://github.com/google/oss-fuzz) with sanitizers. |
53a42e62 | 139 | To add a fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput` |
5a8a9dee FA |
140 | function and add it to the list in `src/fuzz/meson.build`. |
141 | ||
142 | Whenever possible, a seed corpus and a dictionary should also be added with new | |
143 | fuzz targets. The dictionary should be named `src/fuzz/fuzz-foo.dict` and the seed | |
144 | corpus should be built and exported as `$OUT/fuzz-foo_seed_corpus.zip` in | |
145 | `tools/oss-fuzz.sh`. | |
146 | ||
147 | The fuzzers can be built locally if you have libFuzzer installed by running | |
148 | `tools/oss-fuzz.sh`. You should also confirm that the fuzzer runs in the | |
149 | OSS-Fuzz environment by checking out the OSS-Fuzz repo, and then running | |
150 | commands like this: | |
151 | ||
152 | ``` | |
153 | python infra/helper.py build_image systemd | |
154 | python infra/helper.py build_fuzzers --sanitizer memory systemd ../systemd | |
155 | python infra/helper.py run_fuzzer systemd fuzz-foo | |
156 | ``` | |
157 | ||
158 | If you find a bug that impacts the security of systemd, please follow the | |
159 | guidance in [CONTRIBUTING.md](CONTRIBUTING.md) on how to report a security vulnerability. | |
160 | ||
161 | For more details on building fuzzers and integrating with OSS-Fuzz, visit: | |
162 | ||
6cec69fc LK |
163 | - [Setting up a new project - OSS-Fuzz](https://google.github.io/oss-fuzz/getting-started/new-project-guide/) |
164 | - [Tutorials - OSS-Fuzz](https://google.github.io/oss-fuzz/reference/useful-links/#tutorials) | |
4cc06b80 DDM |
165 | |
166 | ## mkosi + clangd | |
167 | ||
168 | [clangd](https://clangd.llvm.org/) is a language server that provides code completion, diagnostics and more | |
169 | right in your editor of choice (with the right plugin installed). When using mkosi, we can run clangd in the | |
170 | mkosi build container to avoid needing to build systemd on the host machine just to make clangd work. To | |
171 | achieve this, create a script with the following contents in systemd's project directory on the host: | |
172 | ||
173 | ```sh | |
174 | #!/usr/bin/env sh | |
175 | tee mkosi-clangd.build > /dev/null << EOF | |
176 | #!/usr/bin/env sh | |
177 | exec clangd \\ | |
178 | --compile-commands-dir=/root/build \\ | |
179 | --path-mappings=\\ | |
180 | "\\ | |
181 | $(pwd)=/root/src,\\ | |
182 | $(pwd)/mkosi.builddir=/root/build,\\ | |
183 | $(pwd)/mkosi.includedir=/usr/include,\\ | |
184 | $(pwd)/mkosi.installdir=/root/dest\\ | |
185 | " \\ | |
186 | --header-insertion=never | |
187 | EOF | |
188 | chmod +x mkosi-clangd.build | |
189 | exec sudo mkosi --source-file-transfer=mount --incremental --skip-final-phase --build-script mkosi-clangd.build build | |
190 | ``` | |
191 | ||
192 | Next, mark the script as executable and point your editor plugin to use this script to start clangd. For | |
193 | vscode's clangd extension, this is done via setting the `clangd.path` option to the path of the | |
194 | mkosi-clangd.sh script. | |
195 | ||
196 | To be able to navigate to include files of systemd's dependencies, we need to make the /usr/include folder of | |
197 | the build image available on the host. mkosi supports this by setting the `IncludeDirectory` option in | |
198 | mkosi's config. The easiest way to set the option is to create a file 20-local.conf in mkosi.default.d/ and | |
199 | add the following contents: | |
200 | ||
201 | ``` | |
202 | [Packages] | |
203 | IncludeDirectory=mkosi.includedir | |
204 | ``` | |
205 | ||
206 | This will make the contents of /usr/include available in mkosi.includedir in the systemd project directory. | |
207 | We already configured clangd to map any paths in /usr/include in the build image to mkosi.includedir/ on the | |
208 | host in the mkosi-clangd.sh script. | |
209 | ||
210 | We also need to make sure clangd is installed in the build image. To have mkosi install clangd in the build | |
211 | image, edit the 20-local.conf file we created earlier and add the following contents under the `[Packages]` | |
212 | section: | |
213 | ||
214 | ``` | |
215 | BuildPackages=<clangd-package> | |
216 | ``` | |
217 | ||
218 | Note that the exact package containing clangd will differ depending on the distribution used. Some | |
219 | distributions have a separate clangd package, others put the clangd binary in a clang-tools-extra package and | |
220 | some bundle clangd in the clang package. | |
221 | ||
222 | Because mkosi needs to run as root, we also need to make sure we can enter the root password when the editor | |
223 | plugin tries to run the mkosi-clangd.sh script. To be able to enter the root password in non-interactive | |
224 | scripts, we use an askpass provider. This is a program that sudo will launch if it detects it's being | |
225 | executed from a non-interactive shell so that the root password can still be entered. There are multiple | |
226 | implementations such as gnome askpass and KDE askpass. Install one of the askpass packages your distro | |
227 | provides and set the `SUDO_ASKPASS` environment variable to the path of the askpass binary you want to use. | |
228 | If configured correctly, a window will appear when your editor plugin tries to run the mkosi-clangd.sh script | |
229 | allowing you to enter the root password. | |
230 | ||
231 | Due to a bug in btrfs, it's currently impossible to mount two mkosi btrfs images at the same time. Because of | |
232 | this, trying to do a regular build while the clangd image is running will fail. To circumvent this, use ext4 | |
233 | instead of btrfs for the images by adding the following contents to 20-local.conf: | |
234 | ||
235 | ``` | |
236 | [Output] | |
237 | Format=gpt_ext4 | |
238 | ``` | |
239 | ||
240 | Finally, to ensure clangd starts up quickly in the editor, run an incremental build with mkosi to make sure | |
241 | the cached images are initialized (`mkosi -i`). | |
242 | ||
243 | Now, your editor will start clangd in the mkosi build image and all of clangd's features will work as | |
244 | expected. | |
66dc9b46 DDM |
245 | |
246 | ## Debugging systemd with mkosi + vscode | |
247 | ||
248 | To simplify debugging systemd when testing changes using mkosi, we're going to show how to attach | |
249 | [VSCode](https://code.visualstudio.com/)'s debugger to an instance of systemd running in a mkosi image | |
250 | (either using QEMU or systemd-nspawn). | |
251 | ||
252 | To allow VSCode's debugger to attach to systemd running in a mkosi image, we have to make sure it can access | |
253 | the container/virtual machine spawned by mkosi where systemd is running. mkosi makes this possible via a | |
254 | handy SSH option that makes the generated image accessible via SSH when booted. The easiest way to set the | |
255 | option is to create a file 20-local.conf in mkosi.default.d/ and add the following contents: | |
256 | ||
257 | ``` | |
258 | [Host] | |
259 | Ssh=yes | |
260 | ``` | |
261 | ||
262 | Next, make sure systemd-networkd is running on the host system so that it can configure the network interface | |
263 | connecting the host system to the container/VM spawned by mkosi. Once systemd-networkd is running, you should | |
264 | be able to connect to a running mkosi image by executing `mkosi ssh` in the systemd repo directory. | |
265 | ||
266 | Now we need to configure VSCode. First, make sure the C/C++ extension is installed. If you're already using | |
267 | a different extension for code completion and other IDE features for C in VSCode, make sure to disable the | |
268 | corresponding parts of the C/C++ extension in your VSCode user settings by adding the following entries: | |
269 | ||
270 | ```json | |
271 | "C_Cpp.formatting": "Disabled", | |
272 | "C_Cpp.intelliSenseEngine": "Disabled", | |
273 | "C_Cpp.enhancedColorization": "Disabled", | |
274 | "C_Cpp.suggestSnippets": false, | |
275 | ``` | |
276 | ||
277 | With the extension set up, we can create the launch.json file in the .vscode/ directory to tell the VSCode | |
278 | debugger how to attach to the systemd instance running in our mkosi container/VM. Create the file and add the | |
279 | following contents: | |
280 | ||
281 | ```json | |
282 | { | |
283 | "version": "0.2.0", | |
284 | "configurations": [ | |
285 | { | |
286 | "type": "cppdbg", | |
287 | "program": "/usr/lib/systemd/systemd", | |
288 | "processId": "${command:pickProcess}", | |
289 | "request": "attach", | |
290 | "name": "systemd", | |
291 | "pipeTransport": { | |
292 | "pipeProgram": "mkosi", | |
293 | "pipeArgs": [ | |
294 | "-C", | |
295 | "/path/to/systemd/repo/directory/on/host/system/", | |
296 | "ssh" | |
297 | ], | |
298 | "debuggerPath": "/usr/bin/gdb" | |
299 | }, | |
300 | "MIMode": "gdb", | |
301 | "sourceFileMap": { | |
302 | "/root/build/../src": { | |
303 | "editorPath": "${workspaceFolder}", | |
304 | "useForBreakpoints": false | |
305 | }, | |
306 | "/root/build/*": { | |
307 | "editorPath": "${workspaceFolder}/mkosi.builddir", | |
308 | "useForBreakpoints": false | |
309 | } | |
310 | } | |
311 | } | |
312 | ] | |
313 | } | |
314 | ``` | |
315 | ||
316 | Now that the debugger knows how to connect to our process in the container/VM and we've set up the necessary | |
317 | source mappings, go to the "Run and Debug" window and run the "systemd" debug configuration. If everything | |
318 | goes well, the debugger should now be attached to the systemd instance running in the container/VM. You can | |
319 | attach breakpoints from the editor and enjoy all the other features of VSCode's debugger. | |
320 | ||
321 | To debug systemd components other than PID 1, set "program" to the full path of the component you want to | |
322 | debug and set "processId" to "${command:pickProcess}". Now, when starting the debugger, VSCode will ask you | |
323 | the PID of the process you want to debug. Run `systemctl show --property MainPID --value <component>` in the | |
324 | container to figure out the PID and enter it when asked and VSCode will attach to that process instead. |