docs: document threading situation in coding style

diff --git a/docs/CODING_STYLE.md b/docs/CODING_STYLE.md
index f76525205fdd261450a539d5c0e2d412e132e38f..f3eefeaae67a394d7ceea349cb26a89f3f0fbf4c 100644 (file)
--- a/docs/CODING_STYLE.md
+++ b/docs/CODING_STYLE.md
@@ -750,3 +750,32 @@ SPDX-License-Identifier: LGPL-2.1-or-later
 - Reasonable use of non-ASCII Unicode UTF-8 characters in code comments is
   welcome. If your code comment contains an emoji or two this will certainly
   brighten the day of the occasional reviewer of your code. Really! 😊
+
+## Threading
+
+- We generally avoid using threads, to the level this is possible. In
+  particular in the service manager/PID 1 threads are not OK to use. This is
+  because you cannot mix memory allocation in threads with use of glibc's
+  `clone()` call, or manual `clone()`/`clone3()` system call wrappers. Only
+  glibc's own `fork()` call will properly synchronize the memory allocation
+  locks around the process clone operation. This means that if a process is
+  cloned via `clone()`/`clone3()` and another thread currently has the
+  `malloc()` lock taken, it will be cloned in locked state to the child, and
+  thus can never be acquired in the child, leading to deadlocks. Hence, when
+  using `clone()`/`clone3()` there are only two ways out: never use threads in the
+  parent, or never do memory allocation in the child. For our uses we need
+  `clone()`/`clone3()` and hence decided to avoid threads. Of course, sometimes the
+  concurrency threads allow is beneficial, however we suggest forking off
+  worker *processes* rather than worker *threads* for this purpose, ideally
+  even with an `execve()` to remove the CoW trap situation `fork()` easily
+  triggers.
+
+- A corollary of the above is: never use `clone()` where a `fork()` would do
+  too. Also consider using `posix_spawn()` which combines `clone()` +
+  `execve()` into one and has nice properties since it avoids becoming a CoW
+  trap by using `CLONE_VORK` and `CLONE_VM` together.
+
+- While we avoid forking off threads on our own, writing thread-safe code is a
+  good idea where it might end up running inside of libsystemd.so or
+  similar. Hence, use TLS (i.e. `thread_local`) where appropriate, and maybe
+  the occasional `pthread_once()`.