bpo-38816: Add notes in the C-API docs about fork in subinterpreters. (GH-17176)

The C-API docs are a bit sparse on the interplay between C `fork()` and the CPython runtime.  This change adds some more information on the subject.

https://bugs.python.org/issue38816

diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
index dc30e496825511fc524a8758808d03213be4cd18..86bf7f9eec65e7e5003782ae6448c227dd23492e 100644 (file)
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -769,9 +769,19 @@ supports the creation of additional interpreters (using
 :c:func:`Py_NewInterpreter`), but mixing multiple interpreters and the
 :c:func:`PyGILState_\*` API is unsupported.
 
+
+.. _fork-and-threads:
+
+Cautions about fork()
+---------------------
+
 Another important thing to note about threads is their behaviour in the face
 of the C :c:func:`fork` call. On most systems with :c:func:`fork`, after a
-process forks only the thread that issued the fork will exist. That also
+process forks only the thread that issued the fork will exist.  This has a
+concrete impact both on how locks must be handled and on all stored state
+in CPython's runtime.
+
+The fact that only the "current" thread remains
 means any locks held by other threads will never be released. Python solves
 this for :func:`os.fork` by acquiring the locks it uses internally before
 the fork, and releasing them afterwards. In addition, it resets any
@@ -786,6 +796,17 @@ being held by a thread that is defunct after the fork.
 :c:func:`PyOS_AfterFork_Child` tries to reset the necessary locks, but is not
 always able to.
 
+The fact that all other threads go away also means that CPython's
+runtime state there must be cleaned up properly, which :func:`os.fork`
+does.  This means finalizing all other :c:type:`PyThreadState` objects
+belonging to the current interpreter and all other
+:c:type:`PyInterpreterState` objects.  Due to this and the special
+nature of the :ref:`"main" interpreter <sub-interpreter-support>`,
+:c:func:`fork` should only be called in that interpreter's "main"
+thread, where the CPython global runtime was originally initialized.
+The only exception is if :c:func:`exec` will be called immediately
+after.
+
 
 High-level API
 --------------

diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst
index 212f4d1066c9c1c1940c724f415c878d7cdaa455..d3bbee23f50812b9af9d70e630a2fdbb5140708e 100644 (file)
--- a/Doc/c-api/sys.rst
+++ b/Doc/c-api/sys.rst
@@ -33,6 +33,12 @@ Operating System Utilities
    that clones the current process.
    Only available on systems where :c:func:`fork` is defined.
 
+   .. warning::
+      The C :c:func:`fork` call should only be made from the
+      :ref:`"main" thread <fork-and-threads>` (of the
+      :ref:`"main" interpreter <sub-interpreter-support>`).  The same is
+      true for ``PyOS_BeforeFork()``.
+
    .. versionadded:: 3.7
 
 
@@ -44,6 +50,12 @@ Operating System Utilities
    of whether process cloning was successful.
    Only available on systems where :c:func:`fork` is defined.
 
+   .. warning::
+      The C :c:func:`fork` call should only be made from the
+      :ref:`"main" thread <fork-and-threads>` (of the
+      :ref:`"main" interpreter <sub-interpreter-support>`).  The same is
+      true for ``PyOS_AfterFork_Parent()``.
+
    .. versionadded:: 3.7
 
 
@@ -55,6 +67,12 @@ Operating System Utilities
    any chance the process will call back into the Python interpreter.
    Only available on systems where :c:func:`fork` is defined.
 
+   .. warning::
+      The C :c:func:`fork` call should only be made from the
+      :ref:`"main" thread <fork-and-threads>` (of the
+      :ref:`"main" interpreter <sub-interpreter-support>`).  The same is
+      true for ``PyOS_AfterFork_Child()``.
+
    .. versionadded:: 3.7
 
    .. seealso::

diff --git a/Misc/NEWS.d/next/Documentation/2019-11-15-11-39-13.bpo-38816.vUaSVL.rst b/Misc/NEWS.d/next/Documentation/2019-11-15-11-39-13.bpo-38816.vUaSVL.rst
new file mode 100644 (file)
index 0000000..49accbc
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2019-11-15-11-39-13.bpo-38816.vUaSVL.rst
@@ -0,0 +1,3 @@
+Provides more details about the interaction between :c:func:`fork` and
+CPython's runtime, focusing just on the C-API.  This includes cautions
+about where :c:func:`fork` should and shouldn't be called.