gh-100228: Document the os.fork threads DeprecationWarning. (#109767)

Document the `os.fork` posix threads detected `DeprecationWarning` in 3.12 What's New, os, multiprocessing, and concurrent.futures docs.

Many reviews and doc cleanup edits by Adam & Hugo. 🥳

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst
index 09c9fc4e6e227a82d86f1aa5daef735a69617707..6503d1fcf70a32554904302aaedd2c1447e424d2 100644 (file)
--- a/Doc/library/concurrent.futures.rst
+++ b/Doc/library/concurrent.futures.rst
@@ -293,6 +293,14 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
       The *max_tasks_per_child* argument was added to allow users to
       control the lifetime of workers in the pool.
 
+   .. versionchanged:: 3.12
+      On POSIX systems, if your application has multiple threads and the
+      :mod:`multiprocessing` context uses the ``"fork"`` start method:
+      The :func:`os.fork` function called internally to spawn workers may raise a
+      :exc:`DeprecationWarning`. Pass a *mp_context* configured to use a
+      different start method. See the :func:`os.fork` documentation for
+      further explanation.
+
 .. _processpoolexecutor-example:
 
 ProcessPoolExecutor Example

diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index 38d24a86072970dbc591df5549ef7a5a3bdcb767..2f0f1f800fdc944c1f57d3d3d3b317457d7a49a4 100644 (file)
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -131,6 +131,12 @@ to start a process.  These *start methods* are
        Code that requires *fork* should explicitly specify that via
        :func:`get_context` or :func:`set_start_method`.
 
+    .. versionchanged:: 3.12
+       If Python is able to detect that your process has multiple threads, the
+       :func:`os.fork` function that this start method calls internally will
+       raise a :exc:`DeprecationWarning`. Use a different start method.
+       See the :func:`os.fork` documentation for further explanation.
+
   *forkserver*
     When the program starts and selects the *forkserver* start method,
     a server process is spawned.  From then on, whenever a new process

diff --git a/Doc/library/os.rst b/Doc/library/os.rst
index c67b966f777db860e8b197cd171ac8517ebb63b7..74897a76b1d20ad24b93394db63440a00eb15c88 100644 (file)
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -4157,15 +4157,38 @@ written in Python, such as a mail server's external command delivery program.
 
    .. audit-event:: os.fork "" os.fork
 
+   .. warning::
+
+      If you use TLS sockets in an application calling ``fork()``, see
+      the warning in the :mod:`ssl` documentation.
+
    .. versionchanged:: 3.8
       Calling ``fork()`` in a subinterpreter is no longer supported
       (:exc:`RuntimeError` is raised).
 
-   .. warning::
-
-      See :mod:`ssl` for applications that use the SSL module with fork().
+   .. versionchanged:: 3.12
+      If Python is able to detect that your process has multiple
+      threads, :func:`os.fork` now raises a :exc:`DeprecationWarning`.
+
+      We chose to surface this as a warning, when detectable, to better
+      inform developers of a design problem that the POSIX platform
+      specifically notes as not supported. Even in code that
+      *appears* to work, it has never been safe to mix threading with
+      :func:`os.fork` on POSIX platforms. The CPython runtime itself has
+      always made API calls that are not safe for use in the child
+      process when threads existed in the parent (such as ``malloc`` and
+      ``free``).
+
+      Users of macOS or users of libc or malloc implementations other
+      than those typically found in glibc to date are among those
+      already more likely to experience deadlocks running such code.
+
+      See `this discussion on fork being incompatible with threads
+      <https://discuss.python.org/t/33555>`_
+      for technical details of why we're surfacing this longstanding
+      platform compatibility problem to developers.
 
-   .. availability:: Unix, not Emscripten, not WASI.
+   .. availability:: POSIX, not Emscripten, not WASI.
 
 
 .. function:: forkpty()
@@ -4178,6 +4201,11 @@ written in Python, such as a mail server's external command delivery program.
 
    .. audit-event:: os.forkpty "" os.forkpty
 
+   .. versionchanged:: 3.12
+      If Python is able to detect that your process has multiple
+      threads, this now raises a :exc:`DeprecationWarning`. See the
+      longer explanation on :func:`os.fork`.
+
    .. versionchanged:: 3.8
       Calling ``forkpty()`` in a subinterpreter is no longer supported
       (:exc:`RuntimeError` is raised).

diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst
index 4ee87974a98d16f5905d7754cd57a0c21a050b7d..22538a476d31b4a175b72394661d2fe7939893b5 100644 (file)
--- a/Doc/whatsnew/3.12.rst
+++ b/Doc/whatsnew/3.12.rst
@@ -1065,6 +1065,18 @@ Deprecated
   contain the creation time, which is also available in the new ``st_birthtime``
   field. (Contributed by Steve Dower in :gh:`99726`.)
 
+* :mod:`os`: On POSIX platforms, :func:`os.fork` can now raise a
+  :exc:`DeprecationWarning` when it can detect being called from a
+  multithreaded process. There has always been a fundamental incompatibility
+  with the POSIX platform when doing so. Even if such code *appeared* to work.
+  We added the warning to to raise awareness as issues encounted by code doing
+  this are becoming more frequent. See the :func:`os.fork` documentation for
+  more details.
+
+  When this warning appears due to usage of :mod:`multiprocessing` or
+  :mod:`concurrent.futures` the fix is to use a different
+  :mod:`multiprocessing` start method such as ``"spawn"`` or ``"forkserver"``.
+
 * :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed
   in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.)