gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread (gh...

If `Py_IsFinalizing()` is true, non-daemon threads (other than the current one)
are done, and daemon threads are prevented from running, so they
cannot finalize themselves and become done. Joining them (without timeout)
would block forever.

Raise PythonFinalizationError instead of hanging.

Raise even when a timeout is given, for consistency with trying to join your own thread.

See gh-123940 for a use case: calling `join()` from `__del__`. This is
ill-advised, but an exception should at least make it easier to diagnose.

diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
index 3597f35e0a26567c1b48b886f217acadf3b14b98..33e9f87a6061df57886170096075b9e328b1dd9d 100644 (file)
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -1131,7 +1131,7 @@ Cautions regarding runtime finalization
 In the late stage of :term:`interpreter shutdown`, after attempting to wait for
 non-daemon threads to exit (though this can be interrupted by
 :class:`KeyboardInterrupt`) and running the :mod:`atexit` functions, the runtime
-is marked as *finalizing*: :c:func:`_Py_IsFinalizing` and
+is marked as *finalizing*: :c:func:`Py_IsFinalizing` and
 :func:`sys.is_finalizing` return true.  At this point, only the *finalization
 thread* that initiated finalization (typically the main thread) is allowed to
 acquire the :term:`GIL`.

diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
index c56604339030d3b92e2bd9ca87494995955cc1d6..4b391ba020a8bfe55a4ce1fab39a5630f7842803 100644 (file)
--- a/Doc/library/exceptions.rst
+++ b/Doc/library/exceptions.rst
@@ -428,6 +428,7 @@ The following exceptions are the exceptions that are usually raised.
    :exc:`PythonFinalizationError` during the Python finalization:
 
    * Creating a new Python thread.
+   * :meth:`Joining <threading.Thread.join>` a running daemon thread.
    * :func:`os.fork`.
 
    See also the :func:`sys.is_finalizing` function.
@@ -435,6 +436,9 @@ The following exceptions are the exceptions that are usually raised.
    .. versionadded:: 3.13
       Previously, a plain :exc:`RuntimeError` was raised.
 
+   .. versionchanged:: next
+
+      :meth:`threading.Thread.join` can now raise this exception.
 
 .. exception:: RecursionError
 

diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst
index 989c7624448847b818ee63ed6b4376c8b8941048..d948493c2103df06dc745c4b4e612538540df92e 100644 (file)
--- a/Doc/library/threading.rst
+++ b/Doc/library/threading.rst
@@ -448,6 +448,14 @@ since it is impossible to detect the termination of alien threads.
       an error to :meth:`~Thread.join` a thread before it has been started
       and attempts to do so raise the same exception.
 
+      If an attempt is made to join a running daemonic thread in in late stages
+      of :term:`Python finalization <interpreter shutdown>` :meth:`!join`
+      raises a :exc:`PythonFinalizationError`.
+
+      .. versionchanged:: next
+
+         May raise :exc:`PythonFinalizationError`.
+
    .. attribute:: name
 
       A string used for identification purposes only. It has no semantics.

diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py
index fa666608263e27edc993efc1418ee68d65ca628b..b768886362654f87d6b04cf1318d33e3c24d5db0 100644 (file)
--- a/Lib/test/test_threading.py
+++ b/Lib/test/test_threading.py
@@ -1171,6 +1171,77 @@ class ThreadTests(BaseTestCase):
         self.assertEqual(out.strip(), b"OK")
         self.assertIn(b"can't create new thread at interpreter shutdown", err)
 
+    def test_join_daemon_thread_in_finalization(self):
+        # gh-123940: Py_Finalize() prevents other threads from running Python
+        # code, so join() can not succeed unless the thread is already done.
+        # (Non-Python threads, that is `threading._DummyThread`, can't be
+        # joined at all.)
+        # We raise an exception rather than hang.
+        for timeout in (None, 10):
+            with self.subTest(timeout=timeout):
+                code = textwrap.dedent(f"""
+                    import threading
+
+
+                    def loop():
+                        while True:
+                            pass
+
+
+                    class Cycle:
+                        def __init__(self):
+                            self.self_ref = self
+                            self.thr = threading.Thread(
+                                target=loop, daemon=True)
+                            self.thr.start()
+
+                        def __del__(self):
+                            assert self.thr.is_alive()
+                            try:
+                                self.thr.join(timeout={timeout})
+                            except PythonFinalizationError:
+                                assert self.thr.is_alive()
+                                print('got the correct exception!')
+
+                    # Cycle holds a reference to itself, which ensures it is
+                    # cleaned up during the GC that runs after daemon threads
+                    # have been forced to exit during finalization.
+                    Cycle()
+                """)
+                rc, out, err = assert_python_ok("-c", code)
+                self.assertEqual(err, b"")
+                self.assertIn(b"got the correct exception", out)
+
+    def test_join_finished_daemon_thread_in_finalization(self):
+        # (see previous test)
+        # If the thread is already finished, join() succeeds.
+        code = textwrap.dedent("""
+            import threading
+            done = threading.Event()
+
+            def loop():
+                done.set()
+
+
+            class Cycle:
+                def __init__(self):
+                    self.self_ref = self
+                    self.thr = threading.Thread(target=loop, daemon=True)
+                    self.thr.start()
+                    done.wait()
+
+                def __del__(self):
+                    assert not self.thr.is_alive()
+                    self.thr.join()
+                    assert not self.thr.is_alive()
+                    print('all clear!')
+
+            Cycle()
+        """)
+        rc, out, err = assert_python_ok("-c", code)
+        self.assertEqual(err, b"")
+        self.assertIn(b"all clear", out)
+
     def test_start_new_thread_failed(self):
         # gh-109746: if Python fails to start newly created thread
         # due to failure of underlying PyThread_start_new_thread() call,

diff --git a/Misc/NEWS.d/next/Library/2025-02-21-15-46-43.gh-issue-130402.Rwu_KK.rst b/Misc/NEWS.d/next/Library/2025-02-21-15-46-43.gh-issue-130402.Rwu_KK.rst
new file mode 100644 (file)
index 0000000..b91d429
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2025-02-21-15-46-43.gh-issue-130402.Rwu_KK.rst
@@ -0,0 +1,2 @@
+Joining running daemon threads during interpreter shutdown
+now raises :exc:`PythonFinalizationError`.

diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c
index 6967f7ef42f17394790cf15f3808a3e29f56c95c..9776a32755db686a2d88f7297500136eb10d4494 100644 (file)
--- a/Modules/_threadmodule.c
+++ b/Modules/_threadmodule.c
@@ -511,11 +511,21 @@ ThreadHandle_join(ThreadHandle *self, PyTime_t timeout_ns)
     // To work around this, we set `thread_is_exiting` immediately before
     // `thread_run` returns.  We can be sure that we are not attempting to join
     // ourselves if the handle's thread is about to exit.
-    if (!_PyEvent_IsSet(&self->thread_is_exiting) &&
-        ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
-        // PyThread_join_thread() would deadlock or error out.
-        PyErr_SetString(ThreadError, "Cannot join current thread");
-        return -1;
+    if (!_PyEvent_IsSet(&self->thread_is_exiting)) {
+        if (ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
+            // PyThread_join_thread() would deadlock or error out.
+            PyErr_SetString(ThreadError, "Cannot join current thread");
+            return -1;
+        }
+        if (Py_IsFinalizing()) {
+            // gh-123940: On finalization, other threads are prevented from
+            // running Python code. They cannot finalize themselves,
+            // so join() would hang forever (or until timeout).
+            // We raise instead.
+            PyErr_SetString(PyExc_PythonFinalizationError,
+                            "cannot join thread at interpreter shutdown");
+            return -1;
+        }
     }
 
     // Wait until the deadline for the thread to exit.