bpo-39511: PyThreadState_Clear() calls on_delete (GH-18296)

PyThreadState.on_delete is a callback used to notify Python when a
thread completes. _thread._set_sentinel() function creates a lock
which is released when the thread completes. It sets on_delete
callback to the internal release_sentinel() function. This lock is
known as Threading._tstate_lock in the threading module.

The release_sentinel() function uses the Python C API. The problem is
that on_delete is called late in the Python finalization, when the C
API is no longer fully working.

The PyThreadState_Clear() function now calls the
PyThreadState.on_delete callback. Previously, that happened in
PyThreadState_Delete().

The release_sentinel() function is now called when the C API is still
fully working.

diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
index 7ea48aec009c927f1ddba09c9edc43d891635014..14049ee64205f12a61e60edd6b7aa432e66491f5 100644 (file)
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -1048,6 +1048,10 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
    Reset all information in a thread state object.  The global interpreter lock
    must be held.
 
+   .. versionchanged:: 3.9
+      This function now calls the :c:member:`PyThreadState.on_delete` callback.
+      Previously, that happened in :c:func:`PyThreadState_Delete`.
+
 
 .. c:function:: void PyThreadState_Delete(PyThreadState *tstate)
 

diff --git a/Misc/NEWS.d/next/C API/2020-01-31-16-35-21.bpo-39511.nv9yEn.rst b/Misc/NEWS.d/next/C API/2020-01-31-16-35-21.bpo-39511.nv9yEn.rst
new file mode 100644 (file)
index 0000000..14a0487
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2020-01-31-16-35-21.bpo-39511.nv9yEn.rst
@@ -0,0 +1,3 @@
+The :c:func:`PyThreadState_Clear` function now calls the
+:c:member:`PyThreadState.on_delete` callback. Previously, that happened in
+:c:func:`PyThreadState_Delete`.

diff --git a/Python/pystate.c b/Python/pystate.c
index d792380de464989c1fc10feda549df65e2411cfb..ebc17ea5a721930f65d9ee246ed6ab67946da935 100644 (file)
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -806,6 +806,10 @@ PyThreadState_Clear(PyThreadState *tstate)
     Py_CLEAR(tstate->async_gen_finalizer);
 
     Py_CLEAR(tstate->context);
+
+    if (tstate->on_delete != NULL) {
+        tstate->on_delete(tstate->on_delete_data);
+    }
 }
 
 
@@ -830,9 +834,7 @@ tstate_delete_common(PyThreadState *tstate,
     if (tstate->next)
         tstate->next->prev = tstate->prev;
     HEAD_UNLOCK(runtime);
-    if (tstate->on_delete != NULL) {
-        tstate->on_delete(tstate->on_delete_data);
-    }
+
     PyMem_RawFree(tstate);
 
     if (gilstate->autoInterpreterState &&