]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
gh-109047: concurrent.futures catches PythonFinalizationError (#109810)
authorVictor Stinner <vstinner@python.org>
Fri, 29 Sep 2023 19:31:19 +0000 (21:31 +0200)
committerGitHub <noreply@github.com>
Fri, 29 Sep 2023 19:31:19 +0000 (19:31 +0000)
commit635184212179b0511768ea1cd57256e134ba2d75
tree7755f29d577b5db379d9027cffd8363d1926fad6
parentf3df8fa669158f89af69b5661e98314d98fb916f
gh-109047: concurrent.futures catches PythonFinalizationError (#109810)

concurrent.futures: The *executor manager thread* now catches
exceptions when adding an item to the *call queue*. During Python
finalization, creating a new thread can now raise RuntimeError. Catch
the exception and call terminate_broken() in this case.

Add test_python_finalization_error() to test_concurrent_futures.

concurrent.futures._ExecutorManagerThread changes:

* terminate_broken() no longer calls shutdown_workers() since the
  call queue is no longer working anymore (read and write ends of
  the queue pipe are closed).
* terminate_broken() now terminates child processes, not only
  wait until they complete.
* _ExecutorManagerThread.terminate_broken() now holds shutdown_lock
  to prevent race conditons with ProcessPoolExecutor.submit().

multiprocessing.Queue changes:

* Add _terminate_broken() method.
* _start_thread() sets _thread to None on exception to prevent
  leaking "dangling threads" even if the thread was not started
  yet.
Lib/concurrent/futures/process.py
Lib/multiprocessing/queues.py
Lib/test/test_concurrent_futures/test_process_pool.py
Misc/NEWS.d/next/Library/2023-09-25-02-11-14.gh-issue-109047.b1TrqG.rst [new file with mode: 0644]