]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure...
authorThomas Moreau <thomas.moreau.2010@gmail.com>
Sun, 16 Feb 2020 18:09:26 +0000 (19:09 +0100)
committerGitHub <noreply@github.com>
Sun, 16 Feb 2020 18:09:26 +0000 (10:09 -0800)
commita5cbab552d294d99fde864306632d7e511a75d3c
tree3f12da3e9bc19c5ae9e836a6694d90cc9ddd35d6
parent1ed61617a4a6632905ad6a0b440cd2cafb8b6414
bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)

As reported initially by @rad-pat in #6084, the following script causes a deadlock.

```
from concurrent.futures import ProcessPoolExecutor

class ObjectWithPickleError():
    """Triggers a RuntimeError when sending job to the workers"""

    def __reduce__(self):
        raise RuntimeError()

if __name__ == "__main__":
    e = ProcessPoolExecutor()
    f = e.submit(id, ObjectWithPickleError())
    e.shutdown(wait=False)
    f.result()  # Deadlock on get
```

This is caused by the fact that the main process is closing communication channels that might be necessary to the `queue_management_thread` later. To avoid this, this PR let the `queue_management_thread` manage all the closing.

https://bugs.python.org/issue39104

Automerge-Triggered-By: @pitrou
Lib/concurrent/futures/process.py
Lib/test/test_concurrent_futures.py
Misc/NEWS.d/next/Library/2020-02-16-18-49-16.bpo-39104.cI5MJY.rst [new file with mode: 0644]