[3.13] gh-71936: Fix race condition in multiprocessing.Pool (GH-124973) (GH-126869)
Proxes of shared objects register a Finalizer in BaseProxy._incref(), and it
will call BaseProxy._decref() when it is GCed. This may cause a race condition
with Pool(maxtasksperchild=None) on Windows.
A connection would be closed and raised TypeError when a GC occurs between
_ConnectionBase._check_writable() and _ConnectionBase._send_bytes() in
_ConnectionBase.send() in the second or later task, and a new object
is allocated that shares the id() of a previously deleted one.
Instead of using the id() of the token (or the proxy), use a unique,
non-reusable number.
gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance (GH-126182)
* gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance
This changes nothing changes for CPython supported platforms,
but hints how to deal with platforms that stick to the letter of
the spec.
It also marks `socket.getaddrinfo` as a wrapper around `getaddrinfo(3)`;
specifically, workarounds to make the function work consistently across
platforms are out of scope in its code.
Include wording similar to the POSIX's “by providing options and by
limiting the returned information”, which IMO suggests that the
hints limit the resulting list compared to the defaults, *but* can
be interpreted differently. Details are added in a note.
Specifically say that this wraps the underlying C function. So, the
details are in OS docs. The “full range of results” bit goes away.
Use `AF_UNSPEC` rather than zero for the *family* default, although
I don't think a system where it's nonzero would be very usable.
Suggest setting proto and/or type (with examples, as the appropriate
values aren't obvious). Say why you probably want to do that that
on all systems; mention the behavior on the “letter of the spec”
systems.
Suggest that the results should be tried in order, which is,
AFAIK best practice -- see RFC 6724 section 2, and its predecessor
from 2003 (which are specific to IP, but indicate how people use this):
> Well-behaved applications SHOULD iterate through the list of
> addresses returned from `getaddrinfo()` until they find a working address.
Discard two leading slashes from the beginning of a `file:` URI if they
introduce an empty authority section. As a result, file URIs like
`///etc/hosts` are correctly parsed as `/etc/hosts`.
(cherry picked from commit cae9d9d20f61cdbde0765efa340b6b596c31b67f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.13] gh-126433: Change channel_info.count to int64_t (GH-126447) (#126826)
gh-126433: Change channel_info.count to int64_t (GH-126447)
Fix compiler warnings on 32-bit Windows: change channel_info.count
type from Py_ssize_t to int64_t in _interpchannelsmodule.c.
(cherry picked from commit 09d7083962062acfef7e7a9a309a01fb70ad8276)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Marshall <jmarshall@hey.com> Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
One of the most common reasons I see the old `pipes` module still in use
when porting to Python 3.13 is for the undocumented `pipes.quote`
function, which can easily be replaced with `shlex.quote`. I think it's
worth specifically calling this out, since being directed to the
`subprocess` module would be confusing in this case.
(cherry picked from commit 73e34b680852794d110cd806505b3d74d9d593db)
Co-authored-by: Colin Watson <cjwatson@debian.org>
Barney Gale [Wed, 13 Nov 2024 23:32:56 +0000 (23:32 +0000)]
[3.13] GH-118289: Fix handling of non-directories in `posixpath.realpath()` (GH-120127) (#126815)
In strict mode, raise `NotADirectoryError` if we encounter a non-directory
while we still have path parts left to process.
We use a `part_count` variable rather than `len(rest)` because the `rest`
stack also contains markers for unresolved symlinks.
(cherry picked from commit fd4b5453df74e249987553b12c14ad75fafa4991)
[3.13] gh-126688: Reinit import lock after fork (GH-126692) (GH-126765)
The PyMutex implementation supports unlocking after fork because we
clear the list of waiters in parking_lot.c. This doesn't work as well
for _PyRecursiveMutex because on some systems, such as SerenityOS, the
thread id is not preserved across fork().
(cherry picked from commit 5610860840aa71b186fc5639211dd268b817d65f)
[3.13] GH-120423: `pathname2url()`: handle forward slashes in Windows paths (GH-126593) (#126764)
GH-120423: `pathname2url()`: handle forward slashes in Windows paths (GH-126593)
Adjust `urllib.request.pathname2url()` so that forward slashes in Windows
paths are handled identically to backward slashes.
(cherry picked from commit bf224bd7cef5d24eaff35945ebe7ffe14df7710f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.13] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) (gh-125709) (GH-125204)
* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb39947093feda3ff85b8dc820922cc5e5f954)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)
They used to be shared, before 3.12. Returning to sharing them resolves a failure on Py_TRACE_REFS builds.
---------
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
[3.13] gh-126505: Fix bugs in compiling case-insensitive character classes (GH-126557) (GH-126689)
* upper-case non-BMP character was ignored
* the ASCII flag was ignored when matching a character range whose
upper bound is beyond the BMP region
(cherry picked from commit 819830f34a11ecaa3aada174ca8eedeb3f260630)
[3.13] gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503) (GH-126571)
gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503)
If `read()` in the ConnectionHandler thread raises `OSError` (except `ConnectionError`),
the ConnectionHandler shuts down the entire ThreadedEchoServer,
preventing further connections.
It also does that for `EPROTOTYPE` in `wrap_conn`.
As far as I can see, this is done to avoid the server thread getting stuck,
forgotten, in its accept loop. However, since 2011 (5b95eb90a7167285b6544b50865227c584943c9a)
the server is used as a context manager, and its `__exit__` does `stop()` and `join()`.
(I'm not sure if we *always* used `with` since that commit, but currently we do.)
Make sure that the context manager *is* used, and remove the `server.stop()`
calls from ConnectionHandler.
(cherry picked from commit c9cda1608edf7664c10f4f467e24591062c2fe62)
[3.13] gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501) (GH-126574)
gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501)
The skipping machinery called `getattr(err, "reason", "")` on an arbitrary
exception. As intermittent Buildbot failures show, sometimes it's set
to None.
[3.13] gh-126664: Use `else` instead of `finally` in "The with statement" documentation. (GH-126665) (#126670)
gh-126664: Use `else` instead of `finally` in "The with statement" documentation. (GH-126665)
(cherry picked from commit 25257d61cfccc3b4189f96390a5c4db73fd5302c)
[3.13] gh-126647: `Doc/using/configure.rst`: Add an entry for ``--enable-experimental-jit`` option (GH-126648) (#126655)
gh-126647: `Doc/using/configure.rst`: Add an entry for ``--enable-experimental-jit`` option (GH-126648)
Add an entry for the ``--enable-experimental-jit`` option in ``Doc/using/configure.rst``.
This was added as an experimental option in CPython 3.13.
Possible values for it:
* `no` - don't build the JIT.
* `yes` - build the JIT.
* `yes-off` - build the JIT but disable it by default.
* `interpreter` - don't build the JIT but enable tier 2 interpreter instead.
[3.13] gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635) (GH-126652)
gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635)
The first version had it running two forkserver and one spawn tests underneath each of the _fork, _forkserver, and _spawn test suites that build off the generic one.
This adds to the existing complexity of the multiprocessing test suite by offering BaseTestCase classes another attribute to control which suites they are invoked under. Practicality vs purity here. :/
[3.13] Skip test in test_socket.py if `sys.getrefcount` isn't available (GH-126640) (#126645)
Skip test in test_socket.py if `sys.getrefcount` isn't available (GH-126640)
Skip `testMakefileCloseSocketDestroy` test if `sys.getrefcount` isn't available. This is necessary for PyPy and other Python implementations that do not have `sys.getrefcount`.
(cherry picked from commit 0f6bb28ff3ba152faf7523ea9aaf0094cc39bdda)
[3.13] gh-126565: Skip `zipfile.Path.exists` check in write mode (GH-126576) (#126642)
gh-126565: Skip `zipfile.Path.exists` check in write mode (GH-126576)
When `zipfile.Path.open` is called, the implementation will check
whether the path already exists in the ZIP file. However, this check is
only required when the ZIP file is in read mode. By swapping arguments
of the `and` operator, the short-circuiting will prevent the check from
being run in write mode.
This change will improve the performance of `open()`, because checking
whether a file exists is slow in write mode, especially when the archive
has many members.
(cherry picked from commit 160758a574d12bf0d965d8206136e7da4f4fd6c3)
Co-authored-by: Jan Hicken <janhicken@users.noreply.github.com>
`sys.path` was not properly being sent from the parent process when launching
the multiprocessing forkserver process to preload imports. This bug has been
there since the forkserver start method was introduced in Python 3.4. It was
always _supposed_ to inherit `sys.path` the same way the spawn method does.
Observable behavior change: A `''` value in `sys.path` will now be replaced in
the forkserver's `sys.path` with an absolute pathname
`os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was
imported in the parent process as it already was when using the spawn start
method. **This will only be observable during forkserver preload imports**.
The code invoked before calling things in another process already correctly sets `sys.path`.
Which is likely why this went unnoticed for so long as a mere performance issue in
some configurations.
A workaround for the bug on impacted Pythons is to set PYTHONPATH in the
environment before multiprocessing's forkserver process was started. Not perfect
as that is then inherited by other children, etc, but likely good enough for many
people's purposes.
[3.13] GH-126212: Fix removal of slashes in file URIs on Windows (GH-126214) (#126590)
GH-126212: Fix removal of slashes in file URIs on Windows (GH-126214)
Adjust `urllib.request.pathname2url()` and `url2pathname()` so that they
don't remove slashes from Windows DOS drive paths and URLs. There was no
basis for this behaviour, and it conflicts with how UNC and POSIX paths are
handled.
(cherry picked from commit 54c63a32d06cb5f07a66245c375eac7d7efb964a)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.13] Doc: C API: Demote sections to subsections for consistency (GH-126535) (#126546)
Doc: C API: Demote sections to subsections for consistency (GH-126535)
The entire file should be a single section; the headings below the
first heading should be subsections.
(cherry picked from commit e3510bd3dd9ea8f2a30cb1128470aee3a48d8880)
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
[3.13] gh-48020: [docs] Remove the logging howto suggested future FileHandler multiprocessing support (GH-126531) (#126540)
gh-48020: [docs] Remove the logging howto suggested future FileHandler multiprocessing support (GH-126531)
Docs: Remove the logging howto potential promise of multiprocessing support in the future.
Stick to the facts and suggestions, don't provide hope where we're not going to
implement complexity that we'd rather the user implement themselves when
needed.
(cherry picked from commit 1fe67df8e373a5177143e4a310c83438e79f9b77)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
[3.13] gh-125631: Enable setting persistent_id and persistent_load of pickler and unpickler (GH-125752) (GH-126528)
pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can
again be overridden as instance attributes.
(cherry picked from commit 223d3dc554dde45f185f7f465753824c6f698b9b)
Co-authored-by: Stephen Morton <git@tungol.org> Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
[3.13] gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (GH-120143) (#126459)
gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (GH-120143)
(cherry picked from commit 83ba8c2bba834c0b92de669cac16fcda17485e0e)
- Explicit memory management for `None` objects (since we still try to treat immortal objects as regular objects)
- Respect possible errors of `sys.monitoring.register_callback` call
(cherry picked from commit 75872605aa78dbdfc5c4f025b0f90a7f37ba10c3)
gh-125436: Doc: Add missing ``allow_unnamed_section`` parameter to ``ConfigParser`` documentation (GH-125437)
Add missing ``allow_unnamed_section`` parameter to ``ConfigParser`` doc, as well as to it's parent ``RawConfigParser``.
Split too long line on ``ConfigParser`` signature.
Add some sections about when some of ``RawConfigParser`` parameters were added.
(cherry picked from commit d9602265479bcd96dc377d92a34556baf34ac3cd)
[3.13] gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126) (GH-126275)
gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126)
(cherry picked from commit 6c67446a6e73ab0e9a26e4360412cbd2f5550e66)
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com>
Co-authored-by: Zhikang Yan <2951256653@qq.com> Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Donghee Na [Sun, 3 Nov 2024 04:10:15 +0000 (13:10 +0900)]
[3.13] gh-125832: Clarify comment for inlined comprehensions as per PEP-709 (gh-126322) (gh-126344)
* gh-125832: Clarify comment for inlined comprehensions as per PEP-709 (#126322)
* Fix comprehensions comment to inlined by pep 709
* Update spacing
Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
* Add reference to PEP 709
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com> Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
(cherry picked from commit 868bfcc02ed42a1042851830b79c6877b7f1c7a8)
[3.13] docs: add a more precise example in enum doc (GH-121015) (#126306)
docs: add a more precise example in enum doc (GH-121015)
* docs: add a more precise example
Previous example used manual integer value assignment in class based declaration but in functional syntax has been used auto value assignment what could be confusing for the new users. Additionally documentation doesn't show how to declare new enum via functional syntax with usage of the manual value assignment.
[3.13] gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (GH-126271) (#126310)
gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (GH-126271)
(cherry picked from commit 28b148fb32e4548b461137d18d1ab6d366395d36)
Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
[3.13] gh-126256: Update time.rst to use the same clock as instead of the same clock than (GH-126257) (#126258)
gh-126256: Update time.rst to use the same clock as instead of the same clock than (GH-126257)
Update time.rst to use `the same clock as` instead of `the same clock than`
The time documentation uses the same clock than time.monotonic instead of the same clock as time.monotonic, which is grammatically false. This PR fixes changes two instances of `the same clock than` to `the same clock as`.
(cherry picked from commit d0abd0b826cfa574d1515c6f8459c9901939388f)
[3.13] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126250)
gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120)
(cherry picked from commit 0e8665554b2f1334e530fd6de5b3a4e908405419)
[3.13] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126229)
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103)
(cherry picked from commit d07dcce6935364cab807e0df931ed09b088ade69)
* Add definitions for "context", "current context", and "context
management protocol".
* Update related definitions to be consistent with the new
definitions.
* Restructure the documentation for the `contextvars.Context` class
to prepare for adding context manager support, and for consistency
with the definitions.
* Use `testcode` and `testoutput` to test the `Context.run` example.
[3.13] Prefer "similar" over "equivalent" in tutorial (GH-125343) (GH-125372)
Prefer "similar" over "equivalent" in tutorial (GH-125343)
In the datastructures tutorial doc, some operations are described as
"equivalent to" others. This has led to some user-confusion -- at
least in the Discourse forums -- about cases in which the operations
differ.
This change doesn't systematically eliminate the word "equivalent"
from the tutorial. It just substitutes "similar to" in several cases
in which "equivalent to" could mislead users into expecting exact
equivalence.
(cherry picked from commit 4a2282b0679bbf7b7fbd36aae1b1565145238961)
Co-authored-by: Stephen Rosen <sirosen@globus.org>
projects / thirdparty / Python / cpython.git / log
