Serhiy Storchaka [Mon, 18 Nov 2024 11:24:13 +0000 (13:24 +0200)]
[3.12] gh-67877: Fix memory leaks in terminated RE matching (GH-126840) (GH-126961)
If SRE(match) function terminates abruptly, either because of a signal
or because memory allocation fails, allocated SRE_REPEAT blocks might
be never released.
[3.12] gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919) (#126916)
gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919)
The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line. It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet. Clearly, though, this is a case that needs to be handled
correctly. The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.
[3.12] gh-126476: Raise IllegalMonthError for calendar.formatmonth() when the input month is not correct (GH-126484) (GH-126878)
gh-126476: Raise IllegalMonthError for calendar.formatmonth() when the input month is not correct (GH-126484)
(cherry picked from commit 3be7498d2450519d5d8f63a35ef298db3b3d935b)
[3.12] Added a warning to the urljoin docs, indicating that it is not safe to use with attacker controlled URLs (GH-126659) (#126889)
Added a warning to the urljoin docs, indicating that it is not safe to use with attacker controlled URLs (GH-126659)
This was flagged to me at a party today by someone who works in red-teaming as a frequently encountered footgun. Documenting the potentially unexpected behavior seemed like a good place to start.
(cherry picked from commit d6bcc154e93a0a20ab97187d3e8b726fffb14f8f)
Co-authored-by: Alex Gaynor <alex.gaynor@gmail.com>
[3.12] gh-71936: Fix race condition in multiprocessing.Pool (GH-124973) (GH-126870)
Proxes of shared objects register a Finalizer in BaseProxy._incref(), and it
will call BaseProxy._decref() when it is GCed. This may cause a race condition
with Pool(maxtasksperchild=None) on Windows.
A connection would be closed and raised TypeError when a GC occurs between
_ConnectionBase._check_writable() and _ConnectionBase._send_bytes() in
_ConnectionBase.send() in the second or later task, and a new object
is allocated that shares the id() of a previously deleted one.
Instead of using the id() of the token (or the proxy), use a unique,
non-reusable number.
gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance (GH-126182)
* gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance
This changes nothing changes for CPython supported platforms,
but hints how to deal with platforms that stick to the letter of
the spec.
It also marks `socket.getaddrinfo` as a wrapper around `getaddrinfo(3)`;
specifically, workarounds to make the function work consistently across
platforms are out of scope in its code.
Include wording similar to the POSIX's “by providing options and by
limiting the returned information”, which IMO suggests that the
hints limit the resulting list compared to the defaults, *but* can
be interpreted differently. Details are added in a note.
Specifically say that this wraps the underlying C function. So, the
details are in OS docs. The “full range of results” bit goes away.
Use `AF_UNSPEC` rather than zero for the *family* default, although
I don't think a system where it's nonzero would be very usable.
Suggest setting proto and/or type (with examples, as the appropriate
values aren't obvious). Say why you probably want to do that that
on all systems; mention the behavior on the “letter of the spec”
systems.
Suggest that the results should be tried in order, which is,
AFAIK best practice -- see RFC 6724 section 2, and its predecessor
from 2003 (which are specific to IP, but indicate how people use this):
> Well-behaved applications SHOULD iterate through the list of
> addresses returned from `getaddrinfo()` until they find a working address.
Discard two leading slashes from the beginning of a `file:` URI if they
introduce an empty authority section. As a result, file URIs like
`///etc/hosts` are correctly parsed as `/etc/hosts`.
(cherry picked from commit cae9d9d20f61cdbde0765efa340b6b596c31b67f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Co-authored-by: John Marshall <jmarshall@hey.com> Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
[3.12] GH-120423: `pathname2url()`: handle forward slashes in Windows paths (GH-126593) (#126763)
GH-120423: `pathname2url()`: handle forward slashes in Windows paths (GH-126593)
Adjust `urllib.request.pathname2url()` so that forward slashes in Windows
paths are handled identically to backward slashes.
(cherry picked from commit bf224bd7cef5d24eaff35945ebe7ffe14df7710f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.12] gh-126505: Fix bugs in compiling case-insensitive character classes (GH-126557) (GH-126690)
* upper-case non-BMP character was ignored
* the ASCII flag was ignored when matching a character range whose
upper bound is beyond the BMP region
(cherry picked from commit 819830f34a11ecaa3aada174ca8eedeb3f260630)
[3.12] gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503) (GH-126572)
gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503)
If `read()` in the ConnectionHandler thread raises `OSError` (except `ConnectionError`),
the ConnectionHandler shuts down the entire ThreadedEchoServer,
preventing further connections.
It also does that for `EPROTOTYPE` in `wrap_conn`.
As far as I can see, this is done to avoid the server thread getting stuck,
forgotten, in its accept loop. However, since 2011 (5b95eb90a7167285b6544b50865227c584943c9a)
the server is used as a context manager, and its `__exit__` does `stop()` and `join()`.
(I'm not sure if we *always* used `with` since that commit, but currently we do.)
Make sure that the context manager *is* used, and remove the `server.stop()`
calls from ConnectionHandler.
(cherry picked from commit c9cda1608edf7664c10f4f467e24591062c2fe62)
[3.12] gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501) (GH-126573)
gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501)
The skipping machinery called `getattr(err, "reason", "")` on an arbitrary
exception. As intermittent Buildbot failures show, sometimes it's set
to None.
[3.12] gh-126664: Use `else` instead of `finally` in "The with statement" documentation. (GH-126665) (#126671)
gh-126664: Use `else` instead of `finally` in "The with statement" documentation. (GH-126665)
(cherry picked from commit 25257d61cfccc3b4189f96390a5c4db73fd5302c)
[3.12] gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635) (GH-126653)
gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635)
The first version had it running two forkserver and one spawn tests underneath each of the _fork, _forkserver, and _spawn test suites that build off the generic one.
This adds to the existing complexity of the multiprocessing test suite by offering BaseTestCase classes another attribute to control which suites they are invoked under. Practicality vs purity here. :/
[3.12] Skip test in test_socket.py if `sys.getrefcount` isn't available (GH-126640) (#126646)
Skip test in test_socket.py if `sys.getrefcount` isn't available (GH-126640)
Skip `testMakefileCloseSocketDestroy` test if `sys.getrefcount` isn't available. This is necessary for PyPy and other Python implementations that do not have `sys.getrefcount`.
(cherry picked from commit 0f6bb28ff3ba152faf7523ea9aaf0094cc39bdda)
[3.12] gh-126565: Skip `zipfile.Path.exists` check in write mode (GH-126576) (#126643)
gh-126565: Skip `zipfile.Path.exists` check in write mode (GH-126576)
When `zipfile.Path.open` is called, the implementation will check
whether the path already exists in the ZIP file. However, this check is
only required when the ZIP file is in read mode. By swapping arguments
of the `and` operator, the short-circuiting will prevent the check from
being run in write mode.
This change will improve the performance of `open()`, because checking
whether a file exists is slow in write mode, especially when the archive
has many members.
(cherry picked from commit 160758a574d12bf0d965d8206136e7da4f4fd6c3)
Co-authored-by: Jan Hicken <janhicken@users.noreply.github.com>
`sys.path` was not properly being sent from the parent process when launching
the multiprocessing forkserver process to preload imports. This bug has been
there since the forkserver start method was introduced in Python 3.4. It was
always _supposed_ to inherit `sys.path` the same way the spawn method does.
Observable behavior change: A `''` value in `sys.path` will now be replaced in
the forkserver's `sys.path` with an absolute pathname
`os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was
imported in the parent process as it already was when using the spawn start
method. **This will only be observable during forkserver preload imports**.
The code invoked before calling things in another process already correctly sets `sys.path`.
Which is likely why this went unnoticed for so long as a mere performance issue in
some configurations.
A workaround for the bug on impacted Pythons is to set PYTHONPATH in the
environment before multiprocessing's forkserver process was started. Not perfect
as that is then inherited by other children, etc, but likely good enough for many
people's purposes.
[3.12] GH-126212: Fix removal of slashes in file URIs on Windows (GH-126214) (#126591)
GH-126212: Fix removal of slashes in file URIs on Windows (GH-126214)
Adjust `urllib.request.pathname2url()` and `url2pathname()` so that they
don't remove slashes from Windows DOS drive paths and URLs. There was no
basis for this behaviour, and it conflicts with how UNC and POSIX paths are
handled.
(cherry picked from commit 54c63a32d06cb5f07a66245c375eac7d7efb964a)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.12] Doc: C API: Demote sections to subsections for consistency (GH-126535) (#126545)
Doc: C API: Demote sections to subsections for consistency (GH-126535)
The entire file should be a single section; the headings below the
first heading should be subsections.
(cherry picked from commit e3510bd3dd9ea8f2a30cb1128470aee3a48d8880)
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Stephen Morton <git@tungol.org> Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
[3.12] gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (GH-120143) (#126460)
gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (GH-120143)
(cherry picked from commit 83ba8c2bba834c0b92de669cac16fcda17485e0e)
- Explicit memory management for `None` objects (since we still try to treat immortal objects as regular objects)
- Respect possible errors of `sys.monitoring.register_callback` call
(cherry picked from commit 75872605aa78dbdfc5c4f025b0f90a7f37ba10c3)
[3.12] gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126) (GH-126276)
gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126)
(cherry picked from commit 6c67446a6e73ab0e9a26e4360412cbd2f5550e66)
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com>
Co-authored-by: Zhikang Yan <2951256653@qq.com> Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
[3.12] docs: add a more precise example in enum doc (GH-121015) (#126307)
docs: add a more precise example in enum doc (GH-121015)
* docs: add a more precise example
Previous example used manual integer value assignment in class based declaration but in functional syntax has been used auto value assignment what could be confusing for the new users. Additionally documentation doesn't show how to declare new enum via functional syntax with usage of the manual value assignment.
[3.12] gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (GH-126271) (#126311)
gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (GH-126271)
(cherry picked from commit 28b148fb32e4548b461137d18d1ab6d366395d36)
Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
[3.12] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126251)
gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120)
(cherry picked from commit 0e8665554b2f1334e530fd6de5b3a4e908405419)
[3.12] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126230)
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103)
(cherry picked from commit d07dcce6935364cab807e0df931ed09b088ade69)
[3.12] Prefer "similar" over "equivalent" in tutorial (GH-125343) (GH-125373)
Prefer "similar" over "equivalent" in tutorial (GH-125343)
In the datastructures tutorial doc, some operations are described as
"equivalent to" others. This has led to some user-confusion -- at
least in the Discourse forums -- about cases in which the operations
differ.
This change doesn't systematically eliminate the word "equivalent"
from the tutorial. It just substitutes "similar to" in several cases
in which "equivalent to" could mislead users into expecting exact
equivalence.
(cherry picked from commit 4a2282b0679bbf7b7fbd36aae1b1565145238961)
Co-authored-by: Stephen Rosen <sirosen@globus.org>
Serhiy Storchaka [Wed, 30 Oct 2024 09:10:10 +0000 (11:10 +0200)]
[3.12] gh-126071: Improve formatting of the argparse documentation (GH-126073) (GH-126174)
* Use appropriate roles for ArgumentParser, Action, etc.
* Remove superfluous repeated links.
* Explicitly document signatures and add index entries for some methods
and classes.
* Make it more clear that some parameters are keyword-only.
* Fix some minor errors.
(cherry picked from commit 2ab377a47c8290f8bf52c8ffb5d7fc4c45452611)
[3.12] gh-116938: Fix `dict.update` docstring and remove erraneous full stop from `dict` documentation (GH-125421) (#126151)
gh-116938: Fix `dict.update` docstring and remove erraneous full stop from `dict` documentation (GH-125421)
(cherry picked from commit 5527c4051c0b58218ce69044f92b45f1d66ed43f)
Co-authored-by: Prometheus3375 <35541026+Prometheus3375@users.noreply.github.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
[3.12] GH-125866: Improve tests for `pathname2url()` and `url2pathname()` (GH-125993) (#126145)
GH-125866: Improve tests for `pathname2url()` and `url2pathname()` (GH-125993)
Merge `URL2PathNameTests` and `PathName2URLTests` test cases (which test
only the Windows-specific implementations from `nturl2path`) into the main
`Pathname_Tests` test case for these functions.
Co-authored-by: Carol Willing <carolcode@willingconsulting.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
[3.12] gh-126105: Fix crash in `ast` module, when `._fields` is delet… (#126132)
[3.12] gh-126105: Fix crash in `ast` module, when `._fields` is deleted (GH-126115)
Previously, if the `ast.AST._fields` attribute was deleted, attempts to create a new `as`t node would crash due to the assumption that `_fields` always had a non-NULL value. Now it has been fixed by adding an extra check to ensure that `_fields` does not have a NULL value (this can happen when you manually remove `_fields` attribute).
(cherry picked from commit b2eaa75b176e07730215d76d8dce4d63fb493391)
Also: Add test_ctypes/_support.py from 3.13+
This partially backports be89ee5649031e08f191bf596fa20a09c5698079
(https://github.com/python/cpython/pull/113727)
by AN Long
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com> Co-authored-by: AN Long <aisk@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
projects / thirdparty / Python / cpython.git / log
