Don't make the assumption that the 'name' argument is a string. Use
repr() to format the 'name' argument instead.
(cherry picked from commit 20657fbdb14d50ca4ec115da0cbef155871d8d33)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.13] gh-124008: Fix calculation of the number of written bytes for the Windows console (GH-124059) (GH-127325)
Since MultiByteToWideChar()/WideCharToMultiByte() is not reversible if
the data contains invalid UTF-8 sequences, use binary search to
calculate the number of written bytes from the number of written
characters.
Co-authored-by: Илья Любавский <100635212+lubaskinc0de@users.noreply.github.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Tomas R. <tomas.roun8@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
[3.13] gh-127208: Reject null character in _imp.create_dynamic() (GH-127400) (#127418)
gh-127208: Reject null character in _imp.create_dynamic() (GH-127400)
_imp.create_dynamic() now rejects embedded null characters in the
path and in the module name.
(cherry picked from commit b14fdadc6c620875a20b7ccc3c9b069e85d8557a)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.13] Improve `pathname2url()` and `url2pathname()` docs (GH-127125) (#127232)
Improve `pathname2url()` and `url2pathname()` docs (GH-127125)
These functions have long sown confusion among Python developers. The
existing documentation says they deal with URL path components, but that
doesn't fit the evidence on Windows:
>>> pathname2url(r'C:\foo')
'///C:/foo'
>>> pathname2url(r'\\server\share')
'////server/share' # or '//server/share' as of quite recently
If these were URL path components, they would imply complete URLs like
`file://///C:/foo` and `file://////server/share`. Clearly this isn't right.
Yet the implementation in `nturl2path` is deliberate, and the
`url2pathname()` function correctly inverts it.
On non-Windows platforms, the behaviour until quite recently is to simply
quote/unquote the path without adding or removing any leading slashes. This
behaviour is compatible with *both* interpretations -- 1) the value is a
URL path component (existing docs), and 2) the value is everything
following `file:` (this commit)
The conclusion I draw is that these functions operate on everything after
the `file:` prefix, which may include an authority section. This is the
only explanation that fits both the Windows and non-Windows behaviour.
It's also a better match for the function names.
(cherry picked from commit 307c63358681d669ae39e5ecd814bded4a93443a)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Łukasz Langa [Sat, 23 Nov 2024 09:40:25 +0000 (10:40 +0100)]
[3.13] Enable aarch64 Ubuntu CI jobs (GH-125786) (GH-127170)
This change enables custom GHA runners for Ubuntu-24.04 that run on Arm hardware. It also prepares for Windows runners on Arm hardware, but doesn't enable that just yet, because the Arm GHA runner images for Windows need to be updated.
(cherry picked from commit 8214e0f709010a0e1fa06dc2ce004b5f6103cc6b)
Serhiy Storchaka [Fri, 22 Nov 2024 19:55:44 +0000 (21:55 +0200)]
[3.13] gh-109746: Make _thread.start_new_thread delete state of new thread on its startup failure (GH-109761) (GH-127171)
If Python fails to start newly created thread
due to failure of underlying PyThread_start_new_thread() call,
its state should be removed from interpreter' thread states list
to avoid its double cleanup.
[3.13] gh-88110: Clear concurrent.futures.thread._threads_queues after fork to avoid joining parent process' threads (GH-126098) (GH-127163)
Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.
[3.13] gh-127001: Fix PATHEXT issues in shutil.which() on Windows (GH-127035) (GH-127156)
* Name without a PATHEXT extension is only searched if the mode does not
include X_OK.
* Support multi-component PATHEXT extensions (e.g. ".foo.bar").
* Support files without extensions in PATHEXT contains dot-only extension
(".", "..", etc).
* Support PATHEXT extensions that end with a dot (e.g. ".foo.").
(cherry picked from commit 8899e85de100557899da05f0b37867a371a73800)
[3.13] gh-127065: Make `methodcaller` thread-safe in free threading build (GH-127109) (GH-127150)
The `methodcaller` C vectorcall implementation uses an arguments array
that is shared across calls. The first argument is modified on every
invocation. This isn't thread-safe in the free threading build. I think
it's also not safe in general, but for now just disable it in the free
threading build.
(cherry picked from commit f83ca6962af973fff6a3124f4bd3d45fea4dd5b8)
[3.13] GH-127078: `url2pathname()`: handle extra slash before UNC drive in URL path (GH-127132) (#127135)
GH-127078: `url2pathname()`: handle extra slash before UNC drive in URL path (GH-127132)
Decode a file URI like `file://///server/share` as a UNC path like
`\\server\share`. This form of file URI is created by software the simply
prepends `file:///` to any absolute Windows path.
(cherry picked from commit 8c98ed846a7d7e50c4cf06f823d94737144dcf6a)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Discard any 'localhost' authority from the beginning of a `file:` URI. As a
result, file URIs like `//localhost/etc/hosts` are correctly decoded as
`/etc/hosts`.
(cherry picked from commit ebf564a1d3e2e81b9846535114e481d6096443d2)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Eric Snow [Thu, 21 Nov 2024 17:33:15 +0000 (10:33 -0700)]
[3.13] gh-126986: Stop Using _PyInterpreterState_FailIfNotRunning() (gh-127112)
This is a pseudo-backport of d6b3e78 (gh-126988). In that change for 3.14+, we dropped _PyInterpreterState_FailIfNotRunning() and added _PyErr_SetInterpreterAlreadyRunning(). Here, we replace usage of _PyInterpreterState_FailIfNotRunning() with the inlined equivalent of _PyErr_SetInterpreterAlreadyRunning(), without adding that function. That way we avoid changing the 3.13 ABI.
[3.13] gh-127020: Make `PyCode_GetCode` thread-safe for free threading (GH-127043) (GH-127107)
Some fields in PyCodeObject are lazily initialized. Use atomics and
critical sections to make their initializations and accesses thread-safe.
(cherry picked from commit 3926842117feffe5d2c9727e1899bea5ae2adb28)
It now returns multiple era description segments separated by semicolons.
Previously it only returned the first segment on platforms with Glibc.
(cherry picked from commit 4803cd0244847f286641c85591fda08b513cea52)
[3.13] gh-126997: Fix support of non-ASCII strings in pickletools (GH-127062) (GH-127094)
* Fix support of STRING and GLOBAL opcodes with non-ASCII arguments.
* dis() now outputs non-ASCII bytes in STRING, BINSTRING and
SHORT_BINSTRING arguments as escaped (\xXX).
(cherry picked from commit eaf217108226633c03cc5c4c90f0b6e4587c8803)
[3.13] GH-85168: Use filesystem encoding when converting to/from `file` URIs (GH-126852) (#127039)
GH-85168: Use filesystem encoding when converting to/from `file` URIs (GH-126852)
Adjust `urllib.request.url2pathname()` and `pathname2url()` to use the
filesystem encoding when quoting and unquoting file URIs, rather than
forcing use of UTF-8.
No changes are needed in the `nturl2path` module because Windows always
uses UTF-8, per PEP 529.
(cherry picked from commit c9b399fbdb01584dcfff0d7f6ad484644ff269c3)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
If PyObject_SetItem() fails in the `load_build()` function of _pickle.c, no DECREF for the `dict` variable.
(cherry picked from commit 29cbcbd73bbfd8c953c0b213fb33682c289934ff)
Co-authored-by: Stan U <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
[3.13] gh-126909: Fix running xattr tests on systems with lower limits (GH-126930) (#126965)
gh-126909: Fix running xattr tests on systems with lower limits (GH-126930)
Modify the extended attribute tests to write fewer and smaller extended
attributes, in order to fit within filesystems with total xattr limit
of 1 KiB (e.g. ext4 with 1 KiB blocks). Previously, the test would
write over 2 KiB, making it fail with ENOSPC on such systems.
(cherry picked from commit 2c0a21c1aad65ab8362491acf856eb574b1257ad)
Serhiy Storchaka [Mon, 18 Nov 2024 11:26:45 +0000 (13:26 +0200)]
[3.13] gh-67877: Fix memory leaks in terminated RE matching (GH-126840) (GH-126960)
If SRE(match) function terminates abruptly, either because of a signal
or because memory allocation fails, allocated SRE_REPEAT blocks might
be never released.
[3.13] gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919) (#126917)
gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919)
The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line. It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet. Clearly, though, this is a case that needs to be handled
correctly. The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.
For dlsym(), a return value of NULL does not necessarily indicate
an error [1].
Therefore, to avoid using stale (or NULL) dlerror() values, we must:
1. clear the previous error state by calling dlerror()
2. call dlsym()
3. call dlerror()
If the return value of dlerror() is not NULL, an error occured.
In ctypes we choose to treat a NULL return value from dlsym()
as a "not found" error. This is the same as the fallback
message we use on Windows, Cygwin or when getting/formatting
the error reason fails.
[3.13] gh-126476: Raise IllegalMonthError for calendar.formatmonth() when the input month is not correct (GH-126484) (GH-126879)
gh-126476: Raise IllegalMonthError for calendar.formatmonth() when the input month is not correct (GH-126484)
(cherry picked from commit 3be7498d2450519d5d8f63a35ef298db3b3d935b)
[3.13] Added a warning to the urljoin docs, indicating that it is not safe to use with attacker controlled URLs (GH-126659) (#126888)
Added a warning to the urljoin docs, indicating that it is not safe to use with attacker controlled URLs (GH-126659)
This was flagged to me at a party today by someone who works in red-teaming as a frequently encountered footgun. Documenting the potentially unexpected behavior seemed like a good place to start.
(cherry picked from commit d6bcc154e93a0a20ab97187d3e8b726fffb14f8f)
Co-authored-by: Alex Gaynor <alex.gaynor@gmail.com>
[3.13] gh-71936: Fix race condition in multiprocessing.Pool (GH-124973) (GH-126869)
Proxes of shared objects register a Finalizer in BaseProxy._incref(), and it
will call BaseProxy._decref() when it is GCed. This may cause a race condition
with Pool(maxtasksperchild=None) on Windows.
A connection would be closed and raised TypeError when a GC occurs between
_ConnectionBase._check_writable() and _ConnectionBase._send_bytes() in
_ConnectionBase.send() in the second or later task, and a new object
is allocated that shares the id() of a previously deleted one.
Instead of using the id() of the token (or the proxy), use a unique,
non-reusable number.
gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance (GH-126182)
* gh-123832: Adjust `socket.getaddrinfo` docs for better POSIX compliance
This changes nothing changes for CPython supported platforms,
but hints how to deal with platforms that stick to the letter of
the spec.
It also marks `socket.getaddrinfo` as a wrapper around `getaddrinfo(3)`;
specifically, workarounds to make the function work consistently across
platforms are out of scope in its code.
Include wording similar to the POSIX's “by providing options and by
limiting the returned information”, which IMO suggests that the
hints limit the resulting list compared to the defaults, *but* can
be interpreted differently. Details are added in a note.
Specifically say that this wraps the underlying C function. So, the
details are in OS docs. The “full range of results” bit goes away.
Use `AF_UNSPEC` rather than zero for the *family* default, although
I don't think a system where it's nonzero would be very usable.
Suggest setting proto and/or type (with examples, as the appropriate
values aren't obvious). Say why you probably want to do that that
on all systems; mention the behavior on the “letter of the spec”
systems.
Suggest that the results should be tried in order, which is,
AFAIK best practice -- see RFC 6724 section 2, and its predecessor
from 2003 (which are specific to IP, but indicate how people use this):
> Well-behaved applications SHOULD iterate through the list of
> addresses returned from `getaddrinfo()` until they find a working address.
Discard two leading slashes from the beginning of a `file:` URI if they
introduce an empty authority section. As a result, file URIs like
`///etc/hosts` are correctly parsed as `/etc/hosts`.
(cherry picked from commit cae9d9d20f61cdbde0765efa340b6b596c31b67f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
[3.13] gh-126433: Change channel_info.count to int64_t (GH-126447) (#126826)
gh-126433: Change channel_info.count to int64_t (GH-126447)
Fix compiler warnings on 32-bit Windows: change channel_info.count
type from Py_ssize_t to int64_t in _interpchannelsmodule.c.
(cherry picked from commit 09d7083962062acfef7e7a9a309a01fb70ad8276)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Marshall <jmarshall@hey.com> Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
One of the most common reasons I see the old `pipes` module still in use
when porting to Python 3.13 is for the undocumented `pipes.quote`
function, which can easily be replaced with `shlex.quote`. I think it's
worth specifically calling this out, since being directed to the
`subprocess` module would be confusing in this case.
(cherry picked from commit 73e34b680852794d110cd806505b3d74d9d593db)
Co-authored-by: Colin Watson <cjwatson@debian.org>
Barney Gale [Wed, 13 Nov 2024 23:32:56 +0000 (23:32 +0000)]
[3.13] GH-118289: Fix handling of non-directories in `posixpath.realpath()` (GH-120127) (#126815)
In strict mode, raise `NotADirectoryError` if we encounter a non-directory
while we still have path parts left to process.
We use a `part_count` variable rather than `len(rest)` because the `rest`
stack also contains markers for unresolved symlinks.
(cherry picked from commit fd4b5453df74e249987553b12c14ad75fafa4991)
projects / thirdparty / Python / cpython.git / log
