Eric Snow [Wed, 18 Jun 2025 23:57:14 +0000 (17:57 -0600)]
gh-133485: Use interpreters.Interpreter in InterpreterPoolExecutor (gh-133957)
Most importantly, this resolves the issues with functions and types defined in __main__.
It also expands the number of supported objects and simplifies the implementation.
Serhiy Storchaka [Wed, 18 Jun 2025 15:26:01 +0000 (18:26 +0300)]
gh-135376: Fix and improve test_random (GH-135377)
* Remove duplicated code. Tests for Random and SystemRandom now share
the code.
* Move implementation agnostic tests that was only run for SystemRandom,
so they are now run for Random too.
* Add tests for __index__() support.
* Add tests for randint().
Duane Griffin [Wed, 18 Jun 2025 12:17:02 +0000 (00:17 +1200)]
gh-135335: flush stdout/stderr in forkserver after preloading modules (#135338)
If a preloaded module writes to stdout or stderr, and the stream is buffered,
child processes will inherit the buffered data after forking. Attempt to
prevent this by flushing the streams after preload.
Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>
Hood Chatham [Wed, 18 Jun 2025 03:20:43 +0000 (20:20 -0700)]
gh-127146: Emscripten: Fix pathlib glob_dotdot test (#135624)
The Emscripten path resolver uses the same mechanism for resolving `..`
at a file system root as for resolving symlinks. This is because
roots don't store their mountpoints. If the parent of a node is itself,
it is a root but it might be a mountpoint in some other file system.
If a path has enough `..`'s at the root, it will return ELOOP.
Enough turns out to be 49.
Emscripten currently `gethostbyname_r()` returns an incorrect
IP address for `localhost`. Will be resolved by upstream PR:
https://github.com/emscripten-core/emscripten/pull/24593
As noted in the new tests, there are a few situations we must carefully accommodate
for functions that get pickled during interp.call(). We do so by running the script
from the main interpreter's __main__ module in a hidden module in the other
interpreter. That hidden module is used as the function __globals__.
Ken Jin [Tue, 17 Jun 2025 15:25:53 +0000 (23:25 +0800)]
gh-134584: Decref elimination for float ops in the JIT (GH-134588)
This PR adds a PyJitRef API to the JIT's optimizer that mimics the _PyStackRef API. This allows it to track references and their stack lifetimes properly. Thus opening up the doorway to refcount elimination in the JIT.
Michał Górny [Tue, 17 Jun 2025 08:05:04 +0000 (10:05 +0200)]
gh-134632: Fix `build-details.json` to use `INCLUDEPY` path (#134633)
* gh-134632: Fix `build-details.json` to use `INCLUDEPY` path
Fix ``build-details.json`` generation to use ``INCLUDEPY``, in order to
reference the ``pythonX.Y`` subdirectory of the include directory, as
required in :pep:`739`, instead of the top-level include directory.
* test_build_details: Add tests for the c_api section
* test_build_details: Expect pkgconfig for CPython unconditionally
Ned Deily [Mon, 16 Jun 2025 23:35:59 +0000 (01:35 +0200)]
gh-119132: Update 'Using Python on macOS' documentation. (#135591)
Remove `experimental` qualification for free-threading in the document text. Note that images included in the document will be updated later in the release cycle.
Eric Snow [Mon, 16 Jun 2025 23:34:19 +0000 (17:34 -0600)]
gh-135443: Sometimes Fall Back to __main__.__dict__ For Globals (gh-135491)
For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true. This allows those functions to be run with Interpreter.call().
Use `ma_used` instead of `ma_keys->dk_nentries` for modification check
so that we only check if the dictionary is modified, not if new keys are
added to a different dictionary that shared the same keys object.
Donghee Na [Mon, 16 Jun 2025 14:32:52 +0000 (23:32 +0900)]
gh-119132: Remove "experimental" tag from the CPython free-threading. (gh-135550)
* gh-119132: Remove "experimental" tag from the CPython free-threading build
* Address code review
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Add NEWS.d
* Regen configure.ac
* Update doc
* Update
* Update
* Update
* Update Doc/howto/free-threading-python.rst
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
* Update ctypes.rst
* Update
* Update Doc/howto/free-threading-python.rst
Co-authored-by: T. Wouters <thomas@python.org>
* Apply suggestions from code review
Co-authored-by: T. Wouters <thomas@python.org> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
---------
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: T. Wouters <thomas@python.org>
Dan Lenski [Sun, 15 Jun 2025 19:29:38 +0000 (12:29 -0700)]
gh-67022: Document bytes/str inconsistency in email.header.decode_header() and suggest email.headerregistry.HeaderRegistry as a sane alternative (#92900)
* gh-67022: Document bytes/str inconsistency in email.header.decode_header()
This function's possible return types have been surprising and error-prone
for the entirety of its Python 3.x history. It can return either:
1. `typing.List[typing.Tuple[bytes, typing.Optional[str]]]` of length >1
2. or `typing.List[typing.Tuple[str, None]]`, of length exactly 1
This means that any user of this function must be prepared to accept either
`bytes` or `str` for the first member of the 2-tuples it returns, which is a
very surprising behavior in Python 3.x, particularly given that the second
member of the tuple is supposed to represent the charset/encoding of the
first member.
This patch documents the behavior of this function, and adds test cases
to demonstrate it.
As discussed in bpo-22833, this cannot be changed in a backwards-compatible
way, and some users of this function depend precisely on the existing
behavior.
Add warnings about obsolescence of 'email.header.decode_header' and 'email.header.make_header' functions.
Recommend use of `email.headerregistry.HeaderRegistry` instead, as suggested
in https://github.com/python/cpython/pull/92900#discussion_r1112472177
Jeremy Cline [Sun, 15 Jun 2025 05:34:29 +0000 (01:34 -0400)]
gh-127319: Disable port reuse on HTTP, XMLRPC, and logging TCP servers (GH-135405)
Prior to issue #120485 these servers did not allow port reuse, which
makes sense as the behavior of port reuse is surprising if you're not
expecting it. It's unclear to me why these services were switched to
allow port reuse, but I believe the desired behavior (unless subclasses
opt in) is to not allow port reuse.
See also: https://bugzilla.redhat.com/show_bug.cgi?id=2323170
Document behaviour of single-phase init. Call it "legacy".
Reorganize PyModule docs.
Move PyInit_modulename docs from the tutorial to reference documentation.
Move PyMODINIT_FUNC docs from generic macros to the new page.
Add doc stubs for `PYTHON_API_VERSION` & `PYTHON_ABI_VERSION`
Remove incorrect refcounts.dat entry for `PyModuleDef_Init`.
This removes the "Return value: Borrowed reference." note.
Instead, note that the function sometimes returns a borrowed reference,
sometimes as strong one.
(IMO, it's best to not think of `PyModuleDef` as a `PyObject` at all,
and act like it can't be reference-counted.)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Petr Viktorin [Wed, 11 Jun 2025 10:44:58 +0000 (12:44 +0200)]
gh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (GH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Ajay Kamdar [Tue, 10 Jun 2025 11:28:31 +0000 (07:28 -0400)]
gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)
When shutdown is called with wait=False, the executor thread keeps running
even after the ProcessPoolExecutor's state is reset. The executor then tries
to replenish the worker processes pool resulting in an error and a potential hang
when it comes across a worker that has died. Fixed the issue by having
_adjust_process_count() return without doing anything if the ProcessPoolExecutor's
state has been reset.
Added unit tests to validate two scenarios:
max_workers < num_tasks (exception)
max_workers > num_tasks (exception + hang)
Petr Viktorin [Mon, 9 Jun 2025 13:50:11 +0000 (15:50 +0200)]
gh-127833: Reword and expand the Notation section (GH-134443)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
Bénédikt Tran [Mon, 9 Jun 2025 07:17:43 +0000 (09:17 +0200)]
gh-135004: rewrite and cleanup `blake2module.c` (#135006)
* Cleanup imports and update module docstring.
* Simplify detection of SIMD support.
* Correctly guard `update()` cases.
* Rewrite `py_blake2b_or_s_new` and rename it to `py_blake2_new`.
* Rewrite `blake2_blake2b_copy_locked` and `py_blake2_clear`.
* Refactor computations of `digest` and `hexdigest`.
* Simplify `py_blake2b_get_name` and `py_blake2b_get_block_size`.
* Add `hacl_get_blake2_info` to extract static BLAKE-2 information.
This new helper is used by `py_blake2b_get_digest_size`, but can
be later used to expose `key_length` more easily.