Neil Schemenauer [Wed, 25 Jun 2025 07:06:32 +0000 (00:06 -0700)]
GH-133136: Revise QSBR to reduce excess memory held (gh-135473)
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.
Take into account the size of the memory block when deciding when to
process QSBR requests.
Also track the amount of memory being held by QSBR for mimalloc pages. Advance the write sequence if this memory exceeds a limit. Advancing the sequence will allow it to be freed more quickly.
Process the held QSBR items from the "eval breaker", rather than from `_PyMem_FreeDelayed()`. This gives a higher chance that the global read sequence has advanced enough so that items can be freed.
Emma Smith [Mon, 23 Jun 2025 10:28:05 +0000 (06:28 -0400)]
gh-134986: Catch PermissionError when trying to call perf in tests (#134987)
Using Ubuntu 24.04 on the Windows Subsystem for Linux, perf will raise a
`PermissionError` instead of `FileNotFoundError`. This commit modifies
the tests to catch that.
Kattni [Sun, 22 Jun 2025 04:51:23 +0000 (00:51 -0400)]
patchcheck: use URL paths to identify upstream remote (GH-135806)
* find defined "(fetch)" remotes with "python/cpython" in their URL
* if there is exactly one, use that remote name
* if there is one named "upstream", "origin", or "python",
use that remote (in that precedence order)
* otherwise report an error listing the defined remotes
Bénédikt Tran [Sat, 21 Jun 2025 12:32:00 +0000 (14:32 +0200)]
gh-135532: optimize calls to `PyMem_Malloc` in SHAKE digest computation (#135744)
- Add a fast path when the digest length is 0 to avoid calling useless functions.
- Directly allocate via `PyBytes_FromStringAndSize(NULL, length)` when possible.
Bénédikt Tran [Sat, 21 Jun 2025 09:43:30 +0000 (11:43 +0200)]
gh-135759: consistently reject negative sizes in SHAKE digests (#135767)
Passing a negative digest length to `_hashilb.HASHXOF.[hex]digest()` now
raises a ValueError instead of a MemoryError or a SystemError. This makes
the behavior consistent with that of `_sha3.shake_{128,256}.[hex]digest`.
Hood Chatham [Sat, 21 Jun 2025 06:06:59 +0000 (23:06 -0700)]
gh-127146: Skip test_os.test_mode for Emscripten (#135764)
Temporarily skip test_os.test_mode on Emscripten; this fails consistently
on the buildbot, but not on other test configurations. Reported as #135783
for follow up.
Eric Snow [Fri, 20 Jun 2025 20:26:32 +0000 (14:26 -0600)]
gh-135698: Fix Cross-interpreter Queue.full() With Negative/Default max_size (gh-135724)
We weren't handling non-positive maxsize values (including the default) properly
in Queue.full(). This change fixes that and adjusts an associated assert.
Eric Snow [Wed, 18 Jun 2025 23:57:14 +0000 (17:57 -0600)]
gh-133485: Use interpreters.Interpreter in InterpreterPoolExecutor (gh-133957)
Most importantly, this resolves the issues with functions and types defined in __main__.
It also expands the number of supported objects and simplifies the implementation.
Serhiy Storchaka [Wed, 18 Jun 2025 15:26:01 +0000 (18:26 +0300)]
gh-135376: Fix and improve test_random (GH-135377)
* Remove duplicated code. Tests for Random and SystemRandom now share
the code.
* Move implementation agnostic tests that was only run for SystemRandom,
so they are now run for Random too.
* Add tests for __index__() support.
* Add tests for randint().
Duane Griffin [Wed, 18 Jun 2025 12:17:02 +0000 (00:17 +1200)]
gh-135335: flush stdout/stderr in forkserver after preloading modules (#135338)
If a preloaded module writes to stdout or stderr, and the stream is buffered,
child processes will inherit the buffered data after forking. Attempt to
prevent this by flushing the streams after preload.
Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>
Hood Chatham [Wed, 18 Jun 2025 03:20:43 +0000 (20:20 -0700)]
gh-127146: Emscripten: Fix pathlib glob_dotdot test (#135624)
The Emscripten path resolver uses the same mechanism for resolving `..`
at a file system root as for resolving symlinks. This is because
roots don't store their mountpoints. If the parent of a node is itself,
it is a root but it might be a mountpoint in some other file system.
If a path has enough `..`'s at the root, it will return ELOOP.
Enough turns out to be 49.
Emscripten currently `gethostbyname_r()` returns an incorrect
IP address for `localhost`. Will be resolved by upstream PR:
https://github.com/emscripten-core/emscripten/pull/24593
As noted in the new tests, there are a few situations we must carefully accommodate
for functions that get pickled during interp.call(). We do so by running the script
from the main interpreter's __main__ module in a hidden module in the other
interpreter. That hidden module is used as the function __globals__.
Ken Jin [Tue, 17 Jun 2025 15:25:53 +0000 (23:25 +0800)]
gh-134584: Decref elimination for float ops in the JIT (GH-134588)
This PR adds a PyJitRef API to the JIT's optimizer that mimics the _PyStackRef API. This allows it to track references and their stack lifetimes properly. Thus opening up the doorway to refcount elimination in the JIT.