Petr Viktorin [Wed, 11 Jun 2025 10:44:58 +0000 (12:44 +0200)]
gh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (GH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Ajay Kamdar [Tue, 10 Jun 2025 11:28:31 +0000 (07:28 -0400)]
gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)
When shutdown is called with wait=False, the executor thread keeps running
even after the ProcessPoolExecutor's state is reset. The executor then tries
to replenish the worker processes pool resulting in an error and a potential hang
when it comes across a worker that has died. Fixed the issue by having
_adjust_process_count() return without doing anything if the ProcessPoolExecutor's
state has been reset.
Added unit tests to validate two scenarios:
max_workers < num_tasks (exception)
max_workers > num_tasks (exception + hang)
Petr Viktorin [Mon, 9 Jun 2025 13:50:11 +0000 (15:50 +0200)]
gh-127833: Reword and expand the Notation section (GH-134443)
Prepare the docs for using the notation used in the `python.gram`
file. If we want to sync the two, the meta-syntax should be the same.
Link the Full Grammar docs here; keep only a few extras.
Also, remove the distinction between lexical and syntactic rules,
except for whitespace handling.
With f- and t-strings, the line between the two is blurry.
Co-authored-by: Blaise Pabon <blaise@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Colin Marquardt <cmarqu42@gmail.com>
Bénédikt Tran [Mon, 9 Jun 2025 07:17:43 +0000 (09:17 +0200)]
gh-135004: rewrite and cleanup `blake2module.c` (#135006)
* Cleanup imports and update module docstring.
* Simplify detection of SIMD support.
* Correctly guard `update()` cases.
* Rewrite `py_blake2b_or_s_new` and rename it to `py_blake2_new`.
* Rewrite `blake2_blake2b_copy_locked` and `py_blake2_clear`.
* Refactor computations of `digest` and `hexdigest`.
* Simplify `py_blake2b_get_name` and `py_blake2b_get_block_size`.
* Add `hacl_get_blake2_info` to extract static BLAKE-2 information.
This new helper is used by `py_blake2b_get_digest_size`, but can
be later used to expose `key_length` more easily.
Victor Stinner [Thu, 5 Jun 2025 12:43:47 +0000 (14:43 +0200)]
gh-134989: Fix Py_RETURN_NONE in the limited C API (GH-135165)
Fix Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE macros in the
limited C API 3.11 and older:
Don't treat Py_None, Py_True and Py_False as immortal.
Victor Stinner [Thu, 5 Jun 2025 09:17:03 +0000 (11:17 +0200)]
gh-135124: Change stdout errors in regrtest worker process (#135138)
Set sys.stdout encoder error handler to backslashreplace in regrtest
workers to avoid UnicodeEncodeError when printing a traceback
or any other non-encodable character.
Move the code from the Regrtest class to setup_process().
Call setup_process() earlier, before displaying regrtest headers.
Malcolm Smith [Thu, 5 Jun 2025 05:46:16 +0000 (06:46 +0100)]
gh-131531: android.py enhancements to support cibuildwheel (#132870)
Modifies the environment handling and execution arguments of the Android management
script to support the compilation of third-party binaries, and the use of the testbed to
invoke third-party test code.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
mpage [Wed, 4 Jun 2025 23:07:58 +0000 (16:07 -0700)]
gh-134889: Fix handling of a few opcodes when optimizing `LOAD_FAST` (#134958)
We were incorrectly handling a few opcodes that leave their operands on the stack. Treat all of these conservatively; assume that they always leave operands on the stack.
Thomas Grainger [Wed, 4 Jun 2025 13:00:25 +0000 (14:00 +0100)]
gh-126483: disable warnings filters mutation in concurrent test (GH-132694)
The `test_ssl_in_multiple_threads` test failed because `test_check_hostname_idn()`
modified the global warnings filters via `warnings_helper.check_no_resource_warning()`.
Only check for warnings when the context aware warnings feature is enabled, which makes
the warnings filter context-local and thread-safe.
Sam Gross [Wed, 4 Jun 2025 07:35:56 +0000 (03:35 -0400)]
gh-135099: Only wait on `_PyOS_SigintEvent()` in main thread (GH-135100)
On Windows, the `_PyOS_SigintEvent()` event handle is used to interrupt
the main thread when Ctrl-C is pressed. Previously, we also waited on
the event from other threads, but ignored the result. However, this can
race with interpreter shutdown because the main thread closes the handle
in `_PySignal_Fini` and threads may still be running and using mutexes
during interpreter shtudown.
Only use `_PyOS_SigintEvent()` in the main thread in parking_lot.c, like
we do in other places in the CPython codebase.
Joe Rickerby [Wed, 4 Jun 2025 00:31:43 +0000 (01:31 +0100)]
gh-135101: When choosing the default simulator device, don't use `simctl --set testing` (#135102)
On a fresh Xcode install (including some CI provider configurations), there is
no pre-existing testing set that can be used to identify simulator models. Use
the default device set to detect available models instead. Live testing
simulators are still created in the testing set.
tpburns [Tue, 3 Jun 2025 16:00:25 +0000 (12:00 -0400)]
gh-134248 test_getallocatedblocks pre-check to ignore immortalized strings (#134871)
When sanity checking against gettotalrefcount(), we exclude the blocks for
immortalized strings since their references are not tracked/reported. This
now matches refleak.py's book-keeping using the same functions.