Sam Gross [Thu, 24 Oct 2024 22:09:59 +0000 (18:09 -0400)]
gh-125900: Clean-up logic around immortalization in free-threading (#125901)
* Remove `@suppress_immortalization` decorator
* Make suppression flag per-thread instead of per-interpreter
* Suppress immortalization in `eval()` to avoid refleaks in three tests
(test_datetime.test_roundtrip, test_logging.test_config8_ok, and
test_random.test_after_fork).
* frozenset() is constant, but not a singleton. When run multiple times,
the test could fail due to constant interning.
Sam Gross [Thu, 24 Oct 2024 16:44:38 +0000 (12:44 -0400)]
gh-124218: Avoid refcount contention on builtins module (GH-125847)
This replaces `_PyEval_BuiltinsFromGlobals` with
`_PyDict_LoadBuiltinsFromGlobals`, which returns a new reference
instead of a borrowed reference. Internally, the new function uses
per-thread reference counting when possible to avoid contention on the
refcount fields on the builtins module.
Sam Gross [Thu, 24 Oct 2024 16:03:50 +0000 (12:03 -0400)]
gh-125842: Fix `sys.exit(0xffff_ffff)` on Windows (#125896)
On Windows, `long` is a signed 32-bit integer so it can't represent
`0xffff_ffff` without overflow. Windows exit codes are unsigned 32-bit
integers, so if a child process exits with `-1`, it will be represented
as `0xffff_ffff`.
Also fix a number of other possible cases where `_Py_HandleSystemExit`
could return with an exception set, leading to a `SystemError` (or
fatal error in debug builds) later on during shutdown.
Sam Gross [Thu, 24 Oct 2024 13:33:11 +0000 (09:33 -0400)]
gh-125859: Fix crash when `gc.get_objects` is called during GC (#125882)
This fixes a crash when `gc.get_objects()` or `gc.get_referrers()` is
called during a GC in the free threading build.
Switch to `_PyObjectStack` to avoid corrupting the `struct worklist`
linked list maintained by the GC. Also, don't return objects that are frozen
(`gc.freeze()`) or in the process of being collected to more closely match
the behavior of the default build.
Serhiy Storchaka [Tue, 22 Oct 2024 10:57:25 +0000 (13:57 +0300)]
gh-125355: Rewrite parse_intermixed_args() in argparse (GH-125356)
* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
now supported.
* The missing arguments report now includes the names of all required
optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
parse_known_intermixed_args().
Serhiy Storchaka [Mon, 21 Oct 2024 18:54:12 +0000 (21:54 +0300)]
gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a string again (GH-125774)
This is a follow up of GH-124974. Only Glibc needed a fix.
Now the returned value is a string consisting of semicolon-separated
symbols on all Posix platforms.
Serhiy Storchaka [Mon, 21 Oct 2024 18:30:45 +0000 (21:30 +0300)]
gh-125767: Fix pickling and copying of super objects (GH-125781)
Previously, copying a super object returned a copy of the instance
invoking super(). Pickling a super object could pickle the instance
invoking super() or fail, depending on its type and protocol.
Now deep copying returns a new super object and pickling pickles the super
object. Shallow copying returns the same super object.
Richard Hansen [Mon, 21 Oct 2024 16:46:37 +0000 (12:46 -0400)]
Doc: C API: Move `tp_dealloc` paragraph to `tp_dealloc` section (#125737)
It looks like commit 43cf44ddcce6b225f959ea2a53e4817244ca6054
(gh-31501) accidentally moved the paragraph to the `tp_finalize`
section when the intent was to move it to the `tp_dealloc` section
(according to the commit message).
Also:
* Convert the paragraph to a warning.
* Apply the appropriate font style to `tp_dealloc`.
* Unlinkify the first mention of `tp_dealloc` since the paragraph is
already in the `tp_dealloc` section.
Sam Gross [Mon, 21 Oct 2024 12:23:38 +0000 (08:23 -0400)]
gh-125608: Trigger dictionary watchers when inline values change (#125611)
Dictionary watchers on an object's attributes dictionary
(`object.__dict__`) were not triggered when the managed dictionary used
the object's inline values.
gh-119311: Add missing magic number (3571) for 3.13.0b1 (#125771)
Add missing magic number 3571 for 3.13b1
It was added after branching in https://github.com/python/cpython/commit/6394a72e99b342d980297ec437ecafea92a044c4#diff-efefe383b3a81d16150c280db0b64eed7569254299418f64cc0d749f8e16f3a4R475
Julien [Sat, 19 Oct 2024 17:34:41 +0000 (13:34 -0400)]
gh-125398: Convert paths in venv activate script when using Git Bash under Windows (GH-125399)
* Convert paths in venv activate script when using Git Bash under Windows
With https://github.com/python/cpython/pull/112508 the check to converts paths when running on Windows was changed from using the non-posix environment variable `$OSTYPE` to using `uname` instead.
However this missed the fact that when running under Git Bash on Windows, uname reports `MINGW*` (`$OSTYPE` is still `msys`).
This results in `$PATH` being set to something like `D:\a\github-actions-shells\github-actions-shells\venv/Scripts:…`, instead of `/d/a/github-actions-shells/github-actions-shells/venv/Scripts`.
Notably, the Git Bash is the bash shell that’s used for GitHub Actions Windows runners, and ships with VSCode.
Eric Snow [Fri, 18 Oct 2024 15:26:08 +0000 (09:26 -0600)]
gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605)
This is essentially a cleanup, moving a handful of API declarations to the header files where they fit best, creating new ones when needed.
We do the following:
* add pycore_debug_offsets.h and move _Py_DebugOffsets, etc. there
* inline struct _getargs_runtime_state and struct _gilstate_runtime_state in _PyRuntimeState
* move struct _reftracer_runtime_state to the existing pycore_object_state.h
* add pycore_audit.h and move to it _Py_AuditHookEntry , _PySys_Audit(), and _PySys_ClearAuditHooks
* add audit.h and cpython/audit.h and move the existing audit-related API there
*move the perfmap/trampoline API from cpython/sysmodule.h to cpython/ceval.h, and remove the now-empty cpython/sysmodule.h
Serhiy Storchaka [Thu, 17 Oct 2024 15:46:59 +0000 (18:46 +0300)]
gh-52551: Fix encoding issues in strftime() (GH-125193)
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.
* Characters not encodable in the current locale are now acceptable in
the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
longer recombinated.
* Embedded null character no longer terminates the format string.
Wulian [Thu, 17 Oct 2024 15:23:37 +0000 (23:23 +0800)]
gh-123370: Fix the canvas not clearing after running turtledemo.clock (#123457)
Rewriting the day and date every tick somehow prevented them from being removed either by clicking STOP or loading another example. The solution is to rewrite them only when they change.
This is an implementation of InterpreterPoolExecutor that builds on ThreadPoolExecutor.
(Note that this is not tied to PEP 734, which is strictly about adding a new stdlib module.)
Possible future improvements:
* support passing a script for the initializer or to submit()
* support passing (most) arbitrary functions without pickling
* support passing closures
* optionally exec functions against __main__ instead of the their original module
Sam Gross [Wed, 16 Oct 2024 15:39:17 +0000 (11:39 -0400)]
gh-125451: Fix deadlock in ProcessPoolExecutor shutdown (#125492)
There was a deadlock when `ProcessPoolExecutor` shuts down at the same
time that a queueing thread handles an error processing a task.
Don't use `_shutdown_lock` to protect the `_ThreadWakeup` pipes -- use
an internal lock instead. This fixes the ordering deadlock where the
`ExecutorManagerThread` holds the `_shutdown_lock` and joins the
queueing thread, while the queueing thread is attempting to acquire the
`_shutdown_lock` while closing the `_ThreadWakeup`.
Diego Russo [Wed, 16 Oct 2024 13:13:07 +0000 (14:13 +0100)]
gh-125444: Fix illegal instruction for older Arm architectures (#125574)
On Arm v5 it is not possible to get the thread ID via c13 register
hence the illegal instruction. The c13 register started to provide
thread ID since Arm v6K architecture variant. Other variants of
Arm v6 (T2, Z and base) don’t provide the thread ID via c13.
For the sake of simplicity we group v5 and v6 together and
consider that instructions for Arm v7 only.
gh-124872: Replace enter/exit events with "switched" (#125532)
Users want to know when the current context switches to a different
context object. Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched". However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit. Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.
The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name. If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future. It is not added here
because YAGNI.
I skipped 0 in the enum as a matter of practice. Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).
Co-authored-by: Richard Hansen <rhansen@rhansen.org> Co-authored-by: Victor Stinner <vstinner@python.org>