Sam Gross [Fri, 10 May 2024 20:29:29 +0000 (16:29 -0400)]
gh-118846: Fix free-threading test failures when run sequentially (#118864)
The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
Victor Stinner [Fri, 10 May 2024 19:08:24 +0000 (21:08 +0200)]
gh-118702: Implement vectorcall for BaseException (#118703)
* BaseException_vectorcall() now creates a tuple from 'args' array.
* Creation an exception using BaseException_vectorcall() is now a
single function call, rather than having to call
BaseException_new() and then BaseException_init().
Calling BaseException_init() is inefficient since it overrides
the 'args' attribute.
* _PyErr_SetKeyError() now uses PyObject_CallOneArg() to create the
KeyError instance to use BaseException_vectorcall().
Jelle Zijlstra [Fri, 10 May 2024 16:55:49 +0000 (09:55 -0700)]
Rename typing._collect_parameters (#118900)
Unfortunately, released versions of typing_extensions
monkeypatch this function without the extra parameter, which makes
it so things break badly if current main is used with typing_extensions.
Fortunately, the monkeypatching is not needed on Python 3.13, because CPython
now implements PEP 696. By renaming the function, we prevent the monkeypatch
from breaking typing.py internals.
We keep the old name (raising a DeprecationWarning) to help other external users who call it.
Barney Gale [Fri, 10 May 2024 15:53:46 +0000 (16:53 +0100)]
GH-78707: Drop deprecated `pathlib.PurePath.[is_]relative_to()` arguments (#118780)
Remove support for supplying additional positional arguments to
`PurePath.relative_to()` and `is_relative_to()`. This has been deprecated
since Python 3.12.
Alex Turner [Fri, 10 May 2024 14:26:35 +0000 (15:26 +0100)]
gh-117657: Fix QSBR race condition (#118843)
`_Py_qsbr_unregister` is called when the PyThreadState is already
detached, so the access to `tstate->qsbr` isn't safe without locking the
shared mutex. Grab the `struct _qsbr_shared` from the interpreter
instead.
_PyWeakref_ClearRef was previously exposed in the public C-API, although
it begins with an underscore and is not documented. It's used by a few
C-API extensions. There is currently no alternative public API that can
replace its use.
_PyWeakref_ClearWeakRefsExceptCallbacks is the only thread-safe way to
use _PyWeakref_ClearRef in the free-threaded build. This exposes the C
symbol, but does not make the API public.
Victor Stinner [Fri, 10 May 2024 10:04:16 +0000 (12:04 +0200)]
gh-117873: Use positional-only parameters in _posixshmem (#118012)
* shm_unlink() parameter becomes positional-only.
* shm_open() first parameter (path) becomes positional-only,
the two following parameters remain positional-or-keyword.
Brett Simmers [Thu, 9 May 2024 21:02:39 +0000 (14:02 -0700)]
gh-117657: Replace TSAN suppresions with more specific rules (#118722)
Using `race:` filters out warnings if the function appears anywhere in
the stack trace. This can hide a lot of unrelated warnings, especially
for a function like `_PyEval_EvalFrameDefault`, which is somewhere on
the stack more often than not.
Change all free-threaded suppressions to `race_top:`, which only matches
the top frame, and add any new suppressions this exposes.
Sam Gross [Thu, 9 May 2024 20:03:45 +0000 (16:03 -0400)]
gh-116984: Make mimalloc header includes relative to the current file (#118808)
Some embedders and extensions include parts of the internal API. The
pycore_mimalloc.h file is transitively include by a number of other
internal headers. This avoids include errors for code that was
already including those headers.
Sam Gross [Thu, 9 May 2024 18:52:27 +0000 (14:52 -0400)]
gh-118561: Fix crash involving list.extend in free-threaded build (#118723)
The `list_preallocate_exact` function did not zero initialize array
contents. In the free-threaded build, this could expose uninitialized
memory to concurrent readers between the call to
`list_preallocate_exact` and the filling of the array contents with
items.
mpage [Wed, 8 May 2024 20:03:39 +0000 (13:03 -0700)]
gh-117657: Fix data races when writing / reading `ob_gc_bits` (#118292)
Use relaxed atomics when reading / writing to the field. There are still a
few places in the GC where we do not use atomics. Those should be safe as
the world is stopped.
Tim Peters [Wed, 8 May 2024 00:09:09 +0000 (19:09 -0500)]
gh-118610: Centralize power caching in `_pylong.py` (#118611)
A new `compute_powers()` function computes all and only the powers of the base the various base-conversion functions need, as efficiently as reasonably possible (turns out that invoking `**`is needed at most once). This typically gives a few % speedup, but the primary point is to simplify the base-conversion functions, which no longer need their own, ad hoc, and less efficient power-caching schemes.
T. Wouters [Tue, 7 May 2024 21:00:25 +0000 (23:00 +0200)]
gh-109975: Make a rough editorial pass over What's New (#118711)
Make a rough editorial pass over Python 3.13's What's New document. Add the
release highlights, remove or merge some duplicated entries, and reorder
some of the sections (removals should really go before future deprecations).
The `pool_in_threads.py` test file may crash in free-threaded builds,
which can lead to the Tsan test hanging. Skip it for now until we fix
the underlying issue.
gh-66410: Do not stringify arguments of Tkinter callback (GH-98592)
Callbacks registered in the tkinter module now take arguments as
various Python objects (int, float, bytes, tuple), not just str.
To restore the previous behavior set tkinter module global wantobject to 1
before creating the Tk object or call the wantobject() method of the Tk object
with argument 1.
Calling it with argument 2 restores the current default behavior.
gh-118310: Fix documentation for `enum.Enum.__new__` (GH-118311)
The provided example was incorrect:
- The example enum was missing the `int` mixin as implied by the context
- The value of `int('1a', 16)` was incorrectly given as 17
(should be 26)
argparse performs a complex formatting of the usage for argument grouping
and for line wrapping to fit the terminal width. This formatting has been
a constant source of bugs for at least 10 years (see linked issues below)
where defensive assertion errors are triggered or brackets and paranthesis
are not properly handeled.
Problem
=======
The current implementation of argparse usage formatting relies on regular
expressions to group arguments usage only to separate them again later
with another set of regular expressions. This is a complex and error prone
approach that caused all the issues linked below. Special casing certain
argument formats has not solved the problem. The following are some of
the most common issues:
- empty `metavar`
- mutually exclusive groups with `SUPPRESS`ed arguments
- metavars with whitespace
- metavars with brackets or paranthesis
Solution
========
The following two comments summarize the solution:
- https://github.com/python/cpython/issues/82091#issuecomment-1093832187
- https://github.com/python/cpython/issues/77048#issuecomment-1093776995
Mainly, the solution is to rewrite the usage formatting to avoid the
group-then-separate approach. Instead, the usage parts are kept separate
and only joined together at the end. This allows for a much simpler
implementation that is easier to understand and maintain. It avoids the
regular expressions approach and fixes the corresponding issues.
This closes the following GitHub issues:
- #62090
- #62549
- #77048
- #82091
- #89743
- #96310
- #98666
Eric Snow [Tue, 7 May 2024 04:21:51 +0000 (22:21 -0600)]
gh-117953: Always Run Extension Init Func in Main Interpreter First (gh-118157)
This change makes sure all extension/builtin modules have their init function run first by the main interpreter before proceeding with import in the original interpreter (main or otherwise). This means when the import of a single-phase init module fails in an isolated subinterpreter, it won't tie any global state/callbacks to the subinterpreter.
Brett Simmers [Tue, 7 May 2024 03:07:23 +0000 (20:07 -0700)]
gh-116322: Enable the GIL while loading C extension modules (#118560)
Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.
We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.