Victor Stinner [Sat, 4 May 2024 07:39:58 +0000 (09:39 +0200)]
gh-110850: Use _PyDeadline_Get() in EnterNonRecursiveMutex() (#118556)
Use _PyDeadline_Init() and _PyDeadline_Get() in
EnterNonRecursiveMutex() of thread_nt.h.
_PyDeadline_Get() uses the monotonic clock which is now the same as
the perf counter clock on all platforms. So this change does not
cause any behavior change. It just reuses existing helper functions.
Sam Gross [Fri, 3 May 2024 22:16:45 +0000 (18:16 -0400)]
gh-118527: Intern filename, name, and qualname in code objects. (#118558)
This interns the strings for `co_filename`, `co_name`, and `co_qualname`
on codeobjects in the free-threaded build. This partially addresses a
reference counting bottleneck when creating closures concurrently. The
closures take the name and qualified name from the code object.
Barney Gale [Fri, 3 May 2024 20:29:25 +0000 (21:29 +0100)]
GH-116380: Move pathlib globbing implementation into `pathlib._glob` (#118562)
Moving this code under the `pathlib` package makes it quite a lot easier
to backport in the `pathlib-abc` PyPI package. It was a bit foolish of me
to add it to `glob` in the first place.
Also add `translate()` to `__all__` in `glob`. This function is new in
3.13, so there's no NEWS needed.
Brett Simmers [Fri, 3 May 2024 15:30:55 +0000 (08:30 -0700)]
gh-116322: Add Py_mod_gil module slot (#116882)
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.
PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.
A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
mpage [Fri, 3 May 2024 15:14:26 +0000 (08:14 -0700)]
gh-118495: Skip test using threads after forking when running with TSAN (#118530)
This is unsupported. Note that `skip_unless_reliable_fork()` checks for
the conditions used by the decorators that were removed, along with checking
for TSAN.
Sam Gross [Fri, 3 May 2024 15:09:57 +0000 (11:09 -0400)]
gh-117514: Add `sys._is_gil_enabled()` function (#118514)
The function returns `True` or `False` depending on whether the GIL is
currently enabled. In the default build, it always returns `True`
because the GIL is always enabled.
Sam Gross [Fri, 3 May 2024 15:05:30 +0000 (11:05 -0400)]
gh-118527: Use `_Py_ID(__main__)` for main module name (#118528)
Most module names are interned and immortalized, but the main
module was not. This partially addresses a scaling bottleneck in the
free-threaded when creating closure concurrently in the main module.
Brett Simmers [Thu, 2 May 2024 22:25:36 +0000 (15:25 -0700)]
gh-116738: Make `_codecs` module thread-safe (#117530)
The module itself is a thin wrapper around calls to functions in
`Python/codecs.c`, so that's where the meaningful changes happened:
- Move codecs-related state that lives on `PyInterpreterState` to a
struct declared in `pycore_codecs.h`.
- In free-threaded builds, add a mutex to `codecs_state` to synchronize
operations on `search_path`. Because `search_path_mutex` is used as a
normal mutex and not a critical section, we must be extremely careful
with operations called while holding it.
- The codec registry is explicitly initialized as part of
`_PyUnicode_InitEncodings` to simplify thread-safety.
Sam Gross [Thu, 2 May 2024 17:41:15 +0000 (13:41 -0400)]
gh-118413: Fix test_release_task_refs on free-threaded build (#118494)
The `time.sleep()` call should happen before the GC to give the worker
threads time to clean-up their remaining references to objs.
Additionally, use `support.gc_collect()` instead of `gc.collect()`
just in case the extra GC calls matter.
Mark Shannon [Thu, 2 May 2024 15:17:59 +0000 (16:17 +0100)]
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR
* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
gh-82062: Fix support of parameter defaults on methods in extension modules (GH-115270)
Now inspect.signature() supports references to the module globals in
parameter defaults on methods in extension modules. Previously it was
only supported in functions. The workaround was to specify the fully
qualified name, including the module name.
This commit uses "file pointed to by" to replace "file pointed by" in
- doc for shutil.copytree
- docstring for shutil.copytree
- docstring _abc.PathBase.open
- docstring for pathlib.Path.open
- doc for os.copy_file_range
- doc for os.splice
The docs use "file pointed to by" more frequently than
"file pointed by". So, this commit replaces the uses of
"file pointed by" in order to make the uses consistent
through the docs.
```bash
$ grep -ri 'pointed to by' cpython/
```
yields more results than
```bash
$ grep -ri 'pointed by' cpython/
```
Separately:
There are two occurrences of "tree pointed by":
- cpython/Doc/library/xml.etree.elementtree.rst for
`xml.etree.ElementInclude.include`
- cpython/Lib/xml/etree/ElementInclude.py for `include`
For those uses of "tree pointed by", I expect "tree pointed to by"
instead. However, I found enough uses online of (a) "tree pointed by"
rather than (b) "tree pointed to by" to convince me that (a) is in
common use.
So, this commit does not replace those occurrences of "tree pointed by"
to "tree pointed to by". But I will replace them if a reviewer
believes it is correct to replace them.
* docs: typo: "exists and executable" -> "exists and is executable"
mpage [Wed, 1 May 2024 20:59:12 +0000 (13:59 -0700)]
gh-118433: Temporarily skip `test_interrupt_main_subthread` in free-threaded builds (#118485)
Free-threaded builds can intermittently tickle a longstanding bug (24 years!)
in the implementation of `threading.Condition`, leading to flakiness in the
test suite. Fixing the underlying issue will require more discussion, and will
likely apply to most of the concurrency primitives in the `threading` module
that are written in Python. See gh-118433 for more details.
* Add documentation and tests. Tests release the GIL while calling
raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.
Irit Katriel [Wed, 1 May 2024 11:01:16 +0000 (12:01 +0100)]
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
da-woods [Wed, 1 May 2024 08:33:28 +0000 (09:33 +0100)]
Test syntax error on comma-less tuple-style sequence patterns (#115485)
Adds a test that length-1 tuple-style sequence patterns must end in a comma, since there isn't currently one.
Spotted while reviewing Cython's proposed implementation of the pattern matching syntax (https://github.com/cython/cython/pull/4897#discussion_r1489177169) where there was a bug my the reimplementation that wasn't caught against the CPython tests here.
Malcolm Smith [Wed, 1 May 2024 06:36:45 +0000 (07:36 +0100)]
gh-116622: Add Android testbed (GH-117878)
Add code and config for a minimal Android app, and instructions to build and run it.
Improve Android build instructions in general.
Add a tool subcommand to download the Gradle wrapper (with its binary blob). Android
studio must be downloaded manually (due to the license).
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.
We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.
On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.
In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
Sam Gross [Tue, 30 Apr 2024 19:01:28 +0000 (15:01 -0400)]
gh-118332: Fix deadlock involving stop the world (#118412)
Avoid detaching thread state when stopping the world. When re-attaching
the thread state, the thread would attempt to resume the top-most
critical section, which might now be held by a thread paused for our
stop-the-world request.
gh-117860: Add tests for resolving names when import rebind names (GH-118176)
Add tests for "import", pkgutil.resolve_name() and unittest.mock.path()
for cases when "import a.b as x" and "from a import b as x" give
different results.