Victor Stinner [Sun, 18 Feb 2024 20:06:39 +0000 (21:06 +0100)]
gh-115122: Add --bisect option to regrtest (#115123)
* test.bisect_cmd now exit with code 0 on success, and code 1 on
failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
-X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
methods.
Dmitry Marakasov [Sat, 17 Feb 2024 14:54:47 +0000 (17:54 +0300)]
Fix ProgramPriorityTests on FreeBSD with high nice value (GH-100145)
It expects priority to be capped with 19, which is the cap for Linux,
but for FreeBSD the cap is 20 and the test fails under the similar
conditions. Tweak the condition to cover FreeBSD as well.
6t8k [Sat, 17 Feb 2024 11:16:06 +0000 (11:16 +0000)]
gh-95782: Fix io.BufferedReader.tell() etc. being able to return offsets < 0 (GH-99709)
lseek() always returns 0 for character pseudo-devices like
`/dev/urandom` (for other non-regular files, e.g. `/dev/stdin`, it
always returns -1, to which CPython reacts by raising appropriate
exceptions). They are thus technically seekable despite not having seek
semantics.
When calling read() on e.g. an instance of `io.BufferedReader` that
wraps such a file, `BufferedReader` reads ahead, filling its buffer,
creating a discrepancy between the number of bytes read and the internal
`tell()` always returning 0, which previously resulted in e.g.
`BufferedReader.tell()` or `BufferedReader.seek()` being able to return
positions < 0 even though these are supposed to be always >= 0.
Invariably keep the return value non-negative by returning
max(former_return_value, 0) instead, and add some corresponding tests.
Stevoisiak [Sat, 17 Feb 2024 08:33:28 +0000 (03:33 -0500)]
gh-101699: Explain using Match.expand with \g<0> (GH-101701)
Update documentation for re library to explain that a backreference `\g<0>` is
expanded to the entire string when using Match.expand().
Note that numeric backreferences to group 0 (`\0`) are not supported.
This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and
quiescent state based reclamation (QSBR). The API provides a mechanism
for callers to detect when it is safe to free memory that may be
concurrently accessed by readers.
Łukasz Langa [Fri, 16 Feb 2024 20:24:56 +0000 (21:24 +0100)]
gh-115556: Remove quotes from command-line arguments in test.bat and rt.bat (#115557)
This change essentially replaces usage of `%1` with `%~1`, which removes
quotes, if any. Without this change, the if statements fail due to
the quotes mangling the syntax.
Additionally, this change works around comma being treated as a parameter
delimiter in test.bat by escaping commas at time of parsing. Tested
combinations of rt and regrtest arguments, all seems to work as before
but now you can specify commas in arguments like "-uall,extralargefile".
mpage [Fri, 16 Feb 2024 18:29:25 +0000 (10:29 -0800)]
gh-114271: Make `thread._rlock` thread-safe in free-threaded builds (#115102)
The ID of the owning thread (`rlock_owner`) may be accessed by
multiple threads without holding the underlying lock; relaxed
atomics are used in place of the previous loads/stores.
The number of times that the lock has been acquired (`rlock_count`)
is only ever accessed by the thread that holds the lock; we do not
need to use atomics to access it.
Sam Gross [Fri, 16 Feb 2024 16:22:27 +0000 (11:22 -0500)]
gh-112529: Make the GC scheduling thread-safe (#114880)
The GC keeps track of the number of allocations (less deallocations)
since the last GC. This buffers the count in thread-local state and uses
atomic operations to modify the per-interpreter count. The thread-local
buffering avoids contention on shared state.
A consequence is that the GC scheduling is not as precise, so
"test_sneaky_frame_object" is skipped because it requires that the GC be
run exactly after allocating a frame object.
David Benjamin [Fri, 16 Feb 2024 00:24:51 +0000 (19:24 -0500)]
gh-114572: Fix locking in cert_store_stats and get_ca_certs (#114573)
* gh-114572: Fix locking in cert_store_stats and get_ca_certs
cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with
X509_STORE_get0_objects, but reading the result requires a lock. See
https://github.com/openssl/openssl/pull/23224 for details.
Instead, use X509_STORE_get1_objects, newly added in that PR.
X509_STORE_get1_objects does not exist in current OpenSSLs, but we can
polyfill it with X509_STORE_lock and X509_STORE_unlock.
Dino Viehland [Thu, 15 Feb 2024 18:54:57 +0000 (10:54 -0800)]
gh-113743: Make the MRO cache thread-safe in free-threaded builds (#113930)
Makes _PyType_Lookup thread safe, including:
Thread safety of the underlying cache.
Make mutation of mro and type members thread safe
Also _PyType_GetMRO and _PyType_GetBases are currently returning borrowed references which aren't safe.
Sam Gross [Thu, 15 Feb 2024 13:37:54 +0000 (08:37 -0500)]
gh-115432: Add critical section variant that handles a NULL object (#115433)
This adds `Py_XBEGIN_CRITICAL_SECTION` and
`Py_XEND_CRITICAL_SECTION`, which accept a possibly NULL object as an
argument. If the argument is NULL, then nothing is locked or unlocked.
Otherwise, they behave like `Py_BEGIN/END_CRITICAL_SECTION`.
T. Wouters [Thu, 15 Feb 2024 13:24:13 +0000 (14:24 +0100)]
gh-115490: Work around test.support.interpreters.channels not handling unloading (#115515)
Work around test.support.interpreters.channels not handling unloading, which
regrtest does when running tests sequentially, by explicitly skipping the
unloading of test.support.interpreters and its submodules.
This can be rolled back once test.support.interpreters.channels supports
unloading, if we are keeping sequential runs in the same process around.
Martijn Pieters [Thu, 15 Feb 2024 11:08:45 +0000 (11:08 +0000)]
bpo-38364: unwrap partialmethods just like we unwrap partials (#16600)
* bpo-38364: unwrap partialmethods just like we unwrap partials
The inspect.isgeneratorfunction, inspect.iscoroutinefunction and inspect.isasyncgenfunction already unwrap functools.partial objects, this patch adds support for partialmethod objects as well.
Also: Rename _partialmethod to __partialmethod__.
Since we're checking this attribute on arbitrary function-like objects,
we should use the namespace reserved for core Python.
gh-114258: Refactor Argument Clinic function name parser (#114930)
Refactor state_modulename_name() of the parsing state machine, by
adding helpers for the sections that deal with ...:
1. parsing the function name
2. normalizing "function kind"
3. dealing with cloned functions
4. resolving return converters
5. adding the function to the DSL parser
mpage [Thu, 15 Feb 2024 08:22:47 +0000 (00:22 -0800)]
gh-112050: Make collections.deque thread-safe in free-threaded builds (#113830)
Use critical sections to make deque methods that operate on mutable
state thread-safe when the GIL is disabled. This is mostly accomplished
by using the @critical_section Argument Clinic directive, though there
are a few places where this was not possible and critical sections had
to be manually acquired/released.
Eric Snow [Wed, 14 Feb 2024 23:07:22 +0000 (16:07 -0700)]
gh-115482: Assume the Main Interpreter is Always Running "main" (gh-115484)
This is a temporary fix to unblock embedders that do not call Py_Main().
_PyInterpreterState_IsRunningMain() will always return true for the main interpreter, even in corner cases where it technically should not. The (future) full solution will do the right thing in those corner cases.
Sam Gross [Wed, 14 Feb 2024 21:41:29 +0000 (16:41 -0500)]
gh-112529: Use _PyThread_Id() in mimalloc in free-threaded build (#115488)
The free-threaded GC uses mimallocs segment thread IDs to restore
the overwritten `ob_tid` thread ids in PyObjects. For that reason, it's
important that PyObjects and mimalloc use the same identifiers.
mpage [Wed, 14 Feb 2024 20:15:05 +0000 (12:15 -0800)]
gh-115041: Add wrappers that are atomic only in free-threaded builds (#115046)
These are intended to be used in places where atomics are required in
free-threaded builds but not in the default build. We don't want to
introduce the potential performance overhead of an atomic operation in the
default build.
T. Wouters [Tue, 13 Feb 2024 23:58:49 +0000 (00:58 +0100)]
gh-115421: Update the list of installed test subdirectories. (#115422)
Update the list of installed test subdirectories with all newly added
subdirectories of Lib/test, so that the tests in those directories are
properly installed.