Kumar Aditya [Tue, 18 Feb 2025 12:08:25 +0000 (17:38 +0530)]
improve `test_tasks` to use correct idiom for starting task in asyncio (#130257)
The test should use the correct idiom for starting the task, `loop._run_once` is private API which should not be used directly, instead use `asyncio.sleep(0)` for 1 event loop cycle.
Barney Gale [Mon, 17 Feb 2025 19:15:25 +0000 (19:15 +0000)]
GH-125413: Add private metadata methods to `pathlib.Path.info` (#129897)
Add the following private methods to `pathlib.Path.info`:
- `_posix_permissions()`: the POSIX file permissions (`S_IMODE(st_mode)`)
- `_file_id()`: the file ID (`(st_dev, st_ino)`)
- `_access_time_ns()`: the access time in nanoseconds (`st_atime_ns`)
- `_mod_time_ns()`: the modify time in nanoseconds (`st_mtime_ns`)
- `_bsd_flags()`: the BSD file flags (`st_flags`)
- `_xattrs()`: the file extended attributes as a list of key, value pairs,
or an empty list if `listxattr()` or `getxattr()` fail in an ignorable
way.
These methods replace `LocalCopyReader.read_metadata()`, and so we can
delete the `CopyReader` and `LocalCopyReader` classes. Rather than reading
metadata via `source._copy_reader.read_metadata()`, we instead call
`source.info._posix_permissions()`, `_access_time_ns()`, etc.
Preserving metadata is only supported for local-to-local copies at the
moment. To support copying metadata between arbitrary `ReadablePath` and
`WritablePath` objects, we'd need to make the new methods public and
documented.
Sam Gross [Mon, 17 Feb 2025 13:15:40 +0000 (08:15 -0500)]
gh-129701: Fix a data race in `intern_common` in the free threaded build (GH-130089)
* gh-129701: Fix a data race in `intern_common` in the free threaded build
* Use a mutex to avoid potentially returning a non-immortalized string,
because immortalization happens after the insertion into the interned
dict.
* Use `Py_DECREF()` calls instead of `Py_SET_REFCNT(s, Py_REFCNT(s) - 2)`
for thread-safety. This code path isn't performance sensistive, so
just use `Py_DECREF()` unconditionally for simplicity.
Bénédikt Tran [Mon, 17 Feb 2025 12:12:03 +0000 (13:12 +0100)]
gh-111178: fix UBSan failures in `Modules/_collectionsmodule.c` (#129773)
Fix some UBSan failures for `dequeobject`, `dequeiterobject`, `defdictobject` and `tuplegetterobject`.
We also perform some cleanup by suppressing unused return values and renaming the
unused argument in `METH_NOARGS` methods to `dummy` for semantic purposes.
Barney Gale [Sun, 16 Feb 2025 17:08:55 +0000 (17:08 +0000)]
pathlib ABCs: remove caching of path parser case sensitivity (#130194)
Remove the caching `_is_case_sensitive()` function.
The cache used to speed up `PurePath.[full_]match()` and `Path.[r]glob()`,
but that's no longer the case - these methods use
`self.parser is posixpath` to determine case sensitivity.
This makes the `pathlib._abc` module a little easier to backport to Python
3.8, where `functools.cache()` is unavailable.
Stan Ulbrych [Fri, 14 Feb 2025 17:16:47 +0000 (17:16 +0000)]
gh-82045: Correct and deduplicate "isprintable" docs; add test. (GH-130118)
We had the definition of what makes a character "printable" documented in three places, giving two different definitions.
The definition in the comment on `_PyUnicode_IsPrintable` was inverted; correct that.
With that correction, the two definitions turn out to be equivalent -- but to confirm that, you have to go look up, or happen to know, that those are the only five "Other" categories and only three "Separator" categories in the Unicode character database. That makes it hard for the reader to tell whether they really are the same, or if there's some subtle difference in the intended semantics.
Fix that by cutting the C API docs' and the C comment's copies of the subtle details, in favor of referring to the Python-level docs. That ensures it's explicit that these are all meant to agree, and also lets us concentrate improvements to the wording in one place.
Speaking of which, borrow some ideas from the C comment, along with other tweaks, to hopefully add a bit more clarity to that one newly-centralized copy in the docs.
Also add a thorough test that the implementation agrees with this definition.
Sam Gross [Thu, 13 Feb 2025 17:29:03 +0000 (12:29 -0500)]
gh-128130: Fix unhandled keyboard interrupt data race (gh-129975)
Use an atomic operation when setting
`_PyRuntime.signals.unhandled_keyboard_interrupt`. We now only clear the
variable at the start of `_PyRun_Main`, which is the same function where
we check it.
This avoids race conditions where previously another thread might call
`run_eval_code_obj()` and erroneously clear the unhandled keyboard
interrupt.
Sam Gross [Thu, 13 Feb 2025 16:50:45 +0000 (11:50 -0500)]
gh-130019: Fix data race in _PyType_AllocNoTrack (gh-130058)
The reference count fields, such as `ob_tid` and `ob_ref_shared`, may be
accessed concurrently in the free threading build by a `_Py_TryXGetRef`
or similar operation. The PyObject header fields will be initialized by
`_PyObject_Init`, so only call `memset()` to zero-initialize the remainder
of the allocation.
Ammar Askar [Thu, 13 Feb 2025 01:43:09 +0000 (20:43 -0500)]
gh-96092: Fix traceback.walk_stack(None) skipping too many frames (#129330)
As it says in its documentation, walk_stack was meant to just
follow `f.f_back` like other functions in the traceback module.
Instead it was previously doing `f.f_back.f_back` and then this
changed to `f_back.f_back.f_back.f_back' in Python 3.11 breaking
its behavior for external users.
This happened because the walk_stack function never really had
any good direct tests and its only consumer in the traceback module was
`extract_stack` which passed the result into `StackSummary.extract`.
As a generator, it was previously capturing the state of the stack
when it was first iterated over, rather than the stack when `walk_stack`
was called. Meaning when called inside the two method deep
`extract` and `extract_stack` calls, two `f_back`s were needed.
When 3.11 modified the sequence of calls in `extract`, two more
`f_back`s were needed to make the tests happy.
This changes the generator to capture the stack when `walk_stack` is
called, rather than when it is first iterated over. Since this is
technically a breaking change in behavior, there is a versionchanged
to the documentation. In practice, this is unlikely to break anyone,
you would have been needing to store the result of `walk_stack` and
expecting it to change.
Sam Gross [Wed, 12 Feb 2025 23:09:15 +0000 (18:09 -0500)]
gh-130030: Fix crash on 32-bit Linux with free threading (gh-130043)
The `gc_get_refs` assertion needs to be after we check the alive and
unreachable bits. Otherwise, `ob_tid` may store the actual thread id
instead of the computed `gc_refs`, which may trigger the assertion if
the `ob_tid` looks like a negative value.
donBarbos [Wed, 12 Feb 2025 14:54:22 +0000 (18:54 +0400)]
gh-109798: Normalize `_datetime` and `datetime` error messages (#127345)
Updates error messages in datetime and makes them consistent between Python and C.
---------
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
Sam Gross [Wed, 12 Feb 2025 14:34:40 +0000 (09:34 -0500)]
gh-128759: Fix accesses to `tp_version_tag`. (GH-129750)
We should use a relaxed atomic load in the free threading build in
`PyType_Modified()` because that's called without the type lock held.
It's not necessary to use atomics in `type_modified_unlocked()`.
We should also use `FT_ATOMIC_STORE_UINT_RELAXED()` instead of the
`UINT32` variant because `tp_version_tag` is declared as `unsigned int`.
Sam Gross [Tue, 11 Feb 2025 22:29:27 +0000 (17:29 -0500)]
gh-129967: Fix race condition in `repr(set)` (gh-129978)
The call to `PySequence_List()` could temporarily unlock and relock the
set, allowing the items to be cleared and return the incorrect
notation `{}` for a empty set (it should be `set()`).
Sam Gross [Tue, 11 Feb 2025 21:53:08 +0000 (16:53 -0500)]
gh-117657: Enable test_opcache under TSAN (GH-129831)
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:
* Use relaxed atomics when clearing `ht->_spec_cache.getitem`
(gh-115999)
* Add temporary suppression for type slot modifications (gh-127266)
* Use atomic load when reading `*dictptr`