Bénédikt Tran [Tue, 25 Feb 2025 10:44:59 +0000 (11:44 +0100)]
gh-111178: fix UBSan failures in `Objects/typeobject.c` (#129799)
Fix UBSan failures for `PyTypeObject`.
Introduce a macro cast for `superobject` and remove redundant casts.
Rename the unused parameter in getter/setter methods to `closure`
for semantic purposes.
Barney Gale [Mon, 24 Feb 2025 19:10:50 +0000 (19:10 +0000)]
GH-125413: Fix stale metadata from `pathlib.Path.copy()` and `move()` (#130424)
In `pathlib.Path.copy()` and `move()`, return a fresh `Path` object with an
unpopulated `info` attribute, rather than a `Path` object with information
recorded *prior* to the path's creation.
Bénédikt Tran [Mon, 24 Feb 2025 12:38:18 +0000 (13:38 +0100)]
gh-111178: fix UBSan failures in `Modules/selectmodule.c` (GH-129792)
Fix some UBSan failures for `pollObject`, `devpollObject`, `pyEpoll_Object` as well as
for `kqueue_event_Object`, `kqueue_queue_Object` and `kqueue_tracking_after_fork`.
Suppress unused return values.
Rename the unused parameter in `METH_NOARGS` and getter/setter methods to
`dummy` and `closure` respectively for semantic purposes.
Explicitly declare `_select_exec` as a `static` function.
Kanishk Pachauri [Mon, 24 Feb 2025 02:02:34 +0000 (07:32 +0530)]
gh-130160: use `.. program::` directive for documenting `idle` CLI (#130278)
--------- Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
mingyu [Sun, 23 Feb 2025 20:07:33 +0000 (05:07 +0900)]
gh-129948: Add `set()` to `multiprocessing.managers.SyncManager` (#129949)
The SyncManager provided support for various data structures such as dict, list, and queue, but oddly, not set.
This introduces support for set by defining SetProxy and registering it with SyncManager.
--- Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
Y5 [Sun, 23 Feb 2025 19:30:33 +0000 (03:30 +0800)]
gh-124096: Enable REPL virtual terminal support on Windows (#124119)
To support virtual terminal mode in Windows PYREPL, we need a scanner
to read over the supported escaped VT sequences.
Windows REPL input was using virtual key mode, which does not support
terminal escape sequences. This patch calls `SetConsoleMode` properly
when initializing and send sequences to enable bracketed-paste modes
to support verbatim copy-and-paste.
Signed-off-by: y5c4l3 <y5c4l3@proton.me> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Dustin L. Howett <dustin@howett.net> Co-authored-by: wheeheee <104880306+wheeheee@users.noreply.github.com>
Bénédikt Tran [Sun, 23 Feb 2025 10:34:11 +0000 (11:34 +0100)]
gh-111178: fix UBSan failures in `Modules/_struct.c` (#129793)
Fix some UBSan failures for `PyStructObject` and `unpackiterobject`.
We also perform some cleanup by suppressing unused return values and renaming the
unused parameter in `METH_NOARGS` and getter methods to `dummy` and `closure`
respectively for semantic purposes.
Barney Gale [Fri, 21 Feb 2025 17:47:45 +0000 (17:47 +0000)]
GH-128520: More consistent type-checking behaviour in pathlib (#130199)
In the following methods, skip casting of the argument to a path object if
the argument has a `with_segments` attribute. In `PurePath`:
`relative_to()`, `is_relative_to()`, `match()`, and `full_match()`. In
`Path`: `rename()`, `replace()`, `copy()`, `copy_into()`, `move()`, and
`move_into()`.
Previously the check varied a bit from method to method. The `PurePath`
methods used `isinstance(arg, PurePath)`; the `rename()` and `replace()`
methods always cast, and the remaining `Path` methods checked for a private
`_copy_writer` attribute.
We apply identical changes to relevant methods of the private ABCs. This
improves performance a bit, because `isinstance()` checks on ABCs are
expensive.
Bénédikt Tran [Fri, 21 Feb 2025 14:08:56 +0000 (15:08 +0100)]
gh-111178: fix UBSan failures in `Modules/_functoolsmodule.c` (GH-129778)
Fix UBSan failures for `keyobject`, `lru_list_elem`, `lru_cache_object`.
Suppress unused return values.
Change `_PyPartialObject_CAST` (from #124733) to `partialobject_CAST`
for consistency with the current style for these macros
Sam Gross [Thu, 20 Feb 2025 16:31:15 +0000 (11:31 -0500)]
gh-111924: Fix data races when swapping allocators (gh-130287)
CPython current temporarily changes `PYMEM_DOMAIN_RAW` to the default
allocator during initialization and shutdown. The motivation is to
ensure that core runtime structures are allocated and freed using the
same allocator. However, modifying the current allocator changes global
state and is not thread-safe even with the GIL. Other threads may be
allocating or freeing objects use PYMEM_DOMAIN_RAW; they are not
required to hold the GIL to call PyMem_RawMalloc/PyMem_RawFree.
This adds new internal-only functions like `_PyMem_DefaultRawMalloc`
that aren't affected by calls to `PyMem_SetAllocator()`, so they're
appropriate for Python runtime initialization and finalization. Use
these calls in places where we previously swapped to the default raw
allocator.
Newer GCC versions accept both __attribute__((no_sanitize("undefined")))
and __attribute__((no_sanitize_undefined)) so check that the macro is
not already defined.
Sam Gross [Thu, 20 Feb 2025 15:13:44 +0000 (10:13 -0500)]
gh-129269: Exclude everything in sys.path in `test_coverage_ignore` (gh-130133)
The `test_trace.test_coverage_ignore` test would fail if you had
`setuptools` installed, such as in `~/.local/lib/python3.xxx/site-packages/`.
Ignore everything in `sys.path` when running the test.
Serhiy Storchaka [Thu, 20 Feb 2025 09:08:49 +0000 (11:08 +0200)]
gh-127750: Restore inspect and pydoc support of singledispatchmethod (GH-130309)
The code is still flawed, because it does not recognize class and static
methods, and the first argument is not removed from the signature of
bound methods, but at least it does not worse than in 3.13 and older.
Stan Ulbrych [Thu, 20 Feb 2025 08:30:57 +0000 (08:30 +0000)]
gh-69001: Convert links to more usable buttons (#129591)
In Lib/idlelib/help_about.py, there are 2 links that are not underlined or colored, hence hard to tell that clicking works. All buttons might get converted to colored and underlined links in the future.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
gh-130293: Ensure test__colorize will pass on dumb terminals. (#130333)
Ensure colorize tests will run on dumb terminals (or environment with TERM=dumb set) Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Kumar Aditya [Wed, 19 Feb 2025 16:34:49 +0000 (22:04 +0530)]
gh-129898: per-thread current task implementation in asyncio (#129899)
Store the current running task on the thread state, it makes it thread safe for the free-threading build and while improving performance as there is no lock contention, this effectively makes it lock free.
When accessing the current task of the current running loop in current thread, no locking is required and can be acessed without locking.
In the rare case of accessing current task of a loop running in a different thread, the stop the world pauses is used in free-threading builds to stop all other running threads and find the task for the specified loop.
This also makes it easier for external introspection to find the current task, and now it will be always correct.
T. Wouters [Wed, 19 Feb 2025 00:52:46 +0000 (16:52 -0800)]
gh-115999: Make list and tuple iteration more thread-safe. (#128637)
Make tuple iteration more thread-safe, and actually test concurrent iteration of tuple, range and list. (This is prep work for enabling specialization of FOR_ITER in free-threaded builds.) The basic premise is:
Iterating over a shared iterable (list, tuple or range) should be safe, not involve data races, and behave like iteration normally does.
Using a shared iterator should not crash or involve data races, and should only produce items regular iteration would produce. It is not guaranteed to produce all items, or produce each item only once. (This is not the case for range iteration even after this PR.)
Providing stronger guarantees is possible for some of these iterators, but it's not always straight-forward and can significantly hamper the common case. Since iterators in general aren't shared between threads, and it's simply impossible to concurrently use many iterators (like generators), better to make sharing iterators without explicit synchronization clearly wrong.
Specific issues fixed in order to make the tests pass:
- List iteration could occasionally fail an assertion when a shared list was shrunk and an item past the new end was retrieved concurrently. There's still some unsafety when deleting/inserting multiple items through for example slice assignment, which uses memmove/memcpy.
- Tuple iteration could occasionally crash when the iterator's reference to the tuple was cleared on exhaustion. Like with list iteration, in free-threaded builds we can't safely and efficiently clear the iterator's reference to the iterable (doing it safely would mean extra, slow refcount operations), so just keep the iterable reference around.
Sam Gross [Tue, 18 Feb 2025 23:02:42 +0000 (18:02 -0500)]
gh-130094: Fix race conditions in `importlib` (gh-130101)
Entries may be added or removed from `sys.meta_path` concurrently. For
example, setuptools temporarily adds and removes the `distutils` finder from
the beginning of the list. The local copy ensures that we don't skip over any
entries.
Some packages modify `sys.modules` during import. For example, `collections`
inserts the entry for `collections.abc` into `sys.modules` during import. We
need to ensure that we re-check `sys.modules` *after* the parent module is
fully initialized.