The 'C' format code expects an `int` as a destination (not a `char`).
This led to test failures on big-endian platforms like s390x. Use the
'c' format code, which expects a `char` as the destination (but requires
a Python byte objects instead of a str).
mpage [Tue, 3 Dec 2024 19:20:20 +0000 (11:20 -0800)]
gh-115999: Enable specialization of `CALL` instructions in free-threaded builds (#127123)
The CALL family of instructions were mostly thread-safe already and only required a small number of changes, which are documented below.
A few changes were needed to make CALL_ALLOC_AND_ENTER_INIT thread-safe:
Added _PyType_LookupRefAndVersion, which returns the type version corresponding to the returned ref.
Added _PyType_CacheInitForSpecialization, which takes an init method and the corresponding type version and only populates the specialization cache if the current type version matches the supplied version. This prevents potentially caching a stale value in free-threaded builds if we race with an update to __init__.
Only cache __init__ functions that are deferred in free-threaded builds. This ensures that the reference to __init__ that is stored in the specialization cache is valid if the type version guard in _CHECK_AND_ALLOCATE_OBJECT passes.
Fix a bug in _CREATE_INIT_FRAME where the frame is pushed to the stack on failure.
A few other miscellaneous changes were also needed:
Use {LOCK,UNLOCK}_OBJECT in LIST_APPEND. This ensures that the list's per-object lock is held while we are appending to it.
Add missing co_tlbc for _Py_InitCleanup.
Stop/start the world around setting the eval frame hook. This allows us to read interp->eval_frame non-atomically and preserves the behavior of _CHECK_PEP_523 documented below.
gh-127271: Replace use of PyCell_GET/SET (gh-127272)
* Replace uses of `PyCell_GET` and `PyCell_SET`. These macros are not
safe to use in the free-threaded build. Use `PyCell_GetRef()` and
`PyCell_SetTakeRef()` instead.
* Since `PyCell_GetRef()` returns a strong rather than borrowed ref, some
code restructuring was required, e.g. `frame_get_var()` returns a strong
ref now.
* Add critical sections to `PyCell_GET` and `PyCell_SET`.
* Move critical_section.h earlier in the Python.h file.
* Add `PyCell_GET` to the free-threading howto table of APIs that return
borrowed refs.
gh-115999: Add free-threaded specialization for `SEND` (gh-127426)
No additional thread safety changes are required. Note that sending to
a generator that is shared between threads is currently not safe in the
free-threaded build.
mpage [Tue, 3 Dec 2024 17:50:58 +0000 (09:50 -0800)]
gh-127421: Fix race in test_start_new_thread_failed (#127549)
Fix race in test_start_new_thread_failed
When we succeed in starting a new thread, for example if setrlimit
was ineffective, we must wait for the newly spawned thread to exit.
Otherwise, we run the risk that the newly spawned thread will race
with runtime finalization and access memory that has already been
clobbered/freed.
`_thread.start_new_thread()` only spawns daemon threads, which the runtime
does not wait for at shutdown, and does not return a handle. Use
`_thread.start_joinable_thread()` and join the resulting handle when
the thread is started successfully.
gh-115999: Specialize `LOAD_SUPER_ATTR` in free-threaded builds (gh-127128)
Use existing helpers to atomically modify the bytecode. Add unit tests
to ensure specializing is happening as expected. Add test_specialize.py
that can be used with ThreadSanitizer to detect data races.
Fix thread safety issue with cell_set_contents().
Previously, `traceback.print_list` didn't have a documentation entry and was not exposed in `traceback.__all__`. Now it has a documentation entry and is exposed in `__all__`.
Hood Chatham [Mon, 2 Dec 2024 23:30:24 +0000 (00:30 +0100)]
gh-127111: Emscripten Make web example work again (#127113)
Moves the Emscripten web example into a standalone folder, and updates
Makefile targets to build the web example. Instructions for usage have
also been added.
Sam Gross [Mon, 2 Dec 2024 19:38:26 +0000 (19:38 +0000)]
gh-127521: Mark list as "shared" before resizing if necessary (#127524)
In the free threading build, if a non-owning thread resizes a list,
it must use QSBR to free the old list array because there may be a
concurrent access (without a lock) from the owning thread.
To match the pattern in dictobject.c, we just mark the list as "shared"
before resizing if it's from a non-owning thread and not already marked
as shared.
Barney Gale [Sat, 30 Nov 2024 18:39:39 +0000 (18:39 +0000)]
GH-127381: pathlib ABCs: remove `PathBase.cwd()` and `home()` (#127427)
These classmethods presume that the user has retained the original
`__init__()` signature, which may not be the case. Also, many virtual
filesystems don't provide current or home directories.
Илья Любавский [Fri, 29 Nov 2024 09:00:50 +0000 (12:00 +0300)]
gh-127303: Add docs for token.EXACT_TOKEN_TYPES (#127304)
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Tomas R. <tomas.roun8@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Petr Viktorin [Thu, 28 Nov 2024 12:29:27 +0000 (13:29 +0100)]
gh-127330: Update for OpenSSL 3.4 & document+improve the update process (GH-127331)
- Add `git describe` output to headers generated by `make_ssl_data.py`
This info is more important than the date when the file was generated.
It does mean that the tool now requires a Git checkout of OpenSSL,
not for example a release tarball.
- Regenerate the older file to add the info.
To the other older file, add a note about manual edits.
Serhiy Storchaka [Wed, 27 Nov 2024 11:38:12 +0000 (13:38 +0200)]
gh-124008: Fix calculation of the number of written bytes for the Windows console (GH-124059)
Since MultiByteToWideChar()/WideCharToMultiByte() is not reversible if
the data contains invalid UTF-8 sequences, use binary search to
calculate the number of written bytes from the number of written
characters.
gh-69639: Add mixed-mode rules for complex arithmetic (C-like) (GH-124829)
"Generally, mixed-mode arithmetic combining real and complex variables should
be performed directly, not by first coercing the real to complex, lest the sign
of zero be rendered uninformative; the same goes for combinations of pure
imaginary quantities with complex variables." (c) Kahan, W: Branch cuts for
complex elementary functions.
This patch implements mixed-mode arithmetic rules, combining real and
complex variables as specified by C standards since C99 (in particular,
there is no special version for the true division with real lhs
operand). Most C compilers implementing C99+ Annex G have only these
special rules (without support for imaginary type, which is going to be
deprecated in C2y).
gh-69639: Add mixed-mode rules for complex arithmetic (C-like) (GH-124829)
"Generally, mixed-mode arithmetic combining real and complex variables should
be performed directly, not by first coercing the real to complex, lest the sign
of zero be rendered uninformative; the same goes for combinations of pure
imaginary quantities with complex variables." (c) Kahan, W: Branch cuts for
complex elementary functions.
This patch implements mixed-mode arithmetic rules, combining real and
complex variables as specified by C standards since C99 (in particular,
there is no special version for the true division with real lhs
operand). Most C compilers implementing C99+ Annex G have only these
special rules (without support for imaginary type, which is going to be
deprecated in C2y).
When handed an absolute Windows path such as `C:\foo` or `//server/share`,
the `urllib.request.pathname2url()` function returns a URL with an
authority section, such as `///C:/foo` or `//server/share` (or before
GH-126205, `////server/share`). Only the `file:` prefix is omitted.
But when handed an absolute POSIX path such as `/etc/hosts`, or a Windows
path of the same form (rooted but lacking a drive), the function returns a
URL without an authority section, such as `/etc/hosts`.
This patch corrects the discrepancy by adding a `//` prefix before
drive-less, rooted paths when generating URLs.
The interpreter now handles `_PyStackRef`s pointing to immortal objects
without the deferred bit set, so `_PyEvalFramePushAndInit_UnTagged` is
no longer necessary.