gh-117505: Run ensurepip in isolated env in Windows installer (GH-118257)
ensurepip forks a subprocess to run pip itself, but that subprocess only inherits a -I isolated mode flag (see _run_pip() in Lib/ensurepip/__init__.py), not the "-E -s" flags that the installer has been using. This means that parts of ensurepip don't actually run in an isolated environment and can make incorrect decisions based on packages installed in the user site-packages.
(cherry picked from commit c9073eb1a99606df1efeb8959e9f11a8ebc23ae2)
Co-authored-by: Michael Vincent <377567+Vynce@users.noreply.github.com>
[3.13] gh-119247: Add macros to use PySequence_Fast safely in free-threaded build (GH-119315) (#119419)
Add `Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST` and
`Py_END_CRITICAL_SECTION_SEQUENCE_FAST` macros and update `str.join` to use
them. Also add a regression test that would crash reliably without this
patch.
(cherry picked from commit baf347d91643a83483bae110092750d39471e0c2)
[3.13] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters (gh-119331) (gh-119410)
_PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed.
This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple.
This change also fixes a bug where only the most recent parser was added to the global linked list.
[3.13] gh-118643: Fix AttributeError in the email module (GH-119099) (GH-119389)
Fix regression introduced in gh-100884: AttributeError when re-fold a long
address list.
Also fix more cases of incorrect encoding of the address separator in the
address list missed in gh-100884.
(cherry picked from commit 858b9e85fcdd495947c9e892ce6e3734652c48f2)
[3.13] GH-119292: Add job to JIT CI to build and test with --disable-gil (GH-119314)
(cherry picked from commit c4722cd0573c83aaa52b63a27022b9048a949f54) Co-authored-by: Savannah Ostrowski <savannahostrowski@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
[3.13] gh-119102: Fix REPL for dumb terminal (GH-119269) (#119308)
gh-119102: Fix REPL for dumb terminal (GH-119269)
Use CAN_USE_PYREPL of _pyrepl.__main__ in the site module to decide
if _pyrepl.write_history_file() can be used.
(cherry picked from commit 73f4a58d36b65ec650e8f00b2affc4a4d3195f0c)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.13] Use `fail-fast: false` in `mypy.yml` (GH-119297) (#119304)
Use `fail-fast: false` in `mypy.yml` (GH-119297)
See docs about this setting: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actionsGH-jobsjob_idstrategyfail-fast
(cherry picked from commit b36533290608aed757f6eb16869a679650d32e17)
[3.13] gh-74929: PEP 667 general docs update (gh-119291)
* expand on What's New entry for PEP 667 (including porting notes)
* define 'optimized scope' as a glossary term
* cover comprehensions and generator expressions in locals() docs
* review all mentions of "locals" in documentation (updating if needed)
* review all mentions of "f_locals" in documentation (updating if needed)
[3.13] gh-119050: Add XML support to libregrtest refleak checker (GH-119148) (#119270)
gh-119050: Add XML support to libregrtest refleak checker (GH-119148)
regrtest test runner: Add XML support to the refleak checker
(-R option).
* run_unittest() now stores XML elements as string, rather than
objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
checking for reference leaks. Save XML into a temporary file.
(cherry picked from commit 9257731f5d3e9d4f99e314b23a14506563e167d7)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.13] gh-92081: Fix for email.generator.Generator with whitespace between encoded words. (GH-92281) (#119245)
* Fix for email.generator.Generator with whitespace between encoded words.
email.generator.Generator currently does not handle whitespace between
encoded words correctly when the encoded words span multiple lines. The
current generator will create an encoded word for each line. If the end
of the line happens to correspond with the end real word in the
plaintext, the generator will place an unencoded space at the start of
the subsequent lines to represent the whitespace between the plaintext
words.
A compliant decoder will strip all the whitespace from between two
encoded words which leads to missing spaces in the round-tripped
output.
The fix for this is to make sure that whitespace between two encoded
words ends up inside of one or the other of the encoded words. This
fix places the space inside of the second encoded word.
A second problem happens with continuation lines. A continuation line that
starts with whitespace and is followed by a non-encoded word is fine because
the newline between such continuation lines is defined as condensing to
a single space character. When the continuation line starts with whitespace
followed by an encoded word, however, the RFCs specify that the word is run
together with the encoded word on the previous line. This is because normal
words are filded on syntactic breaks by encoded words are not.
The solution to this is to add the whitespace to the start of the encoded word
on the continuation line.
[3.13] DOCS: Suggest always calling exec with a globals argument and no locals argument (GH-119235) (#119239)
DOCS: Suggest always calling exec with a globals argument and no locals argument (GH-119235)
Many users think they want a locals argument for various reasons but they do not
understand that it makes code be treated as a class definition. They do not want
their code treated as a class definition and get surprised. The reason not
to pass locals specifically is that the following code raises a `NameError`:
```py
exec("""
def f():
print("hi")
f()
def g():
f()
g()
""", {}, {})
```
The reason not to leave out globals is as follows:
[3.13] marshal docs: Remove reference to "Sun" (GH-119161) (#119167)
Nobody has been using a Sun machine for a long time. When I saw
this sentence in a lightning talk just now, I thought it was talking
about sending Python code on a spacecraft.
(cherry picked from commit 697465ff88e49d98443025474e5b534adfba2cb0)
[3.13] gh-118760: Fix errors in calling Tkinter bindings on Windows (GH-118782) (GH-119072)
For unknown reasons some arguments for Tkinter binding can be created
as a 1-tuple containing a Tcl_Obj when wantobjects is 2.
(cherry picked from commit 5b88d95cc542cf02303c6fe0e8719a93544decdb)
[3.13] 3.13 What's New: Add PEP 702 (GH-118922) (#119062)
I honestly forgot this slipped into 3.13, but I think it's worth highlighting more, as it is a PEP-sized change that makes the type system significantly more powerful.
gh-118876: Ensure PC/layout sets ns.temp before using it (GH-118880)
Fixes an AttributeError that occurs when checking if ns.temp is an absolute path during building from source on Windows.
(cherry picked from commit d8a82cca12e12a6b22bfe6691e9b222f6d276f0a)
[3.13] gh-117655: Prevent `test_strptime` from raising a DeprecationWarning (GH-117668) (GH-118956)
gh-117655: Prevent `test_strptime` from raising a DeprecationWarning (GH-117668)
* Fix `test_strptime` raises a DeprecationWarning
* Ignore deprecation warnings where appropriate.
* Update Lib/test/datetimetester.py
This is follow on work to silence unnecessary warnings from the test suite that changes for https://github.com/python/cpython/issues/70647 added.
(cherry picked from commit abead548af0172dabba13da8bacf2da3c02d4927)
[3.13] gh-117657: Log TSAN warnings to separate files and archive them (GH-118747) (#118931)
This ensures we don't lose races that occur in subprocesses or
interleave races from workers running in parallel.
Log files are collected and packaged into a zipfile that can be
downloaded from the "Artifacts" section of the workflow run.
(cherry picked from commit b88889e9ffd7b2d2bdac75aecbf14e37fd68e337)
[3.13] gh-118846: Fix free-threading test failures when run sequentially (GH-118864) (#118927)
The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
(cherry picked from commit b309c8ebff011f27012367b046ff92eecbdd68a5)
`_Py_qsbr_unregister` is called when the PyThreadState is already
detached, so the access to `tstate->qsbr` isn't safe without locking the
shared mutex. Grab the `struct _qsbr_shared` from the interpreter
instead.
(cherry picked from commit 33d20199af65c741bdc908a968edd8dc179b6974)
_PyWeakref_ClearRef was previously exposed in the public C-API, although
it begins with an underscore and is not documented. It's used by a few
C-API extensions. There is currently no alternative public API that can
replace its use.
_PyWeakref_ClearWeakRefsExceptCallbacks is the only thread-safe way to
use _PyWeakref_ClearRef in the free-threaded build. This exposes the C
symbol, but does not make the API public.
(cherry picked from commit db5af7da092409030c9fbe0a3a986bd0ee441b8b)