[3.12] gh-109496: Detect Py_DECREF() after dealloc in debug mode (GH-109539) (#109545)
gh-109496: Detect Py_DECREF() after dealloc in debug mode (GH-109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().
[3.12] gh-109474: Update two Unix packaging URLs (GH-109307) (#109477)
gh-109474: Update two Unix packaging URLs (GH-109307)
update packaging URLs
fix a broken URL for fedora RPM packaging guide and fix a URL redirect for Slackware packaging guide.
(cherry picked from commit 0b38ce440bd76b3d25b6d042ee9613841fb4a947)
We're not fixing CGIHTTPRequestHandler as it is deprecated in 3.13 to go
away in 3.15. This just removes noise from our test suite when warnings
are rightfully enabled.
If the long pre-existing fork+threading mix here ever causes anyone
deadlocks as is possible, disabling the test entirely on that platform
makes sense rather than attempting to fix
http.server.CGIHTTPRequestHandler or refactor to not use a threaded
server in the test.
[3.12] gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode (GH-109423) (#109426)
gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode (GH-109423)
Use a longer key: FIPS mode requires at least of at least 112 bits.
The previous key was only 32 bits.
(cherry picked from commit e091b9f20fa8e409003af79f3c468b8225e6dcd3)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Update exclude list for 3.12
* Explicitly exclude files which failed to lint/parse
* Sort to avoid future merge conflicts
---------
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
[3.12] Fix variable name in dis documentation example (GH-109343) (#109364)
Fix variable name in dis documentation example (GH-109343)
BINARY_SUBSCR example erroneously uses two different names `key` and `index` to refer to the same variable. STORE_SUBSCR and DELETE_SUBSCR use only `key` in the same context. Changing `index` to `key` for consistency.
(cherry picked from commit a0c06a4f933faccd7f8201701b2491d38464212c)
Co-authored-by: Thomas Grainger <tagrain@gmail.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
[3.12] gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360) (#109361)
gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360)
Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex:
Fedora 38). Search patterns in gdb "bt" command output to detect
when gdb fails to retrieve the traceback. For example, skip a test if
"Backtrace stopped: frame did not save the PC" is found.
(cherry picked from commit 44d9a71ea246e7c3fb478d9be62c16914be6c545)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-109295: Fix test_os.test_access_denied() for TEMP=cwd (GH-109299) (#109304)
gh-109295: Fix test_os.test_access_denied() for TEMP=cwd (GH-109299)
Fix test_os.test_access_denied() when the TEMP environment variable
is equal to the current working directory. Run the test using a
different filename, since self.fname already exists in this case.
(cherry picked from commit 7dedfd36dc16d9e1e15d7d0b0a636dd401a5a543)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-109295: Clean up multiprocessing in test_asyncio and test_compileall (GH-109298) (#109301)
gh-109295: Clean up multiprocessing in test_asyncio and test_compileall (GH-109298)
test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
(cherry picked from commit 09ea4b8706165fd9474165090a0ba86509abd6c8)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-109230: test_pyexpat no longer depends on the current directory (GH-109233) (#109241)
gh-109230: test_pyexpat no longer depends on the current directory (GH-109233)
Fix test_pyexpat.test_exception(): it can now be run from a directory
different than Python source code directory. Before, the test failed
in this case.
Skip the test if Modules/pyexpat.c source is not available. Skip also
the test on Python implementations other than CPython.
(cherry picked from commit e55aab95786e0e9fb36a9a1122d2d0fb3d2403cd)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-109237: Fix test_site for non-ASCII working directory (GH-109238) (#109239)
gh-109237: Fix test_site for non-ASCII working directory (GH-109238)
Fix test_site.test_underpth_basic() when the working directory
contains at least one non-ASCII character: encode the "._pth" file to
UTF-8 and enable the UTF-8 Mode to use UTF-8 for the child process
stdout.
(cherry picked from commit cbb3a6f8ada3d133c3ab9f9465b65067fce5bb42)
Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.12] gh-109118: Fix runtime crash when NameError happens in PEP 695 function (GH-109123).
(cherry picked from commit 17f994174de9211b2baaff217eeb1033343230fc)
[3.12] gh-103186: Suppress RuntimeWarning about unclosed async iterator in test_sys_settrace (GH-109075) (#109138)
gh-103186: Suppress RuntimeWarning about unclosed async iterator in test_sys_settrace (GH-109075)
(cherry picked from commit d485551c9d1792ff3539eef1d6374bd4c01dcd5d)
This un-skips this test now that pythontest.net implements appropriate firewall
rules for it.
(cherry picked from commit 1829a3c9a3712b6a68a3a449e4a08787c73da51d)
Co-authored-by: Colin Watson <cjwatson@debian.org> Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.
* "make regen-global-objects": Make sure that deepfreeze.c is
generated and up to date, and always run "make clinic".
* "make clinic" no longer runs generate_global_objects.py script.
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
"make clinic". They are now outdated, these commands are now
safe to use.
* Document generates files in Doc/using/configure.rst.
* Add get_recursion_available() and get_recursion_depth() functions
to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
recursion limit and compute the maximum nested array/dict depending
on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
and sys.setrecursionlimit().
(cherry picked from commit 8ff11425783806f8cb78e99f667546b1f7f3428e)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (GH-108999)
Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.
(cherry picked from commit fbce43a251488f666be9794c908a6613bf8ae260)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-108962: Skip test_tempfile.test_flags() if not supported (GH-108964) (#108967)
gh-108962: Skip test_tempfile.test_flags() if not supported (GH-108964)
Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).
(cherry picked from commit cd2ef21b076b494224985e266c5f5f8b37c66618)
Co-authored-by: Victor Stinner <vstinner@python.org>
Victor Stinner [Fri, 8 Sep 2023 13:14:17 +0000 (15:14 +0200)]
[3.12] gh-108834: Sync libregrtest with the main branch (#108966)
* gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.
Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.
Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.
Changes:
* Fix and enhance statistics in regrtest summary. Add "(filtered)"
when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
[3.12] gh-107732: Mention dir support in importlib.resources docs (GH-107734) (#109058)
gh-107732: Mention dir support in importlib.resources docs (#107734)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
(cherry picked from commit 9f0c0a46f00d687e921990ee83894b2f4ce8a6e7)
Skip test_asyncio, test_imaplib and test_socket tests if FreeBSD TCP
blackhole is enabled (net.inet.tcp.blackhole=2).
(cherry picked from commit a52a3509770f29f940cda9307704908949912276)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Mikhail Samylov <Samylov-Mikhail@yandex.com> Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Anthony Shaw <anthony.p.shaw@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
[3.12] C API tests: use special markers to test that output parameters were set (GH-109014) (#109023)
[3.12] C API tests: use special markers to test that output parameters were set (GH-109014).
(cherry picked from commit bf414b7fcb7c8ba780a5e1d9f320ecef0c7f9488)
Output with two wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl::Found more than one wheel for package pip.
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Found more than one wheel for package pip.
```
Output without wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=::Could not find a pip wheel on disk.
```
(cherry picked from commit f8a047941f2e4a1848700c21d58a08c9ec6a9c68)