[3.11]GH-112275: Fix HEAD_LOCK deadlock in child process after fork (#112336)
HEAD_LOCK is called from _PyEval_ReInitThreads->_PyThreadState_DeleteExcept before _PyRuntimeState_ReInitThreads reinit runtime->interpreters.mutex which might be locked before fork.
[3.11] gh-121650: Encode newlines in headers, and verify headers are sound (GH-122233) (#122608)
Per RFC 2047:
> [...] these encoding schemes allow the
> encoding of arbitrary octet values, mail readers that implement this
> decoding should also ensure that display of the decoded data on the
> recipient's terminal will not cause unwanted side-effects
It seems that the "quoted-word" scheme is a valid way to include
a newline character in a header value, just like we already allow
undecodable bytes or control characters.
They do need to be properly quoted when serialized to text, though.
Verify that email headers are well-formed.
This should fail for custom fold() implementations that aren't careful
about newlines.
[3.11] gh-122133: Authenticate socket connection for `socket.socketpair()` fallback (GH-122134) (#122426)
Authenticate socket connection for `socket.socketpair()` fallback when the platform does not have a native `socketpair` C API. We authenticate in-process using `getsocketname` and `getpeername` (thanks to Nathaniel J Smith for that suggestion).
Serhiy Storchaka [Thu, 23 May 2024 12:09:03 +0000 (15:09 +0300)]
[3.11] gh-118643: Fix AttributeError in the email module (GH-119099) (#119393)
Fix regression introduced in gh-100884: AttributeError when re-fold a long
address list.
Also fix more cases of incorrect encoding of the address separator in the
address list missed in gh-100884.
(cherry picked from commit 858b9e85fcdd495947c9e892ce6e3734652c48f2)
[3.11] gh-117310: Remove extra DECREF on "no ciphers" error path in `_ssl._SSLContext` constructor (GH-117309) (GH-117318)
gh-117310: Remove extra DECREF on "no ciphers" error path in `_ssl._SSLContext` constructor (GH-117309)
Remove extra self DECREF on ssl "no ciphers" error path.
This doesn't come up in practice because nobody links against a broken
OpenSSL library that provides nothing.
(cherry picked from commit 8cb7d7ff86a1a2d41195f01ba4f218941dd7308c)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
[3.11] [3.12] gh-115538: Use isolate mode when running venv test_multiproces… (GH-117264) (#117265)
[3.12] gh-115538: Use isolate mode when running venv test_multiproces… (GH-117264)
[3.12] gh-115538: Use isolate mode when running venv test_multiprocessing_recursion() (GH-117116)
(cherry picked from commit 4ec347760f98b156c6a2d42ca397af6b0b6ecc50)
[3.11] [3.12] gh-83434: Sync libregrtest and test_regrtest with the main branch (GH-117250) (#117251)
[3.12] gh-83434: Sync libregrtest and test_regrtest with the main branch (GH-117250)
* gh-115122: Add --bisect option to regrtest (GH-115123)
* test.bisect_cmd now exit with code 0 on success, and code 1 on
failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
-X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
methods.
* gh-115720: Show number of leaks in huntrleaks progress reports (GH-115726)
Instead of showing a dot for each iteration, show:
- '.' for zero (on negative) leaks
- number of leaks for 1-9
- 'X' if there are more leaks
This allows more rapid iteration: when bisecting, I don't need
to wait for the final report to see if the test still leaks.
Also, show the full result if there are any non-zero entries.
This shows negative entries, for the unfortunate cases where
a reference is created and cleaned up in different runs.
Test *failure* is still determined by the existing heuristic.
[3.11] gh-117187: Fix XML tests for vanilla Expat <2.6.0 (GH-117203) (GH-117245)
gh-117187: Fix XML tests for vanilla Expat <2.6.0 (GH-117203)
This fixes XML unittest fallout from the https://github.com/python/cpython/issues/115398 security fix. When configured using `--with-system-expat` on systems with older pre 2.6.0 versions of libexpat, our unittests were failing.
* sax|etree: Simplify Expat version guard where simplifiable
Idea by Matěj Cepl
* sax|etree: Fix reparse deferral tests for vanilla Expat <2.6.0
This *does not fix* the case of distros with an older version of libexpat with the 2.6.0 feature backported as a security fix. (Ubuntu is a known example of this with its libexpat1 2.5.0-2ubunutu0.1 package)
(cherry picked from commit 9f74e86c78853c101a23e938f8e32ea838d8f62e)
Co-authored-by: Sebastian Pipping <sebastian@pipping.org>
Serhiy Storchaka [Tue, 19 Mar 2024 20:08:00 +0000 (22:08 +0200)]
[3.11] gh-90300: Improve the Python CLI help output (GH-115853) (GH-117022) (GH-117034)
* document equivalent command-line options for all environment variables
* document equivalent environment variables for all command-line options
* reduce the size of variable and option descriptions to minimum
* remove the ending period in single-sentence descriptions
[3.11] gh-116957: configparser: Do post-process values after DuplicateOptionError (GH-116958) (GH-117012)
If you catch DuplicateOptionError / DuplicateSectionError when reading a
config file (the intention is to skip invalid config files) and then
attempt to use the ConfigParser instance, any values it *had* read
successfully so far, were stored as a list instead of string! Later
`get` calls would raise "AttributeError: 'list' object has no attribute
'find'" from somewhere deep in the interpolation code.
[3.11] gh-90535: Fix support of interval>1 in logging.TimedRotatingFileHandler (GH-116220) (GH-116893)
Fix support of interval values > 1 in logging.TimedRotatingFileHandler
for when='MIDNIGHT' and when='Wx'.
(cherry picked from commit 269051d20e65eda30734cbbbdb07d21df61978d6)
[3.11] gh-116881: Remove erroneous or redundant grammar NULL (GH-116885) (#116952)
In Lexical Analysis f-strings section, NULL in the description
of 'literal character' means '\0'. In the format_spec grammar
production, it is wrong with that meaning and redundant if
instead interpreted as <nothing>. Remove it there.
(cherry picked from commit 4e45c6c54a9457b1ca5b4cf3aa2843b7218d4414)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
[3.11] gh-116851: Remove "from ctypes import *" from a ctypes example (GH-116852) (GH-116906)
It is confusing, because libc is not imported from ctypes,
but defined in previous examples, which already contain the import.
(cherry picked from commit 744c0777952f1e535d1192ee15b286aa67b61533)
[3.11] gh-116484: Fix collisions between Checkbutton and ttk.Checkbutton default names (GH-116495) (GH-116902)
Change automatically generated tkinter.Checkbutton widget names to
avoid collisions with automatically generated tkinter.ttk.Checkbutton
widget names within the same parent widget.
(cherry picked from commit c61cb507c10c5b597928284e087a9a384ab267d0)
[3.11] gh-110918: Fix side effects of regrtest test_match_tests() (GH-116718) (#116727)
gh-110918: Fix side effects of regrtest test_match_tests() (GH-116718)
test_match_tests now saves and restores patterns.
Add get_match_tests() function to libregrtest.filter.
Previously, running test_regrtest multiple times in a row only ran
tests once: "./python -m test test_regrtest -R 3:3.
(cherry picked from commit 612f1ec988314bc0bc42a1b908751950331e2ede)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Fix the description of the "-b" option.
* Add references to environment variables for "-s" and "-X dev" options.
(cherry picked from commit 33662d4e01d73cd4f29a25efc2ef09288129023f)
gh-116307: Create a new import helper 'isolated modules' and use that instead of 'Clean Import' to ensure that tests from importlib_resources don't leave modules in sys.modules.
[3.11] gh-116682: stdout may be empty in test_cancel_futures_wait_false (GH-116683) (#116693)
If the `shutdown()` call happens before the worker thread starts executing
the task, then nothing will be printed to stdout.
(cherry picked from commit 7d1abe9502641a3602e9773aebc29ee56d8f40ae)
[3.11] gh-116563: Update tutorial error example (GH-116569) (#116625)
There now may be multiple carets pointing at a token
rather than just a character. Fix the sentence about
possible causes.
(cherry picked from commit 3e45030076bf2cfab41c4456c73fb212b7322c60)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Petr Viktorin [Mon, 11 Mar 2024 11:02:05 +0000 (12:02 +0100)]
[3.11] gh-115421: List all test/ subdirs in Makefile, and test them (GH-115813)
This backports:
- GH-115813
- GH-115422
Unlike on the main branch, new directories are added to the end,
so they're a bit easier to patch out if a redistributor needs to do so.
On main & 3.12, there's a special case for `idlelib/idle_test`; on
3.11 TESTSUBDIRS has several more entries that are not in `test/`.
This backport ignores all of them (including idlelib).
(The alternative would be list them, as additions to TEST_HOME_DIR.
But that's probably too invasive; people might split stdlib up in
surprising ways.)
[3.11] gh-88352: Make TimedRotatingFileHandler tests more stable (GH-116409) (GH-116586)
The tests failed (with less than 1% probability) if for example the file
was created at 11:46:03.999, but the record was emitted at 11:46:04.001,
with atTime=11:46:04, which caused an unexpected rollover. Ensure that the
tests are always run within the range of the same whole second.