test_netrc, test_pep646_syntax and test_xml_etree now return results
in the test_main() function.
Changes:
* Rewrite TestResult as a dataclass with a new State class.
* Add test.support.TestStats class and Regrtest.stats_dict attribute.
* libregrtest.runtest functions now modify a TestResult instance
in-place.
* libregrtest summary lists the number of run tests and skipped
tests, and denied resources.
* Add TestResult.has_meaningful_duration() method.
* Compute TestResult duration in the upper function.
* Use time.perf_counter() instead of time.monotonic().
* Regrtest: rename 'resource_denieds' attribute to 'resource_denied'.
* Rename CHILD_ERROR to MULTIPROCESSING_ERROR.
* Use match/case syntadx to have different code depending on the
test state.
Notes on the backport: doctest.TestResults.skipped is a new feature
in Python 3.13, so don't use it in the backport.
Co-authored-by: Nikita Sobolev <mail@sobolevn.me> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
[3.12] Improve some C API documentation (GH-108768) (#108785)
* Express functions which take argument as a C string in terms of
functions which take Python object.
* Use "note" directive for PyMapping_HasKey() and
PyMapping_HasKeyString() notes..
(cherry picked from commit 6f97eeec222f81bd7ae836c149872a40b079e2a6)
[3.12] Make test_fcntl quiet (GH-108758) (#108760)
Make test_fcntl quiet (GH-108758)
Running test_fcntl logs two "struct.pack: ..." lines because
multiprocessing imports test_fcntl twice with
test.support.verbose=1. Move get_lockdata() inside TestFcntl test
case and only call it where it's needed, to stop logging these lines.
(cherry picked from commit 23f54c120067c96973619b9501fe4dff1b055188)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] `ast` docs: Fix incorrect link on `keyword` (GH-108728) (#108737)
`ast` docs: Fix incorrect link on `keyword` (GH-108728)
In two places, Sphinx was erroneously adding links to the `keyword` module instead of the `ast.keyword` class
(cherry picked from commit c1e2f3b2f70b8a72ea7e1bf792addf62a94ae65d)
[3.12] gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704) (#108733)
gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .
member = object.__new__(cls)
member = int.__new__(cls, value)
member = str.__new__(cls, value)
Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
(cherry picked from commit d48760b2f1e28dd3c1a35721939f400a8ab619b8)
[3.12] gh-97850: Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed (GH-108719) (#108723)
gh-97850: Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed (GH-108719)
Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed
(cherry picked from commit 013a99a47b3299f48cf7f95aa451a116441b029c)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
[3.12] gh-108520: Fix bad fork detection in nested multiprocessing use case (GH-108568) (#108691)
gh-108520: Fix bad fork detection in nested multiprocessing use case (GH-108568)
gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization.
[3.12] Docs: Resolve Sphinx warnings in dis.rst (GH-108476) (#108546)
Docs: Resolve Sphinx warnings in dis.rst (GH-108476)
- Link to the code objects reference
- Suppress link to deliberately undocumented builtins.__build_class__
- Suppress links for example methods
(cherry picked from commit cb1184280b3fb369a07abb4153aa36829cf1df9b)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Matthew James Kraai <kraai@ftbfs.org> Co-authored-by: KRAAI, MATTHEW [VISUS] <mkraai@its.jnj.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
[3.12] gh-107913: Fix possible losses of OSError error codes (GH-107930) (#108523)
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa94d07e9e286826c23507402313ec7f4)
Guido van Rossum [Sat, 26 Aug 2023 23:22:40 +0000 (16:22 -0700)]
[3.12] gh-108487: Change assert that should've been DEOPT_IF (#108509)
* Remove assert that should've been DEOPT_IF
The assert(method != NULL) in CALL_NO_KW_LIST_APPEND is wrong --
this condition should lead to a deoptimization, and indeed there
is a DEOPT_IF two lines later that will trigger if method == NULL.
This would crash in a devious repro scenario (first seen live
in boto3 tests) when compiled with assertions enabled.
In a production version there is no crash, so impact is limited.
(The crash also appears in main; I will prepare a separate PR.)
Co-authored-by: Philipp A <flying-sheep@web.de> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
The decorator now requires to be called with parenthesis:
@support.requires_legacy_unicode_capi()
instead of:
@support.requires_legacy_unicode_capi
The implementation now only imports _testcapi when the decorator is
called, so "import test.support" no longer imports the _testcapi
extension.
(cherry picked from commit 995f4c48e11349fbfb9233e02b732d4534d3008e)
Co-authored-by: Victor Stinner <vstinner@python.org>
Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
(cherry picked from commit aa9a359ca2663195b0f04eef46109c28c4ff74d3)
Co-authored-by: Victor Stinner <vstinner@python.org>
https://docs.python.org/3/howto/pyporting.htmlGH-porting-python-2-code-to-python-3 was written for another time. In this patch:
- material that frames Python 3 as "new" is removed
- descriptions and directions have been trimmed
(cherry picked from commit 809ea7c4b6c2b818ae510f1f58e82b6b05ed4ef9)
Łukasz Langa [Thu, 24 Aug 2023 10:08:43 +0000 (12:08 +0200)]
[3.12] gh-108342: Make ssl TestPreHandshakeClose more reliable (GH-108370) (#108404)
* In preauth tests of test_ssl, explicitly break reference cycles
invoving SingleConnectionTestServerThread to make sure that the
thread is deleted. Otherwise, the test marks the environment as
altered because the threading module sees a "dangling thread"
(SingleConnectionTestServerThread). This test leak was introduced
by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
test_preauth_data_to_tls_client(): the server now waits until the
client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
(cherry picked from commit 592bacb6fc0833336c0453e818e9b95016e9fd47)
Co-authored-by: Victor Stinner <vstinner@python.org>
Currently, test_asyncio package is only splitted into sub-tests when
using command "./python -m test". With this change, it's also
splitted when passing it on the command line:
"./python -m test test_asyncio".
Remove the concept of "STDTESTS". Python is now mature enough to not
have to bother with that anymore. Removing STDTESTS simplify the
code.
(cherry picked from commit 174e9da0836844a2138cc8915dd305cb2cd7a583)
Co-authored-by: Victor Stinner <vstinner@python.org>
Changed "__setattr__" to "object.__setattr__" in a section that was specifically supposed to refer to the __setattr__ method of the object class. Also suppressed the link to the data model docs for __setattr__, since we're talking about a specific __setattr__ implementation, not __setattr__ methods in general.
(cherry picked from commit 79fdacc0059a3959074d2d9d054653eae1dcfe06)
[3.12] gh-105776: Fix test_cppext when CC contains -std=c11 option (GH-108343) (#108345)
gh-105776: Fix test_cppext when CC contains -std=c11 option (GH-108343)
Fix test_cppext when the C compiler command has the "-std=c11" option.
Remove "-std=" options from the compiler command.
(cherry picked from commit 9173b2bbe13aeccc075b571da05c653a2a91de1b)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.12] gh-108342: Break ref cycle in SSLSocket._create() exc (GH-108344) (#108348)
Explicitly break a reference cycle when SSLSocket._create() raises an
exception. Clear the variable storing the exception, since the
exception traceback contains the variables and so creates a reference
cycle.
* Move test_cppext to its own directory
* Rename setup_testcppext.py to setup.py
* Rename _testcppext.cpp to extension.cpp
* The source (extension.cpp) is now also copied by the test.
(cherry picked from commit 21dda09600848ac280481f7c64f8d9516dc69bb2)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Move Python scripts related to test_module to this new directory:
good_getattr.py and bad_getattrX.py scripts.
* Move Lib/test/test_module.py to Lib/test/test_module/__init__.py.
Łukasz Langa [Tue, 22 Aug 2023 19:03:20 +0000 (21:03 +0200)]
[3.12] gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw (#108316)
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.
The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
[3.12] Docs: align the param spec of sqlite3.Connection methods with the implementation (GH-108285) (#108287)
Docs: align the param spec of sqlite3.Connection methods with the implementation (GH-108285)
- no parameters of create_aggregate() are positional-only
- all parameters of create_collation() are positional-only
(cherry picked from commit 893215a4e7f59eabb8ccdf188c4b9b1de5bd8966)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Previously, depending on existing filters, the test
could modify the warnings and so fail as "env changed".
(cherry picked from commit 531930f47f6b2a548d31e62cb4ad3e215a24bf53)
Co-authored-by: Victor Stinner <vstinner@python.org>
(Spawning subprocesses does not require the event loop to run in the main thread -- only signal handling does.)
(cherry picked from commit 1cc391d9e2ea24ca750005335507b52933fc0b52)
[3.12] gh-107396: tarfiles: set self.exception before _init_read_gz() (GH-107485) (#108207)
gh-107396: tarfiles: set self.exception before _init_read_gz() (GH-107485)
In the stack call of: _init_read_gz()
```
_read, tarfile.py:548
read, tarfile.py:526
_init_read_gz, tarfile.py:491
```
a try;except exists that uses `self.exception`, so it needs to be set before
calling _init_read_gz().
(cherry picked from commit 37135d25e269ede92bc7da363bebfa574782e59a)
[3.12] gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929) (#108205)
gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929)
(cherry picked from commit 80bdebdd8593f007a2232ec04a7729bba6ebf12c)
[3.12] Improve references in the tutorial (GH-108069) (#108203)
Improve references in the tutorial (GH-108069)
* Use full qualified names for references (even if they do not work now,
they will work in future).
* Silence references to examples.
(cherry picked from commit 622ddc41674c2566062af82f7b079aa01d2aae8c)
[3.12] gh-107895: Fix test_asyncio.test_runners when run it in CPython's "development mode" (GH-108168) (#108196)
gh-107895: Fix test_asyncio.test_runners when run it in CPython's "development mode" (GH-108168)
(cherry picked from commit 014a5b71e7538926ae1c03c8c5ea13c96e741be3)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
- Mark up named tuple attributes as attributes
- Remove links for external functions
- io.BufferedIOBase has no 'buffer' attribute;
remove the link and mark up using :attr:`!buffer`
- (Re)format some tables as bullet lists:
- sys._emscripten_info
- sys.hash_info
- sys.int_info
- sys.thread_info
- In the paragraphs mentioning 'f_trace_lines' and 'f_trace_opcodes',
add links to the frame objects reference.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
[3.12] gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918) (#108134)
* gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
(cherry picked from commit 633ea217a85f6b6ba5bdbc73094254d5811b3485)