Fix race conditions in test_threading lock tests. Wait until a
condition is met rather than using time.sleep() with a hardcoded
number of seconds.
* Replace sleeping loops with support.sleeping_retry() which raises
an exception on timeout.
* Add wait_threads_blocked(nthread) which computes a sleep depending
on the number of threads. Remove _wait() function.
* test_set_and_clear(): use a way longer Event.wait() timeout.
* BarrierTests.test_repr(): wait until the 2 threads are waiting for
the barrier. Use a way longer timeout for Barrier.wait() timeout.
* test_thread_leak() no longer needs to count
len(threading.enumerate()): Bunch uses
threading_helper.wait_threads_exit() internally which does it in
wait_for_finished().
* Add BaseLockTests.wait_phase() which implements a timeout.
test_reacquire() and test_recursion_count() use wait_phase().
* gh-109974: Fix more threading lock_tests race conditions (#110089)
* Add context manager on Bunch class.
* Bunch now catchs exceptions on executed functions and re-raise them
at __exit__() as an ExceptionGroup.
* Rewrite BarrierProxy.test_default_timeout(). Use a single thread.
Only check that barrier.wait() blocks for at least default timeout
seconds.
* test_with(): inline _with() function.
* Split test_pycfunction.py: add test_cfunction_full.py.
Split the function into the following 6 functions. In verbose
mode, these "pycfunction" tests now log each tested call.
* Move get_gdb_repr() to PrettyPrintTests.
* Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT.
* Rename checkout_hook_path to CHECKOUT_HOOK_PATH.
* Rename gdb_version to GDB_VERSION_TEXT.
* Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION.
* run_gdb() uses "backslashreplace" error handler instead of "replace".
* Add check_gdb() function to util.py.
* Enhance support.check_cflags_pgo(): check also for sysconfig
PGO_PROF_USE_FLAG (if available) in compiler flags.
* Move some SkipTest checks to test_gdb/__init__.py.
* Elaborate why gdb cannot be tested on Windows: gdb doesn't support
PDB debug symbol files.
* gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360)
Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex:
Fedora 38). Search patterns in gdb "bt" command output to detect
when gdb fails to retrieve the traceback. For example, skip a test if
"Backtrace stopped: frame did not save the PC" is found.
* gh-110166: Fix gdb CFunctionFullTests on ppc64le clang build (GH-110331)
CFunctionFullTests now also runs "bt" command before "py-bt-full",
similar to CFunctionTests which also runs "bt" command before
"py-bt". So test_gdb can skip the test if patterns like "?? ()" are
found in the gdb output.
Victor Stinner [Wed, 4 Oct 2023 10:53:28 +0000 (12:53 +0200)]
[3.11] gh-109972: Split test_gdb.py into test_gdb package (#109977) (… (#110343)
[3.12] gh-109972: Split test_gdb.py into test_gdb package (#109977) (#110339)
gh-109972: Split test_gdb.py into test_gdb package (#109977)
Split test_gdb.py file into a test_gdb package made of multiple
tests, so tests can now be run in parallel.
* Create Lib/test/test_gdb/ directory.
* Split test_gdb.py into multiple files in Lib/test/test_gdb/
directory.
* Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory.
Update get_sample_script(): use __file__ to locate gdb_sample.py.
* Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py.
* Explicitly skip test_gdb on Windows. Previously, test_gdb was
skipped even if gdb was available because of
gdb_has_frame_select().
[3.11] gh-109917: Fix test instability in test_concurrent_futures (GH-110306) (#110316)
gh-109917: Fix test instability in test_concurrent_futures (GH-110306)
The test had an instability issue due to the ordering of the dummy
queue operation and the real wakeup pipe operations. Both primitives
are thread safe but not done atomically as a single update and may
interleave arbitrarily. With the old order of operations this can lead
to an incorrect state where the dummy queue is full but the wakeup
pipe is empty. By swapping the order in clear() I think this can no
longer happen in any possible operation interleaving (famous last
words).
(cherry picked from commit a376a72bd92cd7c9930467dd1aba40045fb75e3b)
* Doc: Specify that os.cpu_count() counts *logicial* CPUs.
* Doc: Specify that os.sched_getaffinity(0) is related to the calling
thread.
* Fix test_posix.test_sched_getaffinity(): restore the old CPU mask
when the test completes!
Don't measure the CI performance: don't test the maximum elapsed
time. The check failed on a slow CI.
(cherry picked from commit c62b49ecc8da13fa9522865ef6fe0aec194fd0d8)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-109748: Fix again venv test_zippath_from_non_installed_posix() (GH-110149) (#110153)
gh-109748: Fix again venv test_zippath_from_non_installed_posix() (GH-110149)
Call also copy_python_src_ignore() on listdir() names.
shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
(cherry picked from commit 0def8c712bb6f66f1081cab71deb3681566b846d)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix test_asyncio timeouts: don't measure the maximum duration, a test
should not measure a CI performance. Only measure the minimum
duration when a task has a timeout or delay. Add CLOCK_RES to
test_asyncio.utils.
On Windows, multiprocessing Popen.terminate() now catchs
PermissionError and get the process exit code. If the process is
still running, raise again the PermissionError. Otherwise, the
process terminated as expected: store its exit code.
(cherry picked from commit bd4518c60c9df356cf5e05b81305e3644ebb5e70)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-109960: Remove test_pty timeout of 10 seconds (GH-110058) (#110061)
gh-109960: Remove test_pty timeout of 10 seconds (GH-110058)
In 2003, test_pty got a hardcoded timeout of 10 seconds to prevent
hanging on AIX & HPUX "if run after test_openpty":
commit 7d8145268ee282f14d6adce9305dc3c1c7ffec14. Since 2003, test_pty
was no longer reported to hang on AIX. But today, the test can fail
simply because a CI is busy running other tests in parallel.
The timeout of 10 seconds is no longer needed, just remove it.
Moreover, regrtest now has multiple built-in generic timeout
mecanisms.
(cherry picked from commit 5fdcea744024c8a19ddb57057bf5ec2889546c98)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-110033: Fix signal test_interprocess_signal() (GH-110035) (#110041)
gh-110033: Fix signal test_interprocess_signal() (GH-110035)
Fix test_interprocess_signal() of test_signal. Make sure that the
subprocess.Popen object is deleted before the test raising an
exception in a signal handler. Otherwise, Popen.__del__() can get the
exception which is logged as "Exception ignored in: ...." and the
test fails.
(cherry picked from commit 7e0fbf5175fcf21dae390ba68b7f49706d62aa49)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix test_timeout() of test_concurrent_futures.test_wait. Remove the
future which may or may not complete depending if it takes longer
than the timeout ot not. Keep the second future which does not
complete before wait(). Make also the test faster: 0.5 second instead
of 6 seconds, so remove @support.requires_resource('walltime')
decorator.
(cherry picked from commit 9be283e5e15d5d5685b78a38eb132501f7f3febb)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] Enhance TypedDict docs around required/optional keys (GH-109547) (#109983)
As discussed in comments to GH-109544, the semantics of this attribute
are somewhat confusing. Add a note explaining its limitations and
steering users towards __required_keys__ and __optional_keys__ instead.
(cherry picked from commit f49958c886a2f2608f1008186d588efc2a98b445)
Victor Stinner [Wed, 27 Sep 2023 11:00:22 +0000 (13:00 +0200)]
[3.11] gh-109615: Fix support test_copy_python_src_ignore() (#109958) (#109962)
gh-109615: Fix support test_copy_python_src_ignore() (#109958)
Fix the test when run on an installed Python: use "abs_srcdir" of
sysconfig, and skip the test if the Python source code cannot be
found.
* Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and
Lib/test/libregrtest/utils.py now first try to get "abs_srcdir"
from sysconfig, before getting "srcdir" from sysconfig.
* test.pythoninfo logs sysconfig "abs_srcdir".
Fix copy_source_tree() function of test_tools.test_freeze:
* Don't copy SRC_DIR/build/ anymore. This directory is modified by
other tests running in parallel.
* Add test.support.copy_python_src_ignore().
* Use sysconfig to get the source directory.
* Use sysconfig.get_config_var() to get CONFIG_ARGS variable.
Remove links to PyAPI_FUNC() and PyAPI_DATA() macros since they are
not documented. These macros should only be used to define the Python
C API. They should not be used outside Python code base.
[3.11] gh-109845: Make test_ftplib more stable under load (GH-109912) (GH-109920)
recv() can return partial data cut in the middle of a multibyte
character. Test raw binary data instead of data incorrectly decoded by parts.
(cherry picked from commit 2ef2fffe3be953b91852585c75188d5475b09474)
test_error_at_task_unpickle() and
test_error_during_result_unpickle_in_result_handler() now restore
sys.stderr which is overriden by _raise_error_ignore_stderr().
(cherry picked from commit 2897142d2ec0930a8991af964c798b68fb6dcadd)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-109370: Fix unexpected traceback output in test_concurrent_futures (GH-109780) (GH-109882)
Follow-up of gh-107219.
* Only close the connection writer on Windows.
* Also use existing constant _winapi.ERROR_OPERATION_ABORTED instead of
WSA_OPERATION_ABORTED.
(cherry picked from commit 0b4e090422db5f959184353d53552d1675f74212)
[3.11] gh-109631: Allow interruption of short repeated regex matches (GH-109867) (GH-109885)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
(cherry picked from commit 8ac2085b80eca4d9b2a1093d0a7da020fd12e11a)
Increase timeouts. Barrier default timeout should be long enough to
spawn 4 threads on a slow CI.
(cherry picked from commit e5186c3de4194de3ea8c80edb182d786f5e20944)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
(cherry picked from commit 25bb266fc876b344e31e0b5634a4db94912c1aba)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-89363: Skip threading test_is_alive_after_fork() if ASAN (GH-109835) (#109856)
gh-89363: Skip threading test_is_alive_after_fork() if ASAN (GH-109835)
Skip test_is_alive_after_fork() of test_threading if Python is built
with Address Sanitizer (ASAN).
(cherry picked from commit bc06743533b5fea2d5ecdad6dd3caa372c67439f)
Co-authored-by: Victor Stinner <vstinner@python.org>
Expect the test to be "short" but don't measure the exact performance
of the CI. SHORT_TIMEOUT is about 30 seconds whereas the cancelled
coroutine takes around 1 hour.
(cherry picked from commit f29bc9c9a0a6794c6b8a9e84a7ba9237b427a10a)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.11] gh-109625: Move _ready_to_import() from test_import to support.import_helper (GH-109626) (#109718)
[3.11] gh-109625: Move _ready_to_import() from test_import to support.import_helper (GH-109626).
(cherry picked from commit 115c49ad5a5ccfb628fef3ae06a566f7a0197f97)
Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
(cherry picked from commit aa9a359ca2663195b0f04eef46109c28c4ff74d3)
Co-authored-by: Victor Stinner <vstinner@python.org>
In Python 2.7, the file was used by Lib/test/test_sgmllib.py to test
Lib/sgmllib.py. The sgmllib module and its tests have been removed in
Python 3.0.
(cherry picked from commit d2879f2095abd5c8186c7f69c964a341c2053572)
Co-authored-by: Victor Stinner <vstinner@python.org>
Victor Stinner [Thu, 21 Sep 2023 18:00:38 +0000 (20:00 +0200)]
[3.11] gh-109613: _pystat_fromstructstat() checks for exceptions (#109618) (#109668)
gh-109613: _pystat_fromstructstat() checks for exceptions (#109618)
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.
_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():
Fatal Python error: _Py_CheckSlotResult:
Slot * of type int succeeded with an exception set
Victor Stinner [Wed, 20 Sep 2023 15:45:23 +0000 (17:45 +0200)]
[3.11] gh-103053: Skip test_freeze_simple_script() on PGO build (#109591) (#109616)
gh-103053: Skip test_freeze_simple_script() on PGO build (#109591)
Skip test_freeze_simple_script() of test_tools.test_freeze if Python
is built with "./configure --enable-optimizations", which means with
Profile Guided Optimization (PGO): it just makes the test too slow.
The freeze tool is tested by many other CIs with other (faster)
compiler flags.
test.pythoninfo now gets also get_build_info() of
test.libregrtests.utils.
SubprocessProtocol process_exited() method can be called before
pipe_data_received() and pipe_connection_lost() methods. Document it
and adapt the example in the doc.
projects / thirdparty / Python / cpython.git / log
