]> git.ipfire.org Git - thirdparty/Python/cpython.git/log
thirdparty/Python/cpython.git
2 years agogh-109162: libregrtest: move code around (#109253)
Victor Stinner [Mon, 11 Sep 2023 08:52:03 +0000 (10:52 +0200)] 
gh-109162: libregrtest: move code around (#109253)

* Move Regrtest.display_header() to utils.py.
* Move cleanup_temp_dir() to utils.py.
* Move list_cases() to findtests.py.

2 years agogh-107219: Fix concurrent.futures terminate_broken() (#109244)
Victor Stinner [Mon, 11 Sep 2023 08:11:31 +0000 (10:11 +0200)] 
gh-107219: Fix concurrent.futures terminate_broken() (#109244)

Fix a race condition in concurrent.futures. When a process in the
process pool was terminated abruptly (while the future was running or
pending), close the connection write end. If the call queue is
blocked on sending bytes to a worker process, closing the connection
write end interrupts the send, so the queue can be closed.

Changes:

* _ExecutorManagerThread.terminate_broken() now closes
  call_queue._writer.
* multiprocessing PipeConnection.close() now interrupts
  WaitForMultipleObjects() in _send_bytes() by cancelling the
  overlapped operation.

2 years agoGH-108614: Increase importlib MAGIC for RESUME_CHECK instruction (#109247)
Victor Stinner [Mon, 11 Sep 2023 07:34:04 +0000 (09:34 +0200)] 
GH-108614: Increase importlib MAGIC for RESUME_CHECK instruction (#109247)

2 years agogh-109162: libregrtest: use relative imports (#109250)
Victor Stinner [Mon, 11 Sep 2023 07:02:35 +0000 (09:02 +0200)] 
gh-109162: libregrtest: use relative imports (#109250)

libregrtest.__init__ no longer exposes any symbol, so
"python -m test.libregrtest.worker" imports less modules.

2 years agogh-109162: libregrtest: rename runtest_mp.py to run_workers.py (#109248)
Victor Stinner [Mon, 11 Sep 2023 03:27:37 +0000 (05:27 +0200)] 
gh-109162: libregrtest: rename runtest_mp.py to run_workers.py (#109248)

* Rename runtest_mp.py to run_workers.py
* Move exit_timeout() and temp_cwd() context managers from
  Regrtest.main() to Regrtest.run_tests(). Actions like --list-tests
  or --list-cases don't need these protections.
* Regrtest: remove selected and tests attributes. Pass 'selected' to
  list_tests(), list_cases() and run_tests(). display_result() now
  expects a TestTuple, instead of TestList.
* Rename setup_tests() to setup_process() and rename setup_support()
  to setup_tests().
* Move _adjust_resource_limits() to utils and rename it to
  adjust_rlimit_nofile().
* Move replace_stdout() to utils.
* Fix RunTests.verbose type: it's an int.

2 years agogh-109162: libregrtest: fix Logger (#109246)
Victor Stinner [Mon, 11 Sep 2023 01:46:26 +0000 (03:46 +0200)] 
gh-109162: libregrtest: fix Logger (#109246)

* Pass results, quiet and pgo to Logger constructor.
* Move display_progress() method from Regrtest to Logger.
* No longer pass Regrtest to RunWorkers, but logger and results.

2 years agogh-109162: libregrtest: add single.py and result.py (#109243)
Victor Stinner [Mon, 11 Sep 2023 00:07:18 +0000 (02:07 +0200)] 
gh-109162: libregrtest: add single.py and result.py (#109243)

* Add single.py and result.py files.
* Rename runtest.py to runtests.py.
* Move run_single_test() function and its helper functions to
  single.py.
* Move remove_testfn(), abs_module_name() and normalize_test_name()
  to utils.py.
* Move setup_support() to setup.py.
* Move type hints like TestName to utils.py.
* Rename runtest.py to runtests.py.

2 years agogh-109162: libregrtest: add worker.py (#109229)
Victor Stinner [Sun, 10 Sep 2023 23:11:22 +0000 (01:11 +0200)] 
gh-109162: libregrtest: add worker.py (#109229)

Add new worker.py file:

* Move create_worker_process() and worker_process() to this file.
* Add main() function to worker.py. create_worker_process() now
  runs the command: "python -m test.libregrtest.worker JSON".
* create_worker_process() now starts the worker process in the
  current working directory. Regrtest now gets the absolute path of
  the reflog.txt filename: -R command line option filename.
* Remove --worker-json command line option.
  Remove test_regrtest.test_worker_json().

Related changes:

* Add write_json() and from_json() methods to TestResult.
* Rename select_temp_dir() to get_temp_dir() and move it to utils.
* Rename make_temp_dir() to get_work_dir() and move it to utils.
  It no longer calls os.makedirs(): Regrtest.main() now calls it.
* Move fix_umask() to utils. The function is now called by
  setup_tests().
* Move StrPath to utils.
* Add exit_timeout() context manager to utils.
* RunTests: Replace junit_filename (StrPath) with use_junit (bool).

2 years agogh-109230: test_pyexpat no longer depends on the current directory (#109233)
Victor Stinner [Sun, 10 Sep 2023 22:16:59 +0000 (00:16 +0200)] 
gh-109230: test_pyexpat no longer depends on the current directory (#109233)

Fix test_pyexpat.test_exception(): it can now be run from a directory
different than Python source code directory. Before, the test failed
in this case.

Skip the test if Modules/pyexpat.c source is not available. Skip also
the test on Python implementations other than CPython.

2 years agogh-109237: Fix test_site for non-ASCII working directory (#109238)
Victor Stinner [Sun, 10 Sep 2023 22:04:35 +0000 (00:04 +0200)] 
gh-109237: Fix test_site for non-ASCII working directory (#109238)

Fix test_site.test_underpth_basic() when the working directory
contains at least one non-ASCII character: encode the "._pth" file to
UTF-8 and enable the UTF-8 Mode to use UTF-8 for the child process
stdout.

2 years agogh-50644: Forbid pickling of codecs streams (GH-109180)
Serhiy Storchaka [Sun, 10 Sep 2023 17:06:09 +0000 (20:06 +0300)] 
gh-50644: Forbid pickling of codecs streams (GH-109180)

Attempts to pickle or create a shallow or deep copy of codecs streams
now raise a TypeError.

Previously, copying failed with a RecursionError, while pickling
produced wrong results that eventually caused unpickling to fail with
a RecursionError.

2 years agogh-109054: Don't use libatomic on cross-compilation (#109211)
Victor Stinner [Sun, 10 Sep 2023 16:21:13 +0000 (18:21 +0200)] 
gh-109054: Don't use libatomic on cross-compilation (#109211)

configure no longer uses libatomic by default when Python is
cross-compiled. The LIBATOMIC variable can be set manually in this
case:

    ./configure LIBATOMIC="-latomic" (...)

2 years agogh-109207: Fix SystemError when printing symtable entry object. (GH-109225)
云line [Sun, 10 Sep 2023 12:04:24 +0000 (20:04 +0800)] 
gh-109207: Fix SystemError when printing symtable entry object. (GH-109225)

2 years agoFix "FSTRING_MIDDLE" typo in py312 "What's New" (#109222)
Delgan [Sun, 10 Sep 2023 10:55:56 +0000 (12:55 +0200)] 
Fix "FSTRING_MIDDLE" typo in py312 "What's New" (#109222)

2 years agogh-93627: Align Python implementation of pickle with C implementation of pickle ...
Pieter Eendebak [Sun, 10 Sep 2023 07:06:08 +0000 (09:06 +0200)] 
gh-93627: Align Python implementation of pickle with C implementation of pickle (GH-103035)

If a method like __reduce_ex_ or __reduce__ is set to None, a TypeError is raised.

2 years agogh-109174: Add support of SimpleNamespace in copy.replace() (GH-109175)
Serhiy Storchaka [Sun, 10 Sep 2023 05:09:25 +0000 (08:09 +0300)] 
gh-109174: Add support of SimpleNamespace in copy.replace() (GH-109175)

2 years agogh-109162: libregrtest: add Logger class (#109212)
Victor Stinner [Sun, 10 Sep 2023 03:04:26 +0000 (05:04 +0200)] 
gh-109162: libregrtest: add Logger class (#109212)

* Add Logger class in a new logger.py file.
* Move Regrtest attributes to Logger:

  * start_time
  * test_count_text
  * test_count_width
  * win_load_tracker

* Move Regrtest method to Logger:

  * log()
  * getloadavg(): rename to get_load_avg()
  * set_tests()

* Add methods to the Logger class:

  * start_load_tracker()
  * stop_load_tracker()

2 years agogh-109162: libregrtest: add TestResults class (#109208)
Victor Stinner [Sun, 10 Sep 2023 02:30:43 +0000 (04:30 +0200)] 
gh-109162: libregrtest: add TestResults class (#109208)

* Add TestResults class.
* Move Regrtest methods to TestResults:

  * accumulate_result(): now takes a RunTests parameter
  * get_executed()
  * no_tests_run()

* Add methods to TestResults:

  * add_junit()
  * display_result()
  * display_summary()
  * need_rerun()
  * prepare_rerun()
  * write_junit()

* Rename 'need_rerun' attribute to 'bad_results'.
* Rename 'total_stats' attribute to 'stats'.

2 years agogh-109162: Refactor libregrtest.Regrtest (#109206)
Victor Stinner [Sun, 10 Sep 2023 01:07:05 +0000 (03:07 +0200)] 
gh-109162: Refactor libregrtest.Regrtest (#109206)

* Add type hint types: TestName, StrPath, StrJSON.
* Add attributes to Regrtest:

  * cmdline_args
  * coverage
  * coverage_dir
  * fail_rerun
  * next_single_filename
  * print_slowest
  * tmp_dir
  * want_rerun
  * want_run_leaks

* Remove Regrtest.ns attribute.
* Rename Regrtest methods:

  * cleanup() => cleanup_temp_dir()
  * create_temp_dir() => make_temp_dir()
  * set_temp_dir() => select_temp_dir()

* Convert Regrtest methods to static methods:

  * cleanup_temp_dir()
  * display_header()
  * fix_umask()
  * get_rerun_match_tests()
  * list_tests()
  * make_temp_dir()
  * select_temp_dir()

* Remove display_sanitizers() method: move code into
  display_header().
* Rename 'test_cwd' variable to 'work_dir'.

2 years agogh-109162: Refactor libregrtest.runtest_mp (#109205)
Victor Stinner [Sun, 10 Sep 2023 00:24:38 +0000 (02:24 +0200)] 
gh-109162: Refactor libregrtest.runtest_mp (#109205)

* Add attributes to Regrtest and RunTests:

  * fail_env_changed
  * num_workers

* Rename MultiprocessTestRunner to RunWorkers. Add num_workers
  parameters to RunWorkers constructor. Remove RunWorkers.ns
  attribute.
* Rename TestWorkerProcess to WorkerThread.
* get_running() now returns a string like: "running (...): ...".
* Regrtest.action_run_tests() now selects the number of worker
  processes, instead of the command line parser.

2 years agogh-109162: libregrtest: remove WorkerJob class (#109204)
Victor Stinner [Sat, 9 Sep 2023 23:41:21 +0000 (01:41 +0200)] 
gh-109162: libregrtest: remove WorkerJob class (#109204)

* Add attributes to Regrtest and RunTests:

  * gc_threshold
  * memory_limit
  * python_cmd
  * use_resources

* Remove WorkerJob class. Add as_json() and from_json() methods to
  RunTests. A worker process now only uses RunTests for all
  parameters.
* Add tests on support.set_memlimit() in test_support. Create
  _parse_memlimit() and also adds tests on it.
* Remove 'ns' parameter from runtest.py.

2 years agogh-109162: libregrtest: fix _decode_worker_job() (#109202)
Victor Stinner [Sat, 9 Sep 2023 22:51:24 +0000 (00:51 +0200)] 
gh-109162: libregrtest: fix _decode_worker_job() (#109202)

Decode also HuntRefleak() object inside the RunTests object.

Add an unit test on huntrleaks with multiprocessing (-R -jN).

2 years agoImprove the sieve() recipe in the itertools docs (gh-109199)
Raymond Hettinger [Sat, 9 Sep 2023 22:50:04 +0000 (17:50 -0500)] 
Improve the sieve() recipe in the itertools docs (gh-109199)

Lazier sieve

2 years agoFix an ironic typo in a code comment. (gh-109186)
Raymond Hettinger [Sat, 9 Sep 2023 15:21:42 +0000 (10:21 -0500)] 
Fix an ironic typo in a code comment. (gh-109186)

2 years agogh-109162: Refactor libregrtest.RunTests (#109177)
Victor Stinner [Sat, 9 Sep 2023 09:18:14 +0000 (11:18 +0200)] 
gh-109162: Refactor libregrtest.RunTests (#109177)

* Rename dash_R() runtest_refleak(). The function now gets
  huntrleaks and quiet arguments, instead of 'ns' argument.
* Add attributes to Regrtest and RunTests:

  * verbose
  * quiet
  * huntrleaks
  * test_dir

* Add HuntRefleak class.

2 years agogh-109050: Remove remaining tests for legacy Unicode C API (GH-109068)
Serhiy Storchaka [Sat, 9 Sep 2023 05:44:46 +0000 (08:44 +0300)] 
gh-109050: Remove remaining tests for legacy Unicode C API (GH-109068)

2 years agogh-109118: Fix runtime crash when NameError happens in PEP 695 function (#109123)
Jelle Zijlstra [Sat, 9 Sep 2023 02:49:20 +0000 (19:49 -0700)] 
gh-109118: Fix runtime crash when NameError happens in PEP 695 function (#109123)

2 years agogh-109162: Refactor libregrtest.runtest (#109172)
Victor Stinner [Sat, 9 Sep 2023 01:37:48 +0000 (03:37 +0200)] 
gh-109162: Refactor libregrtest.runtest (#109172)

* Rename runtest() to run_single_test().
* Pass runtests to run_single_test().
* Add type annotation to Regrtest attributes. Add missing attributes
  to Namespace.
* Add attributes to Regrtest and RunTests:

  * fail_fast
  * ignore_tests
  * match_tests
  * output_on_failure
  * pgo
  * pgo_extended
  * timeout

* Get pgo from 'runtests', rather than from 'ns'.
* Remove WorkerJob.match_tests.
* setup_support() now gets pgo_extended from runtests.
* save_env(): change parameter order, pass test_name first.
* Add setup_test_dir() function.
* Pass runtests to setup_tests().

2 years agogh-109052: Use the base opcode when comparing code objects (gh-109107)
Tian Gao [Sat, 9 Sep 2023 01:24:49 +0000 (18:24 -0700)] 
gh-109052: Use the base opcode when comparing code objects (gh-109107)

2 years agogh-109162: Refactor libregrtest WorkerJob (#109171)
Victor Stinner [Sat, 9 Sep 2023 01:03:39 +0000 (03:03 +0200)] 
gh-109162: Refactor libregrtest WorkerJob (#109171)

* Rename --worker-args command line option to --worker-json.
* Rename _parse_worker_args() to _parse_worker_json().
* WorkerJob:

  * Add runtests attribute
  * Remove test_name and rerun attribute

* Rename run_test_in_subprocess() to create_worker_process().
* Rename run_tests_worker() to worker_process().
* create_worker_process() uses json.dump(): write directly JSON to
  stdout.
* Convert MultiprocessResult to a frozen dataclass.
* Rename RunTests.match_tests to RunTests.match_tests_dict.

2 years agogh-109162: Refactor Regrtest.action_run_tests() (#109170)
Victor Stinner [Sat, 9 Sep 2023 00:30:28 +0000 (02:30 +0200)] 
gh-109162: Refactor Regrtest.action_run_tests() (#109170)

Refator Regrtest class:

* Rename finalize() finalize_tests().
* Pass tracer to run_test() and finalize_tests(). Remove Regrtest.tracer.
* run_test() does less things: move code to its caller.

2 years agogh-108996: Skip broken test_msvcrt for now (#109169)
Victor Stinner [Fri, 8 Sep 2023 23:56:53 +0000 (01:56 +0200)] 
gh-108996: Skip broken test_msvcrt for now (#109169)

2 years agogh-109162: Regrtest copies 'ns' attributes (#109168)
Victor Stinner [Fri, 8 Sep 2023 23:48:54 +0000 (01:48 +0200)] 
gh-109162: Regrtest copies 'ns' attributes (#109168)

* Regrtest.__init__() now copies 'ns' namespace attributes to
  Regrtest attributes. Regrtest match_tests and ignore_tests
  attributes have type FilterTuple (tuple), instead of a list.
* Add RunTests.copy(). Regrtest._rerun_failed_tests() now uses
  RunTests.copy().
* Replace Regrtest.all_tests (list) with Regrtest.first_runtests
  (RunTests).
* Make random_seed maximum 10x larger (9 digits, instead of 8).

2 years agogh-109162: Refactor Regrtest.main() (#109163)
Victor Stinner [Fri, 8 Sep 2023 22:41:26 +0000 (00:41 +0200)] 
gh-109162: Refactor Regrtest.main() (#109163)

* main() now calls _parse_args() and pass 'ns' to Regrtest
  constructor.  Remove kwargs argument from Regrtest.main().
* _parse_args() checks ns.huntrleaks.
* set_temp_dir() is now responsible to call expanduser().
* Regrtest.main() sets self.tests earlier.
* Add TestTuple and TestList types.
* Rename MatchTests to FilterTuple and rename MatchTestsDict
  to FilterTestDict.
* TestResult.get_rerun_match_tests() return type
  is now FilterTuple: return a tuple instead of a list.
  RunTests.tests type becomes TestTuple.

2 years agogh-108996: add tests for msvcrt (#109004)
AN Long [Fri, 8 Sep 2023 22:38:38 +0000 (06:38 +0800)] 
gh-108996: add tests for msvcrt (#109004)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2 years agogh-109054: configure checks if libatomic is needed (#109101)
Victor Stinner [Fri, 8 Sep 2023 21:14:33 +0000 (23:14 +0200)] 
gh-109054: configure checks if libatomic is needed (#109101)

Fix building the _testcapi extension on Linux AArch64 which requires
linking to libatomic when <cpython/pyatomic.h> is used: the
_Py_atomic_or_uint64() function requires libatomic
__atomic_fetch_or_8() on this platform.

The configure script now checks if linking to libatomic is needed and
generates a new LIBATOMIC variable used to build the _testcapi
extension.

Building the _testcapi extension now uses the LIBATOMIC variable in
its LDFLAGS, since Modules/_testcapi/pyatomic.c uses
<cpython/pyatomic.h>.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2 years agogh-108455: peg_generator: enable mypy's `--warn-unreachable` setting and `redundant...
Alex Waygood [Fri, 8 Sep 2023 21:05:40 +0000 (22:05 +0100)] 
gh-108455: peg_generator: enable mypy's `--warn-unreachable` setting and `redundant-expr` error code (#109160)

2 years agogh-91960: FreeBSD Cirrus CI runs configure separately (#109127)
Victor Stinner [Fri, 8 Sep 2023 20:06:34 +0000 (22:06 +0200)] 
gh-91960: FreeBSD Cirrus CI runs configure separately (#109127)

Run configure and make in separated steps to have more readable logs.

2 years agogh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positi...
Pablo Galindo Salgado [Fri, 8 Sep 2023 17:00:23 +0000 (18:00 +0100)] 
gh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positives (#109121)

2 years agoCheck the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
Serhiy Storchaka [Fri, 8 Sep 2023 16:57:41 +0000 (19:57 +0300)] 
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)

2 years agoGH-108614: Unbreak emscripten build (GH-109132)
Mark Shannon [Fri, 8 Sep 2023 16:54:45 +0000 (17:54 +0100)] 
GH-108614: Unbreak emscripten build (GH-109132)

2 years agoGH-109067: fix randomly failing `test_async_gen_asyncio_gc_aclose_09` test (#109142)
Kumar Aditya [Fri, 8 Sep 2023 16:27:58 +0000 (21:57 +0530)] 
GH-109067: fix randomly failing `test_async_gen_asyncio_gc_aclose_09` test (#109142)

Use `asyncio.sleep(0)` instead of short sleeps.

2 years agogh-109136: Fix summarize_stats.py tool (#109137)
Michael Droettboom [Fri, 8 Sep 2023 16:23:58 +0000 (12:23 -0400)] 
gh-109136: Fix summarize_stats.py tool (#109137)

2 years agogh-106922: Fix error location for constructs with spaces and parentheses (#108959)
Pablo Galindo Salgado [Fri, 8 Sep 2023 16:18:35 +0000 (17:18 +0100)] 
gh-106922: Fix error location for constructs with spaces and parentheses (#108959)

2 years agogh-109140: Rename duplicated tests in `test_binascii` (#109141)
Hugo van Kemenade [Fri, 8 Sep 2023 15:30:28 +0000 (09:30 -0600)] 
gh-109140: Rename duplicated tests in `test_binascii` (#109141)

2 years agogh-104690: thread_run() checks for tstate dangling pointer (#109056)
Victor Stinner [Fri, 8 Sep 2023 09:50:46 +0000 (11:50 +0200)] 
gh-104690: thread_run() checks for tstate dangling pointer (#109056)

thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.

Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.

2 years agoGH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)
Victor Stinner [Fri, 8 Sep 2023 09:48:28 +0000 (11:48 +0200)] 
GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)

Symbols of the C API should be prefixed by "Py_" to avoid conflict
with existing names in 3rd party C extensions on "#include <Python.h>".

test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other
_testcapi and _testinternalcapi constants.

2 years agoGH-108716: Turn off deep-freezing of code objects. (GH-108722)
Mark Shannon [Fri, 8 Sep 2023 09:34:40 +0000 (10:34 +0100)] 
GH-108716: Turn off deep-freezing of code objects. (GH-108722)

2 years agoUpdate `CODEOWNERS` for `Tools/wasm/` (#109119)
Brett Cannon [Fri, 8 Sep 2023 05:37:29 +0000 (22:37 -0700)] 
Update `CODEOWNERS` for `Tools/wasm/` (#109119)

2 years agogh-109022: [Enum] require `names=()` to create empty enum type (GH-109048)
Ethan Furman [Fri, 8 Sep 2023 01:19:03 +0000 (18:19 -0700)] 
gh-109022: [Enum] require `names=()` to create empty enum type (GH-109048)

add guard so that ``Enum('bar')`` raises a TypeError instead of
creating a new enum class called `bar`.  To create the new but
empty class, use:

    huh = Enum('bar', names=())

2 years agoUse unittest test runner for doctests in test_statistics (GH-108921)
Serhiy Storchaka [Thu, 7 Sep 2023 20:08:55 +0000 (23:08 +0300)] 
Use unittest test runner for doctests in test_statistics (GH-108921)

2 years agoAdd version directives to ast docs (#108788)
Shantanu [Thu, 7 Sep 2023 18:34:18 +0000 (11:34 -0700)] 
Add version directives to ast docs (#108788)

2 years agogh-68403: Fix test_coverage in test_trace (GH-108910)
Serhiy Storchaka [Thu, 7 Sep 2023 18:28:18 +0000 (21:28 +0300)] 
gh-68403: Fix test_coverage in test_trace (GH-108910)

Its behavior no longer affected by test running options such as -m.

2 years agogh-75743: Restore test_timeout.testConnectTimeout() (#109087)
Ee Durbin [Thu, 7 Sep 2023 18:13:32 +0000 (14:13 -0400)] 
gh-75743: Restore test_timeout.testConnectTimeout() (#109087)

This un-skips this test now that pythontest.net implements appropriate firewall
rules for it.

2 years agogh-103186: Make test_generated_cases less noisy by default (GH-109100)
Serhiy Storchaka [Thu, 7 Sep 2023 17:53:38 +0000 (20:53 +0300)] 
gh-103186: Make test_generated_cases less noisy by default (GH-109100)

Print additional details only when tests are run with -vv.

2 years agogh-109094: remove unnecessary updates of frame->prev_instr in instrumentation functio...
Irit Katriel [Thu, 7 Sep 2023 17:23:11 +0000 (18:23 +0100)] 
gh-109094: remove unnecessary updates of frame->prev_instr in instrumentation functions (#109076)

2 years agogh-107924: re-order os.sendfile() flag documentation (#107926)
Christoph Anton Mitterer [Thu, 7 Sep 2023 15:50:10 +0000 (17:50 +0200)] 
gh-107924: re-order os.sendfile() flag documentation (#107926)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agogh-108732: include comprehension locals in frame.f_locals (#109026)
Carl Meyer [Thu, 7 Sep 2023 14:56:43 +0000 (08:56 -0600)] 
gh-108732: include comprehension locals in frame.f_locals (#109026)

Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2 years agogh-102823: Document return type of floor division on floats (#102824)
Mark Dickinson [Thu, 7 Sep 2023 14:20:33 +0000 (15:20 +0100)] 
gh-102823: Document return type of floor division on floats (#102824)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agogh-88943: Improve syntax error for non-ASCII character that follows a numerical liter...
Serhiy Storchaka [Thu, 7 Sep 2023 14:00:13 +0000 (17:00 +0300)] 
gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081)

It now points on the invalid non-ASCII character, not on the valid numerical literal.

2 years agogh-107544: Add docs about `json.dumps(..., default=)` (#108259)
Nikita Sobolev [Thu, 7 Sep 2023 13:53:33 +0000 (16:53 +0300)] 
gh-107544: Add docs about `json.dumps(..., default=)` (#108259)

2 years agogh-71770: Add more details on behavior of configparser's default_section (#31562)
Stanley [Thu, 7 Sep 2023 13:52:58 +0000 (06:52 -0700)] 
gh-71770: Add more details on behavior of configparser's default_section (#31562)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agoGH-108614: Add `RESUME_CHECK` instruction (GH-108630)
Mark Shannon [Thu, 7 Sep 2023 13:39:03 +0000 (14:39 +0100)] 
GH-108614: Add `RESUME_CHECK` instruction (GH-108630)

2 years agogh-103186: Suppress RuntimeWarning about unclosed async iterator in test_sys_settrace...
Serhiy Storchaka [Thu, 7 Sep 2023 13:34:44 +0000 (16:34 +0300)] 
gh-103186: Suppress RuntimeWarning about unclosed async iterator in test_sys_settrace (GH-109075)

2 years agogh-103186: Remove debug print in test_sys_settrace (GH-109077)
Serhiy Storchaka [Thu, 7 Sep 2023 13:30:41 +0000 (16:30 +0300)] 
gh-103186: Remove debug print in test_sys_settrace (GH-109077)

2 years agoGH-90915: Document that SystemExit doesn't trigger sys.excepthook (#31357)
Colin Watson [Thu, 7 Sep 2023 13:14:27 +0000 (14:14 +0100)] 
GH-90915: Document that SystemExit doesn't trigger sys.excepthook (#31357)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agosocket documentation fix - rename triple to 3-tuple (#24722)
Ori Hoch [Thu, 7 Sep 2023 10:33:02 +0000 (13:33 +0300)] 
socket documentation fix - rename triple to 3-tuple (#24722)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agobpo-38157: Add example about per file output for mock_open. (#16090)
Karthikeyan Singaravelan [Thu, 7 Sep 2023 09:19:13 +0000 (14:49 +0530)] 
bpo-38157: Add example about per file output for mock_open. (#16090)

Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agogh-103186: Suppress and assert expected RuntimeWarnings in test_sys_settrace (GH...
Ijtaba Hussain [Thu, 7 Sep 2023 07:41:38 +0000 (12:41 +0500)] 
gh-103186: Suppress and assert expected RuntimeWarnings in test_sys_settrace (GH-103244)

Caused as a result of frame manipulation where locals are never assigned / initialised.

2 years agotest.pythoninfo logs freedesktop_os_release() (#109057)
Victor Stinner [Thu, 7 Sep 2023 05:43:32 +0000 (07:43 +0200)] 
test.pythoninfo logs freedesktop_os_release() (#109057)

2 years agogh-108915: Removes extra backslashes in str.split docstring (#109044)
Daniel Weiss [Thu, 7 Sep 2023 05:33:51 +0000 (23:33 -0600)] 
gh-108915: Removes extra backslashes in str.split docstring (#109044)

2 years agogh-108753: _Py_PrintSpecializationStats() uses Py_hexdigits (#109040)
Victor Stinner [Thu, 7 Sep 2023 02:47:57 +0000 (04:47 +0200)] 
gh-108753: _Py_PrintSpecializationStats() uses Py_hexdigits (#109040)

2 years agogh-109045: Remove remaining LIMITED_API_AVAILABLE checks in tests (#109046)
Anders Kaseorg [Thu, 7 Sep 2023 02:42:58 +0000 (19:42 -0700)] 
gh-109045: Remove remaining LIMITED_API_AVAILABLE checks in tests (#109046)

Commit 13a00078b81776b23b0b6add69b848382240d1f2 (#108663) made all
Python builds compatible with the Limited API, and removed the
LIMITED_API_AVAILABLE flag.  However, some tests were still checking
for that flag, so they were now being incorrectly skipped.  Remove
these checks to let these tests run again.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2 years agogh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument (gh-108539)
Dong-hee Na [Thu, 7 Sep 2023 00:53:54 +0000 (09:53 +0900)] 
gh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument (gh-108539)

2 years agogh-107211: No longer export internal _PyLong_FromUid() (#109037)
Victor Stinner [Thu, 7 Sep 2023 00:09:06 +0000 (02:09 +0200)] 
gh-107211: No longer export internal _PyLong_FromUid() (#109037)

No longer export _PyLong_FromUid() and _Py_Sigset_Converter()
internal C API function.

2 years agogh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016)
Victor Stinner [Wed, 6 Sep 2023 23:58:03 +0000 (01:58 +0200)] 
gh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016)

Skip test_asyncio, test_imaplib and test_socket tests if FreeBSD TCP
blackhole is enabled (net.inet.tcp.blackhole=2).

2 years agoDocs: Fix typo in datetime.tzinfo docstring (#107257)
Mikhail Samylov [Wed, 6 Sep 2023 21:29:46 +0000 (00:29 +0300)] 
Docs: Fix typo in datetime.tzinfo docstring (#107257)

Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agoGH-108202: Document ``calendar``'s command-line interface (#109020)
Adam Turner [Wed, 6 Sep 2023 21:19:54 +0000 (22:19 +0100)] 
GH-108202: Document ``calendar``'s command-line interface (#109020)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agoGH-104584: Restore frame->stacktop on optimizer error (GH-108953)
Brandt Bucher [Wed, 6 Sep 2023 20:59:50 +0000 (13:59 -0700)] 
GH-104584: Restore frame->stacktop on optimizer error (GH-108953)

2 years agogh-108751: Add copy.replace() function (GH-108752)
Serhiy Storchaka [Wed, 6 Sep 2023 20:55:42 +0000 (23:55 +0300)] 
gh-108751: Add copy.replace() function (GH-108752)

It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.

2 years agogh-107732: Mention dir support in importlib.resources docs (#107734)
Shahriar Heidrich [Wed, 6 Sep 2023 20:53:32 +0000 (22:53 +0200)] 
gh-107732: Mention dir support in importlib.resources docs (#107734)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agoGH-90690: Mention removal of ``PRECALL`` in What's New (#103910)
Anthony Shaw [Wed, 6 Sep 2023 20:28:00 +0000 (06:28 +1000)] 
GH-90690: Mention removal of ``PRECALL`` in What's New (#103910)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2 years agogh-106307: Fix PyMapping_GetOptionalItemString() (GH-108797)
Serhiy Storchaka [Wed, 6 Sep 2023 19:47:38 +0000 (22:47 +0300)] 
gh-106307: Fix PyMapping_GetOptionalItemString() (GH-108797)

The resulting pointer was not set to NULL if the creation of a temporary
string object was failed.

The tests were also missed due to oversight.

2 years agoC API tests: use special markers to test that output parameters were set (GH-109014)
Serhiy Storchaka [Wed, 6 Sep 2023 19:02:01 +0000 (22:02 +0300)] 
C API tests: use special markers to test that output parameters were set (GH-109014)

2 years agogh-108740: Fix "make regen-all" race condition (#108741)
Victor Stinner [Wed, 6 Sep 2023 18:09:21 +0000 (20:09 +0200)] 
gh-108740: Fix "make regen-all" race condition (#108741)

Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.

* "make regen-global-objects": Make sure that deepfreeze.c is
  generated and up to date, and always run "make clinic".
* "make clinic" no longer runs generate_global_objects.py script.
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
  It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
  "make clinic". They are now outdated, these commands are now
  safe to use.
* Document generates files in Doc/using/configure.rst.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2 years agogh-108753: Enhance pystats (#108754)
Victor Stinner [Wed, 6 Sep 2023 15:54:59 +0000 (17:54 +0200)] 
gh-108753: Enhance pystats (#108754)

Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2 years agogh-108851: Fix tomllib recursion tests (#108853)
Victor Stinner [Wed, 6 Sep 2023 15:34:31 +0000 (17:34 +0200)] 
gh-108851: Fix tomllib recursion tests (#108853)

* Add get_recursion_available() and get_recursion_depth() functions
  to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
  recursion limit and compute the maximum nested array/dict depending
  on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
  and sys.setrecursionlimit().

2 years agogh-91960: Add FreeBSD build and test using Cirrus-CI (#91961)
Ed Maste [Wed, 6 Sep 2023 14:57:40 +0000 (10:57 -0400)] 
gh-91960: Add FreeBSD build and test using Cirrus-CI (#91961)

Cirrus-CI is a hosted CI service that supports FreeBSD, Linux, macOS,
and Windows.  Add a .cirrus.yml configuration file to provide CI coverage
on pull requests for FreeBSD 13.2.

Co-authored-by: Victor Stinner <vstinner@python.org>
2 years agogh-109002: Ensure only one wheel for each vendored package (#109003)
Łukasz Langa [Wed, 6 Sep 2023 14:49:44 +0000 (16:49 +0200)] 
gh-109002: Ensure only one wheel for each vendored package (#109003)

Output with one wheel:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
Verifying checksum for /Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl.
Expected digest: 7ccf472345f20d35bdc9d1841ff5f313260c2c33fe417f48c30ac46cccabf5be
Actual digest:   7ccf472345f20d35bdc9d1841ff5f313260c2c33fe417f48c30ac46cccabf5be
::notice file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Successfully verified the checksum of the pip wheel.
```

Output with two wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl::Found more than one wheel for package pip.

::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Found more than one wheel for package pip.
```

Output without wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=::Could not find a pip wheel on disk.
```

2 years agogh-91960: Skip test_gdb if gdb cannot retrive Python frames (#108999)
Victor Stinner [Wed, 6 Sep 2023 14:34:35 +0000 (16:34 +0200)] 
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (#108999)

Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.

2 years agogh-107219: Fix concurrent.futures terminate_broken() (#108974)
Victor Stinner [Wed, 6 Sep 2023 13:57:01 +0000 (15:57 +0200)] 
gh-107219: Fix concurrent.futures terminate_broken() (#108974)

Fix a race condition in _ExecutorManagerThread.terminate_broken():
ignore the InvalidStateError on future.set_exception(). It can happen
if the future is cancelled before the caller.

Moreover, test_crash_big_data() now waits explicitly until the
executor completes.

2 years agogh-108765: Cleanup #include in Python/*.c files (#108977)
Victor Stinner [Wed, 6 Sep 2023 13:56:08 +0000 (15:56 +0200)] 
gh-108765: Cleanup #include in Python/*.c files (#108977)

Mention one symbol imported by each #include.

2 years agogh-108303: Create Lib/test/test_dataclasses/ directory (#108978)
Victor Stinner [Wed, 6 Sep 2023 13:54:16 +0000 (15:54 +0200)] 
gh-108303: Create Lib/test/test_dataclasses/ directory (#108978)

Move test_dataclasses.py and its "dataclass_*.py" modules into the
new Lib/test/test_dataclasses/ subdirectory.

2 years agogh-108983: Add more PEP 526 tests to `test_grammar` (#108984)
Nikita Sobolev [Wed, 6 Sep 2023 13:41:38 +0000 (16:41 +0300)] 
gh-108983: Add more PEP 526 tests to `test_grammar` (#108984)

2 years agogh-108991: replace _PyFrame_GetState by two simpler functions (#108992)
Irit Katriel [Wed, 6 Sep 2023 11:54:59 +0000 (12:54 +0100)] 
gh-108991: replace _PyFrame_GetState by two simpler functions (#108992)

2 years agogh-106670: Fix Pdb handling of chained Exceptions with no stacks. (#108865)
Matthias Bussonnier [Wed, 6 Sep 2023 09:41:56 +0000 (11:41 +0200)] 
gh-106670: Fix Pdb handling of chained Exceptions with no stacks. (#108865)

2 years agoFix a typo in umarshal.py (#108803)
KH [Wed, 6 Sep 2023 09:06:41 +0000 (18:06 +0900)] 
Fix a typo in umarshal.py (#108803)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2 years agogh-108717: Speedup `os.DirEntry.is_junction` function (#108718)
Nikita Sobolev [Wed, 6 Sep 2023 08:52:53 +0000 (11:52 +0300)] 
gh-108717: Speedup `os.DirEntry.is_junction` function (#108718)

2 years agogh-108857: improve markup in inspect.Signature.replace() docs (#108862)
nabin2004 [Wed, 6 Sep 2023 00:27:59 +0000 (06:12 +0545)] 
gh-108857: improve markup in inspect.Signature.replace() docs (#108862)

2 years agogh-108962: Skip test_tempfile.test_flags() if not supported (#108964)
Victor Stinner [Tue, 5 Sep 2023 21:59:40 +0000 (23:59 +0200)] 
gh-108962: Skip test_tempfile.test_flags() if not supported (#108964)

Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).