Barney Gale [Mon, 3 Jul 2023 20:29:44 +0000 (21:29 +0100)]
GH-106330: Fix matching of empty path in `pathlib.PurePath.match()` (GH-106331)
We match paths using the `_lines` attribute, which is derived from the
path's string representation. The bug arises because an empty path's string
representation is `'.'` (not `''`), which is matched by the `'*'` wildcard.
- Tweak uops debugging output
- Fix the bug from gh-106290
- Rename `SET_IP` to `SAVE_IP` (per https://github.com/faster-cpython/ideas/issues/558)
- Add a `SAVE_IP` uop at the start of the trace (ditto)
- Allow `unbound_local_error`; this gives us uops for `LOAD_FAST_CHECK`, `LOAD_CLOSURE`, and `DELETE_FAST`
- Longer traces
- Support `STORE_FAST_LOAD_FAST`, `STORE_FAST_STORE_FAST`
- Add deps on pycore_uops.h to Makefile(.pre.in)
Mario Corchero [Mon, 3 Jul 2023 06:56:54 +0000 (07:56 +0100)]
gh-61215: New mock to wait for multi-threaded events to happen (#16094)
mock: Add `ThreadingMock` class
Add a new class that allows to wait for a call to happen by using
`Event` objects. This mock class can be used to test and validate
expectations of multithreading code.
It uses two attributes for events to distinguish calls with any argument
and calls with specific arguments.
The calls with specific arguments need a lock to prevent two calls in
parallel from creating the same event twice.
The timeout is configured at class and constructor level to allow users
to set a timeout, we considered passing it as an argument to the
function but it could collide with a function parameter. Alternatively
we also considered passing it as positional only but from an API
caller perspective it was unclear what the first number meant on the
function call, think `mock.wait_until_called(1, "arg1", "arg2")`, where
1 is the timeout.
Lastly we also considered adding the new attributes to magic mock
directly rather than having a custom mock class for multi threading
scenarios, but we preferred to have specialised class that can be
composed if necessary. Additionally, having added it to `MagicMock`
directly would have resulted in `AsyncMock` having this logic, which
would not work as expected, since when if user "waits" on a
coroutine does not have the same meaning as waiting on a standard
call.
Remove private _PyThreadState and _PyInterpreterState C API
functions: move them to the internal C API (pycore_pystate.h and
pycore_interp.h). Don't export most of these functions anymore, but
still export functions used by tests.
Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C
API, but keep it in the stable API.
Remove private _PyThreadState and _PyInterpreterState C API
functions: move them to the internal C API (pycore_pystate.h and
pycore_interp.h). Don't export most of these functions anymore, but
still export functions used by tests.
Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C
API, but keep it in the stable API.
Victor Stinner [Sun, 2 Jul 2023 00:49:18 +0000 (02:49 +0200)]
gh-106320: Remove private _PyTraceMalloc C API functions (#106324)
* Remove private _PyTraceMalloc C API functions: move them to the
internal C API.
* Don't export most of these functions anymore, but still export
_PyTraceMalloc_GetTraceback() used by tests.
* Rename Include/tracemalloc.h to Include/cpython/tracemalloc.h
gh-102541: Fix Helper.help("mod") for non-existent mod (#105934)
If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Barney Gale [Sat, 1 Jul 2023 12:33:29 +0000 (13:33 +0100)]
GH-89812: Improve test for `pathlib.Path.stat()` (GH-106064)
Make assertions about the `st_mode`, `st_ino` and `st_dev` attributes of
the stat results from two files and a directory, rather than checking if
`chmod()` affects `st_mode` (which is already tested elsewhere).
Barney Gale [Sat, 1 Jul 2023 12:29:02 +0000 (13:29 +0100)]
GH-89812: Miscellaneous pathlib test improvements (GH-106063)
- Split out dedicated test for unbuffered `open()`
- Split out dedicated test for `is_mount()` at the filesystem root
- Avoid `os.stat()` when checking that empty paths point to '.'
- Remove unnecessary `rmtree()` call
- Remove unused `assertSame()` method
Barney Gale [Sat, 1 Jul 2023 11:24:34 +0000 (12:24 +0100)]
GH-89812: Make symlink support configurable in pathlib tests. (GH-106060)
Adjust the pathlib tests to add a new `PathTest.can_symlink` class
attribute, which allows us to enable or disable symlink support in tests.
A (near-)future commit will add an `AbstractPath` class; its tests will
hard-code the value to `True` or `False` depending on a stub subclass's
capabilities.
Terry Jan Reedy [Sat, 1 Jul 2023 02:34:31 +0000 (22:34 -0400)]
gh-106232: Make timeit doc command lines compatible with Windows. (#106296)
Command Prompt (CMD Shell) and older versions of PowerShell
require double quotes and single quotes inside the string.
This form also works on linux and macOS.
Barney Gale [Fri, 30 Jun 2023 23:46:44 +0000 (00:46 +0100)]
GH-89812: Simplify creation of symlinks in pathlib tests. (GH-106061)
Remove `PathTest.dirlink()` function. Symlinks in `PathTest.setUp()` are
created using `os.symlink()` directly; symlinks in test functions use
`Path.symlink_to()` in order to make the tests applicable to a
(near-)future `AbstractPath` class.
Charlie Zhao [Thu, 29 Jun 2023 10:27:20 +0000 (18:27 +0800)]
gh-106078: Convert `_decimal` types to heap types (#106079)
- Establish global state struct
- Convert static types to heap types and add them to global state:
* PyDecContextManager_Type
* PyDecContext_Type
* PyDecSignalDictMixin_Type
* PyDec_Type
- Add to global state:
* PyDecSignalDict_Type
* DecimalTuple
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend@python.org>
Serhiy Storchaka [Thu, 29 Jun 2023 09:22:19 +0000 (12:22 +0300)]
gh-101006: Improve error handling when read marshal data (GH-101007)
* EOFError no longer overrides other errors such as MemoryError or OSError at
the start of the object.
* Raise more relevant error when the NULL object occurs as a code object
component.
* Minimize an overhead of calling PyErr_Occurred().
Guido van Rossum [Wed, 28 Jun 2023 18:28:07 +0000 (11:28 -0700)]
gh-104584: Emit macro expansions to opcode_metadata.h (#106163)
This produces longer traces (superblocks?).
Also improved debug output (uop names are now printed instead of numeric opcodes). This would be simpler if the numeric opcode values were generated by generate_cases.py, but that's another project.
Refactored some code in generate_cases.py so the essential algorithm for cache effects is only run once. (Deciding which effects are used and what the total cache size is, regardless of what's used.)
Victor Stinner [Wed, 28 Jun 2023 02:41:21 +0000 (04:41 +0200)]
GH-104584: Fix test_capi.test_counter_optimizer() when run twice (#106171)
test_counter_optimizer() and test_long_loop() of test_capi now create
a new function at each call. Otherwise, the optimizer counters are
not the expected values when the test is run more than once.
Victor Stinner [Wed, 28 Jun 2023 02:26:52 +0000 (04:26 +0200)]
gh-101634: regrtest reports decoding error as failed test (#106169)
When running the Python test suite with -jN option, if a worker stdout
cannot be decoded from the locale encoding report a failed testn so the
exitcode is non-zero.
Victor Stinner [Wed, 28 Jun 2023 01:45:57 +0000 (03:45 +0200)]
gh-106168: PyTuple_SET_ITEM() now checks the index (#106164)
PyTuple_SET_ITEM() and PyList_SET_ITEM() now check the index argument
with an assertion if Python is built in debug mode or is built with
assertions.
* list_extend() and _PyList_AppendTakeRef() now set the list size
before calling PyList_SET_ITEM().
* PyStructSequence_GetItem() and PyStructSequence_SetItem() now check
the index argument: must be lesser than REAL_SIZE(op).
* PyStructSequence_GET_ITEM() and PyStructSequence_SET_ITEM() are now
aliases to PyStructSequence_GetItem() and
PyStructSequence_SetItem().
T. Wouters [Wed, 28 Jun 2023 00:55:11 +0000 (02:55 +0200)]
GH-106160: Fix test_gzip failing under WASI, which does not have zlib. (#106167)
Fix test_gzip's failure under WASI, which does not have zlib, by using
test.support.import_helper.import_module to import zlib. (gzip
unconditionally imports zlib, so this does not cause any new skips.)
Guido van Rossum [Tue, 27 Jun 2023 21:17:41 +0000 (14:17 -0700)]
gh-104584: Change DEOPT_IF in uops executor (#106146)
This effectively reverts bb578a0, restoring the original DEOPT_IF() macro in ceval_macros.h, and redefining it in the Tier 2 interpreter. We can get rid of the PREDICTED() macros there as well!
Guido van Rossum [Tue, 27 Jun 2023 02:02:57 +0000 (19:02 -0700)]
gh-104584: Baby steps towards generating and executing traces (#105924)
Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose).
All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.
Barney Gale [Mon, 26 Jun 2023 16:58:17 +0000 (17:58 +0100)]
GH-105793: Add follow_symlinks argument to `pathlib.Path.is_dir()` and `is_file()` (GH-105794)
Brings `pathlib.Path.is_dir()` and `in line with `os.DirEntry.is_dir()`, which
will be important for implementing generic path walking and globbing.
Likewise `is_file()`.