Victor Stinner [Fri, 21 Oct 2022 14:21:36 +0000 (16:21 +0200)]
gh-95027: Fix regrtest stdout encoding on Windows (#98492)
On Windows, when the Python test suite is run with the -jN option,
the ANSI code page is now used as the encoding for the stdout
temporary file, rather than using UTF-8 which can lead to decoding
errors.
Gregory P. Smith [Thu, 20 Oct 2022 22:30:09 +0000 (15:30 -0700)]
gh-97514: Don't use Linux abstract sockets for multiprocessing (#98501)
Linux abstract sockets are insecure as they lack any form of filesystem
permissions so their use allows anyone on the system to inject code into
the process.
This removes the default preference for abstract sockets in
multiprocessing introduced in Python 3.9+ via
https://github.com/python/cpython/pull/18866 while fixing
https://github.com/python/cpython/issues/84031.
Explicit use of an abstract socket by a user now generates a
RuntimeWarning. If we choose to keep this warning, it should be
backported to the 3.7 and 3.8 branches.
MonadChains [Thu, 20 Oct 2022 13:59:10 +0000 (15:59 +0200)]
gh-98421: Clean Up PyObject_Print (GH-98422)
Work on test coverage for `PyObject_Print` made it clear that some lines can't get executed.
Simplify the function by excluding the checks for non-string types.
Also eliminate creating a temporary bytes object.
Victor Stinner [Wed, 19 Oct 2022 22:31:47 +0000 (00:31 +0200)]
gh-98257: Make _PyEval_SetTrace() reentrant (#98258)
Make sys.setprofile() and sys.settrace() functions reentrant. They
can no long fail with: RuntimeError("Cannot install a trace function
while another trace function is being installed").
Make _PyEval_SetTrace() and _PyEval_SetProfile() functions reentrant,
rather than detecting and rejecting reentrant calls. Only delete the
reference to function arguments once the new function is fully set,
when a reentrant call is safe. Call also _PySys_Audit() earlier.
Doc: Remove title text from internal links (#98409)
Rely on the title of the linked internal page instead of putting the title. Sphinx will render with the title correctly, and this will reduce work for translators
Victor Stinner [Tue, 18 Oct 2022 15:52:31 +0000 (17:52 +0200)]
gh-98393: os module reject bytes-like, only accept bytes (#98394)
The os module and the PyUnicode_FSDecoder() function no longer accept
bytes-like paths, like bytearray and memoryview types: only the exact
bytes type is accepted for bytes strings.
Alex Zvorygin [Tue, 18 Oct 2022 13:42:10 +0000 (06:42 -0700)]
gh-98378: Add small format string example to strftime comments (GH-98379)
A small example of what a full date and time would look like would help a lot of developers who may not realize that they should investigate `time.h`'s `strftime`, run `man strftime`, or click through a series of docs on the python docs before they get to the actual [definition here](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) which still doesn't have an obvious copy-pastable example of "what the heck format does this thing actually expect?".
C.A.M. Gerlach [Mon, 17 Oct 2022 19:59:22 +0000 (14:59 -0500)]
gh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (GH-98345)
Part of #95913
Forward port of #93306, which was a backport of #93185, to address #84694
This adds the What's New entry for the removal of the subinterpreter-related env variable, build-time flag, etc. As @ericsnowcurrently was author of the original changes, I added him as a co-author to the commit.
This addition to the Python 3.11 What's New document were only made to the Python 3.11 branch during the backport process, and not added to the version in `main`. Forward-porting it ensures the docs retain these additions for the future, rather than being lost in a legacy Python versions, allows it to be be edited as part of #95913 , and avoids merge conflicts with routine back-ports of PRs touching it.
I've pulled in the addition exactly as-is with no modifications; any editing will be done in future PRs (and therefore can be reviewed and backported accordingly).
The one other such addition is forward-ported in #98344
Kumar Aditya [Mon, 17 Oct 2022 15:27:02 +0000 (20:57 +0530)]
GH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (#98333)
Alas, warnings.catch_warnings() has global scope, not thread scope, so this is still not perfect, but it reduces the time during which warnings are ignored. Better solution welcome.
Ruben Vorderman [Mon, 17 Oct 2022 02:10:58 +0000 (04:10 +0200)]
gh-95534: Improve gzip reading speed by 10% (#97664)
Change summary:
+ There is now a `gzip.READ_BUFFER_SIZE` constant that is 128KB. Other programs that read in 128KB chunks: pigz and cat. So this seems best practice among good programs. Also it is faster than 8 kb chunks.
+ a zlib._ZlibDecompressor was added. This is the _bz2.BZ2Decompressor ported to zlib. Since the zlib.Decompress object is better for in-memory decompression, the _ZlibDecompressor is hidden. It only makes sense in file decompression, and that is already implemented now in the gzip library. No need to bother the users with this.
+ The ZlibDecompressor uses the older Cpython arrange_output_buffer functions, as those are faster and more appropriate for the use case.
+ GzipFile.read has been optimized. There is no longer a `unconsumed_tail` member to write back to padded file. This is instead handled by the ZlibDecompressor itself, which has an internal buffer. `_add_read_data` has been inlined, as it was just two calls.
EDIT: While I am adding improvements anyway, I figured I could add another one-liner optimization now to the python -m gzip application. That read chunks in io.DEFAULT_BUFFER_SIZE previously, but has been updated now to use READ_BUFFER_SIZE chunks.
Terry Jan Reedy [Sun, 16 Oct 2022 14:23:11 +0000 (10:23 -0400)]
gh-97527: IDLE - fix buggy macosx patch (#98313)
#97530 fixed IDLE tests possibly crashing on a Mac without a GUI.
But it resulted in IDLE not starting in 3.10.8, 3.12.0a1, and
Microsoft Python 3.10.2288.0 when test/* is not installed.
After this patch, test.* is only imported when testing on Mac.
Kumar Aditya [Sat, 15 Oct 2022 23:09:30 +0000 (04:39 +0530)]
GH-94597: Deprecate child watcher getters and setters (#98215)
This is the next step for deprecating child watchers.
Until we've removed the API completely we have to use it, so this PR is mostly suppressing a lot of warnings when using the API internally.
Once the child watcher API is totally removed, the two child watcher implementations we actually use and need (Pidfd and Thread) will be turned into internal helpers.
Victor Stinner [Thu, 13 Oct 2022 11:34:55 +0000 (13:34 +0200)]
gh-98178: syslog() is not thread-safe on macOS (#98213)
On macOS, fix a crash in syslog.syslog() in multi-threaded
applications. On macOS, the libc syslog() function is not
thread-safe, so syslog.syslog() no longer releases the GIL to call
it.
Signal wakeup fd errors are now logged with
_PyErr_WriteUnraisableMsg(), rather than PySys_WriteStderr() and
PyErr_WriteUnraisable(), to pass the error message to
sys.unraisablehook. By default, it's still written into stderr (unless
sys.unraisablehook is overriden).
Victor Stinner [Wed, 12 Oct 2022 15:26:21 +0000 (17:26 +0200)]
gh-97669: Fix test_tools reference leak (#98216)
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
Guido van Rossum [Tue, 11 Oct 2022 23:45:53 +0000 (16:45 -0700)]
Fix some incorrect indentation around the main switch (#98177)
The `}` marked with `/* End instructions */` is the end of the switch.
There is another pair of `{}` around the switch, which is vestigial
from ancient times when it was `for (;;) { switch (opcode) { ... } }`.
All `DISPATCH` macro calls should be inside that pair.
Victor Stinner [Tue, 11 Oct 2022 15:59:58 +0000 (17:59 +0200)]
gh-97982: Remove asciilib_count() (#98164)
asciilib_count() is the same than ucs1lib_count(): the code is not
specialized for ASCII strings, so it's not worth it to have a
separated function. Remove asciilib_count() function.