Alex Waygood [Wed, 9 Oct 2024 10:53:57 +0000 (11:53 +0100)]
gh-101100: Consolidate documentation on `ModuleType` attributes (#124709)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Barry Warsaw <barry@python.org> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-107562: make_ssl_certs.py: produce test certificates that expire far in the future by default (GH-107594)
This allows testing Y2038 with system time set to after that,
so that actual Y2038 issues can be exposed, and not masked
by expired certificate errors.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Victor Stinner [Wed, 9 Oct 2024 08:15:17 +0000 (10:15 +0200)]
gh-124502: Remove _PyUnicode_EQ() function (#125114)
* Replace unicode_compare_eq() with unicode_eq().
* Use unicode_eq() in setobject.c.
* Replace _PyUnicode_EQ() with _PyUnicode_Equal().
* Remove unicode_compare_eq() and _PyUnicode_EQ().
mpage [Tue, 8 Oct 2024 14:04:35 +0000 (07:04 -0700)]
gh-115999: Stop the world when invalidating function versions (#124997)
Stop the world when invalidating function versions
The tier1 interpreter specializes `CALL` instructions based on the values
of certain function attributes (e.g. `__code__`, `__defaults__`). The tier1
interpreter uses function versions to verify that the attributes of a function
during execution of a specialization match those seen during specialization.
A function's version is initialized in `MAKE_FUNCTION` and is invalidated when
any of the critical function attributes are changed. The tier1 interpreter stores
the function version in the inline cache during specialization. A guard is used by
the specialized instruction to verify that the version of the function on the operand
stack matches the cached version (and therefore has all of the expected attributes).
It is assumed that once the guard passes, all attributes will remain unchanged
while executing the rest of the specialized instruction.
Stopping the world when invalidating function versions ensures that all critical
function attributes will remain unchanged after the function version guard passes
in free-threaded builds. It's important to note that this is only true if the remainder
of the specialized instruction does not enter and exit a stop-the-world point.
We will stop the world the first time any of the following function attributes
are mutated:
Cody Maloney [Tue, 8 Oct 2024 11:49:50 +0000 (04:49 -0700)]
gh-90102: Fix pyio _isatty_open_only() (#125089)
Spotted by @ngnpope.
`isatty` returns False to indicate the file is not a TTY. The C
implementation of _io does that (`Py_RETURN_FALSE`) but I got the
bool backwards in the _pyio implementaiton.
gh-53203: Improve tests for strptime() (GH-125090)
Run them with different locales and different date and time.
Add the @run_with_locales() decorator to run the test with multiple
locales.
Improve the run_with_locale() context manager/decorator -- it now
catches only expected exceptions and reports the test as skipped if no
appropriate locale is available.
Alyssa Coghlan [Tue, 8 Oct 2024 08:34:11 +0000 (18:34 +1000)]
bpo-34206: Improve docs and test coverage for pre-init functions (#8023)
- move the Py_Main documentation from the very high level API section
to the initialization and finalization section
- make it clear that it encapsulates a full Py_Initialize/Finalize
cycle of its own
- point out that exactly which settings will be read and applied
correctly when Py_Main is called after a separate runtime
initialization call is version dependent
- be explicit that Py_IsInitialized can be called prior to
initialization
- actually test that Py_IsInitialized can be called prior to
initialization
- flush stdout in the embedding tests that run code so it appears
in the expected order when running with "-vv"
- make "-vv" on the subinterpreter embedding tests less spammy
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
gh-69998: Fix decoding error in locale.nl_langinfo() (GH-124963)
The function now sets temporarily the LC_CTYPE locale to the locale
of the category that determines the requested value if the locales are
different and the resulting string is non-ASCII.
This temporary change affects other threads.
Alyssa Coghlan [Tue, 8 Oct 2024 07:52:12 +0000 (17:52 +1000)]
gh-70870: Clarify dual usage of 'free variable' (#122545)
The term "free variable" has unfortunately become genuinely
ambiguous over the years (presumably due to the names of
some relevant code object instance attributes).
While we can't eliminate that ambiguity at this late date, we can
at least alert people to the potential ambiguity by describing
both the formal meaning of the term and the common
alternative use as a direct synonym for "closure variable".
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
is not supported for certificate signing requests (since the issuing
certificate authority is not known).
David von Oheimb, the OpenSSL dev that made the change, commented in:
https://github.com/openssl/openssl/issues/22966#issuecomment-1858396738 :
> This problem did not show up in older OpenSSL versions because of a bug:
> the `req` app ignored the `-extensions` option unless `-x505` is given,
> which I fixed in https://github.com/openssl/openssl/pull/16865.
(I assume `-x505` is a typo for `-x509`.)
In our `make_cert_key` function:
If `sign` is true:
- We don't pass `-x509` to `req`, so in this case it should be safe to
omit the `-extensions` argument. (Old OpenSSL ignores it, new OpenSSL
fails on it.)
- The extensions are passed to the `ca` call later in the function.
There they take effect, and `authorityKeyIdentifier` is valid.
If `sign` is false, this commit has no effect except rearranging the
CLI arguments.
Andrés Delfino [Sun, 6 Oct 2024 22:22:19 +0000 (19:22 -0300)]
Doc: Simplify the definition of 'soft deprecated' (#124988)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
morotti [Fri, 4 Oct 2024 23:51:22 +0000 (00:51 +0100)]
gh-117151: increase default buffer size of shutil.copyfileobj() to 256k. (GH-119783)
* gh-117151: increase default buffer size of shutil.copyfileobj() to 256k.
it was set to 16k in the 1990s.
it was raised to 64k in 2019. the discussion at the time mentioned another 5% improvement by raising to 128k and settled for a very conservative setting.
it's 2024 now, I think it should be revisited to match modern hardware. I am measuring 0-15% performance improvement when raising to 256k on various types of disk. there is no downside as far as I can tell.
this function is only intended for sequential copy of full files (or file like objects). it's the typical use case that benefits from larger operations.
for reference, I came across this function while trying to profile pip that is using it to copy files when installing python packages.
efimov-mikhail [Thu, 3 Oct 2024 12:58:56 +0000 (15:58 +0300)]
gh-124889: Remove redundant artificial rules in PEG parser (#124893)
Cache in C PEG-generator reworked:
we save artificial rules in cache by Node string representation as a key instead of Node object itself.
As a result total count of artificial rules in parsers.c is lowered from 283 to 170.
More natural number ordering is used for the names of artificial rules.
Auxiliary method CCallMakerVisitor._generate_artificial_rule_call is added.
Its purpose is abstracting work with artificial rules cache.
Explicit using of "is_repeat1" kwarg is added to visit_Repeat0 and visit_Repeat1 methods.
Its slightly improve code readabitily.
gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805)
Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Michał Górny [Wed, 2 Oct 2024 14:31:42 +0000 (16:31 +0200)]
gh-124213: Fix incorrect context manager use in in_systemd_nspawn_sync_suppressed() (#124892)
Fix the incorrect use of `os.open()` result as a context manager,
while it is actually a numeric file descriptor.
I have missed the problem, because in the original version the
`os.open()` call would always fail, and I failed to test the final
version in all possible scenarios properly.
Peter Bierma [Wed, 2 Oct 2024 14:31:23 +0000 (10:31 -0400)]
gh-120378: Fix crash caused by integer overflow in `curses` (#124555)
This is actually an upstream problem in curses, and has been reported
to them already:
https://lists.gnu.org/archive/html/bug-ncurses/2024-09/msg00101.html
This is a nice workaround in the meantime to prevent the segfault.
Mark Byrne [Wed, 2 Oct 2024 09:58:39 +0000 (11:58 +0200)]
gh-122864: Fix a ``test_funcattrs.test___builtins__`` when executing directly (#124845)
Previously when executing ``test_functattrs.test___builtins__`` directly, it failed because the fact, that ``__builtins__`` is refers to the built-in module ``builtins`` while it's expects a ``__builtins__.__dict__``. But when this test is being run from another module, then ``__builtins__`` is refers to ``builtins.__dict__``. Now this part of the behaviour is covered.
---------
Co-authored-by: Victor Stinner <vstinner@python.org>