Zackery Spytz [Mon, 19 Oct 2020 22:47:37 +0000 (16:47 -0600)]
bpo-41845: Move PyObject_GenericGetDict() back into the limited API (GH22646)
It was moved out of the limited API in 7d95e4072169911b228c9e42367afb5f17fd3db0.
This change re-enables it from 3.10, to avoid generating invalid extension modules for earlier versions.
Kevin Adler [Fri, 16 Oct 2020 18:03:28 +0000 (13:03 -0500)]
closes bpo-42030: Remove legacy AIX dynload support (GH-22717)
Since c19c5a6, AIX builds have defaulted to using dynload_shlib over
dynload_aix when dlopen is available. This function has been available
since AIX 4.3, which went out of support in 2003, the same year the
previously referenced commit was made. It has been nearly 20 years
since a version of AIX has been supported which has not used
dynload_shlib so there's no reason to keep this legacy code around.
Victor Stinner [Thu, 15 Oct 2020 14:22:19 +0000 (16:22 +0200)]
bpo-1635741: Add a global module state to unicodedata (GH-22712)
Prepare unicodedata to add a state per module: start with a global
"module" state, pass it to subfunctions which access &UCD_Type. This
change also prepares the conversion of the UCD_Type static type to a
heap type.
Kevin Adler [Thu, 15 Oct 2020 01:53:27 +0000 (20:53 -0500)]
bpo-41894: Fix UnicodeDecodeError while loading native module (GH-22466)
When running in a non-UTF-8 locale, if an error occurs while importing a
native Python module (say because a dependent share library is missing),
the error message string returned may contain non-ASCII code points
causing a UnicodeDecodeError.
PyUnicode_DecodeFSDefault is used for buffers which may contain
filesystem paths. For consistency with os.strerror(),
PyUnicode_DecodeLocale is used for buffers which contain system error
messages. While the shortname parameter is always encoded in ASCII
according to PEP 489, it is left decoded using PyUnicode_FromString to
minimize the changes and since it should not affect the decoding (albeit
_potentially_ slower).
In dynload_hpux, since the error buffer contains a message generated
from a static ASCII string and the module filesystem path,
PyUnicode_DecodeFSDefault is used instead of PyUnicode_DecodeLocale as
is used elsewhere.
* bpo-41894: Fix bugs in dynload error msg handling
For both dynload_aix and dynload_hpux, properly handle the possibility
that decoding strings may return NULL and when such an error happens,
properly decrement any previously decoded strings and return early.
In addition, in dynload_aix, ensure that we pass the decoded string
*object* pathname_ob to PyErr_SetImportError instead of the original
pathname buffer.
Kevin Adler [Wed, 14 Oct 2020 01:49:24 +0000 (20:49 -0500)]
closes bpo-42029: Remove dynload_dl (GH-22687)
All references to this dynamic loading method were removed in b9949db,
when support for this method was dropped, but the implementation code
was not dropped (seemingly in oversight).
Kyle Evans [Mon, 12 Oct 2020 23:53:16 +0000 (18:53 -0500)]
bpo-40422: Move _Py_*_SUPPRESS_IPH bits into _Py_closerange (GH-22672)
This suppression is no longer needed in os_closerange_impl, as it just
invokes the internal _Py_closerange implementation. On the other hand,
consumers of _Py_closerange may not have any other reason to suppress
invalid parameter issues, so narrow the scope to here.
Kyle Evans [Sun, 11 Oct 2020 20:18:53 +0000 (15:18 -0500)]
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.
[note that this version does check for ENOSYS, but currently ignores all other errors]
Kyle Evans [Sun, 11 Oct 2020 18:54:11 +0000 (13:54 -0500)]
bpo-40422: create a common _Py_closerange API (GH-19754)
Such an API can be used both for os.closerange and subprocess. For the latter, this yields potential improvement for platforms that have fdwalk but wouldn't have used it there. This will prove even more beneficial later for platforms that have close_range(2), as the new API will prefer that over all else if it's available.
The new API is structured to look more like close_range(2), closing from [start, end] rather than the [low, high) of os.closerange().
Serhiy Storchaka [Sun, 11 Oct 2020 13:51:07 +0000 (16:51 +0300)]
bpo-41993: Fix possible issues in remove_module() (GH-22631)
* PyMapping_HasKey() is not safe because it silences all exceptions and can return incorrect result.
* Informative exceptions from PyMapping_DelItem() are overridden with RuntimeError and
the original exception raised before calling remove_module() is lost.
* There is a race condition between PyMapping_HasKey() and PyMapping_DelItem().
Batuhan Taskaya [Sat, 10 Oct 2020 17:14:59 +0000 (20:14 +0300)]
bpo-42000: Cleanup the AST related C-code (GH-22641)
- Use the proper asdl sequence when creating empty arguments
- Remove reduntant casts (thanks to new typed asdl_sequences)
- Remove MarshalPrototypeVisitor and some utilities from asdl generator
- Fix the header of `Python/ast.c` (kept from pgen times)
Saiyang Gou [Fri, 9 Oct 2020 20:00:15 +0000 (13:00 -0700)]
bpo-39481: Fix duplicate SimpleQueue type in test_genericalias.py (GH-22619)
There are two different `SimpleQueue` types imported (from `multiprocessing.queues` and `queue`) in `Lib/test/test_genericalias.py`, the second one shadowing the first one, making the first one not actually tested. Fix by using different names.
Remove complex special methods __int__, __float__, __floordiv__,
__mod__, __divmod__, __rfloordiv__, __rmod__ and __rdivmod__
which always raised a TypeError.
E-Paine [Thu, 8 Oct 2020 13:30:13 +0000 (14:30 +0100)]
bpo-41306: Allow scale value to not be rounded (GH-21715)
This fixes the test failure with Tk 6.8.10 which is caused by changes to how Tk rounds the `from`, `to` and `tickinterval` arguments. This PR uses `noconv` if the patchlevel is greater than or equal to 8.6.10 (credit to Serhiy for this idea as it is much simpler than what I previously proposed).
Going into more detail for those who want it, the Tk change was made in [commit 591f68c](https://github.com/tcltk/tk/commit/591f68cb382525b72664c6fecaab87742b6cc87a) and means that the arguments listed above are rounded relative to the value of `from`. However, when rounding the `from` argument ([line 623](https://github.com/tcltk/tk/blob/591f68cb382525b72664c6fecaab87742b6cc87a/generic/tkScale.c#L623)), it is rounded relative to itself (i.e. rounding `0`) and therefore the assigned value for `from` is always what is given (no matter what values of `from` and `resolution`).
It was not practical to prevent the function from returning None if user-specific site-packages are disabled, since there are other uses of the function which are relying on this behaviour (e.g. `python -m site`).
Mikhail Golubev [Wed, 7 Oct 2020 21:44:31 +0000 (00:44 +0300)]
bpo-41923: PEP 613: Add TypeAlias to typing module (#22532)
This special marker annotation is intended to help in distinguishing
proper PEP 484-compliant type aliases from regular top-level variable
assignments.
Batuhan Taskaya [Tue, 6 Oct 2020 20:03:02 +0000 (23:03 +0300)]
bpo-38605: Make 'from __future__ import annotations' the default (GH-20434)
The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions.
For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).
Florian Bruhin [Tue, 6 Oct 2020 14:21:56 +0000 (16:21 +0200)]
bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (GH-22575)
Similarly to GH-22566, those tests called eval() on content received via
HTTP in test_named_sequences_full. This likely isn't exploitable because
unicodedata.lookup(seqname) is called before self.checkletter(seqname,
None) - thus any string which isn't a valid unicode character name
wouldn't ever reach the checkletter method.
Still, it's probably better to be safe than sorry.
Enable recursion checks which were disabled when get __bases__ of
non-type objects in issubclass() and isinstance() and when intern
strings. It fixes a stack overflow when getting __bases__ leads
to infinite recursion.
Originally recursion checks was disabled for PyDict_GetItem() which
silences all errors including the one raised in case of detected
recursion and can return incorrect result. But now the code uses
PyDict_GetItemWithError() and PyDict_SetDefault() instead.