bpo-41282: (PEP 632) Load install schemes from sysconfig (GH-24549)
With this patch, `distutils.command.install.INSTALL_SCHEMES` are loaded from
`sysconfig._INSTALL_SCHEMES`.
The distutils module is deprecated and will be removed in 3.12 (PEP 632).
This change makes the `sysconfig._INSTALL_SCHEMES` the single point of truth
for install schemes while keeping `distutils.command.install.INSTALL_SCHEMES`
exactly the same. If we, during the transition to the sysconfig, change
something, this makes sure that it also propagates to distutils until the
module gets removed.
Moreover, as discussed [on Discourse], Linux distros need to patch
distutils/sysconfig to make sure the packages will land in proper locations.
This patch makes it easier because it leaves only one location where install
schemes are defined which is much easier to patch/adjust.
The implementation is slightly different than the plan but I think it's the
easiest way how to do it and it also makes the downstream patch simple,
flexible and easy to maintain.
It's also necessary to implement this before setuptools starts bundling
the distutils module so the default install schemes stay in the standard library.
The removed code from sysconfig does not seem to have any negative effect
because, honestly, it seems that nothing actually uses the install schemes
from sysconfig at all. There were many big changes in these modules where
they were trying to include packaging in stdlib and then reverted that.
Also, the test of distutils install command does not count with the different
locations which is good evidence that the reason to have this piece of code
is no longer valid.
Petr Viktorin [Wed, 14 Apr 2021 14:38:30 +0000 (16:38 +0200)]
bpo-43795: Sort PC/python3dll.c (GH-25312)
Each section is sorted to reduce diffs (review effort) when the file becomes generated.
Sort is done with key=str.lower to preserve most of the original order (underscored items first).
Pablo Galindo [Wed, 14 Apr 2021 14:10:33 +0000 (15:10 +0100)]
bpo-38530: Offer suggestions on NameError (GH-25397)
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:
>>> schwarzschild_black_hole = None
>>> schwarschild_black_hole
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
Pablo Galindo [Wed, 14 Apr 2021 01:36:07 +0000 (02:36 +0100)]
bpo-38530: Offer suggestions on AttributeError (#16856)
When printing AttributeError, PyErr_Display will offer suggestions of similar
attribute names in the object that the exception was raised from:
>>> collections.namedtoplo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
Victor Stinner [Wed, 14 Apr 2021 01:24:33 +0000 (03:24 +0200)]
bpo-43680: Deprecate io.OpenWrapper (GH-25357)
Deprecate io.OpenWrapper and _pyio.OpenWrapper: use io.open and
_pyio.open instead. Until Python 3.9, _pyio.open was not a static
method and builtins.open was set to OpenWrapper to not become a bound
method when set to a class variable. _io.open is a built-in function
whereas _pyio.open is a Python function. In Python 3.10, _pyio.open()
is now a static method, and builtins.open() is now io.open().
Christian Heimes [Tue, 13 Apr 2021 17:23:45 +0000 (19:23 +0200)]
bpo-43811: Test multiple OpenSSL versions on GHA (GH-25360)
The new checks are only executed when one or more OpenSSL-related files are modified. The checks run a handful of networking and hashing test suites. All SSL checks are optional. This PR also introduces ccache to speed up compilation. In common cases it speeds up configure and compile time from about 90 seconds to less than 30 seconds.
Signed-off-by: Christian Heimes <christian@python.org>
Victor Stinner [Tue, 13 Apr 2021 13:25:16 +0000 (15:25 +0200)]
bpo-43770: Cleanup type_ready() (GH-25388)
* Rename functions
* Only pass type parameter to "add_xxx" functions.
* Clarify the role of the type_ready_inherit_as_structs() function.
* Move type_dict_set_doc() code to call it in type_ready_fill_dict().
Pablo Galindo [Mon, 12 Apr 2021 19:53:15 +0000 (20:53 +0100)]
Fix Sphinx errors in the documentation and re-activate the suspicious check (GH-25368)
The suspicious check is still executed as part of the release process and release managers have been
lately fixing some actual errors that the suspicious target can find. For this reason, reactivate the suspicious
until we decide what to do in a coordinated fashion.
The configure --without-cycle-gc option has been removed in Python
2.3 by the commit cccd1e72481106b0a373117f32fda6261f3141a7. It's now
time to remove the last reference to it in the documentation.
Victor Stinner [Mon, 12 Apr 2021 08:44:53 +0000 (10:44 +0200)]
bpo-43680: _pyio.open() becomes a static method (GH-25354)
The Python _pyio.open() function becomes a static method to behave as
io.open() built-in function: don't become a bound method when stored
as a class variable. It becomes possible since static methods are now
callable in Python 3.10. Moreover, _pyio.OpenWrapper becomes a simple
alias to _pyio.open.
init_set_builtins_open() now sets builtins.open to io.open, rather
than setting it to io.OpenWrapper, since OpenWrapper is now an alias
to open in the io and _pyio modules.
bpo-43723: Deprecate camelCase aliases from threading (GH-25174)
The snake_case names have existed since Python 2.6, so there is
no reason to keep the old camelCase names around. One similar
method, threading.Thread.isAlive, was already removed in
Python 3.9 (bpo-37804).
Victor Stinner [Sun, 11 Apr 2021 21:57:09 +0000 (23:57 +0200)]
bpo-43770: Refactor PyType_Ready() function (GH-25336)
* Split PyType_Ready() into sub-functions.
* type_ready_mro() now checks if bases are static types earlier.
* Check tp_name earlier, in type_ready_checks().
* Add _PyType_IsReady() macro to check if a type is ready.
Victor Stinner [Sat, 10 Apr 2021 22:17:39 +0000 (00:17 +0200)]
bpo-43753: Add Py_Is() and Py_IsNone() functions (GH-25227)
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
Victor Stinner [Sat, 10 Apr 2021 21:15:32 +0000 (23:15 +0200)]
bpo-43770: Refactor type_new() function (GH-25325)
* Split type_new() into into many small functions.
* Add type_new_ctx structure to pass variables between subfunctions.
* Initialize some PyTypeObject and PyHeapTypeObject members earlier
in type_new_alloc().
* Rename variables to more specific names.
* Add "__weakref__" identifier for type_new_visit_slots().
* Factorize code to convert a method to a classmethod
(__init_subclass__ and __class_getitem__).
* Add braces to respect PEP 7.
* Move variable declarations where the variables are initialized.
Matthew Suozzo [Sat, 10 Apr 2021 03:45:50 +0000 (23:45 -0400)]
bpo-43478: Restrict use of Mock objects as specs (GH-25326)
* Restrict using Mock objects as specs as this is always a test bug where the resulting mock is misleadingly useless.
* Skip a broken test that exposes a bug elsewhere in mock (noted in the original issue).
Barney Gale [Fri, 9 Apr 2021 20:52:49 +0000 (21:52 +0100)]
bpo-40107: Switch to using io.open() for pathlib.Path.open() (GH-25240)
Previously we had identical behaviour but only allowed accessors to override os.open(). This change allows the override to also construct the IO wrapper as well.
Static methods (@staticmethod) and class methods (@classmethod) now
inherit the method attributes (__module__, __name__, __qualname__,
__doc__, __annotations__) and have a new __wrapped__ attribute.
Changes:
* Add a repr() method to staticmethod and classmethod types.
* Add tests on the @classmethod decorator.
Mark Shannon [Thu, 8 Apr 2021 10:22:55 +0000 (11:22 +0100)]
bpo-43760: Streamline dispatch sequence for machines without computed gotos. (GH-25244)
* Do fetch and decode at end of opcode then jump directly to switch.
Should allow compilers that don't support computed-gotos, specifically MSVC,
to generate better code.
Barney Gale [Wed, 7 Apr 2021 22:50:13 +0000 (23:50 +0100)]
bpo-39899: os.path.expanduser(): don't guess other Windows users' home directories if the basename of the current user's home directory doesn't match their username. (GH-18841)
This makes `ntpath.expanduser()` match `pathlib.Path.expanduser()` in this regard, and is more in line with `posixpath.expanduser()`'s cautious approach.
Also remove the near-duplicate implementation of `expanduser()` in pathlib, and by doing so fix a bug where KeyError could be raised when expanding another user's home directory.
Victor Stinner [Wed, 7 Apr 2021 19:34:22 +0000 (21:34 +0200)]
bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix.
Remove macros creating aliases without prefix. For example, Module()
becomes _PyAST_Module(). Update Grammar/python.gram to use
_PyAST_xxx() functions.
Yeting Li [Wed, 7 Apr 2021 11:27:41 +0000 (19:27 +0800)]
bpo-43075: Fix ReDoS in urllib AbstractBasicAuthHandler (GH-24391)
Fix Regular Expression Denial of Service (ReDoS) vulnerability in
urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex
has quadratic worst-case complexity and it allows cause a denial of
service when identifying crafted invalid RFCs. This ReDoS issue is on
the client side and needs remote attackers to control the HTTP server.