bpo-45183: don't raise an exception when calling zipimport.zipimporter.find_spec() when the zip file is missing and the internal cache has been reset (GH-28435)
This can occur when the zip file gets deleted, you call zipimport.zipimporter.invalidate_cache(), and then try to use zipimport.zipimporter.find_spec() (i.e. you left the zip file path on sys.path).
Victor Stinner [Fri, 17 Sep 2021 18:20:43 +0000 (20:20 +0200)]
bpo-45020: Fix build out of source tree (GH-28410)
* Makefile.pre.in: Add $(srcdir) when needed, remove it when it was
used by mistake.
* freeze_modules.py tool uses ./Programs/_freeze_module if the
executable doesn't exist in the source tree.
Victor Stinner [Fri, 17 Sep 2021 18:12:25 +0000 (20:12 +0200)]
bpo-45231: update_file.py preserves end of line (GH-28411)
The update_file.py tool now preserves the end of line of the updated
file. Fix the "make regen-frozen" command: it no longer changes the
end of line of PCbuild/ files on Unix. Git changes the end of line
depending on the platform.
Victor Stinner [Fri, 17 Sep 2021 12:09:14 +0000 (14:09 +0200)]
bpo-45116: Add the Py_ALWAYS_INLINE macro (GH-28390)
Add the Py_ALWAYS_INLINE macro to ask the compiler to always inline a
static inline function. The compiler can ignore it and decides to not
inline the function.
bpo-45162: Remove many old deprecated unittest features (GH-28268)
* "fail*" and "assert*" aliases of TestCase methods.
* Broken from start TestCase method assertDictContainsSubset().
* Ignored TestLoader.loadTestsFromModule() parameter use_load_tests.
* Old alias _TextTestResult of TextTestResult.
bpo-45187: Collect test_socket tests using unittest (GH-28317)
Previously, test classes ISOTPTest, J1939Test, BasicUDPLITETest and
UDPLITETimeoutTest were not included in the list of tests and
were not run by regrtest.
Eric Snow [Thu, 16 Sep 2021 20:20:52 +0000 (14:20 -0600)]
bpo-45020: Drop the frozen .h files from the repo. (gh-28392)
The main advantage is that the files will no longer show up in diffs and PRs. That means, for a PR, the number of files / lines changed will more clearly reflect the actual change. (This is essentially an un-revert of gh-28375.)
bpo-45212: Fix dangling threads in skipped tests in test_socket (GH-28361)
tearDown() is not called if setUp() raises an exception
(including SkipTest). addCleanup() should be used for guaranteed
execution of the cleanup code.
Eric Snow [Thu, 16 Sep 2021 01:15:26 +0000 (19:15 -0600)]
bpo-45020: Drop the frozen .h files from the repo. (gh-28375)
The main advantage is that the files will no longer show up in diffs and PRs. That means, for a PR, the number of files / lines changed will more clearly reflect the actual change.
Eric Snow [Wed, 15 Sep 2021 20:15:32 +0000 (14:15 -0600)]
bpo-45019: Clean up the frozen __hello__ module. (gh-28374)
Here's one more small cleanup that should have been in PR gh-28319. We eliminate stdout side-effects from importing the frozen __hello__ module, and update tests accordingly. We also move the module's source file into Lib/ from Toos/freeze/flag.py.
Victor Stinner [Wed, 15 Sep 2021 18:20:45 +0000 (20:20 +0200)]
bpo-45205: Make test_compileall quiet (GH-28356)
Make test_compileall quiet: test_year_2038_mtime_compilation() and
test_larger_than_32_bit_times() of test_compileall no longer log
"Compiling ..." messages to stdout.
Steve Dower [Wed, 15 Sep 2021 17:11:12 +0000 (18:11 +0100)]
bpo-45188: Windows now regenerates frozen modules at the start of build instead of late (GH-28322)
This will enable us to drop the frozen module header files from the repository.
It does currently cause many source files to be built twice, which just takes more time. For whoever comes to fix this in the future, the files shared between freeze_module and pythoncore should be put into a static library that is consumed by both.
Eric Snow [Wed, 15 Sep 2021 16:19:30 +0000 (10:19 -0600)]
bpo-45020: Freeze some of the modules imported during startup. (gh-28335)
Doing this provides significant performance gains for runtime startup (~15% with all the imported modules frozen). We don't yet freeze all the imported modules because there are a few hiccups in the build systems we need to sort out first. (See bpo-45186 and bpo-45188.)
Note that in PR GH-28320 we added a command-line flag (-X frozen_modules=[on|off]) that allows users to opt out of (or into) using frozen modules. The default is still "off" but we will change it to "on" as soon as we can do it in a way that does not cause contributors pain.
Victor Stinner [Wed, 15 Sep 2021 12:26:43 +0000 (14:26 +0200)]
bpo-21302: Add _PyTime_AsNanoseconds() (GH-28350)
Refactor pytime.c:
* Add pytime_from_nanoseconds() and pytime_as_nanoseconds(),
and use explicitly these functions
* Add two empty lines between functions
* PEP 7: add braces { ... }
* C99: declare variables where they are set
* Rename private functions to lowercase
* Rename error_time_t_overflow() to pytime_time_t_overflow()
* Rename win_perf_counter_frequency() to py_win_perf_counter_frequency()
* py_get_monotonic_clock(): add an assertion to detect overflow when
mach_absolute_time() unsigned uint64_t is casted to _PyTime_t
(signed int64_t).
Terry Jan Reedy [Wed, 15 Sep 2021 07:13:23 +0000 (03:13 -0400)]
bpo-45193: Restore IDLE completion boxes on Ubuntu (GH-28343)
The line that should not have been needed on macOS tk 8.6.8 but was,
should not be a problem on Ubuntu, but is. It is not needed on macOS
tk 8.6.11, installed with 3.10. Disable it but leave it for
now in case some system needs it.
Eric Snow [Tue, 14 Sep 2021 23:31:45 +0000 (17:31 -0600)]
bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of frozen modules. (gh-28320)
Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file. This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now.
bpo-42135 Correct version slated for importlib.find_loader removal (GH-28312)
importlib.find_loader should also be slated for 3.12 like the others in GH-25169 and as documented in https://docs.python.org/3.11/whatsnew/3.10.html#deprecated.
Eric Snow [Mon, 13 Sep 2021 22:18:37 +0000 (16:18 -0600)]
bpo-45019: Do some cleanup related to frozen modules. (gh-28319)
There are a few things I missed in gh-27980. This is a follow-up that will make subsequent PRs cleaner. It includes fixes to tests and tools that reference the frozen modules.
bpo-21302: Add clock_nanosleep() implementation for time.sleep() (GH-28111)
In Unix operating systems, time.sleep() now uses the clock_nanosleep() function,
if available, which allows to sleep for an interval specified with nanosecond precision.
Co-authored-by: Victor Stinner <vstinner@python.org>
bpo-43413: Fix handling keyword arguments in subclasses of some buitin classes (GH-26456)
* Constructors of subclasses of some buitin classes (e.g. tuple, list,
frozenset) no longer accept arbitrary keyword arguments.
* Subclass of set can now define a __new__() method with additional
keyword parameters without overriding also __init__().
bpo-35474: Fix mimetypes.guess_all_extensions() potentially mutating list (GH-28286)
* Calling guess_all_extensions() with strict=False potentially
mutated types_map_inv.
* Mutating the result of guess_all_extensions() mutated types_map_inv.
David CARLIER [Sat, 11 Sep 2021 08:46:22 +0000 (09:46 +0100)]
bpo-45163: Haiku build fix. (GH-28269)
linkage issues mainly for shared libs and missing system library,
also little nit into the signal extension as strsignal returns
a constant in this platform.
bpo-25894: Always report skipped and failed subtests separately (GH-28082)
* In default mode output separate characters for skipped and failed subtests.
* In verbose mode output separate lines (including description) for skipped
and failed subtests.
* In verbose mode output test description for errors in test cleanup.
bpo-45067 - Verify the version of ncurses for extended color support feature usage. (GH-28260)
* issue45067 - Fix _curses compilation in CentOS 7. Verify the version of ncurses
for extended color support feature usage.
The function extended_color_content was introduced in 2017. The ncurses-devel
package in CentOS 7 had a older version ncurses resulted in compilation error.
For compiling ncurses with extended color support, we verify the version of the
ncurses library.
Vincent Michel [Thu, 9 Sep 2021 13:12:03 +0000 (15:12 +0200)]
bpo-44219: Release the GIL during isatty syscalls (GH-28250)
Release the GIL while performing isatty() system calls on arbitrary
file descriptors. In particular, this affects os.isatty(),
os.device_encoding() and io.TextIOWrapper. By extension,
io.open() in text mode is also affected.
Due to significant security concerns, the reuse_address parameter of
asyncio.loop.create_datagram_endpoint, deprecated in Python 3.9, is
now removed. This is because of the behavior of the socket option
SO_REUSEADDR in UDP.