]> git.ipfire.org Git - thirdparty/Python/cpython.git/log
thirdparty/Python/cpython.git
5 years agobpo-35569: Expose RFC 3542 IPv6 socket options on macOS (GH-19526)
Erlend Egeberg Aasland [Sun, 17 May 2020 06:32:46 +0000 (08:32 +0200)] 
bpo-35569: Expose RFC 3542 IPv6 socket options on macOS (GH-19526)

5 years agobpo-29587: Add another test for the gen.throw() fix. (GH-19859)
Chris Jerdonek [Sun, 17 May 2020 04:14:48 +0000 (21:14 -0700)] 
bpo-29587: Add another test for the gen.throw() fix. (GH-19859)

5 years agobpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)
Lysandros Nikolaou [Sun, 17 May 2020 03:19:23 +0000 (06:19 +0300)] 
bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)

The following improvements are implemented in this commit:
- `p->error_indicator` is set, in case malloc or realloc fail.
- Avoid memory leaks in the case that realloc fails.
- Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
5 years agobpo-38870: Use subTest in test_unparse for better error reporting (GH-20141)
Pablo Galindo [Sun, 17 May 2020 02:53:57 +0000 (03:53 +0100)] 
bpo-38870: Use subTest in test_unparse for better error reporting (GH-20141)

5 years agoUse subTest in test_exceptions for better error reporting (GH-20140)
Pablo Galindo [Sun, 17 May 2020 00:22:00 +0000 (01:22 +0100)] 
Use subTest in test_exceptions for better error reporting (GH-20140)

5 years agobpo-40645: Deprecated internal details of hmac.HMAC (GH-20132)
Christian Heimes [Sat, 16 May 2020 23:05:40 +0000 (01:05 +0200)] 
bpo-40645: Deprecated internal details of hmac.HMAC (GH-20132)

5 years agobpo-38870: Implement round tripping support for typed AST in ast.unparse (GH-17797)
Batuhan Taskaya [Sat, 16 May 2020 23:04:12 +0000 (02:04 +0300)] 
bpo-38870: Implement round tripping support for typed AST in ast.unparse (GH-17797)

5 years agobpo-38870: Correctly handle empty docstrings in ast.unparse (GH-18768)
Batuhan Taskaya [Sat, 16 May 2020 22:49:07 +0000 (01:49 +0300)] 
bpo-38870: Correctly handle empty docstrings in ast.unparse (GH-18768)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
5 years agobpo-40165: Suppress stderr when checking if test_stty_match should be skipped (GH...
Batuhan Taskaya [Sat, 16 May 2020 22:38:02 +0000 (01:38 +0300)] 
bpo-40165: Suppress stderr when checking if test_stty_match should be skipped (GH-19325)

5 years agobpo-40211: Clarify os.preadv and os.pwritev are supported on AIX 7.1+ (GH-19401)
Batuhan Taskaya [Sat, 16 May 2020 22:36:40 +0000 (01:36 +0300)] 
bpo-40211: Clarify os.preadv and os.pwritev are supported on AIX 7.1+ (GH-19401)

5 years agobpo-39740: Early declare devpoll_methods to support old compilers (GH-19281)
Batuhan Taskaya [Sat, 16 May 2020 22:36:14 +0000 (01:36 +0300)] 
bpo-39740: Early declare devpoll_methods to support old compilers (GH-19281)

5 years agobpo-38870: Don't put unnecessary parentheses on class declarations in ast.parse ...
Batuhan Taskaya [Sat, 16 May 2020 21:53:25 +0000 (00:53 +0300)] 
bpo-38870: Don't put unnecessary parentheses on class declarations in ast.parse (GH-20134)

5 years agobpo-38870: Do not separate factor prefixes in ast.unparse (GH-20133)
Batuhan Taskaya [Sat, 16 May 2020 21:46:11 +0000 (00:46 +0300)] 
bpo-38870: Do not separate factor prefixes in ast.unparse (GH-20133)

5 years agobpo-37630: Use SHA3 and SHAKE XOF from OpenSSL (GH-16049)
Christian Heimes [Sat, 16 May 2020 20:27:06 +0000 (22:27 +0200)] 
bpo-37630: Use SHA3 and SHAKE XOF from OpenSSL (GH-16049)

OpenSSL 1.1.1 comes with SHA3 and SHAKE builtin.

Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
5 years agobpo-40503: Add documentation and what's new entry for zoneinfo (GH-20006)
Paul Ganssle [Sat, 16 May 2020 16:14:58 +0000 (12:14 -0400)] 
bpo-40503: Add documentation and what's new entry for zoneinfo (GH-20006)

This adds the documentation for the `zoneinfo` module added in PEP 615: https://www.python.org/dev/peps/pep-0615/

The implementation itself was GH-19909: https://github.com/python/cpython/pull/19909

bpo-40503: https://bugs.python.org/issue40503

Co-authored-by: Victor Stinner <vstinner@python.org>
5 years agobpo-24416: Return named tuple from date.isocalendar() (GH-20113)
Paul Ganssle [Sat, 16 May 2020 14:02:59 +0000 (10:02 -0400)] 
bpo-24416: Return named tuple from date.isocalendar() (GH-20113)

{date, datetime}.isocalendar() now return a private custom named tuple object
IsoCalendarDate rather than a simple tuple.

In order to leave IsocalendarDate as a private class and to improve what
backwards compatibility is offered for pickling the result of a
datetime.isocalendar() call, add a __reduce__ method to the named tuples that
reduces them to plain tuples. (This is the part of this PR most likely to cause
problems — if it causes major issues, switching to a strucseq or equivalent
would be prudent).

The pure python implementation of IsoCalendarDate uses positional-only
arguments, since it is private and only constructed by position anyway; the
equivalent change in the argument clinic on the C side would require us to move
the forward declaration of the type above the clinic import for whatever
reason, so it seems preferable to hold off on that for now.

bpo-24416: https://bugs.python.org/issue24416

Original PR by Dong-hee Na with only minor alterations by Paul Ganssle.

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
5 years agobpo-39305: Update nntplib to merge nntplib.NNTP and nntplib._NNTPBase (GH-19817)
Dong-hee Na [Sat, 16 May 2020 10:31:54 +0000 (19:31 +0900)] 
bpo-39305: Update nntplib to merge nntplib.NNTP and nntplib._NNTPBase (GH-19817)

5 years agobpo-40275: lazy import modules in test.support (GH-20128)
Hai Shi [Sat, 16 May 2020 10:01:39 +0000 (18:01 +0800)] 
bpo-40275: lazy import modules in test.support (GH-20128)

Automerge-Triggered-By: @vstinner
5 years agobpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)
Batuhan Taskaya [Sat, 16 May 2020 09:39:09 +0000 (12:39 +0300)] 
bpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)

On AIX, time.thread_time() is now implemented with thread_cputime()
which has nanosecond resolution, rather than
clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 ms.

5 years agobpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
Paul Ganssle [Sat, 16 May 2020 08:20:06 +0000 (04:20 -0400)] 
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)

This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).

This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:

1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)

Differences from the reference implementation:

- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.

Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:

- Fixed reference and memory leaks

    With much debugging help from Pablo Galindo

- Added smoke tests ensuring that the C and Python modules are built

    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.

- Adjustments to zoneinfo.__dir__

    Suggested by Petr Viktorin.

- Slight refactorings as suggested by Steve Dower.

- Removed unnecessary if check on std_abbr

    Discovered this because of a missing line in branch coverage.

5 years agobpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)
Christian Heimes [Sat, 16 May 2020 01:33:05 +0000 (03:33 +0200)] 
bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)

OpenSSL can be build without support for TLS 1.0 and 1.1. The ssl module
now correctly adheres to OPENSSL_NO_TLS1 and OPENSSL_NO_TLS1_1 flags.

Also update multissltest to test with latest OpenSSL and LibreSSL
releases.

Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
5 years agobpo-39075: types.SimpleNamespace no longer sorts attributes in its repr (GH-19430)
Zackery Spytz [Sat, 16 May 2020 01:27:54 +0000 (19:27 -0600)] 
bpo-39075: types.SimpleNamespace no longer sorts attributes in its repr (GH-19430)

5 years agobpo-31033: Add a msg argument to Future.cancel() and Task.cancel() (GH-19979)
Chris Jerdonek [Fri, 15 May 2020 23:55:50 +0000 (16:55 -0700)] 
bpo-31033: Add a msg argument to Future.cancel() and Task.cancel() (GH-19979)

5 years agoAdd Andrew York to ACKS for GH-19622. (GH-20105)
Chris Jerdonek [Fri, 15 May 2020 22:19:39 +0000 (15:19 -0700)] 
Add Andrew York to ACKS for GH-19622. (GH-20105)

This updates ACKS for commit 003708bcf8f2c58d4b65f68318acf164d713e008
contributed by Andrew York.

5 years agobpo-40637: Add option to disable builtin hashes (GH-20121)
Christian Heimes [Fri, 15 May 2020 21:54:53 +0000 (23:54 +0200)] 
bpo-40637: Add option to disable builtin hashes (GH-20121)

Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
5 years agobpo-40445: Update compileall.compile_dir docs (GH-19806)
Shantanu [Fri, 15 May 2020 21:28:23 +0000 (14:28 -0700)] 
bpo-40445: Update compileall.compile_dir docs (GH-19806)

5 years agobpo-40636: Clarify the zip built-in docstring. (GH-20118)
Gregory P. Smith [Fri, 15 May 2020 21:26:00 +0000 (14:26 -0700)] 
bpo-40636: Clarify the zip built-in docstring. (GH-20118)

Clarify the zip built-in docstring.

This puts much simpler text up front along with an example.

As it was, the zip built-in docstring was technically correct.  But too
technical for the reader who shouldn't _need_ to know about `__next__` and
`StopIteration` as most people do not need to understand the internal
implementation details of the iterator protocol in their daily life.

This is a documentation only change, intended to be backported to 3.8; it is
only tangentially related to PEP-618 which might offer new behavior options
in the future.

Wording based a bit more on enumerate per Brandt's suggestion.

This gets rid of the legacy wording paragraph which seems too tied to
implementation details of the iterator protocol which isn't relevant here.

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
5 years agobpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
Christian Heimes [Fri, 15 May 2020 20:32:25 +0000 (22:32 +0200)] 
bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)

5 years agobpo-40607: Reraise exception during task cancelation in asyncio.wait_for() (GH-20054)
romasku [Fri, 15 May 2020 20:12:05 +0000 (23:12 +0300)] 
bpo-40607: Reraise exception during task cancelation in asyncio.wait_for() (GH-20054)

Currently, if asyncio.wait_for() timeout expires, it cancels
inner future and then always raises TimeoutError. In case
those future is task, it can handle cancelation mannually,
and those process can lead to some other exception. Current
implementation silently loses thoses exception.

To resolve this, wait_for will check was the cancelation
successfull or not. In case there was exception, wait_for
will reraise it.

Co-authored-by: Roman Skurikhin <roman.skurikhin@cruxlab.com>
5 years agobpo-40515: Require OPENSSL_THREADS (GH-19953)
Christian Heimes [Fri, 15 May 2020 18:55:25 +0000 (20:55 +0200)] 
bpo-40515: Require OPENSSL_THREADS (GH-19953)

The ``ssl`` and ``hashlib`` modules now actively check that OpenSSL is
build with thread support. Python 3.7.0 made thread support mandatory and no
longer works safely with a no-thread builds.

Signed-off-by: Christian Heimes <christian@python.org>
5 years agobpo-40479: Test with latest OpenSSL versions (GH-20108)
Christian Heimes [Fri, 15 May 2020 16:48:25 +0000 (18:48 +0200)] 
bpo-40479: Test with latest OpenSSL versions (GH-20108)

* 1.0.2u (EOL)
* 1.1.0l (EOL)
* 1.1.1g
* 3.0.0-alpha2 (disabled for now)

Build the FIPS provider and create a FIPS configuration file for OpenSSL
3.0.0.

Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
5 years agobpo-40479: Fix hashlib issue with OpenSSL 3.0.0 (GH-20107)
Christian Heimes [Fri, 15 May 2020 16:28:05 +0000 (18:28 +0200)] 
bpo-40479: Fix hashlib issue with OpenSSL 3.0.0 (GH-20107)

OpenSSL 3.0.0-alpha2 was released today. The FIPS_mode() function has
been deprecated and removed. It no longer makes sense with the new
provider and context system in OpenSSL 3.0.0.

EVP_default_properties_is_fips_enabled() is good enough for our needs in
unit tests. It's an internal API, too.

Signed-off-by: Christian Heimes <christian@python.org>
5 years agobpo-40055: test_distutils leaves warnings filters unchanged (GH-20095)
Victor Stinner [Fri, 15 May 2020 16:06:23 +0000 (18:06 +0200)] 
bpo-40055: test_distutils leaves warnings filters unchanged (GH-20095)

distutils.tests now saves/restores warnings filters to leave them
unchanged. Importing tests imports docutils which imports
pkg_resources which adds a warnings filter.

5 years agoTrivial typo fix in _tkinter.c (GH-19622)
Andrew York [Fri, 15 May 2020 10:43:58 +0000 (03:43 -0700)] 
Trivial typo fix in _tkinter.c (GH-19622)

Change spelling of a #define in _tkinter.c from HAVE_LIBTOMMAMTH to HAVE_LIBTOMMATH, since this is used to keep track of tclTomMath.h, not tclTomMamth.h. No other file seems to refer to this variable.

5 years agoFix typo in code comment in main_loop label. (GH-20068)
Chris Jerdonek [Fri, 15 May 2020 02:25:45 +0000 (19:25 -0700)] 
Fix typo in code comment in main_loop label. (GH-20068)

5 years agobpo-40612: Fix SyntaxError edge cases in traceback formatting (GH-20072)
Guido van Rossum [Fri, 15 May 2020 02:22:48 +0000 (19:22 -0700)] 
bpo-40612: Fix SyntaxError edge cases in traceback formatting (GH-20072)

This fixes both the traceback.py module and the C code for formatting syntax errors (in Python/pythonrun.c). They now both consistently do the following:

- Suppress caret if it points left of text
- Allow caret pointing just past end of line
- If caret points past end of line, clip to *just* past end of line

The syntax error formatting code in traceback.py was mostly rewritten; small, subtle changes were applied to the C code in pythonrun.c.

There's still a difference when the text contains embedded newlines. Neither handles these very well, and I don't think the case occurs in practice.

Automerge-Triggered-By: @gvanrossum
5 years agoUpdate code comment re: location of struct _is. (GH-20067)
Chris Jerdonek [Fri, 15 May 2020 02:11:00 +0000 (19:11 -0700)] 
Update code comment re: location of struct _is. (GH-20067)

5 years agobpo-40548: github actions: pass the changes check on no source changes (GH-20097)
Filipe Laíns [Fri, 15 May 2020 02:08:21 +0000 (03:08 +0100)] 
bpo-40548: github actions: pass the changes check on no source changes (GH-20097)

Signed-off-by: Filipe Laíns <lains@archlinux.org>
5 years agobpo-40334: Correctly identify invalid target in assignment errors (GH-20076)
Pablo Galindo [Fri, 15 May 2020 01:04:52 +0000 (02:04 +0100)] 
bpo-40334: Correctly identify invalid target in assignment errors (GH-20076)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
5 years agobpo-38872: Document exec symbol for codeop.compile_command (GH-20047)
Joannah Nanjekye [Fri, 15 May 2020 00:59:46 +0000 (21:59 -0300)] 
bpo-38872: Document exec symbol for codeop.compile_command (GH-20047)

* Document exec symbol for codeop.compile_command

* Remove extra statements

Co-authored-by: nanjekyejoannah <joannah.nanjekye@ibm.com>
5 years agobpo-40462: Fix typo in test_json (GH-20094)
Victor Stinner [Thu, 14 May 2020 23:02:10 +0000 (01:02 +0200)] 
bpo-40462: Fix typo in test_json (GH-20094)

5 years agobpo-40460: Fix typo in idlelib/zzdummy.py (GH-20093)
Victor Stinner [Thu, 14 May 2020 22:51:51 +0000 (00:51 +0200)] 
bpo-40460: Fix typo in idlelib/zzdummy.py (GH-20093)

Replace ztest with ztext.

5 years agobpo-40548: GitHub Action workflow: skip jobs on doc only PRs (GH-19983)
Filipe Laíns [Thu, 14 May 2020 22:11:40 +0000 (23:11 +0100)] 
bpo-40548: GitHub Action workflow: skip jobs on doc only PRs (GH-19983)

Signed-off-by: Filipe Laíns <lains@archlinux.org>
5 years agobpo-40602: _Py_hashtable_set() reports rehash failure (GH-20077)
Victor Stinner [Thu, 14 May 2020 20:44:32 +0000 (22:44 +0200)] 
bpo-40602: _Py_hashtable_set() reports rehash failure (GH-20077)

If _Py_hashtable_set() fails to grow the hash table (rehash), it now
fails rather than ignoring the error.

5 years agobpo-40618: Disallow invalid targets in augassign and except clauses (GH-20083)
Lysandros Nikolaou [Thu, 14 May 2020 20:13:50 +0000 (23:13 +0300)] 
bpo-40618: Disallow invalid targets in augassign and except clauses (GH-20083)

This commit fixes the new parser to disallow invalid targets in the
following scenarios:
- Augmented assignments must only accept a single target (Name,
  Attribute or Subscript), but no tuples or lists.
- `except` clauses should only accept a single `Name` as a target.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
5 years agobpo-40619: Correctly handle error lines in programs without file mode (GH-20090)
Pablo Galindo [Thu, 14 May 2020 20:11:48 +0000 (21:11 +0100)] 
bpo-40619: Correctly handle error lines in programs without file mode (GH-20090)

5 years agobpo-40602: Write unit tests for _Py_hashtable_t (GH-20091)
Victor Stinner [Thu, 14 May 2020 19:55:47 +0000 (21:55 +0200)] 
bpo-40602: Write unit tests for _Py_hashtable_t (GH-20091)

Cleanup also hashtable.c.
Rename _Py_hashtable_t members:

* Rename entries to nentries
* Rename num_buckets to nbuckets

5 years agoRevert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" (GH-20089)
Victor Stinner [Thu, 14 May 2020 16:46:24 +0000 (18:46 +0200)] 
Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" (GH-20089)

* Revert "bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)"

This reverts commit fa0a66e62d087765dbc5c1b89d6149a23ecfb0a6.

* Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)"

This reverts commit a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57.

5 years agobpo-40549: posixmodule.c uses defining_class (GH-20075)
Victor Stinner [Thu, 14 May 2020 16:05:58 +0000 (18:05 +0200)] 
bpo-40549: posixmodule.c uses defining_class (GH-20075)

Pass PEP 573 defining_class to os.DirEntry methods. The module state
is now retrieve from defining_class rather than Py_TYPE(self), to
support subclasses (even if DirEntry doesn't support subclasses yet).

* Pass the module rather than defining_class to DirEntry_fetch_stat().
* Only get the module state once in _posix_clear(),
  _posix_traverse() and _posixmodule_exec().

5 years agobpo-40495: compileall option to hardlink duplicate pyc files (GH-19901)
Lumír 'Frenzy' Balhar [Thu, 14 May 2020 14:17:22 +0000 (16:17 +0200)] 
bpo-40495: compileall option to hardlink duplicate pyc files (GH-19901)

compileall is now able to use hardlinks to prevent duplicates in a
case when .pyc files for different optimization levels have the same content.

Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Victor Stinner <vstinner@python.org>
5 years agobpo-40275: Import locale module lazily in gettext (GH-19905)
Hai Shi [Thu, 14 May 2020 01:22:30 +0000 (09:22 +0800)] 
bpo-40275: Import locale module lazily in gettext (GH-19905)

5 years agobpo-40597: email: Use CTE if lines are longer than max_line_length consistently ...
Arkadiusz Hiler [Thu, 14 May 2020 00:53:26 +0000 (03:53 +0300)] 
bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038)

raw_data_manager (default for EmailPolicy, EmailMessage)
does correct wrapping of 'text' parts as long as the message contains
characters outside of 7bit US-ASCII set: base64 or qp
Content-Transfer-Encoding is applied if the lines would be too long
without it.  It did not, however, do this for ascii-only text,
which could result in lines that were longer than
policy.max_line_length or even the rfc 998  maximum.

This changeset fixes the heuristic so that if lines are longer than
policy.max_line_length, it will always apply a
content-transfer-encoding so that the lines are wrapped correctly.

5 years agobpo-40521: Add PyInterpreterState.unicode (GH-20081)
Victor Stinner [Wed, 13 May 2020 23:48:38 +0000 (01:48 +0200)] 
bpo-40521: Add PyInterpreterState.unicode (GH-20081)

Move PyInterpreterState.fs_codec into a new
PyInterpreterState.unicode structure.

Give a name to the fs_codec structure and use this structure in
unicodeobject.c.

5 years agobpo-29587: Make gen.throw() chain exceptions with yield from (GH-19858)
Chris Jerdonek [Wed, 13 May 2020 23:18:27 +0000 (16:18 -0700)] 
bpo-29587: Make gen.throw() chain exceptions with yield from (GH-19858)

The previous commits on bpo-29587 got exception chaining working
with gen.throw() in the `yield` case. This patch also gets the
`yield from` case working.

As a consequence, implicit exception chaining now also works in
the asyncio scenario of awaiting on a task when an exception is
already active.

Tests are included for both the asyncio case and the pure
generator-only case.

5 years agobpo-39465: Remove _PyUnicode_ClearStaticStrings() from C API (GH-20078)
Victor Stinner [Wed, 13 May 2020 23:11:54 +0000 (01:11 +0200)] 
bpo-39465: Remove _PyUnicode_ClearStaticStrings() from C API (GH-20078)

Remove the _PyUnicode_ClearStaticStrings() function from the C API.
Make the function fully private (declare it with "static").

5 years agoissue-25872: Fix KeyError using linecache from multiple threads (GH-18007)
Michael Graczyk [Wed, 13 May 2020 22:41:57 +0000 (17:41 -0500)] 
issue-25872: Fix KeyError using linecache from multiple threads (GH-18007)

The crash that this fixes occurs when using traceback and other modules from multiple threads;
del cache[filename] can raise a KeyError.

5 years agobpo-38787: Fix Argument Clinic defining_class_converter (GH-20074)
Victor Stinner [Wed, 13 May 2020 22:31:31 +0000 (00:31 +0200)] 
bpo-38787: Fix Argument Clinic defining_class_converter (GH-20074)

Don't hardcode defining_class parameter name to "cls":

* Define CConverter.set_template_dict(): do nothing by default
* CLanguage.render_function() now calls set_template_dict() on all
  converters.

5 years agobpo-40334: Always show the caret on SyntaxErrors (GH-20050)
Lysandros Nikolaou [Wed, 13 May 2020 19:36:27 +0000 (22:36 +0300)] 
bpo-40334: Always show the caret on SyntaxErrors (GH-20050)

This commit fixes SyntaxError locations when the caret is not displayed,
by doing the following:

- `col_number` always gets set to the location of the offending
  node/expr. When no caret is to be displayed, this gets achieved
  by setting the object holding the error line to None.

- Introduce a new function `_PyPegen_raise_error_known_location`,
  which can be called, when an arbitrary `lineno`/`col_offset`
  needs to be passed. This function then gets used in the grammar
  (through some new macros and inline functions) so that SyntaxError
  locations of the new parser match that of the old.

5 years agobpo-34790: add version of removal of explicit passing of coros to `asyncio.wait`...
jack1142 [Wed, 13 May 2020 18:55:12 +0000 (20:55 +0200)] 
bpo-34790: add version of removal of explicit passing of coros to `asyncio.wait`'s documentation (#20008)

5 years agobpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)
Dong-hee Na [Wed, 13 May 2020 13:38:27 +0000 (22:38 +0900)] 
bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)

5 years agobpo-40331: Increase test coverage for the statistics module (GH-19608)
Tzanetos Balitsaris [Wed, 13 May 2020 10:29:31 +0000 (13:29 +0300)] 
bpo-40331: Increase test coverage for the statistics module (GH-19608)

5 years agobpo-40602: Optimize _Py_hashtable_get_ptr() (GH-20066)
Victor Stinner [Wed, 13 May 2020 03:36:23 +0000 (05:36 +0200)] 
bpo-40602: Optimize _Py_hashtable_get_ptr() (GH-20066)

_Py_hashtable_get_entry_ptr() avoids comparing the entry hash:
compare directly keys.

Move _Py_hashtable_get_entry_ptr() just after
_Py_hashtable_get_entry_generic().

5 years agobpo-40609: _Py_hashtable_t values become void* (GH-20065)
Victor Stinner [Wed, 13 May 2020 02:40:30 +0000 (04:40 +0200)] 
bpo-40609: _Py_hashtable_t values become void* (GH-20065)

_Py_hashtable_t values become regular "void *" pointers.

* Add _Py_hashtable_entry_t.data member
* Remove _Py_hashtable_t.data_size member
* Remove _Py_hashtable_t.get_func member. It is no longer needed
  to specialize _Py_hashtable_get() for a specific value size, since
  all entries now have the same size (void*).
* Remove the following macros:

  * _Py_HASHTABLE_GET()
  * _Py_HASHTABLE_SET()
  * _Py_HASHTABLE_SET_NODATA()
  * _Py_HASHTABLE_POP()

* Rename _Py_hashtable_pop() to _Py_hashtable_steal()
* _Py_hashtable_foreach() callback now gets key and value rather than
  entry.
* Remove _Py_hashtable_value_destroy_func type. value_destroy_func
  callback now only has a single parameter: data (void*).

5 years agobpo-40609: _tracemalloc allocates traces (GH-20064)
Victor Stinner [Wed, 13 May 2020 01:52:11 +0000 (03:52 +0200)] 
bpo-40609: _tracemalloc allocates traces (GH-20064)

Rewrite _tracemalloc to store "trace_t*" rather than directly
"trace_t" in traces hash tables. Traces are now allocated on the heap
memory, outside the hash table.

Add tracemalloc_copy_traces() and tracemalloc_copy_domains() helper
functions.

Remove _Py_hashtable_copy() function since there is no API to copy a
key or a value.

Remove also _Py_hashtable_delete() function which was commented.

5 years agobpo-40609: Add destroy functions to _Py_hashtable (GH-20062)
Victor Stinner [Wed, 13 May 2020 00:50:18 +0000 (02:50 +0200)] 
bpo-40609: Add destroy functions to _Py_hashtable (GH-20062)

Add key_destroy_func and value_destroy_func parameters to
_Py_hashtable_new_full().

marshal.c and _tracemalloc.c use these destroy functions.

5 years agobpo-40609: Remove _Py_hashtable_t.key_size (GH-20060)
Victor Stinner [Wed, 13 May 2020 00:26:02 +0000 (02:26 +0200)] 
bpo-40609: Remove _Py_hashtable_t.key_size (GH-20060)

Rewrite _Py_hashtable_t type to always store the key as
a "const void *" pointer. Add an explicit "key" member to
_Py_hashtable_entry_t.

Remove _Py_hashtable_t.key_size member.

hash and compare functions drop their hash table parameter, and their
'key' parameter type becomes "const void *".

5 years agobpo-40609: Rewrite how _tracemalloc handles domains (GH-20059)
Victor Stinner [Tue, 12 May 2020 23:36:47 +0000 (01:36 +0200)] 
bpo-40609: Rewrite how _tracemalloc handles domains (GH-20059)

Rewrite how the _tracemalloc module stores traces of other domains.
Rather than storing the domain inside the key, it now uses a new hash
table with the domain as the key, and the data is a per-domain traces
hash table.

* Add tracemalloc_domain hash table.
* Remove _Py_tracemalloc_config.use_domain.
* Remove pointer_t and related functions.

5 years agoFix Wikipedia link (GH-20031)
Allen Guo [Tue, 12 May 2020 22:54:18 +0000 (18:54 -0400)] 
Fix Wikipedia link (GH-20031)

5 years agobpo-40501: Replace ctypes code in uuid with native module (GH-19948)
Steve Dower [Tue, 12 May 2020 22:32:32 +0000 (23:32 +0100)] 
bpo-40501: Replace ctypes code in uuid with native module (GH-19948)

5 years agobpo-40602: Add _Py_HashPointerRaw() function (GH-20056)
Victor Stinner [Tue, 12 May 2020 16:46:20 +0000 (18:46 +0200)] 
bpo-40602: Add _Py_HashPointerRaw() function (GH-20056)

Add a new _Py_HashPointerRaw() function which avoids replacing -1
with -2 to micro-optimize hash table using pointer keys: using
_Py_hashtable_hash_ptr() hash function.

5 years agobpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)
scoder [Tue, 12 May 2020 14:12:41 +0000 (16:12 +0200)] 
bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)

… now that we allow subtypes of PyCFunction.

Also add PyCMethod_CheckExact() and PyCMethod_Check() for checks against the PyCMethod subtype.

5 years agobpo-40596: Fix str.isidentifier() for non-canonicalized strings containing non-BMP...
Serhiy Storchaka [Tue, 12 May 2020 13:18:00 +0000 (16:18 +0300)] 
bpo-40596: Fix str.isidentifier() for non-canonicalized strings containing non-BMP characters on Windows. (GH-20053)

5 years agobpo-40602: Optimize _Py_hashtable for pointer keys (GH-20051)
Victor Stinner [Tue, 12 May 2020 11:31:59 +0000 (13:31 +0200)] 
bpo-40602: Optimize _Py_hashtable for pointer keys (GH-20051)

Optimize _Py_hashtable_get() and _Py_hashtable_get_entry() for
pointer keys:

* key_size == sizeof(void*)
* hash_func == _Py_hashtable_hash_ptr
* compare_func == _Py_hashtable_compare_direct

Changes:

* Add get_func and get_entry_func members to _Py_hashtable_t
* Convert _Py_hashtable_get() and _Py_hashtable_get_entry() functions
  to static nline functions.
* Add specialized get and get entry for pointer keys.

5 years agobpo-40593: Improve syntax errors for invalid characters in source code. (GH-20033)
Serhiy Storchaka [Tue, 12 May 2020 09:42:04 +0000 (12:42 +0300)] 
bpo-40593: Improve syntax errors for invalid characters in source code. (GH-20033)

5 years agobpo-39481: remove generic classes from ipaddress/mmap (GH-20045)
Batuhan Taskaya [Tue, 12 May 2020 02:32:40 +0000 (05:32 +0300)] 
bpo-39481: remove generic classes from ipaddress/mmap (GH-20045)

These were added by mistake (see https://bugs.python.org/issue39481#msg366288).

5 years agobpo-40480: restore ability to join fnmatch.translate() results (GH-20049)
Tim Peters [Tue, 12 May 2020 02:19:20 +0000 (21:19 -0500)] 
bpo-40480: restore ability to join fnmatch.translate() results (GH-20049)

In translate(), generate unique group names across calls.

The restores the undocumented ability to get a valid regexp
by joining multiple translate() results via `|`.

5 years agobpo-40602: _Py_hashtable_new() uses PyMem_Malloc() (GH-20046)
Victor Stinner [Tue, 12 May 2020 01:07:40 +0000 (03:07 +0200)] 
bpo-40602: _Py_hashtable_new() uses PyMem_Malloc() (GH-20046)

_Py_hashtable_new() now uses PyMem_Malloc/PyMem_Free allocator by
default, rather than PyMem_RawMalloc/PyMem_RawFree.

PyMem_Malloc is faster than PyMem_RawMalloc for memory blocks smaller
than or equal to 512 bytes.

5 years agobpo-40602: Rename hashtable.h to pycore_hashtable.h (GH-20044)
Victor Stinner [Tue, 12 May 2020 00:42:19 +0000 (02:42 +0200)] 
bpo-40602: Rename hashtable.h to pycore_hashtable.h (GH-20044)

* Move Modules/hashtable.h to Include/internal/pycore_hashtable.h
* Move Modules/hashtable.c to Python/hashtable.c
* Python is now linked to hashtable.c. _tracemalloc is no longer
  linked to hashtable.c. Previously, marshal.c got hashtable.c via
  _tracemalloc.c which is built as a builtin module.

5 years agobpo-40571: Make lru_cache(maxsize=None) more discoverable (GH-20019)
Raymond Hettinger [Tue, 12 May 2020 00:00:53 +0000 (17:00 -0700)] 
bpo-40571: Make lru_cache(maxsize=None) more discoverable (GH-20019)

5 years agobpo-39465: Don't access directly _Py_Identifier members (GH-20043)
Victor Stinner [Mon, 11 May 2020 23:43:38 +0000 (01:43 +0200)] 
bpo-39465: Don't access directly _Py_Identifier members (GH-20043)

* Replace id->object with _PyUnicode_FromId(&id)
* Use _Py_static_string_init(str) macro to initialize statically
  name_op in typeobject.c.

5 years agobpo-40334: produce specialized errors for invalid del targets (GH-19911)
Shantanu [Mon, 11 May 2020 21:53:58 +0000 (14:53 -0700)] 
bpo-40334: produce specialized errors for invalid del targets (GH-19911)

5 years agobpo-40584: Update PyType_FromModuleAndSpec() to process tp_vectorcall_offset (GH...
Hai Shi [Mon, 11 May 2020 21:38:55 +0000 (05:38 +0800)] 
bpo-40584: Update PyType_FromModuleAndSpec() to process tp_vectorcall_offset (GH-20026)

5 years agobpo-40561: Add docstrings for webbrowser open functions (GH-19999)
Brad Solomon [Mon, 11 May 2020 18:50:11 +0000 (14:50 -0400)] 
bpo-40561: Add docstrings for webbrowser open functions (GH-19999)

Co-authored-by: Brad Solomon <brsolomon@deloitte.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
5 years agobpo-36346: array: Don't use deprecated APIs (GH-19653)
Inada Naoki [Mon, 11 May 2020 06:37:25 +0000 (15:37 +0900)] 
bpo-36346: array: Don't use deprecated APIs (GH-19653)

* Py_UNICODE -> wchar_t
* Py_UNICODE -> unicode in Argument Clinic
* PyUnicode_AsUnicode -> PyUnicode_AsWideCharString
* Don't use "u#" format.

Co-authored-by: Victor Stinner <vstinner@python.org>
5 years agobpo-40575: Avoid unnecessary overhead in _PyDict_GetItemIdWithError() (GH-20018)
scoder [Mon, 11 May 2020 04:04:31 +0000 (06:04 +0200)] 
bpo-40575: Avoid unnecessary overhead in _PyDict_GetItemIdWithError() (GH-20018)

Avoid unnecessary overhead in _PyDict_GetItemIdWithError() by calling
_PyDict_GetItem_KnownHash() instead of the more generic PyDict_GetItemWithError(),
since we already know the hash of interned strings.

5 years agobpo-40585: Normalize errors messages in codeop when comparing them (GH-20030)
Pablo Galindo [Mon, 11 May 2020 00:41:26 +0000 (01:41 +0100)] 
bpo-40585: Normalize errors messages in codeop when comparing them (GH-20030)

With the new parser, the error message contains always the trailing
newlines, causing the comparison of the repr of the error messages
in codeop to fail. This commit makes the new parser mirror the old parser's
behaviour regarding trailing newlines.

5 years agoImprove code clarity for the set lookup logic (GH-20028)
Raymond Hettinger [Sun, 10 May 2020 21:53:29 +0000 (14:53 -0700)] 
Improve code clarity for the set lookup logic (GH-20028)

5 years agobpo-40257: Tweak docstrings for special generic aliases. (GH-20022)
Serhiy Storchaka [Sun, 10 May 2020 12:14:27 +0000 (15:14 +0300)] 
bpo-40257: Tweak docstrings for special generic aliases. (GH-20022)

* Add the terminating period.
* Omit module name for builtin types.

5 years agobpo-40397: Fix subscription of nested generic alias without parameters. (GH-20021)
Serhiy Storchaka [Sun, 10 May 2020 10:39:40 +0000 (13:39 +0300)] 
bpo-40397: Fix subscription of nested generic alias without parameters. (GH-20021)

5 years agobpo-37986: Improve perfomance of PyLong_FromDouble() (GH-15611)
Sergey Fedoseev [Sun, 10 May 2020 09:15:57 +0000 (14:15 +0500)] 
bpo-37986: Improve perfomance of PyLong_FromDouble() (GH-15611)

* bpo-37986: Improve perfomance of PyLong_FromDouble()

* Use strict bound check for safety and symmetry

* Remove possibly outdated performance claims

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
5 years agobpo-40549: Convert posixmodule.c to multiphase init (GH-19982)
Victor Stinner [Sun, 10 May 2020 09:05:29 +0000 (11:05 +0200)] 
bpo-40549: Convert posixmodule.c to multiphase init (GH-19982)

Convert posixmodule.c ("posix" or "nt" module) to the multiphase
initialization (PEP 489).

* Create the module using PyModuleDef_Init().
* Create ScandirIteratorType and DirEntryType with the new
  PyType_FromModuleAndSpec() (PEP 573)
* Get the module state from ScandirIteratorType and DirEntryType with
  the new PyType_GetModule() (PEP 573)
* Pass module to functions which access the module state.
* convert_sched_param() gets a new module parameter. It is now called
  directly since Argument Clinic doesn't support passing the module
  to an argument converter callback.
* Remove _posixstate_global macro.

5 years agobpo-40397: Remove __args__ and __parameters__ from _SpecialGenericAlias (GH-19984)
Serhiy Storchaka [Sun, 10 May 2020 08:53:16 +0000 (11:53 +0300)] 
bpo-40397: Remove __args__ and __parameters__ from _SpecialGenericAlias (GH-19984)

5 years agoAdd link to Enum class (GH-19884)
Andre Delfino [Sun, 10 May 2020 05:15:54 +0000 (02:15 -0300)] 
Add link to Enum class (GH-19884)

5 years agobpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)
Pablo Galindo [Sun, 10 May 2020 04:34:50 +0000 (05:34 +0100)] 
bpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)

This is for the C generator:
- Disallow rule and variable names starting with `_`
- Rename most local variable names generated by the parser to start with `_`

Exceptions:
- Renaming `p` to `_p` will be a separate PR
- There are still some names that might clash, e.g.
  - anything starting with `Py`
  - C reserved words (`if` etc.)
  - Macros like `EXTRA` and `CHECK`

5 years agobpo-40570: Improve compatibility of uname_result with late-bound .platform (#20015)
Jason R. Coombs [Sat, 9 May 2020 14:12:41 +0000 (10:12 -0400)] 
bpo-40570: Improve compatibility of uname_result with late-bound .platform (#20015)

* bpo-40570: Improve compatibility of uname_result with late-bound .platform.

* Add test capturing ability to cast uname to a tuple.

5 years agobpo-40566: Apply PEP 573 to abc module (GH-20005)
Dong-hee Na [Sat, 9 May 2020 08:31:40 +0000 (17:31 +0900)] 
bpo-40566: Apply PEP 573 to abc module (GH-20005)

5 years agobpo-39791: Add files() to importlib.resources (GH-19722)
Jason R. Coombs [Fri, 8 May 2020 23:20:26 +0000 (19:20 -0400)] 
bpo-39791: Add files() to importlib.resources (GH-19722)

* bpo-39791: Update importlib.resources to support files() API (importlib_resources 1.5).

* 📜🤖 Added by blurb_it.

* Add some documentation about the new objects added.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
5 years agobpo-40502: Initialize n->n_col_offset (GH-19988)
Joannah Nanjekye [Fri, 8 May 2020 20:58:28 +0000 (17:58 -0300)] 
bpo-40502: Initialize n->n_col_offset (GH-19988)

* initialize n->n_col_offset

* 📜🤖 Added by blurb_it.

* Move initialization

Co-authored-by: nanjekyejoannah <joannah.nanjekye@ibm.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
5 years agobpo-40541: Add optional *counts* parameter to random.sample() (GH-19970)
Raymond Hettinger [Fri, 8 May 2020 14:53:15 +0000 (07:53 -0700)] 
bpo-40541: Add optional *counts* parameter to random.sample() (GH-19970)

5 years agoMake the first dataclass example more useful (GH-19994)
Ned Batchelder [Fri, 8 May 2020 11:39:57 +0000 (07:39 -0400)] 
Make the first dataclass example more useful (GH-19994)