bpo-46756: Fix authorization check in urllib.request (GH-31353) (GH-31573)
Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and
urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which
allowed to bypass authorization. For example, access to URI "example.org/foobar"
was allowed if the user was authorized for URI "example.org/foo".
(cherry picked from commit e2e72567a1c94c548868f6ee5329363e6036057a)
bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31471)
Curly brackets were never allowed in namespace URIs
according to RFC 3986, and so-called namespace-validating
XML parsers have the right to reject them a invalid URIs.
libexpat >=2.4.5 has become strcter in that regard due to
related security issues; with ET.XML instantiating a
namespace-aware parser under the hood, this test has no
future in CPython.
Łukasz Langa [Tue, 31 Aug 2021 05:11:53 +0000 (07:11 +0200)]
[3.7] bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28042)
Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the
fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy
is most used on Windows and macOS.
Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Łukasz Langa <lukasz@langa.pl>.
(cherry picked from commit 3fc5d84046ddbd66abac5b598956ea34605a4e5d)
bpo-45001: Make email date parsing more robust against malformed input (GH-27946) (GH-27975)
Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.
The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.
In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.
Here's a minimal example:
$ python
Python 3.9.6 (default, Jun 30 2021, 10:22:16)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import email.utils
>>> email.utils.parsedate('foo')
>>> email.utils.parsedate(' ')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate
t = parsedate_tz(data)
File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz
res = _parsedate_tz(data)
File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz
if data[0].endswith(',') or data[0].lower() in _daynames:
IndexError: list index out of range
The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.
(cherry picked from commit 989f6a3800f06b2bd31cfef7c3269a443ad94fac)
bpo-43075: Fix ReDoS in urllib AbstractBasicAuthHandler (GH-24391) (#25249)
Fix Regular Expression Denial of Service (ReDoS) vulnerability in
urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex
has quadratic worst-case complexity and it allows cause a denial of
service when identifying crafted invalid RFCs. This ReDoS issue is on
the client side and needs remote attackers to control the HTTP server.
(cherry picked from commit 7215d1ae25525c92b026166f9d5cac85fb1defe1)
bpo-42988: Remove the pydoc getfile feature (GH-25015) (#25066)
CVE-2021-3426: Remove the "getfile" feature of the pydoc module which
could be abused to read arbitrary files on the disk (directory
traversal vulnerability). Moreover, even source code of Python
modules can contain sensitive data like passwords. Vulnerability
reported by David Schwörer.
(cherry picked from commit 9b999479c0022edfc9835a8a1f06e046f3881048)
Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Victor Stinner <vstinner@python.org>
[3.7] bpo-43285 Make ftplib not trust the PASV response. (GH-24838) (GH-24881) (GH-24883)
The IPv4 address value returned from the server in response to the PASV command
should not be trusted. This prevents a malicious FTP server from using the
response to probe IPv4 address and port combinations on the client network.
Instead of using the returned address, we use the IP address we're
already connected to. This is the strategy other ftp clients adopted,
and matches the only strategy available for the modern IPv6 EPSV command
where the server response must return a port number and nothing else.
For the rare user who _wants_ this ugly behavior, set a `trust_server_pasv_ipv4_address`
attribute on your `ftplib.FTP` instance to True..
(cherry picked from commit 0ab152c6b5d95caa2dc1a30fa96e10258b5f188e)
Guanzhong Chen [Fri, 26 Feb 2021 10:28:43 +0000 (05:28 -0500)]
[3.7] bpo-43293: Doc: move note about GIL to top of threading module (GH-24622) (#24642)
The note about the GIL was buried pretty deep in the threading documentation,
and this made it hard for first time users to discover why their attempts
at using threading to parallelizing their application did not work.
Senthil Kumaran [Mon, 15 Feb 2021 18:34:14 +0000 (10:34 -0800)]
[3.7] bpo-42967: only use '&' as a query string separator (GH-24297) (GH-24531)
bpo-42967: [security] Address a web cache-poisoning issue reported in
urllib.parse.parse_qsl().
urllib.parse will only us "&" as query string separator by default
instead of both ";" and "&" as allowed in earlier versions. An optional
argument seperator with default value "&" is added to specify the
separator.
Co-authored-by: Éric Araujo <merwok@netwok.org> Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Co-authored-by: Adam Goldschmidt <adamgold7@gmail.com>
(cherry picked from commit fcbe0cb04d35189401c0c880ebfb4311e952d776)
Up until now, the `multiprocessing.pool.ThreadPool` class has gone
undocumented, despite being a public class in multiprocessing that is
included in `multiprocessing.pool.__all__`.
(cherry picked from commit 84ebcf271a2cc8bfd1762acb279502b8b6ef236e)
Co-authored-by: Matt Wozniski <mwozniski@bloomberg.net>
bpo-40791: Make compare_digest more constant-time. (GH-23438)
The existing volatile `left`/`right` pointers guarantee that the reads will all occur, but does not guarantee that they will be _used_. So a compiler can still short-circuit the loop, saving e.g. the overhead of doing the xors and especially the overhead of the data dependency between `result` and the reads. That would change performance depending on where the first unequal byte occurs. This change removes that optimization.
Serhiy Storchaka [Tue, 10 Nov 2020 19:54:15 +0000 (21:54 +0200)]
[3.7] bpo-42103: Improve validation of Plist files. (GH-22882) (#23117)
* Prevent some possible DoS attacks via providing invalid Plist files
with extremely large number of objects or collection sizes.
* Raise InvalidFileException for too large bytes and string size instead of returning garbage.
* Raise InvalidFileException instead of ValueError for specific invalid datetime (NaN).
* Raise InvalidFileException instead of TypeError for non-hashable dict keys.
* Add more tests for invalid Plist files..
(cherry picked from commit 34637a0ce21e7261b952fbd9d006474cc29b681f)
bpo-39603: Prevent header injection in http methods (GH-18485) (GH-21538)
reject control chars in http method in http.client.putrequest to prevent http header injection
(cherry picked from commit 8ca8a2e8fb068863c1138f07e3098478ef8be12e)
Co-authored-by: AMIR <31338382+amiremohamadi@users.noreply.github.com>
bpo-39017: Avoid infinite loop in the tarfile module (GH-21454) (GH-21484)
Avoid infinite loop when reading specially crafted TAR files using the tarfile module
(CVE-2019-20907).
(cherry picked from commit 5a8d121a1f3ef5ad7c105ee378cc79a3eac0c7d4)
Tapas Kundu [Tue, 30 Jun 2020 19:20:21 +0000 (00:50 +0530)]
[3.7] bpo-41004: Resolve hash collisions for IPv4Interface and IPv6Interface (GH-21033) (GH-21231)
CVE-2020-14422
The __hash__() methods of classes IPv4Interface and IPv6Interface had issue
of generating constant hash values of 32 and 128 respectively causing hash collisions.
The fix uses the hash() function to generate hash values for the objects
instead of XOR operation
(cherry picked from commit b30ee26e366bf509b7538d79bfec6c6d38d53f28)
Co-authored-by: Ravi Teja P <rvteja92@gmail.com> Signed-off-by: Tapas Kundu <tkundu@vmware.com>
Ned Deily [Wed, 17 Jun 2020 07:18:55 +0000 (03:18 -0400)]
bpo-39580: add check for CLI installation on macOS
Adds a simple check for whether or not the package is being installed in the GUI or using installer on the command line. This addresses an issue where CLI-based software management tools (such as Munki) unexpectedly open Finder windows into a GUI session during installation runs.
ensurepip optionally installs or upgrades 'pip' and 'setuptools' using
the version of those modules bundled with Python. The internal PIP
installation routine by default temporarily uses its cache, if it
exists. This is undesirable as Python builds and installations may be
independent of the user running the build, whilst PIP cache location
is dependent on the user's environment and outside of the build
environment.
At the same time, there's no value in using the cache while installing
bundled modules.
[3.7] bpo-34226: fix cgi.parse_multipart without content_length (GH-8530) (GH-20892)
In Python 3.7 the behavior of parse_multipart changed requiring CONTENT-LENGTH
header, this fix remove this header as required and fix FieldStorage
read_lines_to_outerboundary, by not using limit when it's negative,
since by default it's -1 if not content-length and keeps substracting what
was read from the file object.
bpo-29620: iterate over a copy of sys.modules (GH-4800) (GH-20817)
unittest.TestCase.assertWarns no longer raises a RuntimeException
when accessing a module's ``__warningregistry__`` causes importation of a new
module, or when a new module is imported in another thread.
(cherry picked from commit 46398fba4d66ad342cf2504ef947b5fb857423b2)
Antoine Pitrou [Wed, 10 Jun 2020 20:37:21 +0000 (22:37 +0200)]
[3.7] bpo-40895: Update weakref documentation to remove old warnings (GH-20687) (GH-20793)
The doccumentation at https://docs.python.org/3.10/library/weakref.html cautions that the `WeakKeyDictionary` and `WeakValueDictionary` are susceptible to the problem of dictionary mutation during iteration.
These notes present the user with a problem that has no easy solution.
I dug into the implementation and found that fortunately, Antoine Pitrou already addressed this challenge (10 years ago!) by introducing an `_IterationGuard` context manager to the implementation, which delays mutation while an iteration is in progress.
I asked for confirmation and @pitrou agreed that these notes could be removed:
https://github.com/python/cpython/commit/c1baa601e2b558deb690edfdf334fceee3b03327GH-commitcomment-39514438.
(cherry picked from commit 1642c0ef750f96664a98cadb09301d492098d2fb)
Co-authored-by: Daniel Fortunov <asqui@users.noreply.github.com> Automerge-Triggered-By: @pitrou
Before
```
content.txt is 42 bytes in size and isa regular file.
folder is 420 bytes in size and isa directory.
magic is 4200 bytes in size and issomething else.
```
After:
```
content.txt is 42 bytes in size and is a regular file.
folder is 420 bytes in size and is a directory.
magic is 4200 bytes in size and is something else.
```
bpo-40767: Allow pure Wayland to get default XDG web browser (GH-20382)
Would be nice to backport to python 3.7+. I don't think it's worth the hassle to backport this all the way down to 3.10. But I'll let the maintainers decide.
This is hard to test because the test setup already includes this [environment variable](https://github.com/python/cpython/blob/master/Lib/test/pythoninfo.pyGH-L292)
Let me know if something doesn't match the PR guidelines. This is my first PR in the python source code.
(cherry picked from commit c822efeda9a0afe87cf3429724732fc8e19a01fb)
Co-authored-by: Jeremy Attali <jeremy.attali@gmail.com>
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.
Note: Tests assume full OpenSSL API and fail with limited API.
Signed-off-by: Christian Heimes <christian@python.org> Co-authored-by: Mark Wright <gienah@gentoo.org>
(cherry picked from commit a871f692b4a2e6c7d45579693e787edc0af1a02c)
Co-authored-by: Christian Heimes <christian@python.org>
bpo-39885: Make IDLE context menu cut and copy work again (GH-18951)
Leave selection when right click within. This exception to clearing selections when right-clicking was omitted from the previous commit, 4ca060d. I did not realize that this completely disabled the context menu entries, and I should have merged a minimal fix immediately. An automated test should follow.
(cherry picked from commit 97e4e0f53d6690db6b942678489716a30925b8af)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
[3.8] bpo-25872: Fix KeyError in linecache when multithreaded (GH-18007) (GH-20092)
Backporting to 3.8 and adding a NEWS item (I should have added one to the master branch -- oh well).
(cherry picked from commit b86636bff4b29ce23c886df079715dd951f13a07)
projects / thirdparty / Python / cpython.git / log
