[3.10] gh-122133: Authenticate socket connection for `socket.socketpair()` fallback (GH-122134) (#122427)
Authenticate socket connection for `socket.socketpair()` fallback when the platform does not have a native `socketpair` C API. We authenticate in-process using `getsocketname` and `getpeername` (thanks to Nathaniel J Smith for that suggestion).
The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).
This patch updates the ranges with what was missing or otherwise
incorrect.
100.64.0.0/10 is left alone, for now, as it's been made special in [1].
The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.
In 3.10 and below, is_private checks whether the network and broadcast
address are both private.
In later versions (where the test wss backported from), it checks
whether they both are in the same private network.
For 0.0.0.0/0, both 0.0.0.0 and 255.225.255.255 are private,
but one is in 0.0.0.0/8 ("This network") and the other in
255.255.255.255/32 ("Limited broadcast").
[3.10] gh-117187: Fix XML tests for vanilla Expat <2.6.0 (GH-117203) (GH-117246)
This fixes XML unittest fallout from the https://github.com/python/cpython/issues/115398 security fix. When configured using `--with-system-expat` on systems with older pre 2.6.0 versions of libexpat, our unittests were failing.
Co-authored-by: Sebastian Pipping <sebastian@pipping.org>
Use of a proxy is intended to defer DNS for the hosts to the proxy itself, rather than a potential for information leak of the host doing DNS resolution itself for any reason. Proxy bypass lists are strictly name based. Most implementations of proxy support agree.
(cherry picked from commit c43b26d02eaa103756c250e8d36829d388c5f3be)
Co-authored-by: Weii Wang <weii.wang@canonical.com>
[3.10] gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573) (#115548)
gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573)
* gh-114572: Fix locking in cert_store_stats and get_ca_certs
cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with
X509_STORE_get0_objects, but reading the result requires a lock. See
https://github.com/openssl/openssl/pull/23224 for details.
Instead, use X509_STORE_get1_objects, newly added in that PR.
X509_STORE_get1_objects does not exist in current OpenSSLs, but we can
polyfill it with X509_STORE_lock and X509_STORE_unlock.
[3.10] gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 (GH-115164) (#115525)
gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 (GH-115164)
Feeding the parser by too small chunks defers parsing to prevent
CVE-2023-52425. Future versions of Expat may be more reactive.
(cherry picked from commit 4a08e7b3431cd32a0daf22a33421cd3035343dc4)
[3.10] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) (GH-113914)
Raise BadZipFile when try to read an entry that overlaps with other entry or
central directory.
(cherry picked from commit 66363b9a7b9fe7c99eba3a185b74c5fdbf842eba)
[3.10] bpo-37013: Fix the error handling in socket.if_indextoname() (GH-13503) (GH-112599)
* Fix a crash when pass UINT_MAX.
* Fix an integer overflow on 64-bit non-Windows platforms.
(cherry picked from commit 0daf555c6fb3feba77989382135a58215e1d70a5)
Output with two wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl::Found more than one wheel for package pip.
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Found more than one wheel for package pip.
```
Output without wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=::Could not find a pip wheel on disk.
```
(cherry picked from commit f8a047941f2e4a1848700c21d58a08c9ec6a9c68)
Łukasz Langa [Thu, 24 Aug 2023 10:09:00 +0000 (12:09 +0200)]
[3.10] gh-108342: Make ssl TestPreHandshakeClose more reliable (GH-108370) (#108406)
* In preauth tests of test_ssl, explicitly break reference cycles
invoving SingleConnectionTestServerThread to make sure that the
thread is deleted. Otherwise, the test marks the environment as
altered because the threading module sees a "dangling thread"
(SingleConnectionTestServerThread). This test leak was introduced
by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
test_preauth_data_to_tls_client(): the server now waits until the
client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
(cherry picked from commit 592bacb6fc0833336c0453e818e9b95016e9fd47)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.10] gh-108342: Break ref cycle in SSLSocket._create() exc (GH-108344) (#108350)
Explicitly break a reference cycle when SSLSocket._create() raises an
exception. Clear the variable storing the exception, since the
exception traceback contains the variables and so creates a reference
cycle.
Łukasz Langa [Tue, 22 Aug 2023 17:53:23 +0000 (19:53 +0200)]
[3.10] gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw (#108318)
gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.
The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
[3.10] gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174) (GH-105200) (#105204)
Upgrade builds to OpenSSL 1.1.1u.
This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.
The Mac/BuildScript/build-installer.py was already updated.
Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9.
Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).
[3.10] [3.11] gh-102153: Start stripping C0 control and space chars in `urlsplit` (GH-102508) (GH-104575) (#104592)
gh-102153: Start stripping C0 control and space chars in `urlsplit` (GH-102508)
`urllib.parse.urlsplit` has already been respecting the WHATWG spec a bit GH-25595.
This adds more sanitizing to respect the "Remove any leading C0 control or space from input" [rule](https://url.spec.whatwg.org/GH-url-parsing:~:text=Remove%20any%20leading%20and%20trailing%20C0%20control%20or%20space%20from%20input.) in response to [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329).
I simplified the docs by eliding the state of the world explanatory
paragraph in this security release only backport. (people will see
that in the mainline /3/ docs)
Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> Co-authored-by: Illia Volochii <illia.volochii@gmail.com> Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
[3.10] gh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (GH-104067) (#104119)
gh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (GH-104067)
Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure)
Ned Deily [Tue, 4 Apr 2023 18:42:10 +0000 (14:42 -0400)]
[3.10] gh-103207: Add instructions to the macOS installer welcome display on how to workaround the macOS 13 Ventura “The installer encountered an error” failure. (GH-103253)
Co-authored-by: Bernhard Wagner <github.comNotification20120125@xmlizer.net> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Éric <merwok@netwok.org>
Steve Dower [Wed, 29 Mar 2023 14:33:39 +0000 (15:33 +0100)]
gh-103097: Add workaround for Windows ARM64 compiler bug (GH-103098)
See https://developercommunity.visualstudio.com/t/Regression-in-MSVC-1433-1434-ARM64-co/10224361 for details of the issue. It only applies to version 14.34.
[3.10] GH-87235: Make sure "python /dev/fd/9 9</path/to/script.py" works on macOS (GH-99768) (#99817)
On macOS all file descriptors for a particular file in /dev/fd
share the same file offset, that is ``open("/dev/fd/9", "r")`` behaves
more like ``dup(9)`` than a regular open.
This causes problems when a user tries to run "/dev/fd/9" as a script
because zipimport changes the file offset to try to read a zipfile
directory. Therefore change zipimport to reset the file offset after
trying to read the zipfile directory.
(cherry picked from commit d08fb257698e3475d6f69bb808211d39e344e5b2)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
* Regen zipimport
[3.10] GH-102711: Fix warnings found by clang (GH-102712) (#103076)
There are some warnings if build python via clang:
Parser/pegen.c:812:31: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_clear_memo_statistics()
^
void
Parser/pegen.c:820:29: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_get_memo_statistics()
^
void
[3.10] GH-95494: Fix transport EOF handling in OpenSSL 3.0 (GH-95495) (#103007)
GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment
that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right.
That option causes OpenSSL to treat transport EOF as the same as
close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has
distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is
usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl
module would raise them for transport EOF and close_notify,
respectively. In OpenSSL 3.0, both act like close_notify.
Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING
and mapping that to the other exception type.
There doesn't seem to have been any unit test of this error, so fill in
the missing one. This had to be done with the BIO path because it's
actually slightly tricky to simulate a transport EOF with Python's fd
based APIs. (If you instruct the server to close the socket, it gets
confused, probably because the server's SSL object is still referencing
the now dead fd?)
(cherry picked from commit 420bbb783b43216cc897dc8914851899db37a31d)
Co-authored-by: David Benjamin <davidben@google.com>
projects / thirdparty / Python / cpython.git / log
