Stefan Eissing [Fri, 30 Dec 2022 08:14:55 +0000 (09:14 +0100)]
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.
- eyeballing cfilter that uses sub-filters for performing parallel connects
- socket cfilter for all transport types, including QUIC
- QUIC implementations in cfilter, can now participate in eyeballing
- connection setup is more dynamic in order to adapt to what filter did
really connect. Relevant to see if a SSL filter needs to be added or
if SSL has already been provided
- HTTP/3 test cases similar to HTTP/2
- multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche
- Fix for data attach/detach in VTLS filters that could lead to crashes
during parallel transfers.
- Eliminating setup() methods in cfilters, no longer needed.
- Improving Curl_conn_is_alive() to replace Curl_connalive() and
integrated ssl alive checks into cfilter.
- Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
connection into and persist it at the easy handle.
- Several more cfilter related cleanups and moves:
- stream_weigth and dependency info is now wrapped in struct
Curl_data_priority
- Curl_data_priority members depend is available in HTTP2|HTTP3
- Curl_data_priority members depend on NGHTTP2 support
- handling init/reset/cleanup of priority part of url.c
- data->state.priority same struct, but shallow copy for compares only
- PROTOPT_STREAM has been removed
- Curl_conn_is_mulitplex() now available to check on capability
- Adding query method to connection filters.
- ngtcp2+quiche: implementing query for max concurrent transfers.
- Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
- setting keepalive timestamp on connect
- DATA_SETUP is called after the connection has been completely
setup (but may not connected yet) to allow filters to initialize
data members they use.
- there is no socket to be had with msh3, it is unclear how select
shall work
- manual test via "curl --http3 https://curl.se" fail with "empty
reply from server".
- Various socket/conn related cleanups:
- Curl_socket is now Curl_socket_open and in cf-socket.c
- Curl_closesocket is now Curl_socket_close and in cf-socket.c
- Curl_ssl_use has been replaced with Cur_conn_is_ssl
- Curl_conn_tcp_accepted_set has been split into
Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
with a clearer purpose
Daniel Stenberg [Wed, 28 Dec 2022 10:08:44 +0000 (11:08 +0100)]
transfer: break the read loop when RECV is cleared
When the RECV bit is cleared because the response reading for this
transfer is complete, the read loop should be stopped. data_pending()
can otherwise still return TRUE and another read would be attempted.
Stefan Eissing [Wed, 28 Dec 2022 08:58:09 +0000 (09:58 +0100)]
openssl: remove attached easy handles from SSL instances
- keeping the "current" easy handle registered at SSL* is no longer
necessary, since the "calling" data object is already stored in the
cfilter's context (and used by other SSL backends from there).
- The "detach" of an easy handle that goes out of scope is then avoided.
- using SSL_set0_wbio for clear reference counting where available.
Daniel Stenberg [Tue, 27 Dec 2022 16:19:51 +0000 (17:19 +0100)]
socketpair: allow localhost MITM sniffers
Windows allow programs to MITM connections to localhost. The previous
check here would detect that and error out. This new method writes data
to verify the pipe thus allowing MITM.
Reported-by: SerusDev on github
Fixes #10144
Closes #10169
Daniel Stenberg [Mon, 26 Dec 2022 08:59:20 +0000 (09:59 +0100)]
cfilters:Curl_conn_get_select_socks: use the first non-connected filter
When there are filters addded for both socket and SSL, the code
previously checked the SSL sockets during connect when it *should* first
check the socket layer until that has connected.
Patrick Monnerat [Fri, 23 Dec 2022 14:35:27 +0000 (15:35 +0100)]
typecheck: accept expressions for option/info parameters
As expressions can have side effects, evaluate only once.
To enable deprecation reporting only once, get rid of the __typeof__
use to define the local temporary variable and use the target type
(CURLoption/CURLINFO). This also avoids multiple reports on type
conflicts (if some) by the curlcheck_* macros.
Note that CURLOPT_* and CURLINFO_* symbols may be deprecated, but not
their values: a curl_easy_setopt call with an integer constant as option
will never report a deprecation.
Reported-by: Thomas Klausner
Fixes #10148
Closes #10149
Paul Howarth [Fri, 23 Dec 2022 12:34:49 +0000 (12:34 +0000)]
tests: avoid use of sha1 in certificates
The SHA-1 algorithm is deprecated (particularly for security-sensitive
applications) in a variety of OS environments. This already affects
RHEL-9 and derivatives, which are not willing to use certificates using
that algorithm. The fix is to use sha256 instead, which is already used
for most of the other certificates in the test suite.
Fixes #10135
This gets rid of issues related to sha1 signatures.
Manual steps after "make clean-certs" and "make build-certs":
- Copy tests/certs/stunnel-sv.pem to tests/stunnel.pem
(make clean-certs does not remove the original tests/stunnel.pem)
- Copy tests/certs/Server-localhost-sv.pubkey-pinned into --pinnedpubkey
options of tests/data/test2041 and tests/data/test2087
Radu Hociung [Mon, 26 Dec 2022 05:39:52 +0000 (00:39 -0500)]
http: remove the trace message "Mark bundle... multiuse"
The message "Mark bundle as not supporting multiuse" was added at commit 29364d93 when an http/2-related bug was fixed, and it appears to be a
leftover trace message.
This message should be removed because:
* it conveys no information to the user
* it is enabled in the default build (--enable-verbose)
* it reads like a warning/unexpected condition
* it is equivalent to "Detected http proto < 2", which is
not a useful message.
* it is a time-wasting red-herring for anyone who encounters
it for the first time while investigating some other, real
problem.
This commit removes the trace message "Mark bundle as not
supporting multiuse"
Daniel Stenberg [Thu, 22 Dec 2022 12:09:16 +0000 (13:09 +0100)]
urldata: move the cookefilelist to the 'set' struct
The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.
Daniel Stenberg [Tue, 20 Dec 2022 09:07:36 +0000 (10:07 +0100)]
http: fix the ::1 comparison for IPv6 localhost for cookies
When checking if there is a "secure context", which it is if the
connection is to localhost even if the protocol is HTTP, the comparison
for ::1 was done incorrectly and included brackets.
Reported-by: BratSinot on github
Fixes #10120
Closes #10121
Daniel Stenberg [Mon, 19 Dec 2022 07:36:55 +0000 (08:36 +0100)]
http: use the IDN decoded name in HSTS checks
Otherwise it stores the info HSTS into the persistent cache for the IDN
name which will not match when the HSTS status is later checked for
using the decoded name.
When utf8asn1str fails there is no allocation returned, so freeing
the return pointer in **to is at best a no-op and at worst a double-
free bug waiting to happen. The current coding isn't hiding any such
bugs but to future proof, avoid freeing the return value pointer iff
the function failed.
Closes: #10087 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg [Tue, 6 Dec 2022 12:00:35 +0000 (13:00 +0100)]
KNOWN_BUGS: remove items not considered bugs any more
- CURL_GLOBAL_SSL
This option was changed in libcurl 7.57.0 and clearly it has not caused
too many issues and a lot of time has passed.
- Store TLS context per transfer instead of per connection
This is a possible future optimization. One that is much less important
and interesting since the added support for CA caching.
- Microsoft telnet server
This bug was filed in May 2007 against curl 7.16.1 and we have not
received further reports.
- active FTP over a SOCKS
Actually, proxies in general is not working with active FTP mode. This
is now added in proxy documentation.
- DICT responses show the underlying protocol
curl still does this, but since this is now an established behavior
since forever we cannot change it easily and adding an option for it
seems crazy as this protocol is not so little its not worth it. Let's
just live with it.
- Secure Transport disabling hostname validation also disables SNI
This is an already documented restriction in Secure Transport.
- CURLOPT_SEEKFUNCTION not called with CURLFORM_STREAM
The curl_formadd() function is marked and documented as deprecated. No
point in collecting bugs for it. It should not be used further.
- STARTTRANSFER time is wrong for HTTP POSTs
After close source code inspection I cannot see how this is true or that
there is any special treatment for different HTTP methods. We also have
not received many further reports on this, making me strongly suspect
that this is no (longer an) issue.
- multipart formposts file name encoding
The once proposed RFC 5987-encoding is since RFC 7578 documented as MUST
NOT be used. The since then implemented MIME API allows the user to set
the name on their own and can thus provide it encoded as it wants.
- DoH is not used for all name resolves when enabled
It is questionable if users actually want to use DoH for interface and
FTP port name resolving. This restriction is now documented and we
advice users against using name resolving at all for these functions.
curl.h: reword comment to not use deprecated option
CURLOPT_INFILE was replaced by CURLOPT_READDATA in 7.9.7, reword the
comment mentioning it to make code grepping easier as well as improve
the documentation.
Closes: #10062 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Ryan Schmidt [Wed, 7 Dec 2022 17:00:41 +0000 (11:00 -0600)]
system.h: fix socklen_t, curl_off_t, long long for Classic Mac OS
Change "__MWERKS__" to "macintosh". When this block was originally added
in 3ac6929 it was probably intended to handle classic Mac OS since the
previous classic Mac OS build procedure for curl (which was removed in bf327a9) used Metrowerks CodeWarrior.
But there are other classic Mac OS compilers, such as the MPW compilers,
that were not handled by this case. For classic Mac OS,
CURL_TYPEOF_CURL_SOCKLEN_T needs to match what's provided by the
third-party GUSI library, which does not vary by compiler.
Meanwhile CodeWarrior works on platforms other than classic Mac OS, and
they may need different definitions. Separate blocks could be added
later for any of those platforms that curl doesn't already support.
tool_formparse: avoid clobbering on function params
While perfectly legal to do, clobbering function parameters and using
them as local variables is confusing at best and rarely improves code
readability. Fix by using a local variable instead, no functionality
is changed.
This also renames the parameter from data to mime_data since the term
data is (soft) reserved for the easy handle struct.
Closes: #10046 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
noproxy: guard against empty hostnames in noproxy check
When checking for a noproxy setting we need to ensure that we get
a hostname passed in. If there is no hostname then there cannot be
a matching noproxy rule for it by definition.
Closes: #10057 Reported-by: Geeknik Labs Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Ryan Schmidt [Mon, 5 Dec 2022 19:25:05 +0000 (13:25 -0600)]
config-mac: define HAVE_SYS_IOCTL_H
This is needed to compile nonblock.c on classic Mac OS with Grand
Unified Socket Interface (GUSI) because nonblock.c uses FIONBIO which is
defined in <sys/filio.h> which is included by <sys/ioctl.h>.
projects / thirdparty / curl.git / log
