Stefan Eissing [Mon, 6 Mar 2023 11:44:45 +0000 (12:44 +0100)]
url: fix logic in connection reuse to deny reuse on "unclean" connections
- add parameter to `conn_is_alive()` cfilter method that returns
if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
input pending
- adapt http/2 and http/3 alive checks to digest pending input
to check the connection state
- remove check_cxn method from openssl as that was just doing
what the socket filter now does.
- add tests for connection reuse with special server configs
Stefan Eissing [Mon, 6 Mar 2023 16:16:01 +0000 (17:16 +0100)]
http2: fix handling of RST and GOAWAY to recognize partial transfers
- a reset transfer (HTTP/2 RST) did not always lead to the proper
error message on receiving its response, leading to wrong reports
of a successful transfer
- test_05_02 was able to trigger this condition with increased transfer
count. The simulated response errors did not carry a 'Content-Length'
so only proper RST handling could detect the abort
- When doing such transfers in parallel, a connection could enter the
state where
a) it had been closed (GOAWAY received)
b) the RST had not been "seen" for the transfer yet
or c) the GOAWAY announced an error and the last successful
stream id was not checked against ongoing transfers
Stefan Eissing [Mon, 6 Mar 2023 15:11:11 +0000 (16:11 +0100)]
tests: use dynamic ports numbers in pytest suite
- necessary ports are bound at start of test suite and then
given to server fixtures for use.
- this make parallel use of pytest (in separate directories),
practically safe for use as OS tend to not reuse such port numbers
for a while
Stefan Eissing [Fri, 3 Mar 2023 16:54:44 +0000 (17:54 +0100)]
connect: fix time_connect and time_appconnect timer statistics
- time_connect was not updated when the overall connection failed,
e.g. when SSL verification was unsuccessful, refs #10670
- rework gather those values to interrogate involved filters,
also from all eyeballing attempts, to report the maximum of
those values.
- added 3 test cases in test_06 to check reported values on
successful, partially failed and totally failed connections.
Harry Sintonen [Sat, 4 Mar 2023 07:02:14 +0000 (09:02 +0200)]
rand: use arc4random as fallback when available
Normally curl uses cryptographically strong random provided by the
selected SSL backend. If compiled without SSL support, a naive built-in
function was used instead.
Generally this was okay, but it will result in some downsides for non-
SSL builds, such as predictable temporary file names.
This change ensures that arc4random will be used instead, if available.
Grisha Levit [Sat, 4 Mar 2023 21:32:56 +0000 (16:32 -0500)]
tool: dump headers even if file is write-only
The fixes in #10079 brought a (seemingly unrelated) change of open mode
from `wb`/`ab` to `wb+`/`ab+` for the headerfile. This makes it no
longer possible to write the header file to e.g. a pipe, like:
curl -D >(grep ...) file:///dev/null
Which presently results in `Warning: Failed to open /dev/fd/63`
Viktor Szakats [Sun, 5 Mar 2023 19:55:14 +0000 (19:55 +0000)]
cmake: fix enabling LDAPS on Windows
Before this patch, enabling LDAPS required a manual C flag:
https://github.com/curl/curl-for-win/blob/c1cfc31cfc04f24f7a4f946564d6f0e1b4d7dd36/curl-cmake.sh#L105
Fix this and enable LDAPS automatically when using `wldap32` (and
when not explicitly disabled). This matches autotools and `Makefile.mk`
behavior. Also remove issue from KNOWN_BUGS.
Add workaround for MSVS 2010 warning triggered by LDAPS now enabled
in more CI tests:
`ldap.c(360): warning C4306: 'type cast' : conversion from 'int' to 'void *' of greater size`
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/46408284/job/v8mwl9yfbmoeqwlr#L312
Reported-by: JackBoosY on github Reviewed-by: Jay Satiro Reviewed-by: Marcel Raad
Fixes #6284
Closes #10674
For OpenLDAP builds this macro is not enough to enable LDAPS, and
OpenLDAP is not an option in `Makefile.mk`. For Novell LDAP it might
have helped, but it's also not an option anymore in `Makefile.mk`.
The future for LDAPS is that we should enable it by default without
extra build knobs.
Jan Engelhardt [Thu, 2 Mar 2023 15:08:22 +0000 (16:08 +0100)]
build: drop the use of XC_AMEND_DISTCLEAN
Because automake used to delete depdirs at once (.deps) and there was an issue
with portability, curl's XC_AMEND_DISTCLEAN greps the Makefiles in an attempt
to build a list of all depfiles and delete them individually instead.
Since commit 08849db866b44510f6b8fd49e313c91a43a3dfd3, automake switched from
deleting directories to individual files. curl's custom logic now finds a lot
more results with the grep (the filtering of these results isn't great), which
causes a massive bloating of the Makefile in the order of O(n^2).
Also remove now-unused XC_AMEND_DISTCLEAN macro group
Stefan Eissing [Wed, 1 Mar 2023 12:05:09 +0000 (13:05 +0100)]
socket: detect "dead" connections better, e.g. not fit for reuse
- refs #10646 where reuse was attempted on closed connections in the
cache, leading to an exhaustion of retries on a transfer
- the mistake was that poll events like POLLHUP, POLLERR, etc
were regarded as "not dead".
- change cf-socket filter check to regard such events as inidication
of corpsiness.
- vtls filter checks: fixed interpretation of backend check result
when inconclusive to interrogate status further down the filter
chain.
Reported-by: SendSonS on github
Fixes #10646
Closes #10652
cf-socket: if socket is already connected, return CURLE_OK
In 7.87.0, if callback method for CURLOPT_SOCKOPTFUNCTION returns
CURL_SOCKOPT_ALREADY_CONNECTED then curl library used to return
CURLE_OK. n 7.88.0, now even if callback returns
CURL_SOCKOPT_ALREADY_CONNECTED, curl library still tries to connect to
socket by invoking method do_connect().
This is regression caused by commit
https://github.com/curl/curl/commit/71b7e0161032927cdfb
Fix: Check if we are already connected and return CURLE_OK.
Jay Satiro [Sun, 26 Feb 2023 08:44:38 +0000 (03:44 -0500)]
transfer: limit Windows SO_SNDBUF updates to once a second
- Change readwrite_upload() to call win_update_buffer_size() no more
than once a second to update SO_SNDBUF (send buffer limit).
Prior to this change during an upload readwrite_upload() could call
win_update_buffer_size() anywhere from hundreds of times per second to
an extreme test case of 100k per second (which is likely due to a bug,
see #10618). In the latter case WPA profiler showed
win_update_buffer_size was the highest capture count in
readwrite_upload. In any case the calls were excessive and unnecessary.
Stefan Eissing [Tue, 28 Feb 2023 10:43:50 +0000 (11:43 +0100)]
http2: fix for http2-prior-knowledge when reusing connections
- refs #10634 where errors in the HTTP/2 framing layer are observed.
- the bug was that on connection reuse, the code attempted to switch
in yet another layer of HTTP/2 handling instead of detecting that
this was already in place.
- added pytest testcase reproducing the issue.
Reported-by: rwmjones on github
Fixes #10634
Closes #10643
Stefan Eissing [Tue, 28 Feb 2023 10:05:06 +0000 (11:05 +0100)]
cf-socket: fix handling of remote addr for accepted tcp sockets
- do not try to determine the remote address of a listen socket. There
is none.
- Update remote address of an accepted socket by getpeername() if
available.
Reported-by: Harry Sintonen
Fixes #10622
Closes #10642
Stefan Eissing [Tue, 28 Feb 2023 09:07:21 +0000 (10:07 +0100)]
http: fix unix domain socket use in https connects
- when h2/h3 eyeballing was involved, unix domain socket
configurations were not honoured
- configuring --unix-socket will disable HTTP/3 as candidate for eyeballing
- combinatino of --unix-socket and --http3-only will fail during initialisation
- adding pytest test_11 to reproduce
Reported-by: Jelle van der Waa
Fixes #10633
Closes #10641
積丹尼 Dan Jacobson [Tue, 28 Feb 2023 01:36:30 +0000 (09:36 +0800)]
rate.c: single URLs make no sense in --rate example
Here somehow you need to put more than one URL in these examples, else
they will make no sense, as --rate only affects the second and beyond
URLs. The first URL will always finish the same time no matter what
--rate is given.
Daniel Stenberg [Fri, 24 Feb 2023 17:17:33 +0000 (18:17 +0100)]
idn: return error if the conversion ends up with a blank host
Some IDN sequences are converted into "" (nothing), which can make this
function end up with a zero length host name and we cannot consider that
a valid host to continue with.
This tool seems very restricted in how often it might be used by a
project and thus very quickly start to report fails simply because it
refuses to run when "there are more runs than allowed".
Jay Satiro [Tue, 21 Feb 2023 08:38:37 +0000 (03:38 -0500)]
winbuild: fix makefile clean
- Fix and move 'clean' code that removes the output and obj directories
trees from MakefileBuild.vc to Makefile.vc.
Prior to this change the 'clean' code did not work right because the
variables containing the directory names were not fully initialized and
the rmdir syntax was sometimes incorrect (typos). DIRDIST for example
was set to ..\builds\ and not ..\builds\$(CONFIG_NAME_LIB)\ so it would
remove the former and not the latter. If WITH_PREFIX was set then that
directory was removed instead.
Also, DIRDIST (the output directory) even if initialized should not be
removed by MakefileBuild.vc because by that time it could be set to a
user directory that may contain other files if WITH_PREFIX is set (eg we
don't want rmdir /s /q C:\usr\local). Therefore we remove from
Makefile.vc before any of that happens. I added a comment in both
makefiles explaining this.
Since cbf57176 the Cirrus CI 'macOS arm64 SecureTransport http2' has
been failing due to c89 warnings mixed code/declaration. That commit is
not the cause so I assume something has changed in the CI outside of our
configuration. Anyway, we don't mix code/declaration so this is the fix
for that.
Stefan Eissing [Thu, 9 Feb 2023 15:07:34 +0000 (16:07 +0100)]
CURLOPT_PIPEWAIT: allow waited reuse also for subsequent connections
As tested in test_02_07, when firing off 200 urls with --parallel, 199
wait for the first connection to be established. if that is multiuse,
urls are added up to its capacity.
The first url over capacity opens another connection. But subsequent
urls found the same situation and open a connection too. They should
have waited for the second connection to actually connect and make its
capacity known.
This change fixes that by
- setting `connkeep()` early in the HTTP setup handler. as otherwise
a new connection is marked as closeit by default and not considered
for multiuse at all
- checking the "connected" status for a candidate always and continuing
to PIPEWAIT if no alternative is found.
pytest:
- removed "skip" from test_02_07
- added test_02_07b to check that http/1.1 continues to work as before
projects / thirdparty / curl.git / log
