Stefan Eissing [Thu, 25 May 2023 13:22:12 +0000 (15:22 +0200)]
http3: send EOF indicator early as possible
- ngtcp2 and quiche implementations relied on the DONE_SEND event
to forward the EOF for uploads to the libraries. This often
result in a last 0 length EOF data. Tracking the amount of
data left to upload allows EOF indication earlier.
- refs #11205 where CloudFlare DoH servers did not like to
receive the initial upload DATA without EOF and returned
a 400 Bad Request
Daniel Stenberg [Thu, 25 May 2023 12:59:21 +0000 (14:59 +0200)]
scripts/contri*sh: no longer grep -v ' '
Originally these scripts filtered out names that have no space so that
they better avoid nick names not intended for credits. Such names are
not too commonly used, plus we now give credit even to those.
Additionally: non-latin names, like Asian, don't have spaces at all so
they were also filtered out and had to be manually added which made it
an error-prone operation where Asian names eventually easily fell off by
mistake.
Emanuele Torre [Tue, 23 May 2023 09:59:59 +0000 (11:59 +0200)]
configure: fix build with arbitrary CC and LD_LIBRARY_PATH
Since ./configure and processes that inherit its environment variables
are the only callers of the run-compiler script, we can just save the
current value of the LD_LIBRARY_PATH and CC variables to another pair of
environment variables, and make run-compiler a static script that
simply restores CC and LD_LIBRARY_PATH to the saved value, and before
running the compiler.
This avoids having to inject the values of the variables in the script,
possibly causing problems if they contains spaces, quotes, and other
special characters.
Also add exports in the script just in case LD_LIBRARY_PATH and CC are
not already in the environment.
Stefan Eissing [Wed, 24 May 2023 16:48:16 +0000 (18:48 +0200)]
http2: fix EOF handling on uploads with auth negotiation
- doing a POST with `--digest` does an override on the initial request
with `Content-Length: 0`, but the http2 filter was unaware of that
and expected the originally request body. It did therefore not
send a final DATA frame with EOF flag to the server.
- The fix overrides any initial notion of post size when the `done_send`
event is triggered by the transfer loop, leading to the EOF that
is necessary.
- refs #11194. The fault did not happen in testing, as Apache httpd
never tries to read the request body of the initial request,
sends the 401 reply and closes the stream. The server used in the
reported issue however tried to read the EOF and timed out on the
request.
Reported-by: Aleksander Mazur
Fixes #11194
Cloes #11200
Stefan Eissing [Mon, 22 May 2023 11:20:51 +0000 (13:20 +0200)]
http/2: unstick uploads
- refs #11157 and #11175 where uploads get stuck or lead to RST streams
- fixes our h2 send behaviour to continue sending in the nghttp2 session
as long as it wants to. This will empty our send buffer as long as
the remote stream/connection window allows.
- in case the window is exhausted, the data remaining in the send buffer
will wait for a WINDOW_UPDATE from the server. Which is a socket event
that engages our transfer loop again
- the problem in the issue was that we did not exhaust the window, but
left data in the sendbuffer and no further socket events did happen.
The server was just waiting for us to send more.
- relatedly, there was an issue fixed that closing a stream with KEEP_HOLD
set kept the transfer from shutting down - as it should have - leading
to a timeout.
Emanuele Torre [Sun, 21 May 2023 03:01:35 +0000 (05:01 +0200)]
Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects"
This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c.
(It only keep the test case that checks redirection to an absolute URL
without hostname and CURLU_NO_AUTHORITY).
I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the
hostname only because I thought
curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting
them, and they were only not being accepted in the hostname when
curl_url_set(CURLUPART_URL) was used for a redirection.
That is not actually the case, urlapi never accepted hostnames with
spaces, and a hostname with a space in it never makes sense.
I probably misread the output of my original test when I they were
normally accepted when using CURLU_ALLOW_SPACE, and not redirecting.
Some other URL parsers seems to allow space in the host part of the URL,
e.g. both python3's urllib.parse module, and Chromium's javascript URL
object allow spaces (chromium percent escapes the spaces with %20),
(they also both ignore TABs, and other whitespace characters), but those
URLs with spaces in the hostname are useless, neither python3's requests
module nor Chromium's window.location can actually use them.
There is no reason to add support for URLs with spaces in the host,
since it was not a inconsistency bug; let's revert that patch before it
makes it into release. Sorry about that.
I also reverted the extra check for CURLU_NO_AUTHORITY since that does
not seem to be necessary, CURLU_NO_AUTHORITY already worked for
redirects.
Dan Fandrich [Wed, 17 May 2023 05:34:40 +0000 (22:34 -0700)]
runtests: handle interrupted reads from IPC pipes
These can be interrupted by signals, especially SIGINT to shut down, and
must be restarted so the IPC call arrives correctly. If the read just
returns an error instead, the IPC calling state will go out of sync and
a proper shutdown won't happen.
Stefan Eissing [Sat, 20 May 2023 10:26:04 +0000 (12:26 +0200)]
http2: upload improvements
Make send buffer smaller to have progress and "upload done" reporting
closer to reality. Fix handling of send "drain" condition to no longer
trigger once the transfer loop reports it is done sending. Also do not
trigger the send "drain" on RST streams.
Background:
- a upload stall was reported in #11157 that timed out
- test_07_33a reproduces a problem with such a stall if the
server 404s the request and RSTs the stream.
- test_07_33b verifies a successful PUT, using the parameters
from #11157 and checks success
Stefan Eissing [Fri, 19 May 2023 07:33:59 +0000 (09:33 +0200)]
ngtcp2: fix compiler warning about possible null-deref
- compiler analyzer did not include the call context for this
static function where the condition had already been checked.
- eleminating the problem by making stream a call parameter
Antoine Pitrou [Thu, 18 May 2023 19:39:05 +0000 (21:39 +0200)]
select: avoid returning an error on EINTR from select() or poll()
This was already done for the poll() and select() calls
made directly from Curl_poll(), but was missed in
Curl_wait_ms(), which is called when there are no fds
to wait on.
Daniel Stenberg [Wed, 17 May 2023 22:31:17 +0000 (00:31 +0200)]
urlapi: allow numerical parts in the host name
It can only be an IPv4 address if all parts are all digits and no more than
four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be
passed through as a host name.
Emanuele Torre [Thu, 18 May 2023 19:16:56 +0000 (21:16 +0200)]
configure: fix --help alignment
AC_ARG_ENABLE seems to only trim off whitespace from the start and end
of its help-string argument, while prepending two spaces of indentation
to all lines.
This means that the two spaces of indentation between the --enable-rtsp
and the --disable-rtsp line were not removed causing ./configure --help
to print:
Optional Features:
[...]
--enable-rtsp Enable RTSP support
--disable-rtsp Disable RTSP support
I removed the indentation to fix the issue, now it prints:
Optional Features:
[...]
--enable-rtsp Enable RTSP support
--disable-rtsp Disable RTSP support
The --enable-hsts and --disable-hsts lines had the same problems, and
have been fixed too.
Daniel Stenberg [Tue, 16 May 2023 14:51:14 +0000 (16:51 +0200)]
configure: generate a script to run the compiler
in the CURL_RUN_IFELSE macro, with LD_LIBRARY_PATH set to the value of
the configure invoke, and not the value that might be used later,
intended for the execution of the output the compiler ouputs.
For example when the compiler uses the same library (like libz) that
configure checks for.
Reported-by: Jonas Bülow
Fixes #11114
Closes #11120
Emanuele Torre [Thu, 18 May 2023 08:48:19 +0000 (10:48 +0200)]
urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects
curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing
CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl
when redirecting to an absolute URL.
Emanuele Torre [Thu, 18 May 2023 04:54:18 +0000 (06:54 +0200)]
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.
Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
line
* 1 was indented with the indentation level of the following line minus
three space (probably unintentional)
Hook the new (1.11.0 or newer) libssh2 support for setting a read timeout
into the SERVER_RESPONSE_TIMEOUT option. With this done, clients can use
the standard curl response timeout setting to also control the time that
libssh2 will wait for packets from a slow server. This is necessary to
enable use of very slow SFTP servers.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
Closes #10965
Daniel Stenberg [Thu, 11 May 2023 07:56:51 +0000 (09:56 +0200)]
headers: clear (possibly) lingering pointer in init
The "prevhead" pointer is used for the headers storage but was not
cleared correctly in init, which made it possible to act up when a
handle is reused.
Reported-by: Steve Herrell
Fixes #11101
Closes #11103
Daniel Stenberg [Sun, 7 May 2023 22:14:33 +0000 (00:14 +0200)]
tool_operate: refuse (--data or --form) and --continue-at combo
libcurl assumes that a --continue-at resumption is done to continue an
upload using the read callback and neither --data nor --form use
that and thus won't do what the user wants. Whatever the user wants
with this strange combination.
Add test 426 to verify.
Reported-by: Smackd0wn on github
Fixes #11081
Closes #11083
Daniel Stenberg [Sun, 7 May 2023 22:12:25 +0000 (00:12 +0200)]
transfer: refuse POSTFIELDS + RESUME_FROM combo
The code assumes that such a resume is wanting to continue an upload
using the read callback, and since POSTFIELDS is done without callback
libcurl will just misbehave.
This combo will make the transfer fail with CURLE_BAD_FUNCTION_ARGUMENT
with an explanation in the error message.
Reported-by: Smackd0wn on github
Fixes #11081
Closes #11083
Dan Fandrich [Sun, 30 Apr 2023 05:11:25 +0000 (22:11 -0700)]
runtests: spawn a new process for the test runner
When the -j option is given, a new process is spawned in which the test
programs are run and from which test servers are started. Only one
process can be started at once, but this is sufficient to test that the
infrastructure can isolate those functions in a new task. There should
be no visible difference between the two modes at the moment.
Dan Fandrich [Wed, 26 Apr 2023 01:03:54 +0000 (18:03 -0700)]
runtests: change runner interface to be asynchronous
Program arguments are marshalled and then written to the end of a pipe
which is later read from and the arguments unmarshalled before the
desired function is called normally. The function return values are
then marshalled and written into another pipe when is later read from
and unmarshalled before being returned to the caller.
The implementation is currently blocking but can be made non-blocking
without any changes to the API. This allows calling multiple runners
without blocking in the future.
projects / thirdparty / curl.git / log
