Stefan Eissing [Fri, 1 Dec 2023 12:50:32 +0000 (13:50 +0100)]
lib: replace readwrite with write_resp
This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.
The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.
Curl's protocol handler `readwrite()` method been changed:
The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:
* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
finished
This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.
The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.
HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.
The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:
All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.
All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.
Daniel Stenberg [Thu, 11 Jan 2024 13:11:19 +0000 (14:11 +0100)]
configure: when enabling QUIC, check that TLS supports QUIC
Most importantly perhaps is when using OpenSSL that the used
build/flavor has the QUIC API: the vanilla OpenSSL does not, only
BoringSSL, libressl, AWS-LC and quictls do.
Sergey Markelov [Thu, 11 Jan 2024 00:23:00 +0000 (17:23 -0700)]
multi: remove total timer reset in file_do() while fetching file://
The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts
with resetting the timer that is a start point for further multi states.
If file://, MSTATE_DO calls file_do() that should not reset the total
timer. Otherwise, the total time is always less than the pre-transfer
and the start transfer times.
Stefan Eissing [Tue, 9 Jan 2024 08:29:34 +0000 (09:29 +0100)]
multi: pollset adjust, init with FIRSTSOCKET during connect
- `conn->sockfd` is set by `Curl_setup_transfer()`, but that
is called *after* the connection has been established
- use `conn->sock[FIRSTSOCKET]` instead
Stefan Eissing [Fri, 5 Jan 2024 11:28:09 +0000 (12:28 +0100)]
transfer: adjust_pollset improvements
- let `multi_getsock()` initialize the pollset in what the
transfer state requires in regards to SEND/RECV
- change connection filters `adjust_pollset()` implementation
to react on the presence of POLLIN/-OUT in the pollset and
no longer check CURL_WANT_SEND/CURL_WANT_RECV
- cf-socket will no longer add POLLIN on its own
- http2 and http/3 filters will only do adjustments if the
passed pollset wants to POLLIN/OUT for the transfer on
the socket. This is similar to the HTTP/2 proxy filter
and works in stacked filters.
Daniel Stenberg [Tue, 2 Jan 2024 12:32:18 +0000 (13:32 +0100)]
tool: prepend output_dir in header callback
When Content-Disposition parsing is used and an output dir is prepended,
make sure to store that new file name correctly so that it can be used
for setting the file timestamp when --remote-time is used.
Extended test 3012 to verify.
Co-Authored-by: Jay Satiro Reported-by: hgdagon on github
Fixes #12614
Closes #12617
Viktor Szakats [Mon, 1 Jan 2024 23:31:47 +0000 (23:31 +0000)]
schannel: fix `-Warith-conversion` gcc 13 warning
```
lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' may change the sign of the result [-Warith-conversion]
1201 | *extension_len = *list_len +
| ^
```
Jay Satiro [Tue, 26 Dec 2023 06:55:54 +0000 (01:55 -0500)]
quiche: return CURLE_HTTP3 on send to invalid stream
Prior to this change if a send failed on a stream in an invalid state
(according to quiche) and not marked as closed (according to libcurl)
then the send function would return CURLE_SEND_ERROR.
We already have similar code for ngtcp2 to return CURLE_HTTP3 in this
case.
Caught by test test_07_upload.py: test_07_22_upload_parallel_fail.
Dan Fandrich [Fri, 29 Dec 2023 06:26:02 +0000 (22:26 -0800)]
CI: Fix use of any-glob-to-all-files in the labeler
Despite its name, this atom acts like one-glob-to-all-files and a
different syntax with braces must be used to get
any-glob-to-all-files semantics. Unfortunately, this makes the file
completely unreadable.
Jay Satiro [Thu, 28 Dec 2023 00:01:46 +0000 (19:01 -0500)]
system_win32: fix a function pointer assignment warning
- Use CURLX_FUNCTION_CAST to suppress a function pointer assignment
warning.
a6bbc87f added lookups of some Windows API functions and then cast them
like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn
about that as breaking strict-aliasing rules so this PR changes those
assignments to use CURLX_FUNCTION_CAST.
Jay Satiro [Sat, 23 Dec 2023 21:45:53 +0000 (16:45 -0500)]
verify-examples.pl: fail verification on unescaped backslash
- Check that all backslashes in EXAMPLE are properly escaped.
eg manpage must always use `\\n` never `\n`.
This is because the manpage requires we always double blackslash to show
a single backslash. Prior to this change an erroneous single backslash
would pass through and compile even though it would not show correctly
in the manpage.
Co-authored-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/12588
Daniel Stenberg [Wed, 27 Dec 2023 08:28:48 +0000 (09:28 +0100)]
getinfo: CURLINFO_QUEUE_TIME_T
Returns the time, in microseconds, during which this transfer was held
in a waiting queue before it started "for real". A transfer might be put
in a queue if after getting started, it cannot create a new connection
etc due to set conditions and limits imposed by the application.
Stefan Eissing [Fri, 22 Dec 2023 11:27:59 +0000 (12:27 +0100)]
transfer: fix upload rate limiting, add test cases
- add test cases for rate limiting uploads for all
http versions
- fix transfer loop handling of limits. Signal a re-receive
attempt only on exhausting maxloops without an EAGAIN
- fix `data->state.selectbits` forcing re-receive to also
set re-sending when transfer is doing this.
Reported-by: Karthikdasari0423 on github
Fixes #12559
Closes #12586
- there seems to be a code path that cleans up easy handles without
triggering DONE or DETACH events to the connection filters. This
would explain wh nghttp2 still holds stream user data
- add GOOD check to easy handle used in on_close_callback to
prevent crashes, ASSERTs in debug builds.
- NULL the stream user data early before submitting RST
- add checks in on_stream_close() to identify UNGOOD easy handles
Daniel Stenberg [Fri, 22 Dec 2023 08:51:20 +0000 (09:51 +0100)]
mprintf: overhaul and bugfixes
In a test case using lots of snprintf() calls using many commonly used
%-codes per call, this version is around 30% faster than previous
version.
It also fixes the #12561 bug which made it not behave correctly when
given unknown %-sequences. Fixing that flaw required a different take on
the problem, which resulted in the new two-arrays model.
lib557: extended - Verify the #12561 fix and test more printf features
unit1398: fix test: It used a <num>$ only for one argument, which is not
supported.
Viktor Szakats [Thu, 21 Dec 2023 03:59:20 +0000 (03:59 +0000)]
appveyor: replace PowerShell with bash + parallel autotools
PowerShell works (after a steep development curve), but one property of
it stuck and kept causing unresolvable usability issues: With
`$ErrorActionPreference=Stop`, it does abort on failures, but shows only
the first line of the error message. In `Continue` mode, it shows the
full error message, but doesn't stop on all errors. Another issue is
PowerShell considering any stderr output as if the command failed (this
has been improved in 7.2 (2021-Nov), but fixed versions aren't running
in CI and will not be for a long time in all test images.)
Thus, we're going with bash.
Also:
- use `-j2` with autotools tests, making them finish 5-15 minutes per
job faster.
- omit `POSIX_PATH_PREFIX`.
- use `WINDIR`.
- prefer forward slashes.
projects / thirdparty / curl.git / log
