Stefan Eissing [Thu, 14 Aug 2025 12:12:54 +0000 (14:12 +0200)]
openssl: auto-pause on verify callback retry
When an application install its own OpenSSL verify callback and that
callback invokes `SSL_set_retry_verify()`, the transfer is automatically
paused and does not progress the connect attempt any further until
unpaused via `curl_easy_pause().
Added test758 to verify.
Ref: #18284
Original PR by @Natris
Bug: https://curl.se/mail/lib-2025-08/0012.html
Closes #18288
Viktor Szakats [Thu, 14 Aug 2025 14:45:15 +0000 (16:45 +0200)]
GHA/linux: try improving valgrind job times with cmake
Make the:
- mbedTLS valgrind job finish under 14m, vs 15m before.
- OpenSSL -O3 valgrind job finish in 14m30, vs 16m17.
- OpenSSL libssh2 valgrind job finish in 16m, vs 17m30.
- long valgrind rustls job finish 1 minute earlier, in return
for spending 30s more on the other rustls job.
Keep using autotools for the less slow valgrind job to test this combo.
Viktor Szakats [Thu, 14 Aug 2025 11:36:04 +0000 (13:36 +0200)]
runtests: assume `Time::HiRes`, drop Perl Win32 dependency
`Time::HiRes` was already used unconditionally before this patch in
`servers.pm`. This package, and functions used by runtests (`sleep` and
`gettimeofday`) are supported by the minimum Perl version required for
curl:
https://perldoc.perl.org/5.8.0/Time::HiRes
- Drop the `portable_sleep()` wrapper in favor of `Time::HiRes::sleep()`.
- Use `Time::HiRes` unconditionally in `serverhelp.pm`.
- Stop using the `Win32` package where available. It was included
to provide a Windows fallback for `Time::HiRes::sleep()`. It was never
actually called, but the dependency may have loaded `Win32.dll`, which
often appears in failed fork operations in GHA logs.
Ref: a6fed41f6f12f3b71cfe85609f02a294b972d3d3 #5054 #5034
Ref: https://github.com/curl/curl/discussions/14854
Viktor Szakats [Wed, 13 Aug 2025 08:20:10 +0000 (10:20 +0200)]
mbedtls: check for feature macros instead of version
Drop three interim macros and mbedTLS version checks in favor of feature
macros defined by mbedTLS itself.
These mbedTLS features require mbedTLS 3.6.0/3.6.1 for production.
Earlier versions may have partial/experimental support for them,
disabled by default and (AFAICS) without documented build options
to enable them.
One feature guard already used the native macro instead of the interim
one before this patch. (`MBEDTLS_SSL_SESSION_TICKETS`)
Viktor Szakats [Fri, 8 Aug 2025 22:45:34 +0000 (00:45 +0200)]
openssl: save and restore OpenSSL error queue in two functions
After merging #18228, I reviewed whether the clearing of the error queue
may interfere with preceding code. Turns out there may be a preceding
`SSL_Connect()` call.
This patch replaces the previous fix of clearing the error queue with
saving and restoring it in two functions which may be called between
the connect call and the `SSL_get_error()` call following it:
- `ossl_log_tls12_secret()`
- `Curl_ssl_setup_x509_store()`
The `ERR_set_mark()`, `ERR_pop_to_mark()` functions are present in all
supported OpenSSL and LibreSSL versions. Also in BoringSSL since its
initial commit.
OpenSSL may modify its error queue in all API calls that can fail.
Thanks-to: Viktor Dukhovni
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3167702142
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3169211739
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3169988050
Viktor Szakats [Wed, 13 Aug 2025 13:14:31 +0000 (15:14 +0200)]
GHA/linux: fix thread sanitizer error output
Replace autotools with cmake to avoid libtool wrappers that are changing
`LD_LIBRARY_PATH` in a way incompatible with the thread sanitizer.
To fix the output when the sanitizier is finding something:
```
==51718==WARNING: Can't write to symbolizer at fd 7
/usr/bin/llvm-symbolizer-18: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: no version information available (required by /usr/bin/llvm-symbolizer-18)
/usr/bin/llvm-symbolizer-18: symbol lookup error: /home/runner/openssl/lib/libcrypto.so.3: undefined symbol: __tsan_func_entry
```
Ref: https://github.com/curl/curl/actions/runs/16911402500/job/47913783729#step:39:4466
Also:
- disable memory tracker which turned out to be incompatible with
the thread sanitizer and detaching threads.
Ref: #18263 and #curl IRC.
- the job is ~30 seconds faster after this patch.
Viktor Szakats [Tue, 12 Aug 2025 07:58:13 +0000 (09:58 +0200)]
URL-SYNTAX.md: drop link to codepoints.net to pass linkcheck
The link works in a browser, but started failing the `mdlinkcheck` test:
```
check https://codepoints.net/U+00DF
FAIL
docs/URL-SYNTAX.md:199 ERROR links to missing URL https://codepoints.net/U+00DF
```
Ref: https://github.com/curl/curl/actions/runs/16902543407/job/47884625446?pr=18254#step:3:22
Dan Fandrich [Mon, 11 Aug 2025 20:36:22 +0000 (13:36 -0700)]
CI: update libstdc++ for linux-old build
actions/checkout@v5 requires a newer libstdc++ than the container
contains. Update it to a backwards-compatible version just like we
already do for libc6.
Stefan Eissing [Tue, 8 Jul 2025 07:15:43 +0000 (09:15 +0200)]
websocket: support CURLOPT_READFUNCTION
Add support for CURLOPT_READFUNCTION with WebSocket urls when *not* in
connect-only mode, e.g. when using curl_multi_perform.
Install the callback function and set CURLOPT_UPLOAD. Return
CURL_READFUNC_PAUSE when having nothing more to send and unpause the
transfer when more data is ready.
This will send the read bytes in a WebSocket BINARY frame.
Add support for this mode in the pytest "ws_data" client and have all
tests run in 'curl_ws_send/recv' and 'peform' mode as well.
Add `curl_ws_start_frame()`. Document, cover in libcurl-ws.md and
explain the READFUNCTION mode for websockets.
Stefan Eissing [Mon, 11 Aug 2025 09:39:03 +0000 (11:39 +0200)]
resolving: dns error tracing
* Add more tracing information to c-ares errors.
* remove CURL_ASYNC_SUCCESS, rename `ares->last_status` to
`ares->ares_status`. Give trace explanation for "common"
errors
* add ares "csv" information to tracing on failure
* DoH: invoke `Curl_resolver_error()` on failure to populate
error buf
Viktor Szakats [Wed, 6 Aug 2025 20:17:50 +0000 (22:17 +0200)]
cmake: optimize building examples in CI
In CI we want to ensure that examples build cleanly, but we don't want
to actually run them there. Meaning it's enough to just compile, but not
link them in CI. Saving time up to 2-4x (MSVC), and disk space up
to 1.2GB (or 8-70x).
Add a new cmake target that compiles all examples without linking them
into runnable binaries. Keep a full build for a single example to test
if it links correctly.
Also:
- CI: switch over all `curl-examples` targets to `curl-examples-build`
- GHA/linux-old: build examples in one of the cmake builds.
Viktor Szakats [Fri, 8 Aug 2025 11:07:34 +0000 (13:07 +0200)]
openssl: clear errors after a failed `d2i_X509()`
Without it, subsequent OpenSSL API calls may fail with an error caught
within the OpenSSL `d2i_X509()` (decode) call.
It was seen to happen when importing from the Windows certificate store
(e.g. with `--ca-native`), and any one of the certificates failed while
decoding, then skipped.
Behind the scene (and undocumented), the failed decode call is adding
an error to an internal OpenSSL error queue. This error is picked up
later, at the connect phase, by another OpenSSL API call, which happens
to check the error queue, without clearing it first. It made the connect
fail with the error collected earlier, while decoding the malformed and
discarded certificate.
Fix by explicitly clearing the error queue if the decode call fails.
Ref: https://docs.openssl.org/3.5/man3/d2i_X509/
`-vvvv` output before this patch:
```
[0-0] == Info: successfully imported Windows ROOT store
[0-0] == Info: successfully imported Windows CA store
[0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=1
[0-0] == Info: TLS connect error: error:068000DD:asn1 encoding routines::illegal padding
[0-0] == Info: [SSL] cf_connect() -> 35, done=0
```
Mainline OpenSSL (as of 3.5.2) and quictls (as of 3.3.0) are affected.
LibreSSL is not affected. (I did not test BoringSSL and AWS-LC)
Assisted-by: Stefan Eissing Reported-by: Michał Petryka
Fixes #18190
Daniel Stenberg [Thu, 7 Aug 2025 21:11:10 +0000 (23:11 +0200)]
tool_operate: cleanups
- move the state struct from config to global. It is used as a single
instance anyway so might as well be a single one to save memory.
- simplify and combine several conditions
- set default retry delay inititally
- use better struct field names to make it easier to understand their
purposes
- remove the state->outfiles field as it was not necessary
- remove superfluous glob cleanup call
- move conditions around to remove an indent level
- move the ->url NULL check
Takes single_transfer()'s complexity score down from 78 to 68.
Deduce that the transfer response expects headers by the protocol
handler implementing `write_resp_hd` callback. This eleminates the
`getheader` parameter in the `Curl_xfer_setup_*()` methods.
Add an implementation to RTSP for `write_resp_hd`, joining the HTTP
protocol in the only handlers having it.
Reverse the default of request's `header` bit that signals that headers
are expected. Default is now FALSE, set to TRUE when setting up the
transfer by presence of `write_resp_hd` in the protocol handler.
Daniel Stenberg [Wed, 6 Aug 2025 21:18:46 +0000 (23:18 +0200)]
curl: make global truly global
The GlobalConfig only exists in a single instance and it has worked like
this since the dawn of time. It is about time we stop passing around
pointers to what was already essentially a global object and instead
just use a... global.
Viktor Szakats [Wed, 6 Aug 2025 17:21:16 +0000 (19:21 +0200)]
cmake: make the ExternalProject test work
By micromanaging the project dependency and its inclusion into the test
project. It feels like an awkward construct, but perhaps better than
nothing.
It's also fragile because it's a static build with no assistance from
the external project (curl in this case). Mitigated in test by disabling
all dependencies and some features.
Since there is no special core cmake logic to be tested here, in CI
the test is tested really. To keep CI jobs at minimum, only add 3 of
them, taking 42s in total. (All 6 would take 270s.)
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.
Daniel Stenberg [Wed, 6 Aug 2025 07:03:31 +0000 (09:03 +0200)]
multi: fix bad splay management
The splay tree is a tree where each easy handle can be added *once*. The
expire time for that node is the closest expire time for that easy
handle.
Easy handles can however have more expire times queued up, so when the
node is removed from the splay tree because it is the next in line to
take care of, we must check if there is another expire time in the queue
and then add the node back into the splay.
Failing to do the later part, the calling of add_next_timeout after
Curl_splaygetbest, would leave the state.expiretime on the previous time
stamp, which when could make the next call to Curl_splaygetbest use the
wrong time stamp and get a wrong node out, causing trouble.
Reported-by: letshack9707 on hackerone
Closes #18201
Stefan Eissing [Wed, 6 Aug 2025 09:56:08 +0000 (11:56 +0200)]
lib: xfer_setup simplify
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.
Daniel Stenberg [Tue, 5 Aug 2025 14:00:20 +0000 (16:00 +0200)]
tool_urlglob: polish, cleanups, improvements
- assert instead of printing "internal error" for unlikely events
- avoid allocating the main struct
- convert globerror() from macro to function
- renames to shorter and clearer names
- malloc + copy => memdup0
- change buffer handling to dynbuf
- realloc to handle more globs, but use less memory for few
Stefan Eissing [Tue, 5 Aug 2025 11:11:53 +0000 (13:11 +0200)]
lib: replace conn [write]sockfd members by index
The `connectdata` members `sockfd` and `writesockfd` needed to by either
CURL_SOCKET_BAD or a copy of one of `conn->sock[2]`. When equal to one,
that index was used to send/recv/poll the proper socket or connection
filter chain.
Replace those with `send_idx` and `recv_idx` which are either -1, 0 or 1
to indicate which socket/filter to send/receive on.
Stefan Eissing [Tue, 5 Aug 2025 11:37:12 +0000 (13:37 +0200)]
vtls: set seen http version on successful ALPN
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.
Fixes #18177 Reported-by: IoannisGS on github
Closes #18181
Daniel Stenberg [Tue, 5 Aug 2025 06:47:31 +0000 (08:47 +0200)]
setopt: add helper functions to setopt_long()
- Consistently keep options within ranges
- Reduce the maximum maxredirs value to fit a signed short
- Removed comments as the place to document the options is not here
Stefan Eissing [Wed, 23 Jul 2025 07:18:59 +0000 (09:18 +0200)]
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:
* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever
Add documentation for functions and info enums.
Add use in the curl command line tool to replace two static
variables counting the same "from the outside".
Stefan Eissing [Mon, 4 Aug 2025 14:17:37 +0000 (16:17 +0200)]
lib: replace `getsock()` logic with pollsets
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Viktor Szakats [Mon, 4 Aug 2025 13:41:41 +0000 (15:41 +0200)]
GHA/windows: give more time for Ubuntu installs
Recently sometimes the Ubuntu package repository is very slow to access.
Remove the time limit for the install step, and bump the total limit,
aligning with Linux jobs.
In most cases the `install packages` steps takes 15-25 seconds.
Sometimes this goes up to 30-45 minutes.
Reported-by: nevakrien on github
Bug: https://github.com/curl/curl/discussions/14854#discussioncomment-13988574
Closes #18163
Daniel Stenberg [Mon, 4 Aug 2025 12:15:03 +0000 (14:15 +0200)]
hostip: do DNS cache pruning in milliseconds
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.
Viktor Szakats [Sat, 2 Aug 2025 11:15:28 +0000 (13:15 +0200)]
runtests: add `--ci` option, show `Env:` only when non-empty
To reduce log noise in local test runs:
- move the `buildinfo.txt` dump and header info lines
`OS:`, `Perl:`, `diff:` behind a `--ci` `runtests.pl` option.
- enable this option for the CI test targets.
- hide `Env:` header info line if empty.
- merge `Env:` output into a single `logmsg()` call.
projects / thirdparty / curl.git / log
