Viktor Szakats [Wed, 6 Aug 2025 20:17:50 +0000 (22:17 +0200)]
cmake: optimize building examples in CI
In CI we want to ensure that examples build cleanly, but we don't want
to actually run them there. Meaning it's enough to just compile, but not
link them in CI. Saving time up to 2-4x (MSVC), and disk space up
to 1.2GB (or 8-70x).
Add a new cmake target that compiles all examples without linking them
into runnable binaries. Keep a full build for a single example to test
if it links correctly.
Also:
- CI: switch over all `curl-examples` targets to `curl-examples-build`
- GHA/linux-old: build examples in one of the cmake builds.
Viktor Szakats [Fri, 8 Aug 2025 11:07:34 +0000 (13:07 +0200)]
openssl: clear errors after a failed `d2i_X509()`
Without it, subsequent OpenSSL API calls may fail with an error caught
within the OpenSSL `d2i_X509()` (decode) call.
It was seen to happen when importing from the Windows certificate store
(e.g. with `--ca-native`), and any one of the certificates failed while
decoding, then skipped.
Behind the scene (and undocumented), the failed decode call is adding
an error to an internal OpenSSL error queue. This error is picked up
later, at the connect phase, by another OpenSSL API call, which happens
to check the error queue, without clearing it first. It made the connect
fail with the error collected earlier, while decoding the malformed and
discarded certificate.
Fix by explicitly clearing the error queue if the decode call fails.
Ref: https://docs.openssl.org/3.5/man3/d2i_X509/
`-vvvv` output before this patch:
```
[0-0] == Info: successfully imported Windows ROOT store
[0-0] == Info: successfully imported Windows CA store
[0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=1
[0-0] == Info: TLS connect error: error:068000DD:asn1 encoding routines::illegal padding
[0-0] == Info: [SSL] cf_connect() -> 35, done=0
```
Mainline OpenSSL (as of 3.5.2) and quictls (as of 3.3.0) are affected.
LibreSSL is not affected. (I did not test BoringSSL and AWS-LC)
Assisted-by: Stefan Eissing Reported-by: Michał Petryka
Fixes #18190
Daniel Stenberg [Thu, 7 Aug 2025 21:11:10 +0000 (23:11 +0200)]
tool_operate: cleanups
- move the state struct from config to global. It is used as a single
instance anyway so might as well be a single one to save memory.
- simplify and combine several conditions
- set default retry delay inititally
- use better struct field names to make it easier to understand their
purposes
- remove the state->outfiles field as it was not necessary
- remove superfluous glob cleanup call
- move conditions around to remove an indent level
- move the ->url NULL check
Takes single_transfer()'s complexity score down from 78 to 68.
Deduce that the transfer response expects headers by the protocol
handler implementing `write_resp_hd` callback. This eleminates the
`getheader` parameter in the `Curl_xfer_setup_*()` methods.
Add an implementation to RTSP for `write_resp_hd`, joining the HTTP
protocol in the only handlers having it.
Reverse the default of request's `header` bit that signals that headers
are expected. Default is now FALSE, set to TRUE when setting up the
transfer by presence of `write_resp_hd` in the protocol handler.
Daniel Stenberg [Wed, 6 Aug 2025 21:18:46 +0000 (23:18 +0200)]
curl: make global truly global
The GlobalConfig only exists in a single instance and it has worked like
this since the dawn of time. It is about time we stop passing around
pointers to what was already essentially a global object and instead
just use a... global.
Viktor Szakats [Wed, 6 Aug 2025 17:21:16 +0000 (19:21 +0200)]
cmake: make the ExternalProject test work
By micromanaging the project dependency and its inclusion into the test
project. It feels like an awkward construct, but perhaps better than
nothing.
It's also fragile because it's a static build with no assistance from
the external project (curl in this case). Mitigated in test by disabling
all dependencies and some features.
Since there is no special core cmake logic to be tested here, in CI
the test is tested really. To keep CI jobs at minimum, only add 3 of
them, taking 42s in total. (All 6 would take 270s.)
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.
Daniel Stenberg [Wed, 6 Aug 2025 07:03:31 +0000 (09:03 +0200)]
multi: fix bad splay management
The splay tree is a tree where each easy handle can be added *once*. The
expire time for that node is the closest expire time for that easy
handle.
Easy handles can however have more expire times queued up, so when the
node is removed from the splay tree because it is the next in line to
take care of, we must check if there is another expire time in the queue
and then add the node back into the splay.
Failing to do the later part, the calling of add_next_timeout after
Curl_splaygetbest, would leave the state.expiretime on the previous time
stamp, which when could make the next call to Curl_splaygetbest use the
wrong time stamp and get a wrong node out, causing trouble.
Reported-by: letshack9707 on hackerone
Closes #18201
Stefan Eissing [Wed, 6 Aug 2025 09:56:08 +0000 (11:56 +0200)]
lib: xfer_setup simplify
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.
Daniel Stenberg [Tue, 5 Aug 2025 14:00:20 +0000 (16:00 +0200)]
tool_urlglob: polish, cleanups, improvements
- assert instead of printing "internal error" for unlikely events
- avoid allocating the main struct
- convert globerror() from macro to function
- renames to shorter and clearer names
- malloc + copy => memdup0
- change buffer handling to dynbuf
- realloc to handle more globs, but use less memory for few
Stefan Eissing [Tue, 5 Aug 2025 11:11:53 +0000 (13:11 +0200)]
lib: replace conn [write]sockfd members by index
The `connectdata` members `sockfd` and `writesockfd` needed to by either
CURL_SOCKET_BAD or a copy of one of `conn->sock[2]`. When equal to one,
that index was used to send/recv/poll the proper socket or connection
filter chain.
Replace those with `send_idx` and `recv_idx` which are either -1, 0 or 1
to indicate which socket/filter to send/receive on.
Stefan Eissing [Tue, 5 Aug 2025 11:37:12 +0000 (13:37 +0200)]
vtls: set seen http version on successful ALPN
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.
Fixes #18177 Reported-by: IoannisGS on github
Closes #18181
Daniel Stenberg [Tue, 5 Aug 2025 06:47:31 +0000 (08:47 +0200)]
setopt: add helper functions to setopt_long()
- Consistently keep options within ranges
- Reduce the maximum maxredirs value to fit a signed short
- Removed comments as the place to document the options is not here
Stefan Eissing [Wed, 23 Jul 2025 07:18:59 +0000 (09:18 +0200)]
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:
* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever
Add documentation for functions and info enums.
Add use in the curl command line tool to replace two static
variables counting the same "from the outside".
Stefan Eissing [Mon, 4 Aug 2025 14:17:37 +0000 (16:17 +0200)]
lib: replace `getsock()` logic with pollsets
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Viktor Szakats [Mon, 4 Aug 2025 13:41:41 +0000 (15:41 +0200)]
GHA/windows: give more time for Ubuntu installs
Recently sometimes the Ubuntu package repository is very slow to access.
Remove the time limit for the install step, and bump the total limit,
aligning with Linux jobs.
In most cases the `install packages` steps takes 15-25 seconds.
Sometimes this goes up to 30-45 minutes.
Reported-by: nevakrien on github
Bug: https://github.com/curl/curl/discussions/14854#discussioncomment-13988574
Closes #18163
Daniel Stenberg [Mon, 4 Aug 2025 12:15:03 +0000 (14:15 +0200)]
hostip: do DNS cache pruning in milliseconds
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.
Viktor Szakats [Sat, 2 Aug 2025 11:15:28 +0000 (13:15 +0200)]
runtests: add `--ci` option, show `Env:` only when non-empty
To reduce log noise in local test runs:
- move the `buildinfo.txt` dump and header info lines
`OS:`, `Perl:`, `diff:` behind a `--ci` `runtests.pl` option.
- enable this option for the CI test targets.
- hide `Env:` header info line if empty.
- merge `Env:` output into a single `logmsg()` call.
Viktor Szakats [Fri, 1 Aug 2025 22:18:12 +0000 (00:18 +0200)]
configure: if no perl, disable unity and shell completion, related tidy ups
Also:
- GHA/linux, macos: test `install` with autotools too.
- GHA/linux, macos: enable fish and zsh completion in an autotools job.
- scripts: drop dynamic perl checks redundant after this patch.
- scripts: drop two interim variables.
- autotools: make `build-certs` target check perl first.
- autotools: replace `$(PERL)` with `@PERL@` to match the pattern used
in most automake scripts. For consistency. This makes `PERL` be
defined at configure-time, as opposed to make-time, for these cases
now.
Stefan Eissing [Sat, 2 Aug 2025 10:55:11 +0000 (12:55 +0200)]
unit2600: add another case
Add a case with 1 ipv4 and 3 ipv6 and check that all are attempted with
the correct minimum duration before failures. To check that more ipv6
than ipv4 lead to the correct behaviour.
Viktor Szakats [Sat, 2 Aug 2025 11:10:05 +0000 (13:10 +0200)]
cmake: re-add simple test target, and name it `tests`
Before this patch there was no (easy) way to run tests without various
verbose options, hindering readability of the output due its length.
Unfortunately CMake reserves `test` as a target name, which was
the reason for its removal earlier. Re-add it with the name `tests`,
to have the functionality without the name collision.
Also add a `tests` target as an alias to `test` for autotools.
Viktor Szakats [Fri, 1 Aug 2025 10:49:40 +0000 (12:49 +0200)]
docs: add CURLOPT type change history, drop casts where present
Some CURLOPT constants defined in the curl public headers were initially
enums (= ints), or macros with bare numeric values. Recent curl releases
upgraded them to `long` constants, to make them pass correctly to
`curl_easy_setop()` by default, i.e. without requiring a `(long)` cast.
This patch drops such casts from the examples embedded in the docs. At
the same time it documents which curl release made them `long` types,
to keep them useful when working with previous libcurl versions.
Also:
- drop a `(long)` cast that was never necessary.
- CURLOPT_ALTSVC_CTRL.md: bump local copy of macros to long.
- test1119: make it ignore symbols ending with an underscore, to skip
wildcard, e.g. `**CURLAUTH_***`.
Viktor Szakats [Thu, 31 Jul 2025 14:38:34 +0000 (16:38 +0200)]
build: if no perl, fix to use the pre-built hugehelp, if present
- cmake: Before this patch a missing perl disabled the curl manual.
After this patch, it automatically picks up a pre-built hugehelp,
if present (= when building from a release tarball).
Follow-up to 0035ff45c59437db1a8ffe960b40905c8582b149 #16081
- autotools: Rework behavior when perl is missing. Before this patch
it caused a hard error when docs/manual/ca-embed were enabled.
Of these, docs were enabled by default. After this patch, doc
generation is automatically skipped, with a warning. Manual generation
falls back to using a pre-built hugehelp, or a stub if that's missing.
CA-embed is automatically skipped, with a warning.
Slight difference from cmake: When built with no perl and no pre-built
hugehelp, the manual is enabled, but the content is empty; with cmake
it's disabled proper.
Follow-up to 137aecfbf1e933e7228a4f62cc977eabffb2afbf #13514
Follow-up to 541321507e386744c3ea1200cc83cc98338e27d7 #12857
Also:
- GHA/distcheck: verify if the pre-built hugehelp makes it into curl.
- GHA/distcheck: streamline disabling perl with autotools.
- GHA/distcheck: `--disable-docs` no longer required to build without perl.
Passing it also implicitly disables the curl manual, which is
undesired in these tests.
- cmake: drop redundant `HAVE_MANUAL_TOOLS` interim variable.
- cmake: move two `set()` lines close to their counterparts.
Viktor Szakats [Fri, 1 Aug 2025 14:27:15 +0000 (16:27 +0200)]
CURLOPT: bump remaining macros to `long`
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
Viktor Szakats [Fri, 1 Aug 2025 14:47:41 +0000 (16:47 +0200)]
CURLOPT: bump `CURLPROTO_*` macros to `long`
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
Viktor Szakats [Fri, 1 Aug 2025 14:43:33 +0000 (16:43 +0200)]
CURLOPT: bump `CURL_SSLVERSION_*` macros to `long`
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
Viktor Szakats [Fri, 1 Aug 2025 15:35:42 +0000 (17:35 +0200)]
CURLOPT: bump `CURLWS_NOAUTOPONG`, `CURLWS_RAW_MODE` macros to `long`
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLWS_NOAUTOPONG
- CURLWS_RAW_MODE
Also:
- keep existing cast within the documentation to make sure it applies
to older curl versions as well.
Stefan Eissing [Fri, 1 Aug 2025 12:55:52 +0000 (14:55 +0200)]
openssl: check SSL_write() length on retries
When an SSL_write() blocks we need to retry it with the
same length as before or stupid OpenSSL freaks out. Remember
it, limit any longer sends and fail shorter ones.
Fixes #18121 Reported-by: adamse on github
Closes #18132
Stefan Eissing [Wed, 23 Jul 2025 09:21:36 +0000 (11:21 +0200)]
tls: CURLINFO_TLS_SSL_PTR testing
Add tests of CURLINFO_TLS_SSL_PTR and its returned value in test client
'hx-download'. Use obtained pointer to look up the negotiated TLS
version.
Update manpage of CURLINFO_TLS_SSL_PTR to also describe the behaviour of
wolfSSL similar to OpenSSL. Fix the wolfSSl implementation for TCP to
behave like that. Update the QUIC queries.
Fix rustls `get_internals()` to return the rustls_connection* and not
the address of the pointer.
Viktor Szakats [Thu, 31 Jul 2025 05:44:35 +0000 (07:44 +0200)]
schannel: not supported with UWP, drop redundant code
Schannel is not supported by UWP. SSPI is also required by Schannel in
curl, and SSPI also isn't supported by UWP.
mingw-w64 is able to create such build regardless (my guess: due to API
parts not accurately marked as UWP-only), but the binary is unlikely
to work. With MSVC the failure happens at build-time.
Drop all UWP-related logic, including two related feature checks, that
can now be permanently enabled.
Also:
- build: show fatal error for Schannel in UWP mode.
- build: do not allow enabling SSPI in UWP mode.
- drop undocumented option `DISABLE_SCHANNEL_CLIENT_CERT`. Added without
mention in an unrelated commit. The PR text says to save size. On x64
this is 0.3%, or 4KB out of 1.3MB. The tiny gain doesn't justify
an extra build variant. Ref: 8beff4355956e3d18ceb3afc21c1f3edec82543c
- move `MPROTO_SCHANNEL_CERT_SHARE_KEY` closer to its use.
- replace commented block with `#if 0`.
projects / thirdparty / curl.git / log
