Viktor Szakats [Sat, 29 Nov 2025 15:07:59 +0000 (16:07 +0100)]
lib/sendf.h: forward declare two structs
To fix non-unity builds using certain header orders (seen in ntlm.c with
the include order changed):
```
lib/vauth/../sendf.h:117:27: error: ‘struct Curl_cwriter’ declared inside parameter list will not be visible outside of this definition or declaration [-Werror]
117 | struct Curl_cwriter *writer);
| ^~~~~~~~~~~~
lib/vauth/../sendf.h:215:54: error: ‘struct Curl_creader’ declared inside parameter list will not be visible outside of this definition or declaration [-Werror]
215 | CURLcode (*do_init)(struct Curl_easy *data, struct Curl_creader *reader);
| ^~~~~~~~~~~~
[...]
```
Ref: https://github.com/curl/curl/actions/runs/19785420705/job/56691185397?pr=19760
Viktor Szakats [Thu, 27 Mar 2025 00:15:16 +0000 (01:15 +0100)]
cmake: define dependencies as `IMPORTED` interface targets
Rework the way curl's custom Find modules advertise their properties.
Before this patch, Find modules returned detected dependency properties
(header dirs, libs, libdirs, C flags, etc.) via global variables. curl's
main `CMakeLists.txt` copied their values into global lists, which it
later applied to targets. This solution worked internally, but it was
unsuited for the public, distributed `CURLConfig.cmake` and publishing
curl's Find modules with it, due to polluting the namespace of consumer
projects. It's also impractical to apply the many individual variables
to every targets depending on libcurl.
To allow using Find modules in consumer projects, this patch makes them
define as imported interface targets, named `CURL::<dependency>`. Then
store dependency information as target properties. It avoids namespace
pollution and makes the dependency information apply automatically
to all targets using `CURL::libcurl_static`.
Find modules continue to return `*_FOUND` and `*_VERSION` variables.
For dependencies detected via `pkg-config`, CMake 3.16+ is recommended.
Older CMake versions have a varying degree of support for
propagating/handling library directories. This may cause issues in envs
where dependencies reside in non-system locations and detected via
`pkg-config` (e.g. macOS + Homebrew). Use `CURL_USE_PKGCONFIG=OFF`
to fix these issues. Or upgrade to newer CMake, or link libcurl
dynamically.
Also:
- re-enable `pkg-config` for old cmake `find_library()` integration
tests.
- make `curlinfo` build after these changes.
- distribute local Find modules.
- export the raw list of lib dependencies via `CURL_LIBRARIES_PRIVATE`.
- `CURLconfig.cmake`: use curl's Find modules to detect dependencies in
the consumer env.
- add custom property to target property debug function.
- the curl build process no longer modifies `CMAKE_C_FLAGS`.
Follow-up to e86542038dda88dadf8959584e803895f979310c #17047
Daniel Stenberg [Fri, 28 Nov 2025 16:16:31 +0000 (17:16 +0100)]
memdebug: buffer output data
Instead of writing each line to file immediately, this now stores them
in an in-memory buffer until that gets full or curl exits. To make it
run faster and write to file less often.
Stefan Eissing [Fri, 28 Nov 2025 11:49:16 +0000 (12:49 +0100)]
ssh: tracing and better pollset handling
Remove connection member `waitfor` and keep it in the SSH connection
meta. Add `ssh` to supported tracing features, convert many DEBUGF
printgs to traces.
Viktor Szakats [Fri, 28 Nov 2025 14:16:08 +0000 (15:16 +0100)]
GHA/curl-for-win: drop WINE install, do not run curl after build
To reduce to amount of Debian packages to install, which hopefully
removes some flakiness due to sometimes very slow Azure package
distro servers. Possible also making these jobs finish 20s faster.
Viktor Szakats [Wed, 8 Oct 2025 00:33:19 +0000 (02:33 +0200)]
build: stop overriding standard memory allocation functions
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Viktor Szakats [Thu, 27 Nov 2025 22:23:18 +0000 (23:23 +0100)]
appveyor: add support for using custom CMake versions
To allow more flexibility and not be limited by defaults offered by
the runner machines:
- Visual Studio 2013: CMake 3.12.2
- Visual Studio 2015, 2017: CMake 3.16.2
Ref: https://www.appveyor.com/docs/windows-images-software/
Start using 3.18.4, 3.19.8, 3.20.6 in older VS jobs to add variations.
Viktor Szakats [Thu, 27 Nov 2025 12:44:27 +0000 (13:44 +0100)]
GHA/http3-linux: fix broken h3 server in non-openssl jobs, for more pytests
It also revealed 3 failing earlydata tests with two backends on Linux,
seen earlier on macOS:
```
LibreSSL before: 571 passed, 141 skipped in 45.34s
LibreSSL after: 736 passed, 95 skipped in 68.08s
aws-lc before: 571 passed, 141 skipped in 78.87s
aws-lc after: 736 passed, 95 skipped in 66.71s
BoringSSL before: 511 passed, 201 skipped in 46.47s
BoringSSL after: 676 passed, 155 skipped in 63.96s
GnuTLS before: 515 passed, 197 skipped in 48.31s
GnuTLS after: 688 passed, 140 skipped in 67.79s (3 failed)
wolfSSL before: 541 passed, 171 skipped in 52.49s
wolfSSL after: 714 passed, 114 skipped in 83.84s (3 failed)
OpenSSL before: 757 passed, 74 skipped in 65.43s
OpenSSL after: 757 passed, 74 skipped in 65.06s
OpenSSL-quic before: 741 passed, 90 skipped in 62.85s
OpenSSL-quic after: 741 passed, 90 skipped in 57.20s
Stefan Eissing [Thu, 27 Nov 2025 12:18:09 +0000 (13:18 +0100)]
curlx_base64_encode: use uint8_t* for input
Change `inputbuff` parameter from `const char *` to `const uint8_t *` to
reflect the binary nature of the input bytes. Half the code was casting
unsigned char to signed already in calling.
Stefan Eissing [Wed, 26 Nov 2025 13:05:46 +0000 (14:05 +0100)]
ip_quadruple/proxy: make port uint16_t
Make `port` member in these struct of type `uint16_t`.
add `uint8_t transport` to `struct ip_quadruple
Define TRNSPRT_NONE as 0. By assigning a valid transport only on a
successful connection, it is clear when the ip_quadruple members are
valid. Also, for transports not involving ports, the getinfos for
`CURLINFO_PRIMARY_PORT` and `CURLINFO_LOCAL_PORT` will now always return
-1.
Make all `transport` members and parameters of type `uint8_t`.
Document the return value of `CURLINFO_LOCAL_PORT` and
`CURLINFO_PRIMARY_PORT` in this regard. Add tests that writeout stats
report ports correctly.
Stefan Eissing [Mon, 3 Nov 2025 12:12:50 +0000 (13:12 +0100)]
conncontrol: reuse handling
Add protocol handler flag `PROTOPT_CONN_REUSE` to indicate that the
protocol allows reusing connections for other tranfers. Add that
to all handlers that support it.
Create connections with `conn->bits.close = FALSE` and remove all
the `connkeep()` calls in protocol handlers setup/connect implementations.
`PROTOPT_CONN_REUSE` assures that the default behaviour applies
at the end of a transfer without need to juggle the close bit.
`conn->bits.close` now serves as an additional indication that a
connection cannot be reused. Only protocol handles that allow
reuse need to set it to override the default behaviour.
Remove all `connclose()` and `connkeep()` calls from connection
filters. Filters should not modify connection flags. They are
supposed to run in eyeballing situations where a filter is just
one of many determining the outcome.
Fix http response header handling to only honour `Connection: close`
for HTTP/1.x versions.
Stefan Eissing [Thu, 27 Nov 2025 09:23:43 +0000 (10:23 +0100)]
vquic: do_sendmsg full init
When passing a `msg_ctrl` to sendmsg() as part of GSO handling, zero the
complete array. This fixes any false positives by valgrind that complain
about uninitialised memory, even though the kernel only ever accesses
the first two bytes.
Reported-by: Aleksei Bavshin
Fixes #19714
Closes #19715
BANADDA [Sat, 15 Nov 2025 02:08:10 +0000 (02:08 +0000)]
examples/multi-uv: fix invalid req->data access
The on_uv_timeout callback was trying to access req->data as
a curl_context pointer, but uv.timeout.data was never initialized,
making it always NULL. This rendered the code inside the if(context)
block unreachable.
Viktor Szakats [Tue, 25 Nov 2025 01:34:26 +0000 (02:34 +0100)]
GHA/http3-linux: build nettle manually for GnuTLS 3.8.11+
GnuTLS 3.8.11 started requiring a nettle version new enough to be
missing from Ubuntu LTS released a year ago. To keep up testing it,
build nettle from source. Besides the necessary one time effort this
has the downside that nettle updates now need to be done manually
a couple of times per year when renovate detects one. (if I got the
renovate formula correct to catch the tag format).
Also:
- switch the local GnuTLS build to use the release tarball instead of
the Git repo and calling the script `bootstrap`. The script could
potentially download source code using the cleartext `git:` protocol.
It's also downloading lots of content, including a full OpenSSL repo.
Stefan Eissing [Wed, 19 Nov 2025 10:54:36 +0000 (11:54 +0100)]
multi: simplify admin handle processing
Fold the special connection pool shutdown handling in multi the things
the admin handle cares about. Add the admin handle to the 'process'
bitset, deduce it from the 'running' count.
The admin handle is the processed like any other transfer, but has a
special case in `multi_runsingle()`. Simplifies all other multi
processing parts.
Stefan Eissing [Tue, 25 Nov 2025 09:00:23 +0000 (10:00 +0100)]
lib: timer stats improvements
* move the TIMER_POSTQUEUE to the time a connection is chosen,
so that TIMER_NAMELOOKUP always happens afterwards
* client writer: do not trigger TIMER_STARTTRANSFER on CLIENTWRITE_INFO
as ftp and other pingpong protocols write that before starting anything
that is the tranfer itself
* Elimnating debug trancing of "closed stream/connection - bailing"
as confusing, as connection is not really closed on most cases.
* Setting 'data->req.upload_done` correctly, so that no "abort upload"
is happening at the end of a perfectly fine download.
* Adding test cases with up-/download of 0-length files.
* pytest: add a "timeline" of timer value checks to Resulst in curl.py,
so that this can be used in several test cases, replacing the local
stuff in test_16
* add timeline checks to ftp test cases
Patrick Monnerat [Mon, 24 Nov 2025 18:50:26 +0000 (19:50 +0100)]
doc: some returned in-memory data may not be altered
Some public prototypes do not declare return values or out parameters as
const where they should be. Avoid changing the public interface, but
document those values as read-only.
Daniel Stenberg [Mon, 24 Nov 2025 13:00:09 +0000 (14:00 +0100)]
hostip: make more functions return CURLcode
- Curl_async_getaddrinfo() always returned NULL so it was pointless.
Return proper curlcode instead to distinguish between errors. Same for
Curl_doh().
- simplify the IP address handling
- make Curl_str2addr() function return CURLcode
For better error handling and for using the CRT functions recommended
via warnings suppressed by `_CRT_SECURE_NO_WARNINGS`.
Also:
- add missing `freopen_s()` prototype when building with mingw-w64 <5.
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/a5d824654cdc57f6eac1bb581b078986f3eb6856/
- tests/server: replace `open()` in the signal handler with `_sopen_s()`
on Windows.
- tests/server: reduce scope of a checksrc exception to a single line.
- checksrc: ban replaced functions.
Viktor Szakats [Mon, 24 Nov 2025 14:55:17 +0000 (15:55 +0100)]
tests/server: use curlx file open/close functions
Replace:
- `open()` with `curlx_open()` (1 call).
- `fopen()` with `curlx_fopen()`.
- `fclose()` with `curlx_fclose()`.
To centralize interacting with the CRT in preparation for using "safe"
alternatives on Windows. This also adds long-filename and Unicode
support for these operations on Windows.
Keep using `open()` in the signal handler to avoid any issues with
calling code not allowed in signal handlers.
Viktor Szakats [Fri, 21 Nov 2025 19:36:26 +0000 (20:36 +0100)]
lib: rename internal header `share.h` to `curl_share.h` to avoid collision
Windows CRTs have a `share.h`. Before this patch when trying to
`#include <share.h>` it, the compiler picked up curl's internal
`lib/share.h` instead. Rename it to avoid this issue.
CRT `share.h` has constants necessary for using safe open CRT functions.
Also rename `lib/share.c` to keep matching the header.
Stefan Eissing [Tue, 11 Nov 2025 13:26:48 +0000 (14:26 +0100)]
ratelimit: redesign
Description of how this works in `docs/internal/RATELIMITS.ms`.
Notable implementation changes:
- KEEP_SEND_PAUSE/KEEP_SEND_HOLD and KEEP_RECV_PAUSE/KEEP_RECV_HOLD
no longer exist. Pausing is down via blocked the new rlimits.
- KEEP_SEND_TIMED no longer exists. Pausing "100-continue" transfers
is done in the new `Curl_http_perform_pollset()` method.
- HTTP/2 rate limiting implemented via window updates. When
transfer initiaiting connection has a ratelimit, adjust the
initial window size
- HTTP/3 ngtcp2 rate limitin implemnented via ack updates
- HTTP/3 quiche does not seem to support this via its API
- the default progress-meter has been improved for accuracy
in "current speed" results.
Viktor Szakats [Fri, 21 Nov 2025 14:55:33 +0000 (15:55 +0100)]
curlx/strerr: use `strerror_s()` on Windows
To replace deprecated, unsafe `sys_nerr`, `sys_errlist` global
variables with the function suggested by the CRT warning silenced via
`_CRT_SECURE_NO_WARNINGS`:
```
lib/curlx/strerr.c(291): warning C4996: '__sys_nerr': This function or variable may be unsafe. Consider using strerror instead.
lib/curlx/strerr.c(292): warning C4996: '__sys_errlist': This function or variable may be unsafe. Consider using strerror instead.
```
(where `strerror` in turn suggests `strerror_s`...)
Upside: returns an error and has a Unicode variant. Downaside: happy
to return success when passing unrecognized error codes. Work it around
by looking for the string "Unknown error" returned in such cases and
falling back to other methods to retrieve a description.