Viktor Szakats [Thu, 3 Jul 2025 00:16:11 +0000 (02:16 +0200)]
cmake/FindGSS: fix processing C header path options
When processing `--cflags` received from `krb5-config` for `gssapi`:
- fix to not break on multiple `-I` options. Before this patch only
the first `-I` option was processed as a header directory, subsequent
ones ended up in C flags as a raw directory, without the `-I` arg.
Follow-up to 558814e16d84aa202c5ccc0c8108a9d728e77a58
Daniel Stenberg [Mon, 30 Jun 2025 22:09:37 +0000 (00:09 +0200)]
curlinfo: provide the 'digest' feature
... since the tests check for the feature using this name, we
accidentally had lots tests not run because this provided the
`digest-auth` feature that was not checked for.
Viktor Szakats [Tue, 24 Jun 2025 23:45:04 +0000 (01:45 +0200)]
tests: move GSS-API dynamic stub into debug-mode libcurl
Replace the `libstubgss.so`-based overload solution with one built into
libcurl at compile-time.
The previous, `LD_PRELOAD`-based, solution was non-portable, allowlisted
for Linux, BSD and Solaris. It also required non-debug builds, which
turned out to be an accidental condition: 7d342c723c5ae8e9312210936287810741f40bc5. It also required a curl tool
built against a shared libcurl. Detecting this condition wasn't always
accurate, e.g. with certain cmake configurations.
The overload solution also didn't work on macOS, though it theoretically
should have:
- #17653
- #2394
Experiments on making the overload solution work in more envs:
- #17759
That revealed that it also did not work on NetBSD, in CI.
The replacement solution is overloading the necessary GSS-API functions
for test 2056 and 2057 at compile time. It requires a debug-enabled curl
build (due to its insecure nature).
This makes these tests run on all platforms. Including most GSS jobs in
CI, that are running tests. (the exception is old-linux, non-debug jobs,
where it felt overkill to enable debug for this.)
The refactored GSS stub code needs to overload less than before because
it's free to use the official GSS API. (This didn't work with
the overload solution on Alpine for example). It can also use libcurl
functions, allowing to replace `snprintf()` with `msnprintf()`.
OS/400 is also overloading GSS API functions. I haven't tested how this
works after this PR. In theory it should, because this PR doesn't rely
on preprocessor overrides.
Note that for future GSS tests, it may be necessary to stub these GSS
API functions: `gss_inquire_context()`, `gss_unwrap()`, `gss_wrap()`.
They are on codepaths not (yet) touched by tests.
Also:
- stub-gss: check for token buffer overrun.
- stub-gss: replace size macros with `sizeof()`.
- GHA: enable debug for some jobs with GSS.
- GHA/linux: ignore results for 2056 and 2057 in the valgrind job.
They leak the same way as seen with 2077 and 2078.
Ref: 7020ba797961d38c3bf24539f9bb407e0586274d #17462
Ref: 146759716cbacfd453b9fb13d1096f0595424a6c #14430
- GHA/linux: fix to ignore `gss_import_name()` leaks in valgrind builds.
only.
- lib/vauth/krb5_gssapi: reduce variable scope.
- lib/vauth/spnego_gssapi: reduce variable scope.
- tests/libtest: drop code and build logic dealing with `libstubgss`.
- runtests:
- drop `ld_preload` feature.
- drop special handling of `LD_PRELOAD` env in tests.
- drop logic dealing with shared curl tool detection.
- drop `LD_PRELOAD` envs from tests.
Daniel Stenberg [Sun, 29 Jun 2025 12:33:13 +0000 (14:33 +0200)]
unit tests: extract "private" prototypes at build time
In order to do unit tests for private functions, functions that are
marked UNITTEST but without a global scope in the library, functions
that do not have prototypes in their corresponding header file, unit
tests previously brought their own private prototype *copy* into the
unit test.
This was error-prone when the internal function changes but the change
might be missed in the unit test which then uses an outdated prototype
copy for testing.
This change removes the private prototypes from unit tests and instead
introduces a C file parser that parses the specific C files and extracts
the necessary unit test prototypes into a generated header file for unit
tests to use. This geneated lib/unitprotos.h header is then included by
unit tests that need private prototypes.
Viktor Szakats [Sat, 28 Jun 2025 18:22:11 +0000 (20:22 +0200)]
GHA: fix zizmor 1.10.0 warnings, update names
Job `name:` now mandatory in zizmor.
Also:
- enclose `name:` values in single-quotes, for uniformity.
- drop `name: checkout` where set, for uniformity.
- dist: also install with cmake.
- dist: replace `make` with `cmake --build` for cmake.
(to make this make-tool agnostic)
- appveyor-status: double-quote shell arguments.
- tweak existing names to be shorter, to sync terms and style across
jobs and steps.
Stefan Eissing [Mon, 30 Jun 2025 06:53:31 +0000 (08:53 +0200)]
cf-socket: make socket data_pending a nop
Eliminating the socket readability check in the socket connection
filters for the 'data_pending' callback. Improves performance of
handling of transfers, up to ~30%, depending on parallelism and response
size.
Whatever `data_pending()` once was, its semantics are now:
"Is there anything buffered in the connection filters that needs
receive?"
Any checks of the socket's readability are done via `multi_wait()`
and friends.
Fix the one place in HTTP/1 proxy code that checked `data_pending()` and
did an early return if false. Remove that check and actually try to
receive data every time.
Viktor Szakats [Fri, 27 Jun 2025 12:21:44 +0000 (14:21 +0200)]
checksrc: reduce exceptions, apply again to curlx
- tests/libtest: move exception to `stub_gssapi.h`.
- tests/libtest: move remaining exception to `testtrace.c`.
- tests/server: drop obsolete exception.
- docs/examples: move `BANNEDFUNC` exceptions to local files (3 lines).
- docs/examples: move `ERRNOVAR` exception to `ephiperfifo.c`.
- docs/examples: drop `typedef struct` (8 files).
- lib/curlx: add `.checksrc` with banned funcs copied from lib.
- checksrc: ban `strncpy`, `strtok_r`, `strtoul` by default.
Drop local bans. Add exception for `strtoul` to `tests/server'.
- lib, src: sync banned funcs.
Also:
- REUSE: drop `stunnel.pem`, it no longer exists.
- docs/examples: formatting.
- docs/examples: simplify some `sizeof()`s.
Viktor Szakats [Fri, 27 Jun 2025 07:03:24 +0000 (09:03 +0200)]
lib: replace scache no-op macros with `#ifdef`
To avoid warning/error in no-SSL, non-unity builds:
```
lib/multi.c:273:5: error: code will never be executed [-Werror,-Wunreachable-code]
273 | goto error;
| ^~~~~~~~~~
```
This allows us to write slightly less code and avoids the ssize_t/size_t
conversions in many cases. Also, it gets the function in line with all
the other send/recv signatures.
Added helper functions in `cfilters.h` for sending from/receving into
a bufq.
Fuzzer now fails to build due to these changes and its testing of
the bufq API.
Viktor Szakats [Thu, 26 Jun 2025 21:13:43 +0000 (23:13 +0200)]
runtests: fix `LD_PRELOAD` detection for cmake-built curl binaries
CMake builds by default don't include a triplet in the `curl -V` output,
but a CMake-specific OS string, which is usually capitalized or stylized,
e.g. "Linux", or "FreeBSD". Make the regexp expression case-insensitive
to handle this.
Viktor Szakats [Wed, 25 Jun 2025 11:30:23 +0000 (13:30 +0200)]
build: split `.c` and `.h` file lists in tests
It allows to pass `.h` files only where they are really needed, which is
`EXTRA_DIST` (or `*SOURCES`) for autotools.
Also:
- rename variables to be shorter and consistent.
- drop references to non-local headers. We don't need to pass them as
dist sources.
- drop empty variables and references, after the above.
- server: add placeholder `UTIL_H`.
Keno Fischer [Tue, 24 Jun 2025 22:33:12 +0000 (18:33 -0400)]
docs: reflect that delimiter-separated capath is only OpenSSL
curl passes down the capath directly to the backends. OpenSSL will then
delimiter-separate this path internally to support multiple directories
(using its certificate hash scheme). However, the other backends
(wolfSSL, mbedTLS, gnutls) only expect a single directory (and do not
use the hash scheme, preferring to iterate the directory and load all
files). This adjusts the `--capath` documentation to reflect that
multiple paths is an OpenSSL-specific feature. Alternatively, curl could
delimiter-separate these itself, but I'm not sure it's worth it.
Viktor Szakats [Mon, 23 Jun 2025 21:29:59 +0000 (23:29 +0200)]
build: fix libcurltool with cmake and tunits, related tidy-ups
Sync how libcurltool is built in different modes and build systems.
cmake:
- build libcurltool with curlx when building shared libcurl.
To make it possible to use standard libcurl when linking tunits.
Also syncing this with autotools.
The remaining difference is that cmake allows to select shared or
static for curl tool and tests/examples independently.
- fix to link with libcurl instead of libcurlu.
To sync with autotools and to link with the standard libcurl for
tool unit tests.
- fix `source_group()` to always include curlx sources.
- add missing 'curlx header files' source group.
autotools:
- build libcurltool without curlx when building static libcurl in
non-unity builds.
To avoid double compilation, just to be thrown away at link time.
Also to sync with unity builds.
both:
- sync source order between autotools and cmake.
- make sure to pass all headers with both autotools and cmake.
This is a no-op with cmake. Maybe a future patch should make sure
to not pass those to remove that noise.
Stefan Eissing [Tue, 24 Jun 2025 10:57:04 +0000 (12:57 +0200)]
multi: xfer table/bitset, handle limits
* calculate capacity growth on multi's xfer table and bitsets to
work correctly when approaching UINT_MAX
* uint-bset: track the first 64bit slot used. This avoids slot scans
on empty sets.
* uint-tbl: remove restriction to grow ot UINT_MAX, it is multi's
job to enforce limits suitable for its use
* test751: use curl_mfprintf() for error messages
Joel Depooter [Tue, 24 Jun 2025 01:14:24 +0000 (18:14 -0700)]
cmake: check USE_WINDOWS_SSPI when adding secur32 to CURL_LIBS
Instead of CURL_WINDOWS_SSPI.
When running CMake on Windows with no additional parameters (ie default
build configuration), the generated project files do not include the
`secur32.lib` library in the linker settings. This is because
the relevant check was looking at `CURL_WINDOWS_SSPI` instead of
`USE_WINDOWS_SSPI`.
`USE_WINDOWS_SSPI` is enabled when building with SChannel (the default
on Windows), or if `CURL_WINDOWS_SSPI` is specified on the command line.
Viktor Szakats [Mon, 23 Jun 2025 17:48:19 +0000 (19:48 +0200)]
build: stop checking for `sys/stat.h`
It has been used unconditionally in `src` and `tests` since at least
2011-09-19 via fdecb56cbfcafe5b770c4181133655b89973f41e. There are
earlier unguarded references in `tests`.
Viktor Szakats [Fri, 20 Jun 2025 07:20:45 +0000 (09:20 +0200)]
build: drop explicit curlx from hdr paths, refer headers with `curlx/` prefix
To make all src and test code refer to curlx headers the same way.
Also:
- src: move `curlx.h` include to `tool_setup.h`.
- src/tool_setup.h: drop stray `curlx/timeval.h`.
- servers: de-duplicate `curlx.h` and `curl_setup.h` includes.
- libtests, units: drop stray curlx sub-headers in favor of
`<curlx/curlx.h>`.
- tests: include `curlx.h` with `<>` instead of `""`. To match
other parts of the codebase.
Viktor Szakats [Sun, 22 Jun 2025 01:17:33 +0000 (03:17 +0200)]
cmake: replace the way clang-tidy verifies tests, fix issues found
Replace existing `mk-unity.pl` `--embed` workaround with running
`clang-tidy` manually on individual test source instead. This aligns
with how clang-tidy works and removes `mk-unity.pl` from the solution.
Also:
- mqttd: fix potentially uninitialized buffer by zero filling it.
```
tests/server/mqttd.c:484:41: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
484 | payload_len = (size_t)(buffer[10] << 8) | buffer[11];
| ^
[...]
tests/server/mqttd.c:606:45: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
606 | topiclen = (size_t)(buffer[1 + bytes] << 8) | buffer[2 + bytes];
| ^
```
- sockfilt: fix potential out-of-bound pointer:
```
tests/server/sockfilt.c:1128:33: error: The 2nd argument to 'send' is a buffer
with size 17010 but should be a buffer with size equal to or greater than
the value of the 3rd argument (which is 18446744073709551615)
[clang-analyzer-unix.StdCLibraryFunctions,-warnings-as-errors]
1128 | ssize_t bytes_written = swrite(sockfd, buffer, buffer_len);
| ^
```
- clang-tidy: suppress bogus `bzero()` warnings that happens
inside the notorious `FD_ZERO()` macros, on macOS.
Yedaya Katsman [Sun, 22 Jun 2025 13:02:28 +0000 (16:02 +0300)]
rustls: don't try printing the not provided file
Caught by gcc (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0:
```
vtls/rustls.c: In function ‘cr_connect’:
vtls/rustls.c:857:61: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
857 | failf(data, "rustls: must provide certificate with key '%s'",
| ^~
```
Viktor Szakats [Sun, 22 Jun 2025 18:10:55 +0000 (20:10 +0200)]
cmake: configure c-ares header directory in project root (was: lib)
The c-ares header directory was added to the header path within `lib`,
as opposed to every other dependency which added them in the root
`CMakeLists.txt`. Such exception is no longer necessary. This patch
aligns c-ares header setup with the rest of dependencies. And also with
autotools, which also makes no exception here.
Cherry-picked from #17705
Cherry-picked from #16973
Viktor Szakats [Sat, 21 Jun 2025 23:10:59 +0000 (01:10 +0200)]
tests: make individual test sources compile cleanly
Tidy up headers and includes to ensure all individual test source
compile cleanly (but not link). To allow running clang-tidy (and
possibly other static analyzers) on them. It also improves readability
and allows to verify them locally, without the bundle logic.
clang-tidy ignores #included C files, so it's blind to bundle C files
the include these tests. The current workaround of embedding has
a couple of downsides:. meaningless filenames and line numbers,
missing issues, messing up self header paths. Thus, running it on
individual sources would be beneficial.
Also:
- de-duplicate includes.
- untangle some includes.
- formatting/indentation fixes.
- merge `getpart.h` into `first.h`.
Viktor Szakats [Sun, 22 Jun 2025 09:17:07 +0000 (11:17 +0200)]
tests/libtest: drop `TEST_HANG_TIMEOUT` redefinition hack
Before this patch the code relied on re-initializing `TEST_HANG_TIMEOUT`
macro before compiling each test, to allow them each to override it to
a custom value for single tests. Thie required re-including `test.h`
into each test.
After this patch this macro becomes a global, immutable, default. Tests
which want to override it can now use alternate macros that do accept
a custom timeout. The only test currently affected is lib1501.
Viktor Szakats [Sun, 22 Jun 2025 08:30:07 +0000 (10:30 +0200)]
cmake: add target property dumper helper function
It's pretty rough and a giant hack, but helps debugging and findind ways
while navigating the CMake maze. I find it sad CMake doesn't have
a built-in function for this that works correctly in all situations.
It's invaluable to be able to see what properties and values an object
has.
It's also possible there is a better solution to this, but I could not
find it.
Stefan Eissing [Thu, 22 May 2025 13:09:58 +0000 (15:09 +0200)]
cf: replace the method get_host with query
Connection filters had a method `get_host()` which had not really been
documented. Since then, the cf had the `query()` method added. Replace
the separate get_host with query.
Add `CF_QUERY_HOST_PORT` as query to connection filters to retrieve
which remote hostname and port the filter (or its sub-filter) is talking
to. The query is implemented by HTTP and SOCKS filters, all others pass
it through.
Add `Curl_conn_get_current_host()` to retrieve the remote host and port
for a connection. During connect, this will return the host the
connection is talking to right now. Before/After connect, this will
return `conn->host.name`.
Stefan Eissing [Wed, 18 Jun 2025 10:34:43 +0000 (12:34 +0200)]
multi: add dirty bitset
Add a bitset `dirty` to the multi handle. The presence of a transfer int
he "dirty" set means: this transfer has something to do ASAP.
"dirty" is set by multiplexing protocols like HTTP/2 and 3 when
encountering response data for another transfer than the current one.
"dirty" is set by protocols that want to be called.
Implementation:
* just an additional `uint_bset` in the multi handle
* `Curl_multi_mark_dirty()` to add a transfer to the dirty set.
* `multi_runsingle()` clears the dirty bit of the transfer at
start. Without new dirty marks, this empties the set after
al dirty transfers have been run.
* `multi_timeout()` immediately gives the current time and
timeout_ms == 0 when dirty transfers are present.
* multi_event: marks all transfers tracked for a socket as dirty.
Then marks all expired transfers as dirty. Then it runs
all dirty transfers.
With this mechanism:
* Most uses of `EXPIRE_RUN_NOW` are replaced by `Curl_multi_mark_dirty()`
* `Curl_multi_mark_dirty()` is cheaper than querying if a transfer is
already dirty or set for timeout. There is no need to check, just do it.
* `data->state.select_bits` is eliminated. We need no longer to
simulate a poll event to make a transfer run.
Viktor Szakats [Sat, 21 Jun 2025 09:03:54 +0000 (11:03 +0200)]
build: tidy up `Makefile.inc` use in lib and src
- cmake: use `CURL_RCFILES` instead of literal.
- cmake: use `LIB_RCFILES` instead of literal.
- cmake: fix comments.
- autotools: use `CURL_RCFILES` in `EXTRA_DIST`.
- autotools: use `LIB_RCFILES` in `EXTRA_DIST`.
- autotools: fix comments.
- autotools: fix indentation.
DoI [Tue, 10 Jun 2025 11:13:35 +0000 (23:13 +1200)]
curl: implement non-blocking STDIN read on Windows
Implements a seperate read thread for STDIN on Windows when curl is run
with -T/--upload-file .
This uses a similar technique to the nmap/ncat project, spawning a
seperate thread which creates a loop-back bound socket, sending STDIN
into this socket, and reading from the other end of said TCP socket in a
non-blocking way in the rest of curl.
Joel Depooter [Fri, 20 Jun 2025 22:44:02 +0000 (15:44 -0700)]
libssh: fix incorrect return value in myssh_in_AUTH_PKEY_INIT
In the unlikely case that no SSH auth methods are supported, the
previous code would return 0 from myssh_in_AUTH_PKEY_INIT. However,
following the code path, it seems like it should be returning SSH_ERROR,
as set in myssh_to_ERROR (through myssh_to_GSSAPI_AUTH,
myssh_to_KEY_AUTH and myssh_to_PASSWD_AUTH).
In actuality, this is unlikely to occur, as the similar code in
myssh_in_AUTHLIST would have already returned an error in this scenario.
However setting a return value and then ignoring it is a bit fishy and
should be documented if this is intended.
I believe this used to return an error, but was changed in the recent
re-factoring of this code.
Viktor Szakats [Sat, 21 Jun 2025 00:22:04 +0000 (02:22 +0200)]
tests: drop `BUNDLE_SRC` variable
Derive it from `$BUNDLE` instead. autotools seems to be already relying
on `$BUNDLE_SRC` being equal to `$BUNDLE.c`. (I haven't realized this
before aaebb45f58b3f62876a68c17c71ac37d98f1b3bb.)
Also drop redundant `nodist_<target>_SOURCE` lines in tunits and units.
Viktor Szakats [Sat, 21 Jun 2025 08:00:22 +0000 (10:00 +0200)]
cmake: omit clang-tidy on internal libs curlu and curltool
Skip clang-tidy while compiling curlu and curltool internal libraries.
To save about 1 minute per run. These libraries compile the lib and src
sources a second time, with the `UNITTESTS` macro enabled, which makes
tiny difference, for internal use. I figure it's not worth the extra CI
(and local) time because finding extra issues in these passes is
unlikely, and if found, not critical.
autotools also doesn't check curlu and curltool with clang-tidy.
Fabrício Canedo [Tue, 17 Jun 2025 18:51:57 +0000 (15:51 -0300)]
docs: fix broken link in CODE_REVIEW.md
In CODE_REVIEW.md file, the link that points to CONTRIBUTE was broken,
so I fixed this issue changing the link from only "CONTRIBUTE.md" to
"https://curl.se/dev/contribute.html".
Fabrício Canedo [Tue, 17 Jun 2025 17:46:23 +0000 (14:46 -0300)]
docs: fix broken link in INSTALL.md
In INSTALL.md file, the link that points to CURL-DISABLE was broken,
so I fixed this issue changing the link from only "CURL-DISABLE.md" to
"https://github.com/curl/curl/blob/master/docs/CURL-DISABLE.md".
Stefan Eissing [Mon, 16 Jun 2025 07:54:20 +0000 (09:54 +0200)]
docs: fix documentation of connect_only 2
Setting CURLOPT_CONNECT_ONLY with value 2 is only defined
for WebSocket and the effect on other protocols is undetermined.
That includes the HTTP urls.
projects / thirdparty / curl.git / log
