Daniel Stenberg [Tue, 22 Apr 2025 12:51:49 +0000 (14:51 +0200)]
lib: provide a getaddrinfo wrapper
This uses c-ares under the hood and supports the CURL_DNS_SERVER
environment variable - for debug builds only. The getaddrinfo()
replacement function is only used if CURL_DNS_SERVER is set to make a
debug build work more like a release version without the variable set.
'override-dns' is a new feature for the test suite when curl can be told
to use a dedicated DNS server, and test 2102 is the first to require
this.
Daniel Stenberg [Sun, 27 Apr 2025 15:35:20 +0000 (17:35 +0200)]
buildinfo: move from tests/server/ to src/, rename to curlinfo
Since a16485a42ea5dabe6c, the test servers build with a different set of
options than the tool/lib - for example a different CURLDEBUG. To make
buildinfo better reflect the curl build, move it to src/ and build it
here using the local CURLDEBUG. Renamed to curlinfo to not get confused
with buildinfo.txt
I chose src/ and not lib/ because the file also uses tool-specific headers.
Jay Satiro [Mon, 28 Apr 2025 17:35:44 +0000 (13:35 -0400)]
progress: fix integer overflow check
- Fix logic typo.
Prior to this change the overflow check was reversed, meaning it did
not stop an overflow condition and also if there wasn't an overflow it
erroneously set the total expected transfer size to the maximum value.
Viktor Szakats [Sun, 27 Apr 2025 10:26:41 +0000 (12:26 +0200)]
cmake: extend integration tests
- GHA: add cmake integration tests for Windows.
- make them run faster with prefill, unity, Ninja, omitting curl tool.
- also test static libcurl.
- add old-cmake support with auto-detection.
- auto-detect Ninja.
- run consumer test apps to see if they work.
- add support for Windows.
- make it more verbose.
- re-add `ExternalProject` cmake consumer test. It's broken.
- tidy up terminology.
To help anyone wanting to build/reproduce release candidates, this is
the set git tag naming scheme to use. Similar to, but different, than
the "normal" release tags to not be possible to mixup.
Viktor Szakats [Sat, 26 Apr 2025 07:34:55 +0000 (09:34 +0200)]
cmake: honor individual picky option overrides found in `CMAKE_C_FLAGS`
Also to sync up with similar `./configure` feature via
`CURL_ADD_COMPILER_WARNINGS()`.
Example: `-DCMAKE_C_FLAGS=-Wno-xor-used-as-pow`
It may be useful as a workaround if a specific build combination hits
a picky warning within curl's source code. If such happens, we do
appreciate a report to fix it in curl itself.
Viktor Szakats [Thu, 24 Apr 2025 23:45:23 +0000 (01:45 +0200)]
cmake: stop deleting `-W<n>` from `CMAKE_C_FLAGS` (MSVC)
1. `CMAKE_C_FLAGS` may apply to other projects, and deleting/altering it
may be unexpected.
2. We pass `-W4`/`-Wall` internally now, which do override custom
`-W<n>` options in all supported MSVC versions.
(as tested with Visual Studio generators)
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51945416
Daniel Stenberg [Fri, 25 Apr 2025 06:16:13 +0000 (08:16 +0200)]
tests/buildinfo: former "disabled" now provides more info
This tool now contains ON/OFF information about features in the build.
This way, runtests gets both positive and negative feature presence with
this. Allows for more flexibility and avoids having to duplicate the
names.
Viktor Szakats [Thu, 24 Apr 2025 17:12:42 +0000 (19:12 +0200)]
GHA/windows: limit jobs to 15 minutes
They typically finish (well) within 10 minutes.
A notable exception was vcpkg jobs when a rebuild was triggered.
With caching lost and reducing them to short builds, this is not
an issue at the moment.
The advantage of shorter timeouts is hung/crashed jobs giving back
control earlier for a manual retry.
Viktor Szakats [Wed, 23 Apr 2025 08:26:38 +0000 (10:26 +0200)]
cmake: install shell completions for cross-builds
Also:
- omit auto-detecting `CURL_COMPLETION_FISH_DIR` via `pkg-config`
for cross-builds and when `CMAKE_INSTALL_PREFIX` is set.
- flatten nested `if`s.
Note:
On macOS with Homebrew, `pkg-config --variable completionsdir fish`
returns the version-specific Cellar path instead of the permanent path
`/opt/homebrew/share/fish/vendor_completions.d/`. This mimics what
autotools does, but may need further fixing, possibly upstream.
https://github.com/Homebrew/homebrew-core/blob/9c13e62b009b8e814fda180e0fcc5096318daf31/Formula/f/fish.rb
https://github.com/fish-shell/fish-shell/blob/ce631fd2fb1f5b63f5f0f1b4041a30dfad823d22/cmake/Install.cmake#L15-L21
Stefan Eissing [Wed, 23 Apr 2025 09:24:45 +0000 (11:24 +0200)]
multi: init_do(): check result
Calls to `Curl_init_do()` did not check on result and missed failures to
properly and completely initialize a transfer request.
The main cause of such an init failure is the need to rewind the
READFUNCTION without a SEEKFUNCTION registered. Check the failure to
"rewind" the upload data immediately make test cases 1576 and friends
fail.
Reported-by: Travis Lane
Fixes #17139
Closes #17150
Viktor Szakats [Wed, 23 Apr 2025 10:12:02 +0000 (12:12 +0200)]
GHA/windows: apply MSYS2 runtime downgrades to v3.5.x, leave v3.6.x as-is
windows-runners 20250420.1.0 come with msys2-runtime 3.6.x. It has
the perf regression issue fixed, so stop downgrading it.
This makes CI jobs settle on this version when supplied by
the runner image or the msys2/setup-msys2 action:
MINGW64_NT-10.0-20348 fv-az980-636 3.6.1-0cfedd4f.x86_64 2025-04-12 01:44 UTC x86_64 Msys
With 3.6.1, we've seen issues launching `perl.exe` before this patch:
https://github.com/curl/curl/discussions/14854#discussioncomment-12908214
https://github.com/curl/curl/discussions/14854#discussioncomment-12921007
I don't understand what this error says, why it happens in certain CMake
versions, and why a workaround is necessary for what seems like
a standard export/consume configuration. This patch is based on internet
suggestions and other projects ending up with this workaround.
Stefan Eissing [Tue, 22 Apr 2025 10:53:22 +0000 (12:53 +0200)]
http: fix HTTP/2 handling of TE request header using "trailers"
A "TE" request header is allowed in HTTP/2 when it only carries the
"trailers" value. RFC 9113 ch. 8.2.2. Check client supplied TE values
for the "trailers" token and only pass that one in a HTTP/2 request.
Add test_01_17 to verify.
Fixes #17122 Reported-by: epicmkirzinger on github
Closes #17128
Viktor Szakats [Sun, 20 Apr 2025 09:59:06 +0000 (11:59 +0200)]
GHA/linux: tidy up around wolfSSH
- fix filter expression for cache step.
It did not cause an issue because `wolfssh` is always paired with
a `wolfssl*`.
- build `wolfssh` against `wolfssl-opensslextra` (was: `wolfssl-all`).
It makes `wolfssh` builds `OPENSSL_COEXIST`-compatible, and clarifies
its use in the clang-tidy job. The earlier mixup didn't cause issues
because the clang-tidy job is compile-only (using their headers only.)
Useful side-effect is making the wolfssh build valgrind-tested.
Reported-by: bo0tzz on github
Ref: https://github.com/curl/curl/discussions/16970#discussioncomment-12752019
Viktor Szakats [Thu, 17 Apr 2025 14:25:37 +0000 (16:25 +0200)]
runtests: add retry option to reduce flakiness
Add `--retry=<num>` option to tell runtests to retry the first `<num>`
tests that failed. Retries aren't run right away, but added to the end
of the test queue. Once all retry slots are used, test fail as normal.
In CI, typically a single test fails for flakiness, and rarely over 5.
Make the `ci-test` targets default to `--retry=5`.
Stefan Eissing [Tue, 15 Apr 2025 08:55:59 +0000 (10:55 +0200)]
quic: no local idle connection timeout, ngtcp2 keep-alive
Do not set a transport parameter idle timeout, meaning we have no such
thing from our side. The remote setting then applies.
In ngtcp2, set its "keep-alive" timer to prevent a possible remote idle
timeout to tear down the connection while we have active transfers on
that connection.
Viktor Szakats [Thu, 17 Apr 2025 17:21:38 +0000 (19:21 +0200)]
GHA/windows: add 2 basic MSVC jobs to restore some CI coverage
To keep testing these with MSVC:
- UWP !ssl
- arm64 build on the `windows-11-arm` runner
- examples
- OpenSSH-Windows (fix install on `windows-11-arm`)
- `windows-2025` runner
- cmake pre-fill checker
Surprise: UWP doesn't support SSPI, which is required by curl's Schannel
backend. Thus, no TLS support for this UWP build. It also suggests
the Schannel UWP mingw-w64 binaries may be broken and just a happy build
accident thanks to mingw-w64 headers being inaccurate.
Building zlib + libssh2 might actually take up to 2+ minutes with vcpkg,
instead of the previously estimated <1.5 minutes.
Viktor Szakats [Thu, 17 Apr 2025 14:59:04 +0000 (16:59 +0200)]
GHA/windows: restore a simple vcpkg job (zlib + libssh2)
To keep the basics tested. Building zlib and libssh2 takes <1.5 minutes.
So far it seems the "lost" vcpkg binary cache is not a vcpkg-tool
regression, but a GitHub server-side deprecation combined with lack of
replacement service or lack of vcpkg support for such service.
Stefan Eissing [Thu, 17 Apr 2025 09:05:28 +0000 (11:05 +0200)]
pytest: make test_07_22 more lenient to exit codes
Depending on timing when the server aborting the connection is detected,
the reported curl exit code may vary. Check for the possible set of
expected codes instead of a single one.
Stefan Eissing [Tue, 25 Mar 2025 08:47:40 +0000 (09:47 +0100)]
multi: do transfer book keeping using mid
Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.
`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".
Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.
Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.
This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.
Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.
Stefan Eissing [Wed, 16 Apr 2025 14:16:26 +0000 (16:16 +0200)]
vquic: ngtcp2 + openssl support
With the new addition of OpenSSL QUIC API support and the support in
ngtcp2 main branch, make the necessary adjustments in curl to support
this combination.
- add support in configure.ac to detect the feature OPENSSL_QUIC_API2 in
openssl
- initialise ngtcp2 properly in this combination
- add a Curl_vquic_init() for global initialisation that ngtcp2 likes
for performance reasons
- add documentation on how to build in docs/HTTP3.md
- add CI testing in http3-linux.yml