Daniel Stenberg [Fri, 4 Apr 2025 12:34:10 +0000 (14:34 +0200)]
randdisable: build randomizer
This script makes a "random" build using configure and verifies that it
builds curl correctly. It randomly adds a number of the available
--disable-* flags to configure. When it detects a problem the script
stops, otherwise it continues trying more combinations.
Stefan Eissing [Fri, 4 Apr 2025 08:43:13 +0000 (10:43 +0200)]
http2: fix stream window size after unpausing
When pausing a HTTP/2 transfer, the stream's local window size
is reduced to 0 to prevent the server from sending further data
which curl cannot write out to the application.
When unpausing again, the stream's window size was not correctly
increased again. The attempt to trigger a window update was
ignored by nghttp2, the server never received it and the transfer
stalled.
Add a debug feature to allow use of small window sizes which
reproduces this bug in test_02_21.
Dan Fandrich [Fri, 4 Apr 2025 18:35:33 +0000 (11:35 -0700)]
tests: unify test case keywords
Unify the case, punctuation and name of test case keywords so they can
be more easily selected or skipped when desired. Add a few keywords that
were missing. Fix a couple of typos in test names.
To use curl as a tool for troubleshooting SigV4 signing, it is useful to
have the 'Canonical Request', 'String To Sign' and 'Signature'
calculations output.
SIZE_MAX is an very overkill size for certificates or keys, lower it to
100KiB for both certificate and keys. The default max size of openssl is
100KiB for the entire chain [1], and it seems firefox fails at ~60kb
[2].
Viktor Szakats [Sat, 29 Mar 2025 02:41:45 +0000 (03:41 +0100)]
tests: prefer `--insecure` over `-k`
To make it uniform in all tests, and greppability.
Also:
- replace `-k` flag with `-q` in test 1268. (the actual flag doesn't
matter in this test)
- keep `-k` in test 300 to test its short form.
(also verified to fail without a working `-k`)
Stefan Eissing [Thu, 3 Apr 2025 11:11:32 +0000 (13:11 +0200)]
dnscache: slight refactoring
Slight refactoring around dnscache, e.g. hostcache
- eliminate `data->state.hostcache`. Always look up
relevant dnscache at share/multi.
- unify naming to "dnscache", replacing "hostcache"
- use `struct Curl_dnscache`, even though it just
contains a `Curl_hash` for now.
- add `Curl_dnscache_destroy()` for cleanup in
share/multi.
Viktor Szakats [Tue, 1 Apr 2025 21:32:16 +0000 (23:32 +0200)]
eventfd: fix feature guards
Enable eventfd code consistently when both `HAVE_EVENTFD` and
`HAVE_SYS_EVENTFD_H` macros are defined.
Before this patch `HAVE_EVENTFD` guarded it alone, though the code
also required the header, which was guarded by `HAVE_SYS_EVENTFD_H`.
These should normally be detected in pairs. When they aren't, omit using
`eventfd()` to avoid calling it without a known matching header.
If this disables valid cases (e.g. some system declares this function
via a different header), feature detection and the code may be extended
for those cases. If these are known to come in pairs, always, another
option is detect them both at build stage, and forward a single macro
to C.
Viktor Szakats [Wed, 2 Apr 2025 10:24:43 +0000 (12:24 +0200)]
configure: restore link checks
The omitted link checks were not what I though they were. Omitting one
caused a mis-detection on Solaris, where the compile check alone
mis-detects `CloseSocket` as present.
Restore link checks for these functions:
`closesocket`, `ioctlsocket`, `socket`, `freeaddrinfo`, `getaddrinfo`,
`gethostname`, `getpeername`, `getsockname`,
`CloseSocket` (AmigaOS), `IoctlSocket` (AmigaOS).
Also re-sync link check code snippets with the ones in current master.
Viktor Szakats [Tue, 1 Apr 2025 22:04:56 +0000 (00:04 +0200)]
GHA/windows: move libssh job from vcpkg to MSYS2
To avoid upstream issue where libssh no longer builds with vcpkg:
```
error: building libssh:x64-windows failed with: BUILD_FAILED
```
Ref: https://github.com/curl/curl/actions/runs/14206672441/job/39805869213?pr=16909#step:5:64
Viktor Szakats [Tue, 1 Apr 2025 22:29:56 +0000 (00:29 +0200)]
GHA/windows: make libssh2 install a per job config
To allow making per-job variations for SSH backends.
Also:
- fix Cygwin builds to not ignore per-job `install:` items.
It worked by accident before this patch.
Follow-up to 66313cc036671cd4d3e72db65a79a715c7b8f154 #16629
Daniel Stenberg [Tue, 1 Apr 2025 08:46:07 +0000 (10:46 +0200)]
vtls_scache: remove "Unreachable Call"
The condition required to reach this call could not happen, because
cf_ssl_scache_get() already checks the same condition and returns NULL
for 'scache' prior to this.
Stefan Eissing [Tue, 1 Apr 2025 11:44:24 +0000 (13:44 +0200)]
http2: fix stream assignemnt for pushes
When a PUSH_PROMISE was received, the h2_stream object was assigned
to the wrong `newhandle->mid` and was thereafter not found. This led
to internal confusion, because the nghttp2 stream user_data was not
cleared and an invalid easy handle was use for trace messages,
resulting in a crash.
Reported-by: Viktor Szakats
Fixes #16881
Closes #16905
Viktor Szakats [Tue, 1 Apr 2025 10:36:06 +0000 (12:36 +0200)]
GHA/windows: drop GnuTLS-fork from vcpkg MultiSSL job
curl now has a working GnuTLS CI job, with tests, with MSYS2.
The MultiSSL build scenario is now tested on macOS.
The vcpkg GnuTLS package seems to have a deep dependency tree with large
packages that need to be rebuilt relatively frequently. Since they can't
fit into to the time limit, these cause CI failures.
To stabilize CI, drop the `shiftmedia-libgnutls` dependency.
Viktor Szakats [Tue, 1 Apr 2025 09:02:57 +0000 (11:02 +0200)]
runtests: fix bundled test invocation with `-g` option
Fixes:
```
$ ./runtests.pl -g 1940
./libtest/libtests lib1940: No such file or directory.
Argument list to give program being debugged when it is started is "http://127.0.0.1:44547/1940".
```
Reported-by: Daniel Stenberg
Fixes #16893
Closes #16898
Daniel Stenberg [Mon, 31 Mar 2025 11:49:18 +0000 (13:49 +0200)]
GHA: run random curl command lines for N seconds in CI
In the memory and address sanitizer builds.
Verify that nothing unexpected happens.
Starting out with 60 second runs.
The script does not set any seed so it runs with a new random every
time, meaning that if it fails in a single CI run it might not fail in a
subsequent one: but it should still show the full command that failed to
enable us to reproduce it locally. We can work on improving the seed
situation later if this script turns useful.
Daniel Stenberg [Sat, 29 Mar 2025 18:10:40 +0000 (19:10 +0100)]
urlapi: remove percent encoded dot sequences from the URL path
Treat %2e and %2E to be "dot equivalents" in the function and remove
such sequences as well, according to RFC 3986 section 5.2.4. That is
also what the browsers do.
This DOES NOT consider %2f sequences in the path to be actual slashes,
so there is no removal of dots for those.
This function does not decode nor encode any percent sequences.
Also switched the code to use dynbuf.
Extends test 1395 and 1560 to verify.
Assisted-by: Demi Marie Obenour
Fixes #16869
Closes #16870
Austin Moore [Wed, 19 Mar 2025 03:58:56 +0000 (23:58 -0400)]
aws_sigv4: merge repeated headers in canonical request
When multiple headers share the same name, AWS SigV4 expects them to be
merged into a single header line, with values comma-delimited in the
order they appeared.
Viktor Szakats [Sun, 30 Mar 2025 20:34:26 +0000 (22:34 +0200)]
build: drop `build-certs` as a test-run dependency
After adding it as a test executables dependency, it run twice in
MSBuild jobs. Also there is little reason to try building them in both
build and run tests targets.
(The reason MSBuild building it twice, is our use of
`TrackFileAccess=false` to improve build performance.)
Viktor Szakats [Sat, 29 Mar 2025 13:43:10 +0000 (14:43 +0100)]
genserv.pl: detect `openssl` in `PATH`, omit `command -v`
Before this patch the script relied on Perl `system()` finding `openssl`
in `PATH`, plus tried to display the full path of `openssl` by using
`command -v` (or `which` on Windows). `command -v` did not work in CI
for unknown reasons. To resolve it, this patch detects `openssl` in
`PATH` manually, displays the detected full path and calls `openssl`
with the detected full path, and stops relying on `system` for this.
It also follows how `sshhelp.pm` is detecting executables. Though this
patch uses Perl `-f` instead of `-e && -d` used there .
Silencing this in CI logs:
```
Can't exec "command": No such file or directory at ../../../tests/certs/genserv.pl line 51.
```
Ref: https://github.com/curl/curl/actions/runs/14145795884/job/39632942668?pr=16865#step:39:108
Viktor Szakats [Fri, 28 Mar 2025 18:10:28 +0000 (19:10 +0100)]
test313: disable CRL test for Schannel due to lack of support and flakiness
The source code and documentation says that CRL is not supported by
curl's Schannel TLS backend.
It's also frequently flaky in CI with both MinGW and MSVC jobs, e.g.:
https://github.com/curl/curl/actions/runs/14134841988/job/39603994164 (Schannel)
https://github.com/curl/curl/actions/runs/14134841988/job/39606336445 (Schannel)
https://github.com/curl/curl/actions/runs/13981383629/job/39147183706 (LibreSSL)
```
curl returned 35, when expecting 60
```
This test was passing with Schannel because it misses the `--insecure`
option and thus always returns 60, regardless of passing `--crlfile` or
not:
```
curl: (60) schannel: CertGetCertificateChain trust error CERT_TRUST_REVOCATION_STATUS_UNKNOWN
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
```
Viktor Szakats [Fri, 28 Mar 2025 00:41:28 +0000 (01:41 +0100)]
tests/server: make the signal handler signal-safe
Before this patch the signal handler called `logmsg()` which in turn
called `printf()` variants (internal implementations), and `FILE *`
functions, `localtime()`. Some of these called `malloc`/`free`, which
isn't supported in s signal handler. Replace them with `write` calls,
losing some logging functionality.
Also:
- De-dupe and move `STD*_FILENO` macros to `lib/curl_setup.h`. Revert
the `src` definition to point to `stderr`, instead of `tool_stderr`.
Follow-up to e5bb88b8f824ed87620bd923552534c83c2a516e #11958
POSIX specs with list of functions allowed in a signal handler:
2004: https://pubs.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html#tag_02_04_03
2017: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04_03
2024: https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html#tag_16_04_03
Linux CI run with the thread sanitizer going crazy when
hitting the signal handler in test 1238 and 1242 (TFTP):
```
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=12582)
#0 malloc <null> (servers+0x5ed70)
#1 _IO_file_doallocate <null> (libc.so.6+0x851b4)
#2 formatf /home/runner/work/curl/curl/bld/tests/server/../../lib/../../lib/mprintf.c:886:9 (servers+0xdff77)
[...]
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=12582)
#0 free <null> (servers+0x5f453)
#1 fclose <null> (libc.so.6+0x8532f)
#2 logmsg /home/runner/work/curl/curl/bld/tests/server/../../../tests/server/util.c:134:5 (servers+0xe684d)
```
Ref: https://github.com/curl/curl/actions/runs/14118903372/job/39555309490?pr=16851
Viktor Szakats [Thu, 27 Mar 2025 22:44:51 +0000 (23:44 +0100)]
GHA/linux: move pytests to non-valgrind job variants, drop 2 redundant runs
- move pytest from the valgrind variant of the mbedTLS and Rustls jobs
to their non-valgrind counterpart (they different in C compiler and
build tool respectively). To parallelize more and finish the workflow
faster.
- drop pytest from the valgrind variant of the two identical (other than
the build tool) 'libressl heimdal' jobs. Saves 1.5 minutes CI time.
- drop pytest from the longest valgrind job to make the workflow finish
almost 2 minutes faster. `sync-resolver` is its unique build propery.
It wasn't pytested on Azure.
- explicitly install `libnghttp2-dev` and `libldap-dev` to keep them in
jobs where pytest deps were installing them implicitly before this
patch.
Viktor Szakats [Mon, 24 Mar 2025 21:13:29 +0000 (22:13 +0100)]
runtests: generate certs dynamically, bump to EC-256, tidy up
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.
Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.
As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.
Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.
Details:
- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
```
+ openssl genrsa -out test-ca.key -passout fd:0 2048
Invalid password argument, starting with "fd:"
```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
(`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
the Visual Studio generator miss to create the `clean-certs`,
`build-certs` targets. No target depend on them, so they don't execute
implicitly anyway. Fixes:
```
MSBUILD : error MSB1009: Project file does not exist.
Switch: clean-certs.vcxproj
```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.
projects / thirdparty / curl.git / log
