]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
4 months agotest313: disable via `<features>` for backends without CRL support
Viktor Szakats [Sat, 29 Mar 2025 12:07:57 +0000 (13:07 +0100)] 
test313: disable via `<features>` for backends without CRL support

Instead of via `tests/data/DISABLED` file.

They are all missing CRL feature support, as opposed to being broken.

Follow-up to 8adee8824cba23b7f3738b551a84101009f8a8d1 #16862
Follow-up to 8b1b5cd4d2df3a8c2c0108d1d2b5d519b7ece23e #16660

Closes #16865

4 months agocurl_trc: provide Curl_trc_dns dummy
Daniel Stenberg [Sat, 29 Mar 2025 19:45:38 +0000 (20:45 +0100)] 
curl_trc: provide Curl_trc_dns dummy

Follow-up to 19226f9bb106347e21d1dd

For building without verbose output.

Closes #16871

4 months agocurl/system.h: drop leftover comment about 32 bit curl_off_t
Daniel Stenberg [Sat, 29 Mar 2025 13:13:29 +0000 (14:13 +0100)] 
curl/system.h: drop leftover comment about 32 bit curl_off_t

curl MUST have a 64 bit curl_off_t these days

Closes #16867

4 months agodocs/internals/PORTING.md: new document
Daniel Stenberg [Fri, 28 Mar 2025 14:37:15 +0000 (15:37 +0100)] 
docs/internals/PORTING.md: new document

The basic approach I use when porting libcurl to another OS when the
existing configure or cmake build setups are not suitable.

Closes #16860

4 months agoasync-threaded resolver: use ref counter
Stefan Eissing [Thu, 27 Mar 2025 12:55:49 +0000 (13:55 +0100)] 
async-threaded resolver: use ref counter

Allocate the data shared between a transfer and an aync resolver
thread separately and use a reference counter to determine its
release.

Closes #16847

4 months agoGHA: update awslabs/aws-lc to v1.49.0
renovate[bot] [Sat, 29 Mar 2025 11:37:28 +0000 (11:37 +0000)] 
GHA: update awslabs/aws-lc to v1.49.0

Closes #16864

4 months agoruntests: support running tests under wine or qemu (cont.)
Viktor Szakats [Fri, 28 Mar 2025 21:21:06 +0000 (22:21 +0100)] 
runtests: support running tests under wine or qemu (cont.)

Respect `CURL_TEST_EXE_RUNNER` env in server verifications using
the in-tree curl, in `tests/servers.pm`.

Follow-up to 72b163c30141b3a9afe23cceb1e2a0b2e817de8b #16785
Closes #16863

4 months agotest313: disable CRL test for Schannel due to lack of support and flakiness
Viktor Szakats [Fri, 28 Mar 2025 18:10:28 +0000 (19:10 +0100)] 
test313: disable CRL test for Schannel due to lack of support and flakiness

The source code and documentation says that CRL is not supported by
curl's Schannel TLS backend.

It's also frequently flaky in CI with both MinGW and MSVC jobs, e.g.:
https://github.com/curl/curl/actions/runs/14134841988/job/39603994164 (Schannel)
https://github.com/curl/curl/actions/runs/14134841988/job/39606336445 (Schannel)
https://github.com/curl/curl/actions/runs/13981383629/job/39147183706 (LibreSSL)

```
curl returned 35, when expecting 60
```

This test was passing with Schannel because it misses the `--insecure`
option and thus always returns 60, regardless of passing `--crlfile` or
not:
```
curl: (60) schannel: CertGetCertificateChain trust error CERT_TRUST_REVOCATION_STATUS_UNKNOWN
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
```

Closes #16862

4 months agosshserver.pl: use Perl `chmod`
Viktor Szakats [Fri, 28 Mar 2025 14:21:00 +0000 (15:21 +0100)] 
sshserver.pl: use Perl `chmod`

Instead of relying on the external/shell command. For portability.

Follow-up to 213c5aca7bfc74a6ae02b065c5f38a1e9ff54d62 #3899
Closes #16859

4 months agoruntests: rewrite `genserv.sh` in Perl
Viktor Szakats [Fri, 28 Mar 2025 13:33:02 +0000 (14:33 +0100)] 
runtests: rewrite `genserv.sh` in Perl

To remove POSIX shell as an extra dependency for runtests.

Also fix to `chmod 0600` the `.pem` file (was: `.prm`), and apply it
_before_ writing the keys.

Follow-up to 44341e736a3e2f7a2b25a774be3a9796e81abab9 #16824
Closes #16858

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 29 Mar 2025 10:27:44 +0000 (11:27 +0100)] 
RELEASE-NOTES: synced

4 months agoc-ares httpsrr: fix ifdef
Stefan Eissing [Fri, 28 Mar 2025 14:42:29 +0000 (15:42 +0100)] 
c-ares httpsrr: fix ifdef

Fix the ifdef const when enabling HTTPSRR with c-ares.

Reported-by: Pavel Kropachev
Ref: #16855
Closes #16861

4 months agocmake: add shell completion support
Viktor Szakats [Wed, 22 Jan 2025 14:13:47 +0000 (15:13 +0100)] 
cmake: add shell completion support

Add the last (*) missing bit for feature parity with autotools.

Also test in CI. Add a new `cmake install` step to GHA/macos.

(*) AFAIK. Let us know if there's something else missing.

Closes #16833

4 months agotests/server: make the signal handler signal-safe
Viktor Szakats [Fri, 28 Mar 2025 00:41:28 +0000 (01:41 +0100)] 
tests/server: make the signal handler signal-safe

Before this patch the signal handler called `logmsg()` which in turn
called `printf()` variants (internal implementations), and `FILE *`
functions, `localtime()`. Some of these called `malloc`/`free`, which
isn't supported in s signal handler. Replace them with `write` calls,
losing some logging functionality.

Also:
- De-dupe and move `STD*_FILENO` macros to `lib/curl_setup.h`. Revert
  the `src` definition to point to `stderr`, instead of `tool_stderr`.
  Follow-up to e5bb88b8f824ed87620bd923552534c83c2a516e #11958

POSIX specs with list of functions allowed in a signal handler:
2004: https://pubs.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html#tag_02_04_03
2017: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04_03
2024: https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html#tag_16_04_03

Linux CI run with the thread sanitizer going crazy when
hitting the signal handler in test 1238 and 1242 (TFTP):
```
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=12582)
    #0 malloc <null> (servers+0x5ed70)
    #1 _IO_file_doallocate <null> (libc.so.6+0x851b4)
    #2 formatf /home/runner/work/curl/curl/bld/tests/server/../../lib/../../lib/mprintf.c:886:9 (servers+0xdff77)
[...]
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=12582)
    #0 free <null> (servers+0x5f453)
    #1 fclose <null> (libc.so.6+0x8532f)
    #2 logmsg /home/runner/work/curl/curl/bld/tests/server/../../../tests/server/util.c:134:5 (servers+0xe684d)
```
Ref: https://github.com/curl/curl/actions/runs/14118903372/job/39555309490?pr=16851

Closes #16852

4 months agoGHA/linux: move pytests to non-valgrind job variants, drop 2 redundant runs
Viktor Szakats [Thu, 27 Mar 2025 22:44:51 +0000 (23:44 +0100)] 
GHA/linux: move pytests to non-valgrind job variants, drop 2 redundant runs

- move pytest from the valgrind variant of the mbedTLS and Rustls jobs
  to their non-valgrind counterpart (they different in C compiler and
  build tool respectively). To parallelize more and finish the workflow
  faster.

- drop pytest from the valgrind variant of the two identical (other than
  the build tool) 'libressl heimdal' jobs. Saves 1.5 minutes CI time.

- drop pytest from the longest valgrind job to make the workflow finish
  almost 2 minutes faster. `sync-resolver` is its unique build propery.
  It wasn't pytested on Azure.

- explicitly install `libnghttp2-dev` and `libldap-dev` to keep them in
  jobs where pytest deps were installing them implicitly before this
  patch.

Before: https://github.com/curl/curl/actions/runs/14118080563
After: https://github.com/curl/curl/actions/runs/14118903372?pr=16851

Closes #16851

4 months agoGHA/linux: enable ECH in Rustls jobs
Viktor Szakats [Thu, 27 Mar 2025 22:22:41 +0000 (23:22 +0100)] 
GHA/linux: enable ECH in Rustls jobs

Closes #16850

4 months agodocs: add rustls --ca-native & CURLSSLOPT_NATIVE_CA
Daniel McCarney [Thu, 27 Mar 2025 12:58:12 +0000 (08:58 -0400)] 
docs: add rustls --ca-native & CURLSSLOPT_NATIVE_CA

The one important caveat is that presently _only_ the native platform
verifier/CAs are consulted when this option is used w/ rustls.

Closes #16848

4 months agovquic: obey IOV_MAX
Stefan Eissing [Thu, 27 Mar 2025 13:55:41 +0000 (14:55 +0100)] 
vquic: obey IOV_MAX

When using `recvmmsg()`, check `IOV_MAX` for limits to the iovec array
passed.

Reported-by: Teh Kok How
Fixes #16846
Closes #16849

4 months agotests/makefile: add dependency on build-certs
Stefan Eissing [Thu, 27 Mar 2025 10:24:22 +0000 (11:24 +0100)] 
tests/makefile: add dependency on build-certs

Closes #16845

4 months agoruntests: generate certs dynamically, bump to EC-256, tidy up
Viktor Szakats [Mon, 24 Mar 2025 21:13:29 +0000 (22:13 +0100)] 
runtests: generate certs dynamically, bump to EC-256, tidy up

Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.

Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.

As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.

Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.

Details:

- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
  separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
  confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
  auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
  noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
  ```
  + openssl genrsa -out test-ca.key -passout fd:0 2048
  Invalid password argument, starting with "fd:"
  ```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
  (`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
  the Visual Studio generator miss to create the `clean-certs`,
  `build-certs` targets. No target depend on them, so they don't execute
  implicitly anyway. Fixes:
  ```
  MSBUILD : error MSB1009: Project file does not exist.
  Switch: clean-certs.vcxproj
  ```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.

Follow-up to 556f722fe32e5e9f4e24f0242100c5e9d57c129b #16593
Follow-up to fa461b4eff52b413f88debf543b5350a6cef4724 #14486

Closes #16824

4 months agorustls: support native platform verifier
Daniel McCarney [Mon, 24 Mar 2025 16:43:22 +0000 (12:43 -0400)] 
rustls: support native platform verifier

e.g. `curl --ca-native ...`

4 months agodocs: include rustls-ffi in ECH docs
Daniel McCarney [Mon, 24 Mar 2025 16:32:50 +0000 (12:32 -0400)] 
docs: include rustls-ffi in ECH docs

4 months agorustls: support ECH w/ DoH lookup for config
Daniel McCarney [Mon, 24 Mar 2025 16:26:21 +0000 (12:26 -0400)] 
rustls: support ECH w/ DoH lookup for config

E.g. `curl --tlsv1.3 --doh-url https://one.one.one.one/dns-query --ech true ...`

Closes #16828

4 months agorustls: add ECH support w/ string ECH config
Daniel McCarney [Mon, 24 Mar 2025 16:11:54 +0000 (12:11 -0400)] 
rustls: add ECH support w/ string ECH config

e.g. `curl --tlsv1.3 --ech ecl:<BASE64 encoded ECH config list> ...`

Closes #16828

4 months agorustls: support ECH GREASE
Daniel McCarney [Mon, 24 Mar 2025 16:01:30 +0000 (12:01 -0400)] 
rustls: support ECH GREASE

e.g. `curl --tlsv1.3 --ech grease ...`

4 months agorustls: use client cert and key if available
Yedaya Katsman [Wed, 1 Jan 2025 19:14:49 +0000 (21:14 +0200)] 
rustls: use client cert and key if available

4 months agorustls: Add support for SSLKEYLOGFILE
Yedaya Katsman [Mon, 21 Oct 2024 20:28:28 +0000 (23:28 +0300)] 
rustls: Add support for SSLKEYLOGFILE

With rustls-ffi 0.15+ we can set up a callback for writing TLS secrets
hooked up to call Curl_tls_keylog_write. To make sure the associated
file is cleaned up we update the Curl_ssl struct for the rustls-ffi vtls
backend to have a cleanup callback.

Closes #16828

4 months agokeylog: move some constants to header file
Yedaya Katsman [Mon, 21 Oct 2024 20:26:08 +0000 (23:26 +0300)] 
keylog: move some constants to header file

This allows TLS implementations to use the constants for defining sizes
and verification.

4 months agoci: use rustls-ffi 0.15 deb
Daniel McCarney [Sun, 16 Mar 2025 16:48:54 +0000 (12:48 -0400)] 
ci: use rustls-ffi 0.15 deb

Take the latest rustls-ffi version and install it via the upstream
project's `.deb` for x86_64 linux.

4 months agodocs: rework RUSTLS install instructions
Daniel McCarney [Sun, 16 Mar 2025 16:38:24 +0000 (12:38 -0400)] 
docs: rework RUSTLS install instructions

Describes using a package manager or pre-built binaries and adopts the
0.15 installation from source instructions.

Previously the rustls docs described installing rustls-ffi from source
using the GNU Makefile. The upstream project has switched to using
cargo-c as a cross-platform solution that works well with the rust
toolchain, pkg-config, and Windows and so this needs an update.

Similarly, for folks that want to avoid the extra cargo-c tool
requirement, rustls-ffi provides binary releases for common platforms,
and some Linux distributions/package managers offer pre-built packages.

The install instructions are expanded to cover these options since
they're generally better for end users than building the dep. from
source (no `rustc` required).

4 months agotests: replace remaining non-ASCII bytes with hex markup
Viktor Szakats [Wed, 26 Mar 2025 16:27:42 +0000 (17:27 +0100)] 
tests: replace remaining non-ASCII bytes with hex markup

Also drop from `binarycheck`.

Closes #16837

4 months agoconfigure: tidy up shell completion rules
Viktor Szakats [Wed, 22 Jan 2025 12:26:53 +0000 (13:26 +0100)] 
configure: tidy up shell completion rules

- allow generating completions in cross-builds.
  Follow-up to d055a01ce90510a6f8ff44bb7fceace9b2dbcf97 #16789

- fix warning messages.

- language, quotes, whitespace.

Closes #16836

4 months agocmake: add custom command scripts as dependencies where missing
Viktor Szakats [Wed, 26 Mar 2025 12:22:14 +0000 (13:22 +0100)] 
cmake: add custom command scripts as dependencies where missing

Also some formatting.

Closes #16835

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 26 Mar 2025 06:54:09 +0000 (07:54 +0100)] 
RELEASE-NOTES: synced

4 months agoGHA/non-native: revert to bare builds for Android 21
Viktor Szakats [Wed, 26 Mar 2025 01:53:40 +0000 (02:53 +0100)] 
GHA/non-native: revert to bare builds for Android 21

vcpkg requires Android 28 by default after a recent update that's being
deployed onto CI runs (with `libiconv:arm64-android@1.18#1`).

Revert to bare, no-ssl, no-psl configuration for Android 21 jobs to make
them work again.

Bug: https://github.com/curl/curl/pull/16824#issuecomment-2750912507
Ref: https://github.com/microsoft/vcpkg/pull/44424#issuecomment-2753027630

Closes #16832

4 months agocmake: document `SH_EXECUTABLE` option
Viktor Szakats [Tue, 25 Mar 2025 23:39:20 +0000 (00:39 +0100)] 
cmake: document `SH_EXECUTABLE` option

Closes #16830

4 months agotests: remove base64 encoded sections
Daniel Stenberg [Mon, 24 Mar 2025 13:41:41 +0000 (14:41 +0100)] 
tests: remove base64 encoded sections

The base64 sections were added in a time when there was no other way to
do these tests. Now we remove them again to make less "hidden" data in
the test suite.

For several test cases we instead then provide a sequence of binary
octets which is not really better, but I have tried to minimize them.

Closes #16816

4 months agomqtt: cleanups
Daniel Stenberg [Mon, 24 Mar 2025 13:21:32 +0000 (14:21 +0100)] 
mqtt: cleanups

- make a variable const
- use calloc instead of malloc + memset
- fix comment language
- improve error messages

Closes #16826

4 months agomqtt: convert sendleftovers to dynbuf
Daniel Stenberg [Mon, 24 Mar 2025 10:46:34 +0000 (11:46 +0100)] 
mqtt: convert sendleftovers to dynbuf

Avoid frequent strdups/free calls, including the double-free risk.

Reported-by: Ronald Crane
Closes #16823

4 months agoSSLCERTS: list support for SSL_CERT_FILE and SSL_CERT_DIR
Ben Bodenmiller [Tue, 18 Mar 2025 20:31:10 +0000 (13:31 -0700)] 
SSLCERTS: list support for SSL_CERT_FILE and SSL_CERT_DIR

Closes #16762

4 months agoscripts: update completion.pl to parse options from docs
tiymat [Fri, 21 Mar 2025 23:15:57 +0000 (20:45 -0230)] 
scripts: update completion.pl to parse options from docs

Reported-by: kpcyrd on github
Fixes #16072
Closes #16789

4 months agoeasy: add 'populate_fds' func to reduce size of 'wait_or_timeout'
Gabriel Marin [Mon, 24 Mar 2025 16:18:49 +0000 (18:18 +0200)] 
easy: add 'populate_fds' func to reduce size of 'wait_or_timeout'

Closes #16820

4 months agorustls: tidy up
Daniel McCarney [Fri, 21 Mar 2025 18:43:07 +0000 (14:43 -0400)] 
rustls: tidy up

Closes #16796

4 months agoopenssl: fix crash on missing cert password
Jay Satiro [Mon, 24 Mar 2025 06:48:01 +0000 (02:48 -0400)] 
openssl: fix crash on missing cert password

- Return 0 for password length if OpenSSL is expecting a certificate
  password but the user did not provide one.

Prior to this change libcurl would crash if OpenSSL called the certificate
password callback in libcurl but no password was provided (NULL).

Reported-by: Roman Zharkov
Fixes https://github.com/curl/curl/issues/16806
Closes https://github.com/curl/curl/pull/16807

4 months agoasyn-thread: minor cleanups
Daniel Stenberg [Mon, 24 Mar 2025 09:13:44 +0000 (10:13 +0100)] 
asyn-thread: minor cleanups

- Use 'td->' in more places instead of the full thing.

- Remove an assert that is always true since the extra dynamic malloc
  was removed

- Ignore Curl_ares_perform() errors to prioritize the "real" resolver
  info and leave RR as "less important"

Closes #16808

4 months agotool_operate: fix pluralization of seconds
Dave Nicolson [Mon, 17 Mar 2025 18:48:02 +0000 (19:48 +0100)] 
tool_operate: fix pluralization of seconds

Closes #16751

4 months agoconncache: eliminate cpool's diconnect callback
Stefan Eissing [Mon, 24 Mar 2025 09:51:00 +0000 (10:51 +0100)] 
conncache: eliminate cpool's diconnect callback

The callback, provided from url.c did the work that the cshutdn
functionality also implemented. Remove it.

Change some DEBUGF(infof()) to CURL_TRC_M().

Closes #16810

4 months agoGHA: update mbedtls to v3.6.3
renovate[bot] [Mon, 24 Mar 2025 17:14:48 +0000 (17:14 +0000)] 
GHA: update mbedtls to v3.6.3

Closes #16822

4 months agotest615: fix for Cygwin, unignore in CI
Viktor Szakats [Sun, 23 Mar 2025 23:31:33 +0000 (00:31 +0100)] 
test615: fix for Cygwin, unignore in CI

Setting a server-side file read-only by `chmod 0444` has does not
prevent overwriting it via SFTP upload (as tested in CI).

Fix it by setting its MS-DOS read-only attribute in addition. It
requires the Cygwin tool `chattr`.

Also unignore in CI.

Fixes:
```
test 0615...[SFTP put remote failure]
curl returned 0, when expecting 9
 615: exit FAILED
=== Start of file stderr615
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 100    30    0     0  100    30      0     93 --:--:-- --:--:-- --:--:--    95

 100    30    0     0  100    30      0     92 --:--:-- --:--:-- --:--:--    92
=== End of file stderr615
```
Ref: https://github.com/curl/curl/actions/runs/14037991918/job/39300723214#step:12:1269

Closes #16818

4 months agoGHA/windows: dl-mingw `PATH` follow-up
Viktor Szakats [Mon, 24 Mar 2025 14:22:52 +0000 (15:22 +0100)] 
GHA/windows: dl-mingw `PATH` follow-up

Follow-up to 468bfc26183cc4e65b2494acf82d256571cb8178 #16813

Closes #16817

4 months agoINFRASTRUCTURE.md: add IRC and Matrix details
Daniel Stenberg [Mon, 24 Mar 2025 09:35:33 +0000 (10:35 +0100)] 
INFRASTRUCTURE.md: add IRC and Matrix details

Closes #16809

4 months agoGHA/windows: use a pure Cygwin environment
Viktor Szakats [Sun, 23 Mar 2025 22:43:14 +0000 (23:43 +0100)] 
GHA/windows: use a pure Cygwin environment

Use the `PATH` `/usr/bin` to avoid any Windows system or 3rd-party tool
installed on the runner machine that may interfere with or add undesired
dependencies to the builds and tests.

Follow-up to d838d434305bd97c0ce5488f24945beff878adc9 #16465
Ref: #16437

Closes #16814

4 months agoGHA/windows: boost dl-mingw and cygwin install performance
Viktor Szakats [Sun, 23 Mar 2025 21:42:29 +0000 (22:42 +0100)] 
GHA/windows: boost dl-mingw and cygwin install performance

Install on drive `D:` which has much better write performance than `C:`,
on GitHub Windows runner machines.

- It's bringing down `dl-mingw` installation steps to 5-15s per job,
  from 15s-130s before this patch.

- Saving 30-90s per job in the Cygwin install step.

The before values were fluctuating, but it seems reasonable to expect
saving at least a couple of minutes for each workflow run.

Closes #16813

4 months agolibssh2: minor tidy-ups
Viktor Szakats [Sun, 23 Mar 2025 13:58:18 +0000 (14:58 +0100)] 
libssh2: minor tidy-ups

Fix indentation and drop an unnecessary assigment.

Closes #16812

4 months agoGHA/windows: install OpenSSH-Windows manually for transparency
Viktor Szakats [Sun, 23 Mar 2025 15:49:06 +0000 (16:49 +0100)] 
GHA/windows: install OpenSSH-Windows manually for transparency

To have the current latest version, and to avoid the stale, misleading
versions installed by Chocolatey. It also installs transparently, faster,
and making the source of the binaries clear. Install on drive `D:` for
best performance.

After much detective work it turns out that the OpenSSH Windows versions
installed by Chocolatey aren't what they seem:

- The latest pre-release named 9.5.0-beta20240403:
  https://community.chocolatey.org/packages/openssh/9.5.0-beta20240403
  is in reality 8.6.0.0p1-Beta from 2021-05-27:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/V8.6.0.0p1-Beta/OpenSSH-Win64.zip

- The latest "stable" version 8.0.0.1 is in reality:
  https://community.chocolatey.org/packages/openssh/8.0.0.1
  is in reality 8.0.0.0p1-Beta:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/v8.0.0.0p1-Beta/OpenSSH-Win64.zip

Ref: https://github.com/curl/curl/pull/16803#issuecomment-2746365654
Follow-up to 67a7775d1233d702964bff9a0a6b5b9fa036c47a #16704
Follow-up to 0ec72c1ef8d87a29bf2eaa5e36ab173147a4d015 #16672

Closes #16811

4 months agoGHA: extend clang-tidy jobs with more build options, add Windows job
Viktor Szakats [Wed, 19 Mar 2025 17:42:54 +0000 (18:42 +0100)] 
GHA: extend clang-tidy jobs with more build options, add Windows job

- linux: wolfssl, wolfssh (replacing libssh2), ech, kerberos/GSSAPI,
  ssls-export (libssh2 remains tested on macos.)

- macos: brotli, zstd, c-ares, gnutls, mbedtls, gsasl, rtmp, ssls-export

- windows: new job with schannel, sspi, winidn, winldap, ssls-export

- unit3205: fix/silence remaining NULL dereferences.

Commits fixing the issues found:
cbbccb8b3aa357926131375e7082b2579fa3fe82 #16766
554e4c14bee82a604e08d51767f49fe11580e700 #16777

Closes #16764

4 months agomulti_ev: fixes regarding connection shutdowns
Stefan Eissing [Fri, 21 Mar 2025 12:45:37 +0000 (13:45 +0100)] 
multi_ev: fixes regarding connection shutdowns

When handling connections (not transfers) related events, always use the
passed `conn` and not `data->conn` as the transfer does not need to have
the same connection.

Fix handling of conn pollset diffs to disregard the transfer.

Closes #16782

4 months agomemdebug: drop dynamic allocation from `curl_dbg_log()`
Viktor Szakats [Sun, 16 Mar 2025 18:45:58 +0000 (19:45 +0100)] 
memdebug: drop dynamic allocation from `curl_dbg_log()`

Closes #16745

4 months agowindows: fix issues detected by clang-tidy, and some more
Viktor Szakats [Thu, 20 Mar 2025 12:17:39 +0000 (13:17 +0100)] 
windows: fix issues detected by clang-tidy, and some more

- digest_sspi: memory leak.
- digest_sspi: free buffers on `calloc()` fail.
  (not detected by clang-tidy)
- schannel_verify: avoid a `NULL` `alt_name_info`.
- schannel: fix potential `NULL` deref for `backend->cred`.
- schannel: fix uninitialized result value.
  Follow-up to 7f4c358541fdadcf29ba20bcdff71c5554e5f69c #3197
- schannel: drop unused assigment.
- tool_doswin: drop unused assigment.
- testutil: fix memory leak on error.
- testutil: fix memory leak on non-error.
  (not detected by clang-tidy)

Cherry-picked from #16764
Closes #16777

4 months agotests: build non-debug unit tests with autotools, run them
Viktor Szakats [Wed, 19 Mar 2025 23:55:55 +0000 (00:55 +0100)] 
tests: build non-debug unit tests with autotools, run them

Before this patch, autotools disabled building unit tests for
non-debug-enabled (`DEBUGBUILD`) builds. runtests skipped running this
combination, though they were built in cmake builds. There seems to be
no technical reason anymore for these restrictions. This patch removes
them, allowing to build and run unit tests for non-debug-enabled builds.

To improve unit test build and run coverage.

- autotools: do not disable building unit tests for non-debug-enabled
  build. Bringing behavior closer to cmake builds. (There are still
  exceptions in autotools, something for another PR)

- runtests: run unit tests for non-debug-enabled builds.
  It extends coverage by 50 tests.

- `lib/altsvc.c`: fix to include `CURL_TIME` support in libcurlu, for
  unit tests. It fixes test 1654, and syncs `CURL_TIME` behavior with
  test 1660 and `lib/hsts.c`.

Ref: 10a7d05be3921b9c9762477a18d41aff232d94d8
Ref: fc8e0dee3045658f293452121f5290d81ba3aa1e #13694
Ref: 99f78cbf6ed376391f519de18d289a6924228591 #16770

Closes #16771

4 months agombedtls: re-enable an error check
Viktor Szakats [Wed, 19 Mar 2025 17:58:56 +0000 (18:58 +0100)] 
mbedtls: re-enable an error check

While adding support for key blobs, a check and error code update moved
after some logic, resulting in the updated code not checked anymore.

Detected by clang-tidy:
```
lib/vtls/mbedtls.c:768:7: error: Value stored to 'ret' is never read [clang-analyzer-deadcode.DeadStores,-warnings-as-errors]
  768 |       ret = MBEDTLS_ERR_PK_TYPE_MISMATCH;
      |       ^
```
Ref: https://github.com/curl/curl/actions/runs/13953249156/job/39057979349?pr=16764#step:12:178

Regression from 05e0453050def7323fbc8210886f909819edcc5f #7157
Cherry-picked from #16764
Closes #16766

4 months agocmake: fix clang-tidy builds to verify tests, fix fallouts
Viktor Szakats [Mon, 17 Mar 2025 23:39:57 +0000 (00:39 +0100)] 
cmake: fix clang-tidy builds to verify tests, fix fallouts

- cmake: disable test bundles for clang-tidy builds.
  clang-tidy ignores #included .c sources, and incompatible with unity
  and bundles. It caused clang-tidy ignoring all test sources. It also
  means this is the first time tests sources are checked with
  clang-tidy. (autotools doesn't run it on tests.)

- cmake: update description for `CURL_TEST_BUNDLES` option.

- fix tests using special `CURLE_*` enums that were missing from
  `curl/curl.h`. Add them as reserved codes.

- fix about ~50 other issues detected by clang-tidy: unchecked results,
  NULL derefs, memory leaks, casts to enums, unused assigments,
  uninitialized `errno` uses, unchecked `open`, indent, and more.

- drop unnecessary casts (lib1533, lib3207).

- suppress a few impossible cases with detailed `NOLINT`s.

- lib/escape.c: drop `NOLINT` no longer necessary.
  Follow-up to 72abf7c13a479edcde80afa60faad3f35f672c0b #13862 (possibly)

- extend two existing `NOLINT` comments with details.

Follow-up to fabfa8e4024473035b3e5c3c30c330be726d9bb4 #15825

Closes #16756

4 months agotest613: make it pass on Windows, fix postprocess, unignore in CI
Viktor Szakats [Sat, 22 Mar 2025 09:41:44 +0000 (10:41 +0100)] 
test613: make it pass on Windows, fix postprocess, unignore in CI

- on native Windows (also when using MSYS2 openssh), the group and other
  permissions do not end up as requested by Perl's chmod:
  ```diff
  --- log/8/check-expected
  +++ log/8/check-generated
  @@ -1,3 +1,3 @@
   d?????????    N U         U               N ???  N NN:NN asubdir[LF]
  --rw?rw?rw?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
  +-rw?r-?r-?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
   -r-?r-?r-?    1 U         U              47 Dec 31  2000 rofile.txt[LF]
  ```
  Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1596
  Fix it by ignoring group and other attributes.

- fix failing postprocess cleanup by making the read-only test file
  writeable again before deleting it. Fixing:
  ```
  Directory not empty at ../../tests/libtest/test613.pl line 83.
  ```
  (seen on Windows with Git for Windows `perl.exe`)

- unignore in GHA/windows.

Closes #16791

4 months agoruntests: use `setfacl` on Cygwin/MSYS, if present
Viktor Szakats [Mon, 24 Feb 2025 23:49:21 +0000 (00:49 +0100)] 
runtests: use `setfacl` on Cygwin/MSYS, if present

To use a native Cygwin tool instead of the Windows `icacls`. It allows
running under Cygwin/MSYS without Windows system folders in the `PATH`.

Also: fix indentation and tidy up syntax of the `icacls` branch.

Note: As of this commit, these `setfacl` and `icacls` calls are not
necessary for a successful CI run. This includes OpenSSH for Windows
tests, that aren't run by default. Keep them anyway, because locally
they may be necessary depending on environment.

Reported-by: Brian Inglis
Fixes #16437
Ref: #16803
Closes #16465

4 months agoasyn-thread: check thread_data->init in Curl_resolver_getsock
Weng Xuetian [Sun, 23 Mar 2025 05:42:26 +0000 (22:42 -0700)] 
asyn-thread: check thread_data->init in Curl_resolver_getsock

resolver may call destroy_async_data after the name is resolved and
corresponding socketpair is already closed at this point. Any following
call to Curl_resolver_getsock should not set the fd.

Fixes #16799
Closes #16802

4 months agoconn: eliminate `conn->now`
Stefan Eissing [Sat, 22 Mar 2025 10:22:22 +0000 (11:22 +0100)] 
conn: eliminate `conn->now`

it was only used in pingpong.c to check if the overall transfer has
timed out and we do that with `Curl_timeleft()` in all other places.

Closes #16793

4 months agoconncache: eliminate `conn->destination_len` as premature optimization
Stefan Eissing [Sat, 22 Mar 2025 09:47:08 +0000 (10:47 +0100)] 
conncache: eliminate `conn->destination_len` as premature optimization

Closes #16792

4 months agodynbuf: assert init on free
Stefan Eissing [Thu, 20 Mar 2025 09:31:30 +0000 (10:31 +0100)] 
dynbuf: assert init on free

Add a DEBUGASSERT() in Curl_dyn_free() that checks that Curl_dyn_init()
has been performed before.

Fix code places that did it wrong.

Fixes #16725
Closes #16775

4 months agolibssh2: fix to ignore `known_hosts` if SHA256 host public key is set
Viktor Szakats [Sun, 23 Mar 2025 20:53:49 +0000 (21:53 +0100)] 
libssh2: fix to ignore `known_hosts` if SHA256 host public key is set

Syncing behavior with MD5 host public keys.

libcurl implemented to force a host key type for hosts is present in
`known_hosts`, and disabled this logic when an MD5 host public key is
explicitly set. libcurl later received support for SHA256 host public
keys. This update missed to extend the `known_hosts` logic with the new
key type.

This caused test 3022 to fail if a pre-existing `known_hosts` listed
the test server IP (127.0.0.1) with a non-RSA host key algo.

Follow-up to d1e7d9197b7fe417fb4d62aad5ea8f15a06d906c #7646
Follow-up to 272282a05416e42d2cc4a847a31fd457bc6cc827 #4747

Closes #16805

4 months agotest612: SCP `rm` the uploaded remote file (not the local source), unignore in CI
Viktor Szakats [Sat, 22 Mar 2025 21:35:33 +0000 (22:35 +0100)] 
test612: SCP `rm` the uploaded remote file (not the local source), unignore in CI

It accidentally worked on all CI-tested operating systems, except on
native Windows.

Fixing:
```
=== Start of file stderr612
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
[...]
 curl: (21) rm command failed: Operation failed
```
Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1424

Also remove this test from the ignore list in GHA/windows.

Closes #16801

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 24 Mar 2025 08:33:30 +0000 (09:33 +0100)] 
RELEASE-NOTES: synced

4 months agosshserver.pl: adjust `AuthorizedKeysFile2` cutoff version
Viktor Szakats [Fri, 21 Mar 2025 16:18:24 +0000 (17:18 +0100)] 
sshserver.pl: adjust `AuthorizedKeysFile2` cutoff version

To avoid in `ssh_server.log`:
```
reprocess config line 6: Deprecated option AuthorizedKeysFile2
```
with openssh 7.1

Closes #16787

4 months agoruntests: support running tests under wine or qemu
Viktor Szakats [Fri, 21 Mar 2025 13:39:55 +0000 (14:39 +0100)] 
runtests: support running tests under wine or qemu

To run curl, tests and servers via `wine`:
```shell
export CURL_TEST_EXE_RUNNER=wine
```
runtests prefixes commands with the specified runner. For systems where
this isn't automatic or supported, e.g. macOS.

Closes #16785

4 months agosshserver: fix excluding obsolete client config lines
Viktor Szakats [Fri, 21 Mar 2025 13:36:46 +0000 (14:36 +0100)] 
sshserver: fix excluding obsolete client config lines

It was already excluded for OpenSSH-Windows. Extend it to all OpenSSH
above v7.3. Syncing up this logic with the sshd server config.

Fixing, in `sftp_server.log`:
```
log/server/curl_sftp_config line 33: Unsupported option "rhostsrsaauthentication"
log/server/curl_sftp_config line 34: Unsupported option "rsaauthentication"
```

`no` has been the default for these since OpenSSH 3.3 (2002-06-21).

Closes #16784

4 months agosecureserver.pl, genserv.sh: cleanups
Yedaya Katsman [Sat, 15 Mar 2025 21:02:27 +0000 (23:02 +0200)] 
secureserver.pl, genserv.sh: cleanups

stunnel: print configuration actually in file
Instead of duplicating all the logic of writing the stunnel
configuration, and having to keep it up to date, when running in verbose
mode print the actual contents of the file after writing it.

certs/genserv.sh: Fix typo in error message

Closes #16783

4 months agoruntests: fix test key format for libssh2 WinCNG (and others)
Viktor Szakats [Fri, 21 Mar 2025 10:48:27 +0000 (11:48 +0100)] 
runtests: fix test key format for libssh2 WinCNG (and others)

SFTP/SCP tests were failing in CI with WinCNG libssh2 since we first
added such job. With `curl: (67) Authentication failure`.

The reason is that the default `ssh-keygen` RSA private key format
changed to OpenSSH (RFC4716) in 2018. libssh2 does not support this
format with some of its crypto backends.

Fix it by generating keys explicitly in PEM format as necessary via
the `-m` option. This format is universally recognized for RSA keys.

2018-08-24: https://www.openssh.com/txt/release-7.8: OpenSSH format becomes default
2010-08-23: https://www.openssh.com/txt/release-5.6: `-m` option first supported

This fixed the auth issue, just to reveal a known flakiness issue in
libssh2 + WinCNG, causing:
```
curl: (2) Failure establishing ssh session: -8, Unable to exchange encryption keys
```
Ref: https://github.com/curl/curl/actions/runs/14000494428/job/39205633258?pr=16781#step:15:1796
Tracked here: https://github.com/libssh2/libssh2/issues/804
Mitigated in libssh2 tests by retrying them.

Due to this, keep ignoring these test results.

Also:
- add an env to customize key format: `CURL_TEST_SSH_KEY_FORMAT`
- display the generated format in the log.
- GHA/linux: document the wolfSSH error code causing it to fail tests:
  ```
  curl: (79) wolfssh SFTP connect error -1051 / WS_MATCH_KEY_ALGO_E / cannot match key algo with peer
  ```

Follow-up to 4911e7af119c1b7efd46a742d47bca44832c3041 #16735
Follow-up to 0ec72c1ef8d87a29bf2eaa5e36ab173147a4d015 #16672
Follow-up to e53523fef07894991c69d907a7c7794c7ada4ff4 #14859
Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979

Closes #16781

4 months agocmake: replace CMAKE_COMPILER_IS_GNUCC with CMAKE_C_COMPILER_ID
Peter Kokot [Sat, 22 Mar 2025 15:02:28 +0000 (16:02 +0100)] 
cmake: replace CMAKE_COMPILER_IS_GNUCC with CMAKE_C_COMPILER_ID

Variable `CMAKE_COMPILER_IS_GNUCC` is deprecated and
`CMAKE_C_COMPILER_ID` should be used instead.

Closes #16797

4 months agocmake: replace exec_program() with execute_process()
Peter Kokot [Thu, 20 Mar 2025 23:37:42 +0000 (00:37 +0100)] 
cmake: replace exec_program() with execute_process()

The `exec_program()` is deprecated as of CMake 3.0.

This also removes the `rm_out` variable as it isn't used in the output.
In `execute_process()` the `ERROR_QUIET` and `OUTPUT_QUIET` resemble
the behavior of `exec_program(OUTPUT_VARIABLE)` behavior in this case.

Closes #16779

4 months agotests: Remove unused variables
Samuel Henrique [Sat, 22 Mar 2025 15:55:59 +0000 (15:55 +0000)] 
tests: Remove unused variables

* tests/directories.pm:
  - %diroutput
* tests/ftpserver.pl:
  - %exit_signal
* tests/test1135.pl:
  - @syms
  - %doc
  - %rem
* tests/test1167.pl:
  - %doc
  - %rem
* tests/tftpserver.pl:
  - %fork

Closes #16798

4 months agocmake: fix typo in ECH config error msg
Daniel McCarney [Fri, 21 Mar 2025 13:54:10 +0000 (09:54 -0400)] 
cmake: fix typo in ECH config error msg

Just a small typo I noticed while working on wiring up rustls-ffi vTLS
ECH support.

Closes #16786

4 months agoautotools: add dependency of libtest and unit bundles to Makefile.inc
Stefan Eissing [Wed, 19 Mar 2025 13:48:38 +0000 (14:48 +0100)] 
autotools: add dependency of libtest and unit bundles to Makefile.inc

So that changes in libtests and unit tests are picked up.

Closes #16763

4 months agoconfigure: fix ECH detection with MultiSSL
Viktor Szakats [Thu, 20 Mar 2025 01:03:38 +0000 (02:03 +0100)] 
configure: fix ECH detection with MultiSSL

Detect OpenSSL and wolfSSL support independently. Pass detection if
either of them has support. Before this patch wolfSSL results overwrote
OpenSSL detection results when both backends were enabled.

Also fix output message when both of them support ECH.

Closes #16774

4 months agoGHA/linux: fix filter expressions for skipall/skiprun
Viktor Szakats [Wed, 19 Mar 2025 21:13:40 +0000 (22:13 +0100)] 
GHA/linux: fix filter expressions for skipall/skiprun

For cases when `install_steps` contains extra components.

After this patch, msh3 and rustls CM jobs skip building and running
tests, saving 2 minutes CI time, as originally intended.

Closes #16772

4 months agoGHA/linux: enable ECH in wolfssl-opensslextra
Viktor Szakats [Wed, 19 Mar 2025 20:44:05 +0000 (21:44 +0100)] 
GHA/linux: enable ECH in wolfssl-opensslextra

To have it in the coexist-capable wolfSSL local build. This allows
to test ECH combinations in MultiSSL builds with OpenSSL.

Also enable ECH in the wolfssl-opensslextra consumer job.

Closes #16773

4 months agoruntests: fix bundle detection by not looking for `units`
Viktor Szakats [Wed, 19 Mar 2025 21:35:26 +0000 (22:35 +0100)] 
runtests: fix bundle detection by not looking for `units`

`units` may be not be built in certain cases when using autotools.

Fixes:
https://app.circleci.com/pipelines/github/curl/curl/12669/workflows/8516da2b-b351-40b2-bf13-7c4ab4bcdd55/jobs/127197

Bug: https://github.com/curl/curl/pull/16750#issuecomment-2738041943
Follow-up to a9b7cbf34f8db80e8c05ee3680cafdce67ca9430 #16750

Closes #16770

4 months agoruntests: drop logic calling the `handle` tool (Windows)
Viktor Szakats [Tue, 25 Feb 2025 17:13:38 +0000 (18:13 +0100)] 
runtests: drop logic calling the `handle` tool (Windows)

In the cases observed throughout the last year, `handle64` run once per
test run, but with no action (match or task kill). It did not help with
flakiness and seems redundant.

runtests launched it (if present) in Cygwin/MSYS jobs too, where it
probably shouldn't have, because we have seen no flakiness there. In CI
the tool was present and launched in MSYS2 jobs, but not in Cygwin.

After this patch the "clearlocks" warning remain in the log. They are
consistently appearing once in every MSVC CI log, early in the tests:
```
  test 3207 SKIPPED: curl lacks OpenSSL support
[...START-OF-TESTS...]
  test 0003...[HTTP POST with auth and contents but with content-length set to 0]
  --pd---e--- OK (3   out of 1596, remaining: 17:50, took 1.423s, duration: 00:02)
  test 0007...[HTTP with cookie parser and header recording]
  --pd--oe--- OK (7   out of 1596, remaining: 07:51, took 1.485s, duration: 00:02)
  test 0006...[HTTP with simple cookie send]
  --pd---e--- OK (6   out of 1596, remaining: 09:11, took 1.488s, duration: 00:02)
  test 0005...[HTTP over proxy]
  --pd---e--- OK (5   out of 1596, remaining: 11:03, took 1.491s, duration: 00:02)
CUSTOMBUILD : error : 169: cleardir(log/8/lock) failed [D:\a\curl\curl\bld\tests\test-ci.vcxproj]
  test 0001...[HTTP GET]
  --pd---e--- OK (1   out of 1596, remaining: 55:34, took 1.466s, duration: 00:02)
  test 0004...[Replaced internal and added custom HTTP headers]
```
Ref: https://github.com/curl/curl/actions/runs/13546192228/job/37858323380?pr=16484#step:14:167

Ref: e53523fef07894991c69d907a7c7794c7ada4ff4 #14859
Ref: 311c31ec8e721b11ba77adf7a3865cf0cd30aa42 #6179
Follow-up to 3a8920e5edaead8304a818594f54485a5564f976 #16600
Closes #16484

4 months agoconfig-win32: set `HAVE_STDINT_H` where available
Viktor Szakats [Tue, 18 Mar 2025 10:17:01 +0000 (11:17 +0100)] 
config-win32: set `HAVE_STDINT_H` where available

For winbuild and VS project builds.

Follow-up to 1ecf2f11136f967deff6d2c124127250d2756c48 #16585

Closes #16759

4 months agoruntests: auto-detect test bundle builds
Viktor Szakats [Mon, 17 Mar 2025 17:46:42 +0000 (18:46 +0100)] 
runtests: auto-detect test bundle builds

If libtests, units and servers binaries are all present, auto-enable
bundle mode.

Drop manual runtests option.

Note: Make sure to "make clean" before changing the test bundle build
setting.

Also fix to append executable extension to all libtest and unit test
executables when launching them. This should make it a tiny bit faster
on Windows.

Follow-up to f4f25505df1d2ede727071eb0c518368a206f64b #15000
Follow-up to 71cf0d1fca9e1f53524e1545ef0c08d174458d80 #14772

Closes #16750

4 months agoGHA/macos: verbose cmake in test steps
Viktor Szakats [Tue, 18 Mar 2025 03:02:47 +0000 (04:02 +0100)] 
GHA/macos: verbose cmake in test steps

To sync with other workflows, and autotools. To see build details,
including clang-tidy invocations.

Closes #16757

4 months agolib696: fix building on Windows in non-bundle mode
Viktor Szakats [Mon, 17 Mar 2025 21:10:51 +0000 (22:10 +0100)] 
lib696: fix building on Windows in non-bundle mode

```
FAILED: tests/libtest/lib696.exe
[...] undefined reference to `curlx_write'
```
Ref: https://github.com/curl/curl/actions/runs/13909717720/job/38921073575?pr=16750#step:12:469

Closes #16753

4 months agoruntests: add feature-based filtering
Aquila Macedo [Fri, 7 Mar 2025 21:40:34 +0000 (18:40 -0300)] 
runtests: add feature-based filtering

This commit introduces support for features in the test selection
process by adding them to the keywords list with the `feat:` prefix. It
allows users to specify features to run only tests with them, or exclude
tests using `!feat:<feature>`, similar to how keywords are handled.

Fixes #16533
Closes #16619

Signed-off-by: Aquila Macedo <aquilamacedo@riseup.net>
4 months agotests: git ignore the bundle executables built
Stefan Eissing [Mon, 17 Mar 2025 14:36:34 +0000 (15:36 +0100)] 
tests: git ignore the bundle executables built

Closes #16749

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 17 Mar 2025 15:45:23 +0000 (16:45 +0100)] 
RELEASE-NOTES: synced

4 months agoGHA/windows: skip flaky test 498
Viktor Szakats [Mon, 17 Mar 2025 13:05:12 +0000 (14:05 +0100)] 
GHA/windows: skip flaky test 498

On suspect of strain on the runtime env/pipes, disable this test, which
is flaky due to `runtests` detecting a 2009 result code from curl, while
curl is returning the expected 56:

```
test 0498...[Reject too large HTTP response headers on endless redirects]

curl returned 2009, when expecting 56
 498: exit FAILED
== Contents of files in the log/5/ dir after test 498
[...]
   0     0    0     0    0     0      0      0 --:--:--  0:00:10 --:--:--     0
 curl: (56) Too large response headers: 6144086 > 6144000
```

In such cases the number of log lines for this single test is 4800. In
comparison the total number of log lines for a clear test run is 3800.

Seen with mingw, dl-mingw, msvc CI jobs.

Follow-up to 4911e7af119c1b7efd46a742d47bca44832c3041 #16735
Ref: https://github.com/curl/curl/discussions/14854#discussioncomment-12503065

Closes #16748

4 months agomemdebug: revert setting gcc 11+ deallocator attribute
Viktor Szakats [Mon, 17 Mar 2025 00:05:14 +0000 (01:05 +0100)] 
memdebug: revert setting gcc 11+ deallocator attribute

These attributes were causing unexplained warnings while playing with
PR #16738: In `CURLDEBUG` builds with mingw, gcc (14.2.0), and `-O3`,
while building `libcurlu`. `-O3` is required. May be related to having
the `CURLDEBUG` allocators in the same source file as their callers
(unity mode). PR #16738 moves `memdebug.c` into the main unity unit.
Unclear why it doesn't affect `libcurl`.

E.g. CI job `mingw, CM ucrt-x86_64 schannel R TrackMemory` CI job:
https://github.com/curl/curl/actions/runs/13888662354/job/38856868429

It also reproduces in an isolated example.

Drop this attribute till we learn more about it.

Ref: https://github.com/curl/curl/pull/16737#issuecomment-2727681306
Partial revert of d5b403074ed2b28e41e0c17dcd436252d38adfa3 #16737

Closes #16740

4 months agoGHA/windows: drop redundant manual changes to MSYS2 config
Viktor Szakats [Sun, 16 Mar 2025 22:14:31 +0000 (23:14 +0100)] 
GHA/windows: drop redundant manual changes to MSYS2 config

It's done automatically by the `msys2/setup-msys2` action.

Suggested-by: Jeremy Drake
Ref: https://github.com/curl/curl/pull/16672#discussion_r1997699803

Closes #16739

4 months agotests/server: support bundle binary
Viktor Szakats [Sun, 8 Sep 2024 22:33:04 +0000 (00:33 +0200)] 
tests/server: support bundle binary

Extend existing `--enable-test-bundles` and `-DCURL_TEST_BUNDLES=ON`
options to also bundle test server programs into a single binary. With
autotools, also bundle auxiliary libcurl sources for a "unity"-style
build.

It saves almost 10 minutes per run, across all CI jobs.
On average it makes `build tests` steps 25% faster.

With CMake, it brings down `testdeps` build steps to 32 to 37, from
45 to 64 before this patch, with unity. Without unity it brings it down
from 400-420 to 280-300. For comparison, without unity and bundles,
the number of build steps is around 1850.

With autotools the gain is possibly larger because this patch does unity
and bundle for test servers.

The total reduction of build steps / log lines is 12000. It's 44% of
reduction on average across all CI jobs.

Follow-up to 77401af4852103b215fe74989e89be61bbf435d2 #16695
Follow-up to 71cf0d1fca9e1f53524e1545ef0c08d174458d80 #14772

Comparison of 'build tests' targets in S(econds) and L(ines/steps),
between before and after this patch:

CI job                                    |BefS|AftS|Gain|BefL|AftL|Gain| Comments
:---------------------------------------- | -: | -: | -: | --:| --:| --:| :-------
GHA/windows                               |    |    |    |    |    |    |
Cygwin AM                                 |110 | 95 | 15 | 237|  88| 149| l(ines)
Cygwin CM                                 | 34 | 27 |  7 |  63|  36|  27| s(teps, with Ninja)
msys2, AM x86_64 !proxy                   | 35 | 21 | 14 | 255| 108| 147| l
msys2, AM x86_64 default                  | 35 | 22 | 13 | 255| 108| 147| l
msys2, CM x86_64 default                  | 14 | 11 |  3 |  63|  36|  27| s
msys2, AM x86_64 default R                | 41 | 29 | 12 | 245|  96| 149| l
mingw, AM x86_64 default                  | 69 | 33 | 36 | 331| 156| 175| l
mingw, AM x86_64 c-ares U                 | 69 | 33 | 36 | 331| 156| 175| l
mingw, CM x86_64 schannel c-ares U        | 32 | 24 |  8 |  63|  36|  27| s
mingw, CM clang-x86_64 gnutls             |  9 |  6 |  3 |  63|  36|  27| s
mingw, CM ucrt-x86_64 schannel R TrackMem | 43 | 35 |  8 |  64|  37|  27| s
mingw, CM clang-x86_64 openssl            | 20 | 18 |  2 |  63|  36|  27| s
mingw, CM ucrt-x86_64 schannel uwp        | 36 | 34 |  2 |  47|  34|  13| s
mingw, CM x86_64 schannel dev debug       | 32 | 24 |  8 |  64|  37|  27| s
mingw, CM i686 schannel R                 | 38 | 37 |  1 |  47|  34|  13| s
dl-mingw, CM 9.5.0-x86_64 schan           |102 | 94 |  8 |  63|  36|  27| s
dl-mingw, CM 7.3.0-x86_64 schan mbedtls U | 32 | 24 |  8 |  94|  60|  34| l
dl-mingw, CM 6.4.0-i686 schannel !unity   | 42 | 31 | 11 | 427| 297| 130| l (no unity, yes bundle)
linux-mingw, AM gcc                       |  - |  - |    |   -|   -|    |
linux-mingw, CM gcc                       | 12 | 10 |  2 |  46|  33|  13| s
mingw32ce, AM 4.4.0-arm schannel          |  - |  - |    |   -|   -|    |
mingw32ce, CM 4.4.0-arm schannel          | 30 | 31 | -1 |  81|  61|  20| l
msvc, CM x64-uwp openssl +examples        | 13 | 10 |  3 | 102|  68|  34| l
msvc, CM x64-windows openssl              | 13 | 10 |  3 | 102|  68|  34| l
msvc, CM x64-windows schannel MultiSSL U  | 13 | 10 |  3 | 102|  68|  34| l
msvc, CM x64-windows libressl             | 14 | 13 |  1 | 102|  68|  34| l
msvc, CM x64-windows boringssl            | 11 | 11 |  0 |  84|  64|  20| l
msvc, CM x64-windows wolfssl +examples    | 13 | 10 |  3 | 102|  68|  34| l
msvc, CM x64-windows mbedtls libssh       | 14 | 10 |  4 | 102|  68|  34| l
GHA/old-linux:                            |    |    |    |    |    |    |
linux (cmake & autoconf) AM               | 14 | 11 |  3 | 217|  75| 142| l
linux (cmake & autoconf) CM               | 15 | 13 |  2 | 545| 380| 165| l
GHA/non-native:                           |    |    |    |    |    |    |
AmigaOS, AM gcc AmiSSL m68k               |  - |  - |    |   -|   -|    |
AmigaOS, CM gcc AmiSSL m68k               |  8 |  7 |  1 |  78|  58|  20| l
Android 21, AM openssl arm64              |  7 |  5 |  2 | 222|  80| 142| l
Android 21, CM openssl arm64              |  4 |  5 | -1 |  81|  61|  20| l
Android 35, AM openssl arm64              |  7 |  5 |  2 | 222|  80| 142| l
Android 35, CM boringssl !zstd arm64      |  4 |  3 |  1 |  81|  61|  20| l
Android 35, CM openssl arm64              |  5 |  4 |  1 |  81|  61|  20| l
FreeBSD, AM clang openssl arm64           |  - |  - |    |   -|   -|    |
FreeBSD, AM clang openssl x86_64          |  7 |  3 |  4 | 513| 133| 380| l
FreeBSD, CM clang openssl !unity !bundle..| 49 | 47 |  2 |1841|1841|    | s (no unity, no bundle)
FreeBSD, CM clang openssl arm64           |  - |  - |    |   -|   -|    |
MS-DOS, AM djgpp openssl i586             |  - |  - |    |   -|   -|    |
MS-DOS, CM djgpp openssl i586             |  6 |  7 | -1 |  45|  32|  13| s
NetBSD, CM clang openssl x86_64           | 15 | 13 |  2 |  62|  35|  27| s
OmniOS, AM gcc openssl amd64              | 20 | 11 |  9 | 216|  74| 142| l
OpenBSD, CM clang libressl x86_64         |  9 |  7 |  2 |  62|  35|  27| s
iOS, AM libressl arm64                    |  6 |  3 |  3 | 217|  68| 149| l
iOS, CM libressl arm64                    | 10 | 10 |  0 | 236| 175|  61| l
iOS, CM-Xcode libressl arm64              | 13 |  8 |  5 | 899| 627| 272| l
GHA/linux:                                |    |    |    |    |    |    |
AM bearssl                                |  9 |  8 |  1 | 258| 111| 147| l
AM bearssl clang                          |  6 |  4 |  2 | 258| 111| 147| l
AM libressl heimdal                       | 11 |  8 |  3 | 267| 120| 147| l
CM libressl heimdal valgrind              |  8 |  5 |  3 |  62|  35|  27| s
AM libressl clang                         |  7 |  4 |  3 | 258| 111| 147| l
AM wolfssl-all                            | 11 |  8 |  3 | 258| 111| 147| l
AM wolfssl-opensslextra valgrind          | 11 |  7 |  4 | 258| 111| 147| l
AM mbedtls valgrind                       | 10 |  7 |  3 | 258| 111| 147| l
AM mbedtls clang                          |  6 |  3 |  3 | 258| 111| 147| l
CM mbedtls                                |  7 |  6 |  1 |  62|  35|  27| s
CM mbedtls-pkg                            |  7 |  7 |  0 |  62|  35|  27| s
CM mbedtls-pkg !pc                        |  - |  - |    |   -|   -|    |
AM msh3                                   | 10 |  7 |  3 | 258| 111| 147| l
CM msh3                                   |  7 |  7 |  0 |  62|  35|  27| s
AM awslc                                  | 18 | 15 |  3 | 247|  98| 149| l
CM awslc                                  | 13 | 11 |  2 | 400| 277| 123| s (no unity, yes bundle)
AM openssl default                        | 10 |  7 |  3 | 258| 111| 147| l
AM openssl libssh2 sync-resolver valgrind | 10 |  7 |  3 | 258| 111| 147| l
AM openssl                                | 11 |  7 |  4 | 258| 111| 147| l
AM openssl -O3 valgrind                   | 26 | 21 |  5 | 258| 111| 147| l
AM openssl clang krb5                     |  7 |  3 |  4 | 265| 118| 147| l
CM openssl clang krb5 LTO                 |  - |  - |    |   -|   -|    |
AM openssl !ipv6 !--libcurl               | 10 |  7 |  3 | 258| 111| 147| l
AM openssl https-only                     | 11 |  8 |  3 | 258| 111| 147| l
CM openssl torture !FTP                   |  7 |  6 |  1 |  62|  35|  27| s
CM openssl torture FTP                    |  8 |  6 |  2 |  62|  35|  27| s
AM openssl i686                           | 12 |  9 |  3 | 258| 111| 147| l
AM !ssl !http !smtp !imap                 | 10 |  7 |  3 | 258| 111| 147| l
AM clang-tidy                             |  - |  - |    |   -|   -|    |
AM scanbuild                              |  - |  - |    |   -|   -|    |
AM address-sanitizer                      | 10 |  6 |  4 | 258| 111| 147| l
AM thread-sanitizer                       |  7 |  4 |  3 | 258| 111| 147| l
AM memory-sanitizer                       | 10 |  6 |  4 | 258| 111| 147| l
AM event-based                            | 10 |  7 |  3 | 253| 106| 147| l
AM duphandle                              | 10 |  8 |  2 | 253| 106| 147| l
AM rustls valgrind                        | 10 |  8 |  2 | 258| 111| 147| l
CM rustls                                 |  7 |  5 |  2 |  62|  35|  27| s
AM IntelC openssl                         | 18 | 14 |  4 | 572| 282| 290| l
AM Slackware openssl gssapi gcc           | 17 | 14 |  3 | 252| 103| 149| l
AM Alpine MUSL https-rr                   | 15 | 11 |  4 | 266| 119| 147| l
AM Alpine MUSL c-ares https-rr            | 15 | 11 |  4 | 266| 119| 147| l
GHA/linux-http3:                          |    |    |    |    |    |    |
AM quictls                                | 12 |  8 |  4 | 258| 111| 147| l
AM gnutls                                 | 11 |  8 |  3 | 257| 110| 147| l
AM wolfssl                                | 11 |  9 |  2 | 257| 110| 147| l
CM wolfssl                                |  8 |  7 |  1 |  62|  35|  27| s
AM openssl-quic                           | 11 |  8 |  3 | 257| 110| 147| l
AM quiche                                 | 11 |  9 |  2 | 257| 110| 147| l
CM quiche                                 |  8 |  6 |  2 |  62|  35|  27| s
GHA/macos:                                |    |    |    |    |    |    |
AM clang !ssl !debug brotli zstd          |  5 |  3 |  2 | 218|  69| 149| l
AM clang !ssl                             |  4 |  5 | -1 | 227|  80| 147| l
AM clang !ssl libssh2 AppleIDN            |  5 |  1 |  4 | 227|  80| 147| l
AM clang OpenSSL libssh c-ares            |  4 |  2 |  2 | 227|  80| 147| l
AM clang !ssl c-ares                      |  4 |  5 | -1 | 227|  80| 147| l
AM clang !ssl HTTP-only                   |  4 |  2 |  2 | 222|  75| 147| l
AM clang SecureTransport libssh2          |  3 |  3 |  0 | 227|  80| 147| l
AM clang SecureTransport libssh2 10.12    |  4 |  2 |  2 | 227|  80| 147| l
AM clang LibreSSL +examples               |  5 |  2 |  3 | 227|  80| 147| l
AM clang OpenSSL                          |  6 |  1 |  5 | 227|  80| 147| l
AM clang OpenSSL event-based              |  5 |  3 |  2 | 227|  80| 147| l
AM clang quictls libssh2 !ldap 10.15      |  4 |  2 |  2 | 227|  80| 147| l
CM clang OpenSSL gsasl rtmp AppleIDN      |  2 |  2 |  0 |  45|  32|  13| s
CM clang OpenSSL AppleIDN clang-tidy +e   |150 | 76 | 74 | 400| 277| 123| s (clang-tidy)
CM clang quictls +static libssh +examples |  2 |  2 |  0 |  45|  32|  13| s
CM clang SecureTransport debug            |  2 |  2 |  0 |  62|  35|  27| s
CM clang LibreSSL !ldap heimdal c-ares +e |  4 |  2 |  2 |  45|  32|  13| s
CM clang wolfSSL !ldap brotli zstd        |  1 |  2 | -1 |  45|  32|  13| s
CM clang mbedTLS openldap brotli zstd     |  1 |  2 | -1 |  45|  32|  13| s
CM clang GnuTLS !ldap krb5                |  0 |  1 |  0 |  45|  32|  13| s
CM clang OpenSSL torture !FTP             |  2 |  1 |  1 |  61|  34|  27| s
CM clang OpenSSL torture FTP              |  2 |  1 |  1 |  61|  34|  27| s
AM llvm@15 OpenSSL libssh                 |  5 |  4 |  1 | 227|  80| 147| l
CM llvm@15 OpenSSL gsasl rtmp AppleIDN    |  3 |  2 |  1 |  45|  32|  13| s
CM llvm@15 quictls +static libssh +e      |  2 |  2 |  0 |  45|  32|  13| s
CM llvm@15 SecureTransport debug          |  3 |  2 |  1 |  62|  35|  27| s
CM llvm@15 LibreSSL !ldap heimdal c-ares..|  3 |  1 |  2 |  45|  32|  13| s
CM llvm@15 wolfSSL !ldap brotli zstd      |  3 |  2 |  1 |  45|  32|  13| s
CM llvm@15 mbedTLS openldap brotli zstd   |  2 |  2 |  0 |  45|  32|  13| s
CM llvm@15 GnuTLS !ldap krb5              |  2 |  1 |  1 |  45|  32|  13| s
AM gcc-12 !ssl !debug                     | 18 | 12 |  6 | 218|  69| 149| l
AM gcc-12 SecureTransport libssh2         | 13 |  6 |  7 | 227|  80| 147| l
CM gcc-12 OpenSSL gsasl rtmp AppleIDN     |  7 |  5 |  2 |  45|  32|  13| s
CM gcc-12 quictls +static libssh +e       |  8 |  6 |  2 |  45|  32|  13| s
CM gcc-12 SecureTransport debug           |  8 |  6 |  2 |  62|  35|  27| s
CM gcc-12 LibreSSL !ldap heimdal c-ares...|  7 |  6 |  1 |  45|  32|  13| s
CM gcc-12 wolfSSL !ldap brotli zstd       | 16 |  6 | 10 |  45|  32|  13| s
CM gcc-12 mbedTLS openldap brotli zstd    | 11 |  6 |  5 |  45|  32|  13| s
CM gcc-12 GnuTLS !ldap krb5               |  7 |  5 |  2 |  45|  32|  13| s
AppVeyor:                                 |    |    |    |    |    |    |
VS2008 D x86 OpenSSL + Schannel SH +e     | 56 | 39 | 17 |  21|  14|   7| s (VCBuild)
VS2010 D x64 Schannel SH +e               | 14 | 12 |  2 | 105|  78|  27| l
VS2012 R x86 OpenSSL + Schannel SH        | 23 | 21 |  2 |  84|  64|  20| l
VS2013 D x64 OpenSSL SH Build-only        |  - |  - |    |   -|   -|    | (no build tests)
VS2015 D x64 OpenSSL ST Build-only        |  - |  - |    |   -|   -|    | (no build tests)
VS2017 D x64 OpenSSL SH Build-only        |  - |  - |    |   -|   -|    | (no build tests)
VS2019 D x64 OpenSSL + Schannel SH        | 33 | 26 |  7 |  81|  61|  20| l
VS2022 D x64 !SSL ST                      | 32 | 25 |  7 |  79|  59|  20| l
VS2022 D x64 !SSL ST HTTP-only            | 33 | 22 | 11 |  79|  59|  20| l
VS2022 R arm64 Schannel ST                | 74 | 69 |  5 |  79|  59|  20| l
VS2022 R x64 Schannel SH U DBGBLD !CURLDBG| 66 | 61 |  5 |  78|  58|  20| l
VS2022 D x64 Schannel ST U                | 34 | 25 |  9 |  79|  59|  20| l
VS2022 D x64 Schannel ST U clang-cl +e    | 42 | 34 |  8 |  53|  40|  13| l
VS2022 R x64 OpenSSL SH                   | 60 | 58 |  2 |  83|  63|  20| l
Total                                     |2394|1807|-587| 26k| 14k|-12k|
in %                                      |    |    |-24%|    |    |-45%|
in minutes                                |    |    |9m47|    |    |    |

Before:
GHA/windows: https://github.com/curl/curl/actions/runs/13854983424
GHA/old-linux: https://github.com/curl/curl/actions/runs/13854983399
GHA/non-native: https://github.com/curl/curl/actions/runs/13854983427
GHA/linux-http3: https://github.com/curl/curl/actions/runs/13854983409
GHA/linux: https://github.com/curl/curl/actions/runs/13854983406
GHA/macos: https://github.com/curl/curl/actions/runs/13854983401
Appveyor: https://ci.appveyor.com/project/curlorg/curl/builds/51703551

After:
GHA/windows: https://github.com/curl/curl/actions/runs/13860433850?pr=15000
GHA/old-linux: https://github.com/curl/curl/actions/runs/13860433809?pr=15000
GHA/non-native: https://github.com/curl/curl/actions/runs/13860433828?pr=15000
GHA/linux-http3: https://github.com/curl/curl/actions/runs/13860433806?pr=15000
GHA/linux: https://github.com/curl/curl/actions/runs/13860433848?pr=15000
GHA/macos: https://github.com/curl/curl/actions/runs/13860433835?pr=15000
Appveyor: https://ci.appveyor.com/project/curlorg/curl/builds/51704222

Closes #15000

4 months agomemdebug: set gcc 11+ deallocator attribute, extend alloc attributes to clang
Viktor Szakats [Sat, 15 Mar 2025 22:22:19 +0000 (23:22 +0100)] 
memdebug: set gcc 11+ deallocator attribute, extend alloc attributes to clang

To make `-Wfree-nonheap-object` and `-Wmismatched-dealloc` work in
`CURLDEBUG` builds.

Also extend `ALLOC_FUNC` and `ALLOC_SIZE` attribute support
to llvm/clang.

llvm/clang is missing the deallocator attribute, tracked here:
https://github.com/llvm/llvm-project/issues/129068

Ref: https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Common-Function-Attributes.html#Common-Function-Attributes
Ref: https://www.gnu.org/software/gcc/gcc-11/changes.html
Ref: 6b143d9cc13fcd208480f678dfd06bf97bde4998 #16734

Closes #16737

4 months agoGHA/windows: unignore 2310, disable SCP/FTP for vcpkg `libssh2[core]`
Viktor Szakats [Sat, 15 Mar 2025 11:03:56 +0000 (12:03 +0100)] 
GHA/windows: unignore 2310, disable SCP/FTP for vcpkg `libssh2[core]`

Skipping these tests saves time and reduces test logs from 11500 lines
to 3800.

Tests are permanently broken due to `curl: (67) Authentication failure`.
This libssh2 is built with WinCNG. Builds using libcrypto from OpenSSL
work fine.

Closes #16735

4 months agoasyn-thread: drop `free()` on non-heap address
Viktor Szakats [Sat, 15 Mar 2025 01:39:05 +0000 (02:39 +0100)] 
asyn-thread: drop `free()` on non-heap address

seen with mingw-w64 gcc 14.2.0 while playing with other modifications:
```
lib/asyn-thread.c: In function 'init_resolve_thread':
lib/asyn-thread.c:447:5: warning: 'free' called on pointer 'data' with nonzero offset 3264 [-Wfree-nonheap-object]
  447 |     free(td);
      |     ^~~~~~~~
```

Where `td` is:
```c
  struct thread_data *td = &data->state.async.thdata;
```

Follow-up to d9fc64d3ab289a84548e952183d7eba79ccc846e #16241
Closes #16734