]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
3 months agoasyn resolver code improvements
Stefan Eissing [Fri, 11 Apr 2025 12:43:45 +0000 (14:43 +0200)] 
asyn resolver code improvements

"asyn" is the internal name under which both c-ares and threaded
resolver operate. Make the naming more consistent. Implement the c-ares
resolver in `asyn-ares.*` and the threaded resolver in `asyn-thrdd.*`.
The common functions are in `asyn-base.c`.

When `CURLRES_ASYNCH` is defined, either of the two is used and
`data->state.async` exists. Members of that struct vary for the selected
implementation, but have the fields `hostname`, `port` and `ip_version`
always present. This are populated when the async resolving starts and
eliminate the need to pass them again when checking on the status and
processing the results of the resolving.

Add a `Curl_resolv_blocking()` to `hostip.h` that relieves FTP and SOCKS
from having to repeat the same code.

`Curl_resolv_check()` remains the function to check for status of
ongoing resolving. Now it also performs internally the check if the
needed DNS entry exists in the dnscache and if so, aborts the asnyc
operation. (libcurl right now does not check for duplicate resolve
attempts. an area for future improvements).

The number of functions in `asyn.h` has been reduced. There were subtle
difference in "cancel()" and "kill()" calls, both replaced by
`Curl_async_shutdown()` now. This changes behaviour for threaded
resolver insofar as the resolving thread is now always joined unless
`data->set.quick_exit` is set. Before this was only done on some code
paths. A future improvement would be a thread pool that keeps a limit
and also could handle joins more gracefully.

DoH, not previously tagged under "asny", has its struct `doh_probes` now
also in `data->state.async`, moved there from `data->req` because it
makes more sense. Further integration of DoH underneath the "asyn"
umbrella seems like a good idea.

Closes #16963

3 months agoGHA/windows: switch a job to ARM64 on native runner
Viktor Szakats [Tue, 15 Apr 2025 16:17:47 +0000 (18:17 +0200)] 
GHA/windows: switch a job to ARM64 on native runner

Switch a build-only MSYS2/mingw-w64 job to ARM64 using the ARM64 Windows
runner. This avoids most downsides of native ARM64 builds. Side-effect
is switching this job from GCC to clang. This adds speed, which offsets
the slower MSYS2 install step.

The new ARM64 runner hits a bunch of corner cases and inefficiencies:
- MSYS2/Cygwin misses native ARM64 support and this isn't expected to
  change. It means GH action installs x64 binaries on ARM64. vcpkg does
  the same. It runs, but not native, so slower. It affects runtests and
  possibly vcpkg build performance.
  https://github.com/msys2/MSYS2-packages/discussions/2889
  https://cygwin.com/pipermail/cygwin/2021-December/250156.html
- MSYS2 diffutils package missing for CLANGARM64.
- vcpkg boringssl build broken for ARM64.

Also:
- bump the vcpkg build time limit, as it seems the native x86_64 jobs
  also can't fit into 35 minutes.
- prepare MSVC jobs for the ARM64 runner.

Ref: https://github.com/github/roadmap/issues/1098#issuecomment-2806476117
Ref: https://github.blog/changelog/2025-04-14-windows-arm64-hosted-runners-now-available-in-public-preview/

Closes #17067

3 months agoschannel: handle pkcs12 client certificates which contain CA certificates
Joel Depooter [Mon, 24 Mar 2025 23:06:24 +0000 (16:06 -0700)] 
schannel: handle pkcs12 client certificates which contain CA certificates

The SChannel code uses the CertFindCertificateInStore function to
retrieve the client certificate from a pkcs12 certificate store.
However, when called with the CERT_FIND_ANY flag, this function does not
provide any guarantees on the order in which certificates are retrieved.
If a pkcs12 file contains an entire certificate chain instead of a
single client certificate, the CertFindCertificateInStore function may
return the CA or an intermediate certificate instead of the desired
client certificate. Since there is no associated private key for such a
certificate, the TLS handshake fails.

With this change, we now pass the CERT_FIND_HAS_PRIVATE_KEY flag. This
ensures that the CertFindCertificateInStore function will return a
certificate which has a corresponding private key. This will stop the CA
and intermediate certificates from being selected. I don't think there
would be much use in a client certificate which has no associated
private key, so this should ensure the client certificate is selected. I
suppose it may be possible for a pkcs12 file to contain multiple
certificates with private keys and the new behaviour may not guarantee
which is selected. However, this is no worse that the previous behaviour
in which any certificate may been selected.

The CERT_FIND_HAS_PRIVATE_KEY is only available in Windows 8 / Server
2012 (aka Windows NT6.2). For older versions, we will fall back to using
the CERT_FIND_ANY flag.

Closes #16825

3 months agohttp: in alt-svc negotiation only allow supported HTTP versions
Sören Tempel [Sat, 12 Apr 2025 16:09:47 +0000 (18:09 +0200)] 
http: in alt-svc negotiation only allow supported HTTP versions

Without this patch, the handling of the alt-svc header added via
279a4772ae67dd4d9770e11e60040f9113b1c345 in curl-8.13.0 attempts to
connect to alternative services via different HTTP versions, even if the
target HTTP version is not supported by curl (i.e., not enabled at
compile-time). If I understand the code and RFC 7838 correctly, then we
should only attempt to migrate to supported protocols. Therefore,
`allowed_apns` should only contain such protocols, and we need to guard
its modification with `ifdefs` for supported HTTP versions.

This was discovered in a downstream bug report in Alpine Linux [1] where
it was reported that a Matrix client (using libcurl) was defunct after
the upgrade to curl-8.13.0. Further debugging revealed that this was due
to the Matrix server sending a `alt-svc: h3=":443";` HTTP header,
causing curl to attempt migration to HTTP3 even though Alpine's curl
version is compiled without HTTP3 support.

I am not sure if this is the best place in the code to address this
or if the `allowed` bitmask shouldn't contain unsupported versions
in the first place. However, since there are existing `ifdefs` in
this function for source (not destination) ALP selection, it may
be a good fit to address this here.

[1]: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17062

Closes #17037

3 months agoGHA/windows: bump a job to windows-2025
Viktor Szakats [Tue, 15 Apr 2025 14:59:11 +0000 (16:59 +0200)] 
GHA/windows: bump a job to windows-2025

- bump an MSYS2/mingw job to windows-2025 runner.
  (MSVC is possible, but vcpkg needs to build for windows-2025, and
  can't share these with windows-2022 builds, so not optimal for
  a single canary job.)
- skip installing OpenSSH-Windows-builtin on windows-2025.
  It's preinstalled:
  ```
  ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
  ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 9.5.0
  ```
  Still older than the manual preview install (9.8.1), so keep using that.

Closes #17066

3 months agocurl_get_line: handle lines ending on the buffer boundary
Cole Helbling [Sat, 12 Apr 2025 04:37:11 +0000 (21:37 -0700)] 
curl_get_line: handle lines ending on the buffer boundary

Very similar to 9f8bdd0eae5c1d441d9d901a7cf917a8ee215c7f, but affects
e.g. netrc file parsing.

Suggested-by: Graham Christensen <graham@grahamc.com>
Add test 744 to verify

Closes #17036

3 months agoeasy_reset: fix dohfor_mid member
Stefan Eissing [Tue, 15 Apr 2025 09:27:12 +0000 (11:27 +0200)] 
easy_reset: fix dohfor_mid member

On an easy reset, the dohfor_mid must be reset to -1.

Reported-by: epicmkirzinger on github
Fixes #17052
Closes #17058

3 months agodict: move internal defines to dict.c
Stefan Eissing [Tue, 15 Apr 2025 10:24:30 +0000 (12:24 +0200)] 
dict: move internal defines to dict.c

Move defines only used in dict.c from urldata.h to implementation.

Closes #17060

3 months agotool_cb_write.c: handle EINTR on flush
Stefan Eissing [Tue, 15 Apr 2025 12:01:19 +0000 (14:01 +0200)] 
tool_cb_write.c: handle EINTR on flush

Report-and-patch-by: Nils Goroll
Fixes #17061
Closes #17063

3 months agocmake: merge `CURL_WERROR` logic into `PickyWarnings.cmake`
Viktor Szakats [Tue, 15 Apr 2025 11:12:35 +0000 (13:12 +0200)] 
cmake: merge `CURL_WERROR` logic into `PickyWarnings.cmake`

Safe to do this now, as the code no longer relies on setting these
options after feature detection.

Also: Tidy up the way we handle options not to be passed to feature
checks, and make sure to show them in the configure log.

Follow-up to e86542038dda88dadf8959584e803895f979310c #17047
Closes #17062

3 months agowolfssl: fix to enable ALPN when available
Viktor Szakats [Tue, 15 Apr 2025 00:17:07 +0000 (02:17 +0200)] 
wolfssl: fix to enable ALPN when available

wolfSSL headers publish the `HAVE_ALPN` macro to tell if it has ALPN
support compiled in. Use that instead of `HAS_ALPN`, which was never
set.

Follow-up to edd573d98046963eb970a78bd1cd02de7f20349b #16167
Closes #17056

3 months agotests/server: fix typo in comment [ci skip]
Viktor Szakats [Tue, 15 Apr 2025 00:24:58 +0000 (02:24 +0200)] 
tests/server: fix typo in comment [ci skip]

3 months agocmake: append picky warnings to `CMAKE_REQUIRED_FLAGS` as string
Viktor Szakats [Mon, 14 Apr 2025 22:38:49 +0000 (00:38 +0200)] 
cmake: append picky warnings to `CMAKE_REQUIRED_FLAGS` as string

Also:
- drop unnecessary type conversion. `CMAKE_REQUIRED_FLAGS` is already
  space-separated.
  https://cmake.org/cmake/help/latest/module/CheckCSourceCompiles.html

Follow-up to e86542038dda88dadf8959584e803895f979310c #17047
Closes #17055

3 months agoprocesshelp.pm: always call `taskkill` with `-f` (force)
Viktor Szakats [Mon, 14 Apr 2025 21:47:30 +0000 (23:47 +0200)] 
processhelp.pm: always call `taskkill` with `-f` (force)

In the hope this avoid a possible hang in `taskkill`.

To kill processes, `runtests` first tries to kill them gently (with
"TERM", or on Windows `taskkill`), then waits some time for them
to disappear and then kills them with `KILL`, or on Windows with
`taskkill -f`. This happens within `killpid()`.

This patch bumps the gentle phase to `taskkill -f`. On the obervation
that a non-forced `taskkill` may hang in cases:

msvc, CM x64-windows wolfssl +examples:
```
  [...]
  test 3006...[SMTP with multiple invalid (all) --mail-rcpt and --mail-rcpt-allowfails]
  --p----e--- OK (1682 out of 1718, remaining: 00:04, took 0.524s, duration: 03:13)
  test 3005...[SMTP with multiple and invalid (all but one) --mail-rcpt and --mail-rcpt-allowfails]
  --p-u--e-Executing: 'taskkill -t -pid 1196 >nul 2>&1'
```
Ref: https://github.com/curl/curl/actions/runs/14445993473/job/40508986059?pr=17051#step:15:4176

Cancelling the job worked, resulting in a greyed out status, with the above
step and log entries lost.

If this change causes issues or does nothing at all, we may revert it
or limit it to CI runs.

Ref: #14854
Closes #17054

3 months agomake: clean tests better (tunits)
Viktor Szakats [Mon, 14 Apr 2025 21:44:57 +0000 (23:44 +0200)] 
make: clean tests better (tunits)

Sync clean target with other test bundles.

Follow-up to d3761bb84013ffd356753f943d687283afe203b5 #16986
Closes #17053

3 months agotests: fixup tunit tests for cmake
Viktor Szakats [Mon, 14 Apr 2025 12:11:49 +0000 (14:11 +0200)] 
tests: fixup tunit tests for cmake

Follow-up to 461ebbd336280ff843ec7971775e15f35134647d #16983

Closes #17051

3 months agocmake: prefer `COMPILE_OPTIONS` over `CMAKE_C_FLAGS` for custom C options
Viktor Szakats [Mon, 14 Apr 2025 07:50:30 +0000 (09:50 +0200)] 
cmake: prefer `COMPILE_OPTIONS` over `CMAKE_C_FLAGS` for custom C options

Also:
- pass `-D_GNU_SOURCE` via `COMPILE_DEFINITIONS`.
- make it explicit to pass these C flags to feature checks.
- update `_GNU_SOURCE` comment with `pipe2()`.
- enable `-pedantic-errors` picky option for GCC with CMake <3.23.
- drop redundant condition when stripping existing MSVC `/Wn` options.

CMake passes `CMAKE_C_FLAGS` to targets, feature checks and raw
`try_compile()` calls. With `COMPILE_OPTIONS`, this is limited to
targets, and we must explicitly pass them to feature checks. This
makes the build logic clearer, and offers more control. It also
reduces log noise by omitting these options from linker commands,
and from `CMAKE_C_FLAGS` dumps in feature checks.

Closes #17047

3 months agotests: add git ignores for tests/tunit generated files
Stefan Eissing [Mon, 14 Apr 2025 10:44:29 +0000 (12:44 +0200)] 
tests: add git ignores for tests/tunit generated files

Closes #17049

3 months agocmake: revert `CURL_LTO` behavior for multi-config generators
Viktor Szakats [Sat, 12 Apr 2025 18:49:13 +0000 (20:49 +0200)] 
cmake: revert `CURL_LTO` behavior for multi-config generators

To avoid having LTO enabled for Debug configurations with multi-config
generators (e.g. MSVC.)

Reported-by: PleaseJustDont
Fixes #17042
Ref: ##17034
Follow-up to a1eaa12a8397026aa34e04d6ed4280ddfe6c4c93 #15829
Closes #17043

3 months agoruntests: split `SSH_PWD` into `SCP_PWD` and `SFTP_PWD`, and more
Viktor Szakats [Sun, 23 Mar 2025 15:01:11 +0000 (16:01 +0100)] 
runtests: split `SSH_PWD` into `SCP_PWD` and `SFTP_PWD`, and more

To allow configuring paths styles for SCP and SFTP servers separately.

- make `scp://` URLs use `%SCP_PWD` (was: `%SSH_PWD`).
- make `%SCP_PWD` equal to `%POSIX_PWD`.
  To fix test 3022 with OpenSSH-Windows 9.8.0 server.
  The fix works on a local machine. Remains broken in CI.
  Before this patch, it was equal to `%FILE_PWD` when using
  OpenSSH-Windows, otherwise it was `%POSIX_PWD`.
  Notice that no matter what path-style we pass, test 3022
  was and still is broken with earlier OpenSSH-Windows versions.
  (as tested with 9.5.0, 9.5.0-beta20240403, 8.0.0.1)
- rename rest of `%SSH_PWD` uses to `%SFTP_PWD`.
- drop unused `%POSIX_PWD`.
- GHA/windows: test with OpenSSH-Windows server again.
  In the LibreSSL MSVC job. This job is short enough to fit the slow
  install of the built-in OpenSSH-Windows tools, if needed.

Follow-up to 1abb087a9c8f1e613b0b38b7afeffb61c18c2fed #5298
Ref: #16803
Closes #17041

3 months agoGHA/windows: add support for built-in OpenSSH-Windows
Viktor Szakats [Sun, 13 Apr 2025 10:01:00 +0000 (12:01 +0200)] 
GHA/windows: add support for built-in OpenSSH-Windows

On the windows-2022 runner it installs these client/server versions:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 8.1.0
```

Not currently enabled. Slight downside (when enabled) that Windows needs
over 1 minute to install these two tiny programs.

Closes #17046

3 months agoGHA/windows: bump Cygwin action, move package store to `D:`
Viktor Szakats [Sat, 12 Apr 2025 15:44:43 +0000 (17:44 +0200)] 
GHA/windows: bump Cygwin action, move package store to `D:`

- to benefit from the new download retry mechanism.
  https://github.com/cygwin/cygwin-install-action/pull/26

- to use a new setting that not only moves the Cygwin install target
  directory to the faster `D:` drive, but also the package download
  directory. Expecting a little performance improvement from this for
  the Cygwin install step.
  https://github.com/cygwin/cygwin-install-action/commit/d3a7464b92425a95438e794ec49927871dde78d2
  https://github.com/cygwin/cygwin-install-action/pull/27

Closes #17040

3 months agocmake: use the `LINK_OPTIONS` property with CMake 3.13+
Viktor Szakats [Sat, 12 Apr 2025 15:36:02 +0000 (17:36 +0200)] 
cmake: use the `LINK_OPTIONS` property with CMake 3.13+

Replacing the superseded `LINK_FLAGS` in these versions.

Follow-up to 7b1444979094a365c82c665cce0e2ebc6b69467b #14378
Closes #17039

3 months agocmake: fix nghttp3 static linking with `USE_OPENSSL_QUIC=ON`
bruce.yoon(윤병조)/kakao [Thu, 10 Apr 2025 06:22:06 +0000 (15:22 +0900)] 
cmake: fix nghttp3 static linking with `USE_OPENSSL_QUIC=ON`

Though cmake finds nghttp3 by pkg-config, nghttp3 isn't linked properly.
Because library directory is not given by -L.

Closes #17010

3 months agocpool/cshutdown: force close connections under pressure
Stefan Eissing [Fri, 11 Apr 2025 10:05:05 +0000 (12:05 +0200)] 
cpool/cshutdown: force close connections under pressure

when CURLMOPT_MAX_HOST_CONNECTIONS or CURLMOPT_MAX_TOTAL_CONNECTIONS
limits are reached, force close connections in shutdown to go below
limit when possible.

Fixes #17020
Reported-by: Fujii Hironori
Closes #17022

3 months agotool_parsecfg: make get_line handle lines ending on the buffer boundary
Daniel Stenberg [Fri, 11 Apr 2025 15:10:37 +0000 (17:10 +0200)] 
tool_parsecfg: make get_line handle lines ending on the buffer boundary

Add test 743 to verify.

Fixes #17030
Reported-by: Marius Kleidl
Closes #17031

4 months agoconfigure: catch asking for double resolver without https-rr
Daniel Stenberg [Fri, 11 Apr 2025 12:15:11 +0000 (14:15 +0200)] 
configure: catch asking for double resolver without https-rr

It is probably an unintentionally bad setup.

Found-by: Stefan Eissing
Closes #17025

4 months agodocs/INSTALL.md: drop reference to removed configure option
Daniel Stenberg [Fri, 11 Apr 2025 12:18:25 +0000 (14:18 +0200)] 
docs/INSTALL.md: drop reference to removed configure option

--disable-pthreads has been removed

Fixes #17023
Reported-by: mschroeder-fzj on github
Closes #17026

4 months agodocs/libcurl: make examples build with picky compiler options
Daniel Stenberg [Fri, 11 Apr 2025 12:40:48 +0000 (14:40 +0200)] 
docs/libcurl: make examples build with picky compiler options

Found by improving verify-examples.pl:

- Operate directly on markdown files to remove the need to render nroff files
  first.

- Add -Wall as a compiler option to find more issues

Closes #17028

4 months agodocs: add missing return statement in examples
Pavel Kropachev [Fri, 11 Apr 2025 10:57:14 +0000 (13:57 +0300)] 
docs: add missing return statement in examples

Closes #17024

4 months agoGHA/macos: re-generate gcc-13 hacklayer on macos-15
Viktor Szakats [Thu, 10 Apr 2025 14:27:11 +0000 (16:27 +0200)] 
GHA/macos: re-generate gcc-13 hacklayer on macos-15

Add workaround for an issue related to the gcc "hacklayer" after the
GitHub macos-15-arm64 runner bumped to 20250408.1231.

Fixes:
```
configure:5175: gcc-13 -o conftest  --sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk  -w conftest.c  >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:71,
                 from /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin24/13/include-fixed/stdio.h:75,
                 from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h: In function 'fmemopen':
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:457:107: error: expected declaration specifiers before '__API_AVAILABLE_GET_MACRO_93585900'
  457 | FILE *fmemopen(void * __restrict __buf _LIBC_SIZE(__size), size_t __size, const char * __restrict __mode) __API_AVAILABLE(macos(10.13), ios(11.0), tvos(11.0), watchos(4.0));
      |                                                                                                           ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/14378524390/job/40316589059?pr=17012#step:7:169

Assisted-by: Bo Anderson
Bug: https://github.com/curl/curl/pull/17012#issuecomment-2792572344
Bug: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2792601570

Closes #17017

4 months agotests: require IPv6 for 1265, 1324, 2086
Viktor Szakats [Thu, 10 Apr 2025 12:15:17 +0000 (14:15 +0200)] 
tests: require IPv6 for 1265, 1324, 2086

And also require HTTP. Also add `IPv6` to the keywords.

Fixing:

Linux AM openssl !ipv6 !--libcurl:
```
FAIL 1265: 'NO_PROXY with IPv6 numerical address' HTTP, HTTP proxy, http_proxy, NO_PROXY, noproxy
FAIL 1324: 'HTTP with --resolve and [ipv6address]' HTTP, HTTP GET, --resolve
FAIL 2086: 'Pre-request callback for HTTP IPv6' HTTP, IPv6
```
Ref: https://github.com/curl/curl/actions/runs/14378524385/job/40318328714?pr=17012#step:41:3789

Follow-up to a09e49168aae63640868b4fb9fa55eb384ff41c7 #17005

Closes #17014

4 months agodist: drop duplicate entry from `CMAKE_DIST`
Viktor Szakats [Thu, 10 Apr 2025 10:44:56 +0000 (12:44 +0200)] 
dist: drop duplicate entry from `CMAKE_DIST`

Closes #17012

4 months agoruntests: remove server verification after start
Daniel Stenberg [Wed, 9 Apr 2025 13:22:28 +0000 (15:22 +0200)] 
runtests: remove server verification after start

Since we start the server on our own port we know the server running is
us. By removing unnecessary verification we speed up tests a little.

Closes #17005

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 10 Apr 2025 06:45:02 +0000 (08:45 +0200)] 
RELEASE-NOTES: synced

4 months agoopenssl-quic: fix shutdown when stream not open
Stefan Eissing [Wed, 9 Apr 2025 09:01:54 +0000 (11:01 +0200)] 
openssl-quic: fix shutdown when stream not open

Check that h3 stream had been opened before telling nghttp3 to
shut it down.

Fixes #16998
Reported-by: Demi Marie Obenour
Closes #17003

4 months agoKNOWN_BUGS: fix link in sivg4 issue 16.3
Daniel Stenberg [Thu, 10 Apr 2025 06:14:14 +0000 (08:14 +0200)] 
KNOWN_BUGS: fix link in sivg4 issue 16.3

Fixes #17007
Reported-by: Demi Marie Obenour
Closes #17009

4 months agoHTTP3.md: fix incorrect variable placeholders
Jake Yuesong Li [Thu, 10 Apr 2025 02:07:53 +0000 (11:37 +0930)] 
HTTP3.md: fix incorrect variable placeholders

Closes #17008

4 months agocmake: quotes, whitespace, use `VERSION_GREATER_EQUAL`
Viktor Szakats [Tue, 8 Apr 2025 22:10:30 +0000 (00:10 +0200)] 
cmake: quotes, whitespace, use `VERSION_GREATER_EQUAL`

- `NOT` + `VERSION_LESS` -> `VERSION_GREATER_EQUAL`
  Available since 3.7, which is the minimum required for curl:
  https://cmake.org/cmake/help/latest/command/if.html#version-greater-equal
- make `CMAKE_REQUIRED_*` argument quotes consistent.
- make `CMAKE_REQUIRED_*` space alignment consistent.
- drop quote from version value for consistency with other cases.
- formatting

Closes #17002

4 months agotests/ech_tests.sh: sync shebang with rest of bash scripts
Viktor Szakats [Tue, 1 Apr 2025 11:56:22 +0000 (13:56 +0200)] 
tests/ech_tests.sh: sync shebang with rest of bash scripts

Closes #17001

4 months agocerts: drop unused `default_bits` from `.prm` files
Viktor Szakats [Wed, 9 Apr 2025 06:10:19 +0000 (08:10 +0200)] 
certs: drop unused `default_bits` from `.prm` files

Cert generation do not use these default values, some were also low,
and they were RSA-specific, and the generator recently switched to ECC.

Closes #16999

4 months agobuild: check required rustls-ffi version
Daniel McCarney [Wed, 2 Apr 2025 13:09:57 +0000 (09:09 -0400)] 
build: check required rustls-ffi version

Try to enforce that the Rustls vTLS backend is only used with
rustls-ffi 0.15 - the documentation already describes this as
the required version.

Follow-up from https://github.com/curl/curl/issues/16890

Closes #16922

4 months agocmake: use `INCLUDE_DIRECTORIES` prop to specify local header dirs
Viktor Szakats [Tue, 8 Apr 2025 09:01:09 +0000 (11:01 +0200)] 
cmake: use `INCLUDE_DIRECTORIES` prop to specify local header dirs

To use more modern cmake, and make it somewhat more obvious where these
header directories should apply.

Also move setting the directory property _before_ defining targets,
to make them inherit this directory property.

Ref: https://cmake.org/cmake/help/latest/command/include_directories.html
Ref: https://cmake.org/cmake/help/latest/prop_dir/INCLUDE_DIRECTORIES.html

Follow-up to 45f7cb7695b0fe0c61f71bdfbe31d161d50a5f51 #16238

Closes #16993

4 months agolib: include files using known path
Daniel Stenberg [Tue, 8 Apr 2025 06:00:12 +0000 (08:00 +0200)] 
lib: include files using known path

by including headers using "../[header]" when done from C files in
subdirectories, we do not need to specify the lib source dir as an
include path and we reduce the risk of header name collisions with
headers in the SDK using the same file names.

Idea-by: Kai Pastor
Ref: #16949
Closes #16991

4 months agoGHA: bump rojopolis/spellcheck-github-actions to 0.48.0
dependabot[bot] [Mon, 7 Apr 2025 15:59:10 +0000 (15:59 +0000)] 
GHA: bump rojopolis/spellcheck-github-actions to 0.48.0

Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases)
- [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rojopolis/spellcheck-github-actions/compare/ed0756273a1658136c36d26e3d0353de35b98c8b...23dc186319866e1de224f94fe1d31b72797aeec7)

---
updated-dependencies:
- dependency-name: rojopolis/spellcheck-github-actions
  dependency-version: 0.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #16988

4 months agoGHA: update openssl/openssl to v3.5.0
renovate[bot] [Tue, 8 Apr 2025 14:12:32 +0000 (14:12 +0000)] 
GHA: update openssl/openssl to v3.5.0

Closes #16997

4 months agodocs: fix incorrect shell substitution in docker run example command
Johan Eliasson [Mon, 7 Apr 2025 20:20:29 +0000 (22:20 +0200)] 
docs: fix incorrect shell substitution in docker run example command

Corrected the volume mount path in the Docker run example by replacing
`(pwd)` with the shell substitution syntax `$(pwd)`. This ensures the
current working directory is properly mounted into the container.

Closes #16990

4 months agoDockerfile: update debian:bookworm-slim Docker digest to 4b44499
renovate[bot] [Tue, 8 Apr 2025 06:09:44 +0000 (06:09 +0000)] 
Dockerfile: update debian:bookworm-slim Docker digest to 4b44499

Closes #16992

4 months agomk-ca-bundle.pl: follow redirects
Daniel Stenberg [Tue, 8 Apr 2025 09:30:17 +0000 (11:30 +0200)] 
mk-ca-bundle.pl: follow redirects

The Mozilla hosted files have started to redirect. Follow them to restore
script functionality.

Reported-by: Harry Sintonen
Closes #16995

4 months agotests/tunit: make a separate directory for tool-based unit tests
Daniel Stenberg [Sat, 5 Apr 2025 20:42:09 +0000 (22:42 +0200)] 
tests/tunit: make a separate directory for tool-based unit tests

Separated from library based unit tests to not confuse memory management
etc. Move 1394 and 1604 there.

Closes #16983

4 months agocurl_multibyte: fixup low-level calls, include in unity builds
Viktor Szakats [Sun, 16 Mar 2025 13:19:38 +0000 (14:19 +0100)] 
curl_multibyte: fixup low-level calls, include in unity builds

Also adjust `()` around low-level calls preventing macro overrides via
e.g. `memdebug.h`:
- add for `malloc` and `free`.
- drop for `_open`. (We do not override `_open` in curl.)

Tidy-up: also sync libcurlu custom macro order in cmake with autotools.

Follow-up to f42a279ee32d3db3ab529da8dfb833edb5a088ca #11928

Closes #16742

4 months agosocketpair: support pipe2 where available
Andy Pan [Sun, 6 Apr 2025 12:37:10 +0000 (20:37 +0800)] 
socketpair: support pipe2 where available

By replacing pipe with pipe2, it would save us 4 extra system calls of
setting O_NONBLOCK and O_CLOEXEC. This system call is widely supported
across UNIX-like OS's: Linux, *BSD, and SunOS derivatives - Solaris,
illumos, etc.

Ref:
https://man7.org/linux/man-pages/man2/pipe.2.html
https://man.freebsd.org/cgi/man.cgi?query=pipe
https://man.dragonflybsd.org/?command=pipe2
https://man.netbsd.org/pipe.2
https://man.openbsd.org/pipe.2
https://docs.oracle.com/cd/E88353_01/html/E37841/pipe2-2.html
https://illumos.org/man/2/pipe2
https://www.gnu.org/software/gnulib/manual/html_node/pipe2.html

Closes #16987

4 months agotests: Add https-mtls server to force client auth
Yedaya Katsman [Thu, 3 Apr 2025 18:51:32 +0000 (21:51 +0300)] 
tests: Add https-mtls server to force client auth

- test2088 verifies that mutual tls works

This adds a new certificate to generate which has the clientAuth key
usage enabled, and uses it to connect to a https-mtls server.

Closes #16923

4 months agotests: removes CApath in stunnel
Yedaya Katsman [Thu, 3 Apr 2025 18:47:32 +0000 (21:47 +0300)] 
tests: removes CApath in stunnel

It wasn't used, and didn't do anything since the folder it got didn't
have files with names of the hash of the subjects. [1]

[1] https://www.stunnel.org/static/stunnel.html#CApath-CA_DIRECTORY

Closes #16923

4 months agoKNOWN_BUGS: wolfssh: all tests fail
Daniel Stenberg [Sun, 6 Apr 2025 11:19:30 +0000 (13:19 +0200)] 
KNOWN_BUGS: wolfssh: all tests fail

Closes #16794

4 months agomake: clean tests better
Daniel Stenberg [Sun, 6 Apr 2025 10:04:33 +0000 (12:04 +0200)] 
make: clean tests better

1. 'make clean' in the root dir now also invokes 'make clean' in the
tests subdir so that it cleans up better recursively. The Makefile.am
does not list 'tests' as a normal subdir to avoid building that
directory for a normal make invoke.

2. 'make clean' in the tests/libtest and tests/unit directories now
*explicitly* remove the unity build executables even if this is not a
unit build. This, because those files may be leftovers from previous
builds and such leftovers can otherwise linger around and since
'runtests.pl' dynamically acts differently based on the mere *presence*
of those files, they can keep you fooled for a while until you
realize...

Closes #16986

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sun, 6 Apr 2025 11:11:26 +0000 (13:11 +0200)] 
RELEASE-NOTES: synced

4 months agosocket: use accept4 when available
Andy Pan [Sat, 5 Apr 2025 15:55:32 +0000 (23:55 +0800)] 
socket: use accept4 when available

Linux, *BSD, and Solaris support accept4 system call that enables the
caller to assign additional flags and save some extra system calls. It
can come in handy when O_NONBLOCK or/and FD_CLOEXEC is/are required on a
socket after being accepted.

Ref:
https://man7.org/linux/man-pages/man2/accept.2.html
https://man.freebsd.org/cgi/man.cgi?query=accept4
https://man.dragonflybsd.org/?command=accept&section=2
https://man.openbsd.org/accept.2
https://man.netbsd.org/accept.2
https://docs.oracle.com/cd/E88353_01/html/E37843/accept4-3c.html
https://www.gnu.org/software/gnulib/manual/html_node/accept4.html

Closes #16979

4 months agoscripts: completion.pl: sort the completion file for all shells
Carlos Henrique Lima Melara [Sat, 5 Apr 2025 22:53:51 +0000 (19:53 -0300)] 
scripts: completion.pl: sort the completion file for all shells

The reproducible builds effort in Debian has caught a regression in curl
8.13.0-rc1 but we were a bit slow to realize it. The ordering of the
completion file for fish is not deterministic so it can differ between
builds. Since there is no restriction about the order of the completion
file for fish, let's just sort it too.

Closes #16985

4 months agoCURLOPT_HTTP_TRANSFER_DECODING: fixed
Daniel Stenberg [Sat, 5 Apr 2025 22:40:56 +0000 (00:40 +0200)] 
CURLOPT_HTTP_TRANSFER_DECODING: fixed

The fix in b8bd019c6a02182 (#16959) broke the
CURLOPT_HTTP_TRANSFER_DECODING handling, shown in test 319 and curl's
--raw option.

This is a follow-up that restores the functionality.

Enable test 319 again.

Fixes #16974
Closes #16984

4 months agoGHA/configure-vs-cmake: dump generated configs to log
Viktor Szakats [Sat, 5 Apr 2025 18:23:17 +0000 (20:23 +0200)] 
GHA/configure-vs-cmake: dump generated configs to log

Sometimes it's useful to have a look at the generated `libcurl.pc` and
`curl-config` files.

`cmp-config.pl` normalizes them before diffing, thus doesn't show their
original content.

Closes #16981

4 months agocmake/FindNGTCP2: simplify multi-pkg-config detection
Viktor Szakats [Sat, 5 Apr 2025 18:09:50 +0000 (20:09 +0200)] 
cmake/FindNGTCP2: simplify multi-pkg-config detection

Use a single `pkg_check_modules` call to detect the main & crypto libs.

Follow-up to 3b501976a9adcf20218ffb96d9041806432227e4 #16479
Closes #16980

4 months agotest1658: add unit test for the HTTPS RR decoder
Daniel Stenberg [Fri, 4 Apr 2025 21:21:41 +0000 (23:21 +0200)] 
test1658: add unit test for the HTTPS RR decoder

Made the HTTPS-RR parser a little stricter while at it.

Drop the ALPN escape handling, that was not needed.

Make the hode handle (and ignore) duplicate ALPN entries.

Closes #16972

4 months agowinbuild: add the deprecation warning to the README
Jay Satiro [Fri, 4 Apr 2025 07:50:15 +0000 (03:50 -0400)] 
winbuild: add the deprecation warning to the README

- Mention in README.md and INSTALL-CMAKE.md that the winbuild build
  system is going to be removed in September 2025.

Closes https://github.com/curl/curl/pull/16957

4 months agotests/globalconfig.pm: remove the qw
Daniel Stenberg [Sat, 5 Apr 2025 14:54:38 +0000 (16:54 +0200)] 
tests/globalconfig.pm: remove the qw

Fixes the warning.

Fixes #16976
Closes #16977

4 months agocontent_encoding: Transfer-Encoding parser improvements
Daniel Stenberg [Fri, 4 Apr 2025 07:54:49 +0000 (09:54 +0200)] 
content_encoding: Transfer-Encoding parser improvements

- allow and ignore "identity" as an encoding

- fail if any other encoder than chunked follows after chunked

- fail on unsolicited encodings - when the server encodes but curl did
  not ask for it

Add test 1493 to 1496 to verify.

Disable test 319 as that is now broken: issue #16974

Reported-by: Jonathan Rosa
Fixes #16956
Closes #16959

4 months agolibtest/first: stop defining MEMDEBUG_NODEFINES
Daniel Stenberg [Sat, 5 Apr 2025 15:53:02 +0000 (17:53 +0200)] 
libtest/first: stop defining MEMDEBUG_NODEFINES

It causes problems in unity builds, in particular when this file is used
for unit tests.

Bonus: switch to plain getenv() instead of curl_getenv() to avoid extra
malloc/free rounds.

Closes #16978

4 months agoranddisable: build randomizer
Daniel Stenberg [Fri, 4 Apr 2025 12:34:10 +0000 (14:34 +0200)] 
randdisable: build randomizer

This script makes a "random" build using configure and verifies that it
builds curl correctly. It randomly adds a number of the available
--disable-* flags to configure. When it detects a problem the script
stops, otherwise it continues trying more combinations.

Closes #16962

4 months agohttp2: fix stream window size after unpausing
Stefan Eissing [Fri, 4 Apr 2025 08:43:13 +0000 (10:43 +0200)] 
http2: fix stream window size after unpausing

When pausing a HTTP/2 transfer, the stream's local window size
is reduced to 0 to prevent the server from sending further data
which curl cannot write out to the application.

When unpausing again, the stream's window size was not correctly
increased again. The attempt to trigger a window update was
ignored by nghttp2, the server never received it and the transfer
stalled.

Add a debug feature to allow use of small window sizes which
reproduces this bug in test_02_21.

Fixes #16955
Closes #16960

4 months agotests: remove some unused test case sections
Dan Fandrich [Fri, 4 Apr 2025 19:24:12 +0000 (12:24 -0700)] 
tests: remove some unused test case sections

4 months agotests: unify test case keywords
Dan Fandrich [Fri, 4 Apr 2025 18:35:33 +0000 (11:35 -0700)] 
tests: unify test case keywords

Unify the case, punctuation and name of test case keywords so they can
be more easily selected or skipped when desired. Add a few keywords that
were missing. Fix a couple of typos in test names.

4 months agorand: update comment on Curl_rand_bytes weak random
Yedaya Katsman [Fri, 4 Apr 2025 14:04:21 +0000 (17:04 +0300)] 
rand: update comment on Curl_rand_bytes weak random

From what I understand both rustls and mbedTLS expose a strong random
function, so as long as you have a TLS library random will be strong.

Followup 89728451234a "vtls/rustls: support strong CSRNG data"
Followup a90a5bccd47b "mbedtls: implement CTR-DRBG and HAVEGE random generators"

Closes #16965

4 months agohttps-connect, fix httpsrr target check
Stefan Eissing [Fri, 4 Apr 2025 16:10:28 +0000 (18:10 +0200)] 
https-connect, fix httpsrr target check

The HTTPSRR check on the record's target was not working as it used the
wrong index on the NUL byte if the target was not NULL.

Fixes #16966
Reported-by: Pavel Kropachev
Closes #16968

4 months agotests: fix some test tag mismatches
Dan Fandrich [Fri, 4 Apr 2025 16:17:08 +0000 (09:17 -0700)] 
tests: fix some test tag mismatches

4 months agotests: move a boolean variable out of the path section
Dan Fandrich [Fri, 4 Apr 2025 15:42:43 +0000 (08:42 -0700)] 
tests: move a boolean variable out of the path section

4 months agohostip: show the correct name on proxy resolve error
Daniel Stenberg [Fri, 4 Apr 2025 10:34:09 +0000 (12:34 +0200)] 
hostip: show the correct name on proxy resolve error

Regression, probably from 8ded8e5f3f4b6586399 (#16451)

Fixes #16958
Reported-by: Jean-Christophe Amiel
Closes #16961

4 months agotests: use a more portable null device path
Jay Satiro [Thu, 3 Apr 2025 03:16:49 +0000 (23:16 -0400)] 
tests: use a more portable null device path

- Use File::Spec->devnull() to get the null device path.

Prior to this change we used NUL for Windows native perl and /dev/null
otherwise.

Bug: https://github.com/curl/curl/pull/16929#discussion_r2025718160
Reported-by: Viktor Szakats
Closes https://github.com/curl/curl/pull/16930

4 months agohttp_aws_sigv4: add additional verbose log statements
Nigel Brittain [Thu, 3 Apr 2025 22:42:04 +0000 (22:42 +0000)] 
http_aws_sigv4: add additional verbose log statements

To use curl as a tool for troubleshooting SigV4 signing, it is useful to
have the 'Canonical Request', 'String To Sign' and 'Signature'
calculations output.

Closes #16952

4 months agocmake: use absolute paths for completion targets
Viktor Szakats [Thu, 3 Apr 2025 23:54:46 +0000 (01:54 +0200)] 
cmake: use absolute paths for completion targets

Fixing potential:
```
CMake Error at scripts/CMakeLists.txt:72 (install):
  install FILES given directory "/usr/ports/ftp/curl/work/.build/scripts/" to
  install.
```

Reported-by: Daniel Engberg
Fixes #16946
Follow-up to c8b0f0c9ad78eafc6c8f0005113de346ee797c21 #16833
Closes #16954

4 months agoscripts: fix --opts-dir help in completion.pl
Dan Fandrich [Thu, 3 Apr 2025 23:12:45 +0000 (16:12 -0700)] 
scripts: fix --opts-dir help in completion.pl

The help text gave the wrong option name.

Reported-by: Daniel Engberg
Ref: #16946

4 months agorustls: make max size of cert and key reasonable
Yedaya Katsman [Thu, 3 Apr 2025 20:36:05 +0000 (23:36 +0300)] 
rustls: make max size of cert and key reasonable

SIZE_MAX is an very overkill size for certificates or keys, lower it to
100KiB for both certificate and keys. The default max size of openssl is
100KiB for the entire chain [1], and it seems firefox fails at ~60kb
[2].

Found by https://github.com/curl/curl/pull/16923

[0] https://docs.openssl.org/3.2/man3/SSL_CTX_set_max_cert_list/#notes
[2] https://0x00.cl/blog/2024/exploring-tls-certs/

Closes #16951

4 months agoGHA/curl-for-win: switch to `podman` (from `docker`)
Viktor Szakats [Fri, 24 Jan 2025 13:42:15 +0000 (14:42 +0100)] 
GHA/curl-for-win: switch to `podman` (from `docker`)

Closes #16727

4 months agotests: prefer `--insecure` over `-k`
Viktor Szakats [Sat, 29 Mar 2025 02:41:45 +0000 (03:41 +0100)] 
tests: prefer `--insecure` over `-k`

To make it uniform in all tests, and greppability.

Also:
- replace `-k` flag with `-q` in test 1268. (the actual flag doesn't
  matter in this test)
- keep `-k` in test 300 to test its short form.
  (also verified to fail without a working `-k`)

Closes #16878

4 months agolib: make Curl_easyopts const 16950/head
Dan Fandrich [Thu, 3 Apr 2025 20:41:50 +0000 (13:41 -0700)] 
lib: make Curl_easyopts const

Also, make the optiontable rule work in an out-of-tree build.

Closes #16950

4 months agolib: add const to clientwriter tables
Dan Fandrich [Thu, 3 Apr 2025 19:42:06 +0000 (12:42 -0700)] 
lib: add const to clientwriter tables

Unlike the connection filter tables that hold a writable log level, the
client writer tables can be const.

4 months agoINSTALL.md: update the minimal libcurl size example
Dan Fandrich [Thu, 3 Apr 2025 19:04:56 +0000 (12:04 -0700)] 
INSTALL.md: update the minimal libcurl size example

It's only gone up by 7 KiB over the dozen releases in the last year.

4 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 3 Apr 2025 18:37:17 +0000 (20:37 +0200)] 
RELEASE-NOTES: synced

4 months agotests: mark ipfs tests to require ipfs
Daniel Stenberg [Thu, 3 Apr 2025 17:34:33 +0000 (19:34 +0200)] 
tests: mark ipfs tests to require ipfs

Fixes #16947
Reported-by: Daniel Engberg
Closes #16948

4 months agoGHA: update cloudflare/quiche to v0.23.5
renovate[bot] [Wed, 2 Apr 2025 02:03:30 +0000 (02:03 +0000)] 
GHA: update cloudflare/quiche to v0.23.5

Closes #16913

4 months agotests/README.md: document --test-duphandle
Daniel Stenberg [Thu, 3 Apr 2025 14:30:42 +0000 (16:30 +0200)] 
tests/README.md: document --test-duphandle

Follow-up to cbafcec50bc99ba0dd0490e6

Closes #16944

4 months agoasyn-thread: fix build without socketpair
Daniel Stenberg [Thu, 3 Apr 2025 14:47:41 +0000 (16:47 +0200)] 
asyn-thread: fix build without socketpair

Follow-up to 9b6148e9d95db54a752b03b571296c40d66e97fe

Closes #16945

4 months agoasync-threaded resolver: use ref counter
Stefan Eissing [Wed, 2 Apr 2025 10:25:21 +0000 (12:25 +0200)] 
async-threaded resolver: use ref counter

Allocate the data shared between a transfer and an aync resolver thread
separately and use a reference counter to determine its release.

Change `Curl_thread_destroy()` to clear the thread handle, so that the
thread is considered "gone" and we do not try to join (and fail to)
afterwards.

Retake of the revert in fb15a986c0d947ae6b9dd6

Closes #16916

4 months agodnscache: slight refactoring
Stefan Eissing [Thu, 3 Apr 2025 11:11:32 +0000 (13:11 +0200)] 
dnscache: slight refactoring

Slight refactoring around dnscache, e.g. hostcache

- eliminate `data->state.hostcache`. Always look up
  relevant dnscache at share/multi.
- unify naming to "dnscache", replacing "hostcache"
- use `struct Curl_dnscache`, even though it just
  contains a `Curl_hash` for now.
- add `Curl_dnscache_destroy()` for cleanup in
  share/multi.

Closes #16941

4 months agotests/README.md: list the openssl tool among the prerequisites
Daniel Stenberg [Thu, 3 Apr 2025 12:28:37 +0000 (14:28 +0200)] 
tests/README.md: list the openssl tool among the prerequisites

Used for test cert generation since 8.13.0

Closes #16942

4 months agoparsedate: provide Curl_wkday also for GnuTLS builds
Daniel Stenberg [Thu, 3 Apr 2025 13:39:44 +0000 (15:39 +0200)] 
parsedate: provide Curl_wkday also for GnuTLS builds

Otherwise --disable-dateparse + --with-gnutls builds might fail.

Found with randdisable

Closes #16943

4 months agogenserv.pl: fail with a message if `openssl` is missing or failing
Viktor Szakats [Wed, 2 Apr 2025 21:40:14 +0000 (23:40 +0200)] 
genserv.pl: fail with a message if `openssl` is missing or failing

Reported-by: Tomas Volf
Fixes #16926
Follow-up to 44341e736a3e2f7a2b25a774be3a9796e81abab9 #16824
Ref: #16928
Co-authored-by: Daniel Stenberg
Closes #16929

4 months agoconncache: make Curl_cpool_init return void
Daniel Stenberg [Thu, 3 Apr 2025 08:48:25 +0000 (10:48 +0200)] 
conncache: make Curl_cpool_init return void

Since it cannot fail, removing the return code simplifies the code paths
calling this function.

Closes #16936

4 months agohttp: fix a build error when all auths are disabled
Daniel Stenberg [Thu, 3 Apr 2025 10:10:45 +0000 (12:10 +0200)] 
http: fix a build error when all auths are disabled

error: ‘result’ undeclared (first use in this function)

Found with randdisable

Closes #16939

4 months agohostip: fix build without threaded-resolver and without DoH
Daniel Stenberg [Thu, 3 Apr 2025 09:35:06 +0000 (11:35 +0200)] 
hostip: fix build without threaded-resolver and without DoH

Closes #16938

4 months agovtls: fix build with ssl but without http
Stefan Eissing [Thu, 3 Apr 2025 09:09:01 +0000 (11:09 +0200)] 
vtls: fix build with ssl but without http

Fixes #16935
Closes #16937