Daniel McCarney [Thu, 12 Sep 2024 16:38:51 +0000 (12:38 -0400)]
vtls/rustls: support strong CSRNG data
Now that the curl rustls vtls backend is using rustls 0.14 we can
address the weak random situation by using
`rustls_default_crypto_provider_random()` to provide a `Curl_ssl`
`random` callback that fills the provided buffer with cryptographically
secure random data.
The mentions in `docs/` about weak RNG when using rustls are removed as
they are no longer applicable.
Daniel McCarney [Thu, 12 Sep 2024 16:31:59 +0000 (12:31 -0400)]
vtls/rustls: simplify ciphersuite skipping
Now that the rustls vtls backend is using rustls 0.14 we can take
advantage of `rustls_supported_ciphersuite_protocol_version()` to skip
TLS 1.3 and TLS 1.2 ciphersuites as required without needing to
interrogate the ciphersuite names as `rustls_str`s.
Daniel McCarney [Thu, 12 Sep 2024 16:18:26 +0000 (12:18 -0400)]
vtls/rustls: rustls-ffi 0.14.0 update
* Documentation is updated to describe new required version, and to link
to the upstream README about cryptography providers.
* GitHub workflow is updated to fetch 0.14.0.
* Breaking changes in`lib/vtls/rustls.c` are addressed:
* The `rustls_client_config_builder_build()` function now uses an out
parameter for the built config instead of returning it directly.
This allows the building process to fail if the default crypto
provider state isn't appropriate, or another error condition occurs.
* Default ciphersuites are collected using renamed functions named to
make it clear the ciphersuites are associated with the default
crypto provider.
* Customization of ciphersuites is now done via
a `rustls_crypto_provider_builder` used to instantiate a
`rustls_crypto_provider`. The customized provider can then can be
used with `rustls_client_config_builder_new_custom` in place of
providing ciphersuites directly.
* `rustls_connection_get_negotiated_ciphersuite()` now returns the
ciphersuite ID directly.
Daniel McCarney [Tue, 6 Aug 2024 17:49:03 +0000 (13:49 -0400)]
vtls/rustls: differentiate error messages
It's easier to diagnose a problem when there is one place where the
error message can be emitted. For that reason this commit updates two
errors that were shared between other fallible operations to use unique
messages.
Daniel McCarney [Fri, 19 Jul 2024 16:23:18 +0000 (12:23 -0400)]
vtls/rustls: simplify builder cleanup
Don't build `config_builder` just to free the resulting config, free the
builder directly.
When `cr_init_backend` encounters an error condition setting up the
Rustls client configuration it must do something with the
`config_builder` that was constructed earlier to avoid a memory leak.
The previous implementation preferred to use a pattern of building the
builder (thus consuming it) and then freeing the built config (to avoid
a memory leak). However, the purpose/intent is clearer when we just free
the builder directly instead of building it and freeing the result.
Stefan Eissing [Mon, 9 Sep 2024 11:52:44 +0000 (13:52 +0200)]
tests: tweak lock file handling and timers
- when server are killed by a test case, do not wait for the server lock
file to go away. These tests are mostly about client timeouts and the
server will hang until killed.
- when killing a server successfully, check for a remaining lock file,
log its existence and remove it.
- lower the delay timings on SLOWDOWN by half
- add SLOWDOWNDATA server command to only slow down the FTP data bytes,
not the control ones.
- lower some timeout values
Viktor Szakats [Thu, 12 Sep 2024 13:08:05 +0000 (15:08 +0200)]
autotools: fix `--with-ca-embed` build rule
Add missing rule dependency on the user-specified CA bundle. This fixes
including it when using the curl distro tarball, and other cases.
Also:
- fix the internal name of the CA bundle to avoid nested quotes.
It broke broke the rule dependency for the make tool.
- exclude the generated (empty) `tool_ca_embed.c` file from the distro
tarball. Patch-by: Daniel Stenberg
Follow-up to 8a3740bc8e558b9a9d4a652b74cf27a0961d7010 #14059 Reported-by: rampageX on github
Fixes #14879
Closes #14882
Stefan Eissing [Thu, 12 Sep 2024 08:03:33 +0000 (10:03 +0200)]
http2: when uploading data from stdin, fix eos forwarding
When uploading data from stdin ('-T -'), and the EOS was only detected
on a 0-length read, the EOS was not forwarded to the filters. This led
HTTP/2 to hang on not forwarding this to the server.
Added test_07_14 to reproduce and verify.
Fixes #14870 Reported-by: nekopsykose on github
Closes #14877
Viktor Szakats [Wed, 11 Sep 2024 21:42:43 +0000 (23:42 +0200)]
cmake: ensure `CURL_USE_OPENSSL`/`USE_OPENSSL_QUIC` are set in sync
OpenSSL must be selected (manually or by default) and detected for
`USE_OPENSSL_QUIC=ON` to work. This was documented before this patch,
but with CMake it was possible to use `USE_OPENSSL_QUIC=ON` without
an explicit or default `CURL_USE_OPENSSL=ON`, leading to a confused
internal state.
Fix by checking the QUIC condition early, showing a warning if enabled
without OpenSSL, and ignoring QUIC in such case.
Also: Fix casing of OpenSSL in option description.
Stefan Eissing [Wed, 11 Sep 2024 10:10:10 +0000 (12:10 +0200)]
test537: cap the rlimit max this test runs
Modern debian reports a soft limit of 134217724 which makes this test
run for minutes to allocate all file descriptors. Impose a cap of 256k,
so we do not run this tests on systems with larger rlimits.
Stefan Eissing [Wed, 11 Sep 2024 11:53:44 +0000 (13:53 +0200)]
QUIC: on connect, keep on trying on draining server
Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routing still
hitting the instance shutting down.
Instead keep on connecting until the overall TIMEOUT fires.
Daniel Stenberg [Wed, 11 Sep 2024 12:12:41 +0000 (14:12 +0200)]
multi: check that the multi handle is valid in curl_multi_assign
By requiring that the multi handle is fine, it can detect bad usage
better and by that avoid crashes. Like in the #14860 case, which is an
application calling curl_multi_assign() with a NULL pointer multi
handle.
Reported-by: Carlo Cabrera
Fixes #14860
Closes #14862
Dan Fandrich [Fri, 6 Sep 2024 17:29:58 +0000 (10:29 -0700)]
GHA/linux-old: add an autoconf/automake build
This verifies that using older autotools still works. The results of the
autoconf build should end up being identical to the second of the two
cmake builds (the difference in the platform string notwithstanding).
Jay Satiro [Thu, 5 Sep 2024 06:18:25 +0000 (02:18 -0400)]
asyn-thread: stop using GetAddrInfoExW on Windows
- For the threaded resolver backend on Windows, revert back to
exclusively use the threaded resolver with libcurl-owned threading
instead of GetAddrInfoExW with Windows-owned threading.
Winsock (the Windows sockets library) has a bug where it does not wait
for all of the name resolver threads it is managing to terminate before
returning from WSACleanup. The threads continue to run and may cause a
crash.
This commit is effectively a revert of several commits that encompass
all GetAddrInfoExW code in libcurl. A manual review of merge conflicts
was used to resolve minor changes that had modified the code for
aesthetic or build reasons in other commits.
Prior to this change if libcurl was built with the threaded resolver
backend for Windows, and Windows 8 or later was the operating system at
runtime, and the caller was not impersonating another user, then libcurl
would use GetAddrInfoExW to handle asynchronous name lookups.
GetAddrInfoExW support was added in a6bbc87f, which preceded 8.6.0, and
prior to that the threaded resolver backend used libcurl-owned threading
exclusively on Windows.
Dan Fandrich [Sat, 7 Sep 2024 01:17:19 +0000 (18:17 -0700)]
maketgz: fix RELEASE-TOOLS.md for daily tarballs
The daily snapshots have no associated git tag, so provide a commit hash
instead in these cases. Fix the dpkg detection since the shell would
exit immediately without showing an error message if it weren't found.
Viktor Szakats [Thu, 5 Sep 2024 21:23:47 +0000 (23:23 +0200)]
build: generate `buildinfo.txt` for test logs
Also:
- read `buildinfo.txt` from `runtests.pl` and dump it to the log.
- cmake: show `CROSS` target flag for cross-builds.
- cmake: add logic to detect arguments passed via the command-line.
It is meant to help filling out missing datapoints in the testclutch
matrix.
Daniel Stenberg [Thu, 5 Sep 2024 13:58:45 +0000 (15:58 +0200)]
GHA/distcheck: keep upload artifacts for one day only
The uploads are only used as a cache mechanism between jobs to save them
from having to re-run maketgz multiple times, so there is no need to
save the artifact longer than this.
Daniel Stenberg [Wed, 4 Sep 2024 09:11:06 +0000 (11:11 +0200)]
openssl: certinfo errors now fail correctly
If there is a (memory) error when creating the certinfo data, the code
would previously continue which could lead to a partial/broken response.
Now, the first error aborts and cleans up the entire thing.
A certinfo "collection" error is however still not considered an error
big enough to stop the handshake.
Bonus 1: made two functions static (and removed the Curl_ prefix) that
were not used outside of openssl.c
Bonus 2: removed the unused function Curl_ossl_set_client_cert
Stefan Eissing [Tue, 3 Sep 2024 07:54:59 +0000 (09:54 +0200)]
cf-socket: fix pollset for listening
When FTP does an active data connection, the socket connection
filter is instantiated with a listening socket. When the filter
adjusts its pollset, it needs to POLLIN, not OUT.
Aki [Sat, 31 Aug 2024 03:48:18 +0000 (11:48 +0800)]
openssl: fix the data race when sharing an SSL session between threads
The SSL_Session object is mutated during connection inside openssl,
and it might not be thread-safe. Besides, according to documentation
of openssl:
```
SSL_SESSION objects keep internal link information about the session
cache list, when being inserted into one SSL_CTX object's session
cache. One SSL_SESSION object, regardless of its reference count,
must therefore only be used with one SSL_CTX object (and the SSL
objects created from this SSL_CTX object).
```
If I understand correctly, it is not safe to share it even in a
single thread.
Instead, serialize the SSL_SESSION before adding it to the cache,
and deserialize it after retrieving it from the cache, so that no
concurrent write to the same object is infeasible.
Also
- add a ci test for thread sanitizer
- add a test for sharing ssl sessions concurrently
- avoid redefining memory functions when not building libcurl, but
including the soruce in libtest
- increase the concurrent connections limit in sws
Notice that there are fix for a global data race for openssl which
is not yet release. The fix is cherry pick for the ci test with
thread sanitizer.
https://github.com/openssl/openssl/commit/d8def79838cd0d5e7c21d217aa26edb5229f0ab4
Stefan Eissing [Mon, 2 Sep 2024 11:29:54 +0000 (13:29 +0200)]
haproxy: send though next filter
Small but, instead of sending the initial data though the connection
method, send it to the next filter in the chain. While the connection
methods accomodates for such use, by ignoring unconnected filters, it is
better to follow the filter chain explicitly.
Viktor Szakats [Thu, 22 Aug 2024 08:54:42 +0000 (10:54 +0200)]
printf: fix mingw-w64 format checks
Change mingw-w64 printf format checks in public curl headers to use
`__MINGW_PRINTF_FORMAT` instead of `gnu_printf`. This syncs the format
checker with format string macros published via `curl/system.h`. (Also
disable format checks for mingw-w64 older than 3.0.0 (2013-09-20) and
classic-mingw, which do not support this macro.)
This fixes bogus format checker `-Wformat` warnings in 3rd party code
using curl format strings with the curl printf functions, when using
mingw-w64 7.0.0 (2019-11-10) and older (with GCC, MSVCRT).
It also allows to delete two workaounds for this within curl itself:
- setting `-D__USE_MINGW_ANSI_STDIO=1` for mingw-w64 via cmake and
configure for `docs/examples` and `tests/http/clients`.
Ref: c730c8549b5b67e7668ca5d2cd82c3cc183e125d #14640
The format check macro is incompatible (depending on mingw-w64 version
and configuration) with the C99 `%z` (`size_t`) format string used
internally by curl.
To work around this problem, override the format check style in curl
public headers to use `gnu_printf`. This is compatible with `%z` in all
mingw-w64 versions and allows keeping the C99 format strings internally.
Also:
- lib/ws.c: add missing space to an error message.
- docs/examples/ftpgetinfo.c: fix to use standard printf.
Daniel Stenberg [Sat, 31 Aug 2024 21:07:12 +0000 (23:07 +0200)]
lib: remove use of RANDOM_FILE
It could previously be set with configure/cmake and used in rare cases
for reading randomness: with ancient mbedTLS or rustls without
arc4random.
We now get randomness in this order:
1. The TLS library's way to provide random
2. On Windows: Curl_win32_random
3. if arc4random exists, use that
4. weak non-crytographically strong pseudo-random
Stefan Eissing [Fri, 30 Aug 2024 11:25:26 +0000 (13:25 +0200)]
url: fix connection reuse for HTTP/2 upgrades
Normally, when a connection's filters have all connected, the
multiplex status is determined. However, HTTP/2 Upgrade:
requests will only do this when the first server response
has been received.
The current connection reuse mechanism does not accomodate
that and when the time between connect and response is large
enough, connection reuse may not happen as desired.
See test case 2405 failures, such as in
https://github.com/curl/curl/actions/runs/10629497461/job/29467166451
Add 'conn->bits.asks_multiplex' as indicator that a connection is
still being evaluated for mulitplexing, so that new transfers
may wait on this to be cleared.
Daniel Stenberg [Sat, 31 Aug 2024 15:24:01 +0000 (17:24 +0200)]
curl.h: make CURLOPT_WRITEINFO and CURLOPT_CLOSEPOLICY compile
The symbols have not been in use for 17+ years and they did not do
anything for several years before that, but apparently there are still
code using them.
Follow-up to 3b057d4b7a7
Fixes #14747 Reported-by: Kai Pastor
Closes #14748
Fixes:
```
Present in CMakeLists.txt, not propagated via curl_config.h.cmake: CURL_DISABLE_HTTP_AUTH
Present in CMakeLists.txt, not propagated via curl_config.h.cmake: CURL_DISABLE_SHUFFLE_DNS
```
Ref: https://github.com/curl/curl/actions/runs/10655027540/job/29532054141?pr=14754#step:11:2090
Viktor Szakats [Sat, 31 Aug 2024 13:39:17 +0000 (15:39 +0200)]
configure: break indentation to fix `--help` output
For recently added/updated options:
windows-unicode, winidn, apple-idn
It looks like the second `AS_HELP_STRING()` must start in the first
column, otherwise its indentation will appear in the `--help` output,
and break unalignment with the rest.
Viktor Szakats [Fri, 30 Aug 2024 15:40:33 +0000 (17:40 +0200)]
cmake: allow disabling `RANDOM_FILE`
`./configure` allows `--random-file=no`. Allow this with CMake too,
using `-DRANDOM_FILE=OFF` (other boolean values work too: no, false, 0,
case insensitive.)
Viktor Szakats [Thu, 29 Aug 2024 23:04:18 +0000 (01:04 +0200)]
build: add `poll()` detection for cross-builds
For cross-builds rely on `_POSIX_C_SOURCE` to decide if `poll()` is
supported, rather than just assuming it isn't.
This may still miss to detect `poll()` support, as seen for example with
Linux MUSL cross-builds.
Also:
- GHA/curl-for-win: enable RISC-V 64 cross-target for Linux MUSL.
(to test this case with cmake, with a false-negative.)
The first RISC-V 64 build in curl's CI.
- GHA/curl-for-win: add arm64/intel64 job for Linux glibc.
(to test this case with cmake, and succeed.)
Stefan Eissing [Wed, 28 Aug 2024 09:09:43 +0000 (11:09 +0200)]
POP3: fix multi-line responses
Some POP3 commands are multi-line, e.g. have responses terminated by a
last line with '.', but some are not. Define the known command
properties and fix response handling.
Add test case for STAT.
Fixes #14677 Reported-by: ralfjunker on github
Closes #14707
projects / thirdparty / curl.git / log
