Stefan Eissing [Thu, 12 Sep 2024 08:03:33 +0000 (10:03 +0200)]
http2: when uploading data from stdin, fix eos forwarding
When uploading data from stdin ('-T -'), and the EOS was only detected
on a 0-length read, the EOS was not forwarded to the filters. This led
HTTP/2 to hang on not forwarding this to the server.
Added test_07_14 to reproduce and verify.
Fixes #14870 Reported-by: nekopsykose on github
Closes #14877
Viktor Szakats [Wed, 11 Sep 2024 21:42:43 +0000 (23:42 +0200)]
cmake: ensure `CURL_USE_OPENSSL`/`USE_OPENSSL_QUIC` are set in sync
OpenSSL must be selected (manually or by default) and detected for
`USE_OPENSSL_QUIC=ON` to work. This was documented before this patch,
but with CMake it was possible to use `USE_OPENSSL_QUIC=ON` without
an explicit or default `CURL_USE_OPENSSL=ON`, leading to a confused
internal state.
Fix by checking the QUIC condition early, showing a warning if enabled
without OpenSSL, and ignoring QUIC in such case.
Also: Fix casing of OpenSSL in option description.
Stefan Eissing [Wed, 11 Sep 2024 10:10:10 +0000 (12:10 +0200)]
test537: cap the rlimit max this test runs
Modern debian reports a soft limit of 134217724 which makes this test
run for minutes to allocate all file descriptors. Impose a cap of 256k,
so we do not run this tests on systems with larger rlimits.
Stefan Eissing [Wed, 11 Sep 2024 11:53:44 +0000 (13:53 +0200)]
QUIC: on connect, keep on trying on draining server
Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routing still
hitting the instance shutting down.
Instead keep on connecting until the overall TIMEOUT fires.
Daniel Stenberg [Wed, 11 Sep 2024 12:12:41 +0000 (14:12 +0200)]
multi: check that the multi handle is valid in curl_multi_assign
By requiring that the multi handle is fine, it can detect bad usage
better and by that avoid crashes. Like in the #14860 case, which is an
application calling curl_multi_assign() with a NULL pointer multi
handle.
Reported-by: Carlo Cabrera
Fixes #14860
Closes #14862
Dan Fandrich [Fri, 6 Sep 2024 17:29:58 +0000 (10:29 -0700)]
GHA/linux-old: add an autoconf/automake build
This verifies that using older autotools still works. The results of the
autoconf build should end up being identical to the second of the two
cmake builds (the difference in the platform string notwithstanding).
Jay Satiro [Thu, 5 Sep 2024 06:18:25 +0000 (02:18 -0400)]
asyn-thread: stop using GetAddrInfoExW on Windows
- For the threaded resolver backend on Windows, revert back to
exclusively use the threaded resolver with libcurl-owned threading
instead of GetAddrInfoExW with Windows-owned threading.
Winsock (the Windows sockets library) has a bug where it does not wait
for all of the name resolver threads it is managing to terminate before
returning from WSACleanup. The threads continue to run and may cause a
crash.
This commit is effectively a revert of several commits that encompass
all GetAddrInfoExW code in libcurl. A manual review of merge conflicts
was used to resolve minor changes that had modified the code for
aesthetic or build reasons in other commits.
Prior to this change if libcurl was built with the threaded resolver
backend for Windows, and Windows 8 or later was the operating system at
runtime, and the caller was not impersonating another user, then libcurl
would use GetAddrInfoExW to handle asynchronous name lookups.
GetAddrInfoExW support was added in a6bbc87f, which preceded 8.6.0, and
prior to that the threaded resolver backend used libcurl-owned threading
exclusively on Windows.
Dan Fandrich [Sat, 7 Sep 2024 01:17:19 +0000 (18:17 -0700)]
maketgz: fix RELEASE-TOOLS.md for daily tarballs
The daily snapshots have no associated git tag, so provide a commit hash
instead in these cases. Fix the dpkg detection since the shell would
exit immediately without showing an error message if it weren't found.
Viktor Szakats [Thu, 5 Sep 2024 21:23:47 +0000 (23:23 +0200)]
build: generate `buildinfo.txt` for test logs
Also:
- read `buildinfo.txt` from `runtests.pl` and dump it to the log.
- cmake: show `CROSS` target flag for cross-builds.
- cmake: add logic to detect arguments passed via the command-line.
It is meant to help filling out missing datapoints in the testclutch
matrix.
Daniel Stenberg [Thu, 5 Sep 2024 13:58:45 +0000 (15:58 +0200)]
GHA/distcheck: keep upload artifacts for one day only
The uploads are only used as a cache mechanism between jobs to save them
from having to re-run maketgz multiple times, so there is no need to
save the artifact longer than this.
Daniel Stenberg [Wed, 4 Sep 2024 09:11:06 +0000 (11:11 +0200)]
openssl: certinfo errors now fail correctly
If there is a (memory) error when creating the certinfo data, the code
would previously continue which could lead to a partial/broken response.
Now, the first error aborts and cleans up the entire thing.
A certinfo "collection" error is however still not considered an error
big enough to stop the handshake.
Bonus 1: made two functions static (and removed the Curl_ prefix) that
were not used outside of openssl.c
Bonus 2: removed the unused function Curl_ossl_set_client_cert
Stefan Eissing [Tue, 3 Sep 2024 07:54:59 +0000 (09:54 +0200)]
cf-socket: fix pollset for listening
When FTP does an active data connection, the socket connection
filter is instantiated with a listening socket. When the filter
adjusts its pollset, it needs to POLLIN, not OUT.
Aki [Sat, 31 Aug 2024 03:48:18 +0000 (11:48 +0800)]
openssl: fix the data race when sharing an SSL session between threads
The SSL_Session object is mutated during connection inside openssl,
and it might not be thread-safe. Besides, according to documentation
of openssl:
```
SSL_SESSION objects keep internal link information about the session
cache list, when being inserted into one SSL_CTX object's session
cache. One SSL_SESSION object, regardless of its reference count,
must therefore only be used with one SSL_CTX object (and the SSL
objects created from this SSL_CTX object).
```
If I understand correctly, it is not safe to share it even in a
single thread.
Instead, serialize the SSL_SESSION before adding it to the cache,
and deserialize it after retrieving it from the cache, so that no
concurrent write to the same object is infeasible.
Also
- add a ci test for thread sanitizer
- add a test for sharing ssl sessions concurrently
- avoid redefining memory functions when not building libcurl, but
including the soruce in libtest
- increase the concurrent connections limit in sws
Notice that there are fix for a global data race for openssl which
is not yet release. The fix is cherry pick for the ci test with
thread sanitizer.
https://github.com/openssl/openssl/commit/d8def79838cd0d5e7c21d217aa26edb5229f0ab4
Stefan Eissing [Mon, 2 Sep 2024 11:29:54 +0000 (13:29 +0200)]
haproxy: send though next filter
Small but, instead of sending the initial data though the connection
method, send it to the next filter in the chain. While the connection
methods accomodates for such use, by ignoring unconnected filters, it is
better to follow the filter chain explicitly.
Viktor Szakats [Thu, 22 Aug 2024 08:54:42 +0000 (10:54 +0200)]
printf: fix mingw-w64 format checks
Change mingw-w64 printf format checks in public curl headers to use
`__MINGW_PRINTF_FORMAT` instead of `gnu_printf`. This syncs the format
checker with format string macros published via `curl/system.h`. (Also
disable format checks for mingw-w64 older than 3.0.0 (2013-09-20) and
classic-mingw, which do not support this macro.)
This fixes bogus format checker `-Wformat` warnings in 3rd party code
using curl format strings with the curl printf functions, when using
mingw-w64 7.0.0 (2019-11-10) and older (with GCC, MSVCRT).
It also allows to delete two workaounds for this within curl itself:
- setting `-D__USE_MINGW_ANSI_STDIO=1` for mingw-w64 via cmake and
configure for `docs/examples` and `tests/http/clients`.
Ref: c730c8549b5b67e7668ca5d2cd82c3cc183e125d #14640
The format check macro is incompatible (depending on mingw-w64 version
and configuration) with the C99 `%z` (`size_t`) format string used
internally by curl.
To work around this problem, override the format check style in curl
public headers to use `gnu_printf`. This is compatible with `%z` in all
mingw-w64 versions and allows keeping the C99 format strings internally.
Also:
- lib/ws.c: add missing space to an error message.
- docs/examples/ftpgetinfo.c: fix to use standard printf.
Daniel Stenberg [Sat, 31 Aug 2024 21:07:12 +0000 (23:07 +0200)]
lib: remove use of RANDOM_FILE
It could previously be set with configure/cmake and used in rare cases
for reading randomness: with ancient mbedTLS or rustls without
arc4random.
We now get randomness in this order:
1. The TLS library's way to provide random
2. On Windows: Curl_win32_random
3. if arc4random exists, use that
4. weak non-crytographically strong pseudo-random
Stefan Eissing [Fri, 30 Aug 2024 11:25:26 +0000 (13:25 +0200)]
url: fix connection reuse for HTTP/2 upgrades
Normally, when a connection's filters have all connected, the
multiplex status is determined. However, HTTP/2 Upgrade:
requests will only do this when the first server response
has been received.
The current connection reuse mechanism does not accomodate
that and when the time between connect and response is large
enough, connection reuse may not happen as desired.
See test case 2405 failures, such as in
https://github.com/curl/curl/actions/runs/10629497461/job/29467166451
Add 'conn->bits.asks_multiplex' as indicator that a connection is
still being evaluated for mulitplexing, so that new transfers
may wait on this to be cleared.
Daniel Stenberg [Sat, 31 Aug 2024 15:24:01 +0000 (17:24 +0200)]
curl.h: make CURLOPT_WRITEINFO and CURLOPT_CLOSEPOLICY compile
The symbols have not been in use for 17+ years and they did not do
anything for several years before that, but apparently there are still
code using them.
Follow-up to 3b057d4b7a7
Fixes #14747 Reported-by: Kai Pastor
Closes #14748
Fixes:
```
Present in CMakeLists.txt, not propagated via curl_config.h.cmake: CURL_DISABLE_HTTP_AUTH
Present in CMakeLists.txt, not propagated via curl_config.h.cmake: CURL_DISABLE_SHUFFLE_DNS
```
Ref: https://github.com/curl/curl/actions/runs/10655027540/job/29532054141?pr=14754#step:11:2090
Viktor Szakats [Sat, 31 Aug 2024 13:39:17 +0000 (15:39 +0200)]
configure: break indentation to fix `--help` output
For recently added/updated options:
windows-unicode, winidn, apple-idn
It looks like the second `AS_HELP_STRING()` must start in the first
column, otherwise its indentation will appear in the `--help` output,
and break unalignment with the rest.
Viktor Szakats [Fri, 30 Aug 2024 15:40:33 +0000 (17:40 +0200)]
cmake: allow disabling `RANDOM_FILE`
`./configure` allows `--random-file=no`. Allow this with CMake too,
using `-DRANDOM_FILE=OFF` (other boolean values work too: no, false, 0,
case insensitive.)
Viktor Szakats [Thu, 29 Aug 2024 23:04:18 +0000 (01:04 +0200)]
build: add `poll()` detection for cross-builds
For cross-builds rely on `_POSIX_C_SOURCE` to decide if `poll()` is
supported, rather than just assuming it isn't.
This may still miss to detect `poll()` support, as seen for example with
Linux MUSL cross-builds.
Also:
- GHA/curl-for-win: enable RISC-V 64 cross-target for Linux MUSL.
(to test this case with cmake, with a false-negative.)
The first RISC-V 64 build in curl's CI.
- GHA/curl-for-win: add arm64/intel64 job for Linux glibc.
(to test this case with cmake, and succeed.)
Stefan Eissing [Wed, 28 Aug 2024 09:09:43 +0000 (11:09 +0200)]
POP3: fix multi-line responses
Some POP3 commands are multi-line, e.g. have responses terminated by a
last line with '.', but some are not. Define the known command
properties and fix response handling.
Add test case for STAT.
Fixes #14677 Reported-by: ralfjunker on github
Closes #14707
Daniel Stenberg [Thu, 29 Aug 2024 21:59:17 +0000 (23:59 +0200)]
managen: fix superfluous leading blank line in quoted sections
When a markdown quoted section using 4-space indentation was converted
to nroff, managen previously caused a newline to appear after the
leading .nf. This fix makes sure that newline is inserted *before* .nf
as intended.
This is perhaps most notable in the HTML version of rendered manpages if
the quoted sections use different colors or similar.
Daniel Stenberg [Thu, 29 Aug 2024 14:00:21 +0000 (16:00 +0200)]
runtests: remove "has_textaware"
All sections defined with the mode="text" attribute now get line endings
normalized so that comparisons become line ending agnostic. Removes the
previous problem of figuring out how exactly different Windows
environments should be treated in this regard.
Viktor Szakats [Thu, 29 Aug 2024 08:15:26 +0000 (10:15 +0200)]
build: drop unused feature-detection code for Apple `poll()`
Drop Apple-specific detection logic for `poll()`. This detection snippet
has been disabled for Apple in both configure and cmake, for `poll()`
being broken on Apple since 10.12 Sierra (2016).
Also replace `exit(1);` with `return 1;` in configure, to make the
snippets match.