Daniel Stenberg [Mon, 16 Oct 2023 12:46:36 +0000 (14:46 +0200)]
gnutls: support CURLSSLOPT_NATIVE_CA
Remove the CURL_CA_FALLBACK logic. That build option was added to allow
primarily OpenSSL to use the default paths for loading the CA certs. For
GnuTLS it was instead made to load the "system certs", which is
different and not desirable.
The native CA store loading is now asked for with this option.
Stefan Eissing [Sat, 7 Oct 2023 13:13:09 +0000 (15:13 +0200)]
RTSP: improved RTP parser
- fix HTTP header parsing to report incomplete
lines it buffers as consumed!
- re-implement the RTP parser for interleave RTP
messages for robustness. It is now keeping its
state at the connection
- RTSP protocol handler "readwrite" implementation
now tracks if the response is before/in/after
header parsing or "in" a bod by calling
"Curl_http_readwrite_headers()" itself. This
allows it to know when non-RTP bytes are "junk"
or HEADER or BODY.
- tested with #12035 and various small receive
sizes where current master fails
Dan Fandrich [Fri, 20 Oct 2023 22:32:21 +0000 (15:32 -0700)]
test1683: remove commented-out check alternatives
Python precheck/postcheck alternatives were included but commented out.
Since these are not used and perl is guaranteed to be available to run
the perl versions anyway, the Python ones are removed.
Dan Fandrich [Fri, 13 Oct 2023 18:46:39 +0000 (11:46 -0700)]
tests: Fix Windows test helper tool search & use it for handle64
The checkcmd() and checktestcmd() functions would not have worked on
Windows due to hard-coding the UNIX PATH separator character and not
adding .exe file extension. This meant that tools like stunnel, valgrind
and nghttpx would not have been found and used on Windows, and
inspection of previous test runs show none of those being found in pure
Windows CI builds.
With this fixed, they can be used to detect the handle64.exe program
before attempting to use it. When handle64.exe was called
unconditionally without it existing, it caused perl to abort the test
run with the error
The running command stopped because the preference variable
"ErrorActionPreference" or common parameter is set to Stop:
sh: handle64.exe: command not found
Jay Satiro [Sun, 15 Oct 2023 01:55:42 +0000 (21:55 -0400)]
build: fix 'threadsafe' feature detection for older gcc
- Add 'threadsafe' to the feature list shown during build if POSIX
threads are being used.
This is a follow-up to 5adb6000 which added support for building a
thread-safe libcurl with older versions of gcc where atomic is not
available but pthread is.
Reported-by: Dan Fandrich Co-authored-by: Dan Fandrich
Fixes https://github.com/curl/curl/issues/12125
Closes https://github.com/curl/curl/pull/12127
Viktor Szakats [Mon, 9 Oct 2023 08:31:04 +0000 (08:31 +0000)]
cmake: replace `check_library_exists_concat()`
The idea of `check_library_exists_concat()` is that it detects an
optional component and adds it to the list of libs that we also use in
subsequent component checks. This caused problems when detecting
components with unnecessary dependencies that were not yet built.
CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used
for component checks, which we already use in most cases. That left 4
uses of `check_library_exists_concat()`. Only one of these actually
needed the 'concat' feature (ldap/lber).
Delete this function and replace it with standard
`check_library_exists()` and manual management of our `CURL_LIBS`
list we use when linking build targets. And special logic to handle the
ldap/lber case.
(We have a similar function for headers: `check_include_file_concat()`.
It works, but problematic for performance reasons and because it hides
the actual headers required in `check_symbol_exists()` calls.)
Jay Satiro [Sat, 14 Oct 2023 05:45:28 +0000 (01:45 -0400)]
tests/server: add more SOCKS5 handshake error checking
- Add additional checking for missing and too-short SOCKS5 handshake
messages.
Prior to this change the SOCKS5 test server did not check that all parts
of the handshake were received successfully. If those parts were missing
or too short then the server would access uninitialized memory.
This issue was discovered in CI job 'memory-sanitizer' test results.
Test 2055 was failing due to the SOCKS5 test server not running. It was
not running because either it crashed or memory sanitizer aborted it
during Test 728. Test 728 connects to the SOCKS5 test server on a
redirect but does not send any data on purpose. The test server was not
prepared for that.
Reported-by: Dan Fandrich
Fixes https://github.com/curl/curl/issues/12117
Closes https://github.com/curl/curl/pull/12118
Sohom Datta [Fri, 13 Oct 2023 21:01:16 +0000 (23:01 +0200)]
tool_getparam: limit --rate to be smaller than number of ms
Currently, curl allows users to specify absurd request rates that might
be higher than the number of milliseconds in the unit (ex: curl --rate 3600050/h http://localhost:8080 does not error out despite there being
only 3600000ms in a hour).
This change adds a conditional check before the millisecond calculation
making sure that the number is not higher than the numerator (the unit)
If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE
Jay Satiro [Fri, 13 Oct 2023 00:50:45 +0000 (20:50 -0400)]
curl_trc: remove a bad assertion
- Remove DEBUGASSERT that an internal handle must not have user
private_data set before calling the user's debug callback.
This is a follow-up to 0dc40b2a. The user can distinguish their easy
handle from an internal easy handle by setting CURLOPT_PRIVATE on their
easy handle. I had wrongly assumed that meant the user couldn't then
set CURLOPT_PRIVATE on an internal handle as well.
Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 Reported-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/12104
Dan Fandrich [Sat, 14 Oct 2023 03:24:08 +0000 (20:24 -0700)]
test613: stop showing an error on missing output file
This test would show an error message if the output was missing during
the log post-processing step, but the message was not captured by the
test harness and wasn't useful since the normal golden log file
comparison would the problem more clearly.
Stefan Eissing [Tue, 10 Oct 2023 10:51:25 +0000 (12:51 +0200)]
quic: manage connection idle timeouts
- configure a 120s idle timeout on our side of the connection
- track the timestamp when actual socket IO happens
- check IO timestamp to our *and* the peer's idle timeouts
in "is this connection alive" checks
Reported-by: calvin2021y on github
Fixes #12064
Closes #12077
Dan Fandrich [Fri, 13 Oct 2023 02:33:19 +0000 (19:33 -0700)]
CI: ignore test 286 on Appveyor gcc 9 build
This test fails sometimes with a super fast retry loop due to what may
just be a compiler bug. The test results are ignored on the one CI job
where it occurs because there seems to be nothing we can do to fix it.
Viktor Szakats [Wed, 11 Oct 2023 04:02:45 +0000 (04:02 +0000)]
lib: fix gcc warning in printf call
Do not pass NULL to printf %s.
Seen with gcc 13.2.0 on Debian:
```
.../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wformat-overflow=]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/17584426483#step:3:11104
Ref: #10284 Co-authored-by: Jay Satiro
Closes #12082
Alex Klyubin [Thu, 12 Oct 2023 18:27:47 +0000 (12:27 -0600)]
http2: safer invocation of populate_binsettings
populate_binsettings now returns a negative value on error, instead of a
huge positive value. Both places which call this function have been
updated to handle this change in its contract.
The way populate_binsettings had been used prior to this change the huge
positive values -- due to signed->unsigned conversion of the potentially
negative result of nghttp2_pack_settings_payload which returns negative
values on error -- are not possible. But only because http2.c currently
always provides a large enough output buffer and provides H2 SETTINGS
IVs which pass the verification logic inside nghttp2. If the
verification logic were to change or if http2.c started passing in more
IVs without increasing the output buffer size, the overflow could become
reachable, and libcurl/curl might start leaking memory contents to
servers/proxies...
Daniel Stenberg [Fri, 13 Oct 2023 07:00:50 +0000 (09:00 +0200)]
wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA
This define is set in wolfssl's options.h file when this function and
feature is present. Handles both builds with the feature explicitly
disabled and wolfSSL versions before 5.5.2 - which introduced this API
call.
Viktor Szakats [Wed, 11 Oct 2023 16:06:12 +0000 (16:06 +0000)]
cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection
Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet,
aligning this with autotools. This fixes an error when doing
cross-builds and also actually detects this feature. It affected systems
not allowlisted into this, e.g. SerenityOS.
We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`.
Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979
Ref: #12095 (closed in favour of this patch)
Ref: #11964 (effort to sync cmake detections with autotools)
Reported-by: Kartatz on Github Assisted-by: Kartatz on Github
Fixes #12093
Closes #12094
Jay Satiro [Wed, 11 Oct 2023 05:34:19 +0000 (07:34 +0200)]
socks: return error if hostname too long for remote resolve
Prior to this change the state machine attempted to change the remote
resolve to a local resolve if the hostname was longer than 255
characters. Unfortunately that did not work as intended and caused a
security issue.
Stefan Eissing [Tue, 10 Oct 2023 08:50:17 +0000 (10:50 +0200)]
CI: remove slowed-network tests
- remove these tests as they are currently not reliable in our CI
setups.
curl handles the test cases, but CI sometimes fails on these due to
additional conditions. Rather than mix them in, an additional CI job
will be added in the future that is specific to them.
Stefan Eissing [Mon, 9 Oct 2023 09:36:37 +0000 (11:36 +0200)]
MQTT: improve receive of ACKs
- add `mq->recvbuf` to provide buffering of incomplete
ACK responses
- continue ACK reading until sufficient bytes available
- fixes test failures on low network receives
Stefan Eissing [Mon, 9 Oct 2023 08:18:20 +0000 (10:18 +0200)]
test2302: improve reliability
- make result print collected write data, unless
change in meta flags is detected
- will show same result even when data arrives via
several writecb invocations
Viktor Szakats [Sun, 8 Oct 2023 15:37:41 +0000 (15:37 +0000)]
build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros
Syncing this up with CMake.
Source code uses the built-in `OPENSSL_IS_AWSLC` and
`OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is
necessary from the build tools.
The one use of `HAVE_BORINGSSL` in the source turned out to be no longer
necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634
autotools detects this anyway for display purposes.
CMake detects this to decide whether to use the BoringSSL-specific
crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection
result just yet (planned in #12066).
Ref: #11964
Reviewed-by: Daniel Stenberg Reviewed-by: Jay Satiro
Closes #12065
Stefan Eissing [Thu, 5 Oct 2023 08:05:12 +0000 (10:05 +0200)]
cf-socket: simulate slow/blocked receives in debug
add 2 env variables for non-UDP sockets:
1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly
should return EAGAIN
2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket
Jay Satiro [Thu, 5 Oct 2023 07:19:47 +0000 (03:19 -0400)]
CURLOPT_DEBUGFUNCTION.3: warn about internal handles
- Warn that the user's debug callback may be called with the handle
parameter set to an internal handle.
Without this warning the user may assume that the only handles their
debug callback receives are the easy handles on which they set
CURLOPT_DEBUGFUNCTION.
This is a follow-up to f8cee8cc which changed DoH handles to inherit
the debug callback function set in the user's easy handle. As a result
those handles are now passed to the user's debug callback function.
Daniel Stenberg [Sat, 7 Oct 2023 18:10:36 +0000 (20:10 +0200)]
multi: do CURLM_CALL_MULTI_PERFORM at two more places
... when it does a state transition but there is no particular socket or
timer activity. This was made apparent when commit b5bb84c removed a
superfluous timer expiry.
Reported-by: Dan Fandrich.
Fixes #12033
Closes #12056
Dan Fandrich [Sat, 7 Oct 2023 01:18:49 +0000 (18:18 -0700)]
tests: fix a race condition in ftp server disconnect
If a client disconnected and reconnected quickly, before the ftp server
had a chance to respond, the protocol message/ack (ping/pong) sequence
got out of sync, causing messages sent to the old client to be delivered
to the new. A disconnect must now be acknowledged and intermediate
requests thrown out until it is, which ensures that such synchronization
problems can't occur. This problem could affect ftp, pop3, imap and smtp
tests.
David Benjamin [Thu, 5 Oct 2023 16:50:55 +0000 (12:50 -0400)]
openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR
While the struct is still public in OpenSSL, there is a (somewhat
inconvenient) accessor. Use it to remain compatible if it becomes opaque
in the future.
Viktor Szakats [Thu, 5 Oct 2023 23:04:40 +0000 (23:04 +0000)]
cmake: re-add missed C89 headers for specific detections
We removed C89 `setjmp.h` and `signal.h` detections and excluded them
from the global header list we use when detecting functions [1]. Then
missed to re-add these headers to the specific functions which need
them to be detected [2]. Fix this omission in this patch.
Dan Fandrich [Thu, 5 Oct 2023 20:11:08 +0000 (13:11 -0700)]
test1903: actually verify the cookies after the test
The test otherwise could do just about anything (except leak memory in
debug mode) and its bad behaviour wouldn't be detected. Now, check the
resulting cookie file to ensure the cookies are still there.
Jay Satiro [Fri, 29 Sep 2023 07:15:19 +0000 (03:15 -0400)]
idn: fix WinIDN null ptr deref on bad host
- Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from
UTF-8 to UTF-16.
Prior to this change a failed conversion erroneously returned CURLE_OK
which meant 'decoded' pointer (what would normally point to the
punycode) would not be written to, remain NULL and be dereferenced
causing an access violation.
Daniel Stenberg [Mon, 2 Oct 2023 13:10:55 +0000 (15:10 +0200)]
base64: also build for curl
Since the tool itself now uses the base64 code using the curlx way, it
needs to build also when the tool needs it. Starting now, the tool build
defines BULDING_CURL to allow lib-side code to use it.
Eduard Strehlau [Tue, 3 Oct 2023 15:10:46 +0000 (11:10 -0400)]
tests: Fix zombie processes left behind by FTP tests.
ftpserver.pl correctly cleans up spawned server processes,
but forgets to wait for the shell used to spawn them.
This is barely noticeable during a normal testrun,
but causes process exhaustion and test failure
during a complete torture run of the FTP tests.
Dan Fandrich [Fri, 29 Sep 2023 23:32:48 +0000 (16:32 -0700)]
tests: propagate errors in libtests
Use the test macros to automatically propagate some errors, and check
and log others while running the tests. This can help in debugging
exactly why a test has failed.
Dan Fandrich [Fri, 29 Sep 2023 22:27:21 +0000 (15:27 -0700)]
tests: set --expect100-timeout to improve test reliability
On an overloaded server, the default 1 second timeout can go by without
the test server having a chance to respond with the expected headers,
causing tests to fail. Increase the 1 second timeout to 99 seconds so
this failure mode is no longer a problem on test 1129. Some other tests
already set a high value, but make them consistently 99 seconds so if
something goes wrong the test is stalled for less time.
Viktor Szakats [Tue, 3 Oct 2023 18:17:37 +0000 (18:17 +0000)]
cmake: improve OpenLDAP builds
- cmake: detect OpenLDAP based on function `ldap_init_fd`.
autotools does this. autotools also publishes this detection result
in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source
doesn't use this value. (it might need to be remove-listed in
`scripts/cmp-config.pl` for future OpenLDAP test builds.)
This also deletes existing self-declaration method via the
CMake-specific `CURL_USE_OPENLDAP` configuration.
- cmake: define `LDAP_DEPRECATED=1` for OpenLDAP.
Like autotools does. This fixes a long list of these warnings:
```
/usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' is not defined, evaluates to 0 [-Wundef]
```
- cmake: delete LDAP TODO comment no longer relevant.
Also:
- autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature
detection functions.
Ref: #11964 (effort to sync cmake detections with autotools)
Daniel Stenberg [Wed, 4 Oct 2023 09:35:54 +0000 (11:35 +0200)]
tests: remove leading spaces from some tags
The threee tags `<name>`, `</name>` and `<command>` were frequently used
with a leading space that this removes. The reason this habbit is so
widespread in testcases is probably that they have been copy and pasted.
Hence, fixing them all now might curb this practice from now on.
projects / thirdparty / curl.git / log
