Daniel Stenberg [Tue, 5 Nov 2024 10:52:38 +0000 (11:52 +0100)]
mbedtls: remove failf() use from mbedtls_random
Since data can be NULL in here, calling failf() can be bad. This should
also be a terribly rare error so the lack of error message for this
should be manageable.
Reported-by: wxiaoguang on github
Fixes #15485
Closes #15486
Daniel Stenberg [Fri, 1 Nov 2024 16:05:15 +0000 (17:05 +0100)]
openssl: extend the OpenSSL error messages
Previously there were three error situations that only added the (unhelpful)
OpenSSL error strings, now we prefix those with a short explanation for the
error situation.
curl_addrinfo: support operating systems with only getaddrinfo(3)
The gethostbyname(3) family was removed in POSIX-1.2008 in favor of
getaddrinfo(3) introduced in POSIX-1.2001. Modern POSIX systems such as
Sortix does not have gethostbyname nor the related definitions and
structures.
curl already only uses getaddrinfo(3) if available and thread safe,
although there is mild breakage if the related gethostbyname definitions
are missing.
This change attempts to fix that breakage:
Remove an unnecessary configure error if gethostbyname is missing since
getaddrinfo is enough as a fallback.
Rewrite Curl_ip2addr to not use struct hostent as it no longer is
standardized and create the struct Curl_addrinfo directly.
Only define the Curl_he2ai function on non-getaddrinfo systems where it
is going to be used with struct hoestent.
Revoke the fallback logic for when it's unknown whether getaddrinfo is
thread safe. It doesn't appear to make any sense since h_errno is
unrelated to getaddrinfo. The logic prevents new POSIX.1-2024 systems
from passing the thread safety test since h_errno does not exist anymore
and POSIX already requires getaddrinfo to be thread safe. There's
already a denylist in place for operating systems with known buggy
implementations.
Stefan Eissing [Thu, 31 Oct 2024 11:47:05 +0000 (12:47 +0100)]
ngtcp2: do not loop on recv
The vquic_recv_packets() function already loops when not all requested
packets can be received (until EAGAIN) and there is not need to do that
again in ngtcp2.
Viktor Szakats [Tue, 29 Oct 2024 15:13:45 +0000 (16:13 +0100)]
build: fix clang-cl builds, add CI job
- appveyor: add build-only job for clang-cl.
- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
automatically, which makes `__int64` detection fail.
Explictly disable this compiler warning for clang-cl to make the
feature detection work and to accept `__int64` in the source code.
- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
to fix these when encountering `__int64`:
```
lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
```
- make `__GNUC__` warning suppressions apply to `__clang__` too.
Necessary for clang-cl, which defines the latter, but not the former.
(Regular clang defines both.)
- examples: fix clang-cl compiler warning in `http2-upload.c`.
```
docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
```
Marwan Yassini [Tue, 29 Oct 2024 17:22:03 +0000 (17:22 +0000)]
mqtt: fix mqtt.md wording and add clearer explanation
Some of the wording in the mqtt.md confused me as to how the commands
were used and what they did, so I cleared up some of the wording to
better explain what each command does.
- Remove reference to 'Developer Command Prompt for Visual Studio'
shortcut since it opens in x86 mode.
That prompt may confuse users since it is not easily switched to x64.
Our instruction says vcvarsall can be used to change the platform but it
is not in the path in any version that I checked (VS 2010, 2013, 2022).
Instead users will now only see the remaining instruction to use a
platform specific command prompt to build curl, like "x64 Native Tools".
There's several links as well to Microsoft documentation for users that
have more complicated requirements, such as using vcvarsall.
Daniel Stenberg [Tue, 29 Oct 2024 08:21:37 +0000 (09:21 +0100)]
tool_operate: url_proto improvements
- renamed to url_proto_and_rewrite to better reveal what it does
- clarify the functionality in the top comment
- make it return CURLE_OUT_OF_MEMORY appropriately
- remove check for URL being set, use assert instead
Daniel Stenberg [Tue, 29 Oct 2024 07:01:47 +0000 (08:01 +0100)]
setopt_cptr: make overflow check only done when needed
An overflow check for if the value of a curl_off_t is larger than a
size_t can hold, is only necessary if the two types are actually
differently sized, now checked by the preprocessor. To avoid
"Unreachable Conditional".
Daniel Stenberg [Mon, 28 Oct 2024 12:27:58 +0000 (13:27 +0100)]
tls: avoid abusing CURLE_SSL_ENGINE_INITFAILED
That error code was introduced and has been used for OpenSSL ENGINE
things and not others, so switch the other use cases over to other TLS
related error codes.
Stefan Eissing [Mon, 28 Oct 2024 10:59:06 +0000 (11:59 +0100)]
tests/http: add --insecure tests
Add two test cases that connection using a hostname the server has no
certificate for. First, verify that the peer verification fail, as
expected. Second, provide '--insecure' to test that the connection
succeeded and returned some data.
Daniel Stenberg [Sun, 27 Oct 2024 10:50:48 +0000 (11:50 +0100)]
tool_getparam: drop unused time() call
The second argument to curl_getdate() once took a time argument, but
that feature has been gone for decades, thus passing in a date there
makes no difference.
Viktor Szakats [Fri, 25 Oct 2024 08:47:25 +0000 (10:47 +0200)]
cmake: tweaks around debug mode and hidden symbols
- emit warning for `ENABLE_DEBUG` builds.
- add words to clarify that `ENABLE_DEBUG` is meant for developing curl
itself.
- add comment saying `CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON`
CMake option may export extra, non-curl symbols.
Ref: https://github.com/microsoft/vcpkg/issues/41761
Unexplained exports seen also in curl CI:
```
[ 742] _tcschr
[ 743] _tcsncmp
[ 744] _tcsncpy
[ 745] _tcspbrk
```
https://ci.appveyor.com/project/curlorg/curl/builds/50864041/job/lolledrg4h7hu6e4?fullLog=true#L2160
CMake extracts these symbols from `.obj` files:
https://gitlab.kitware.com/cmake/cmake/-/issues/22092#note_943718
I have not found any new MSVC option that helps fixing this without
decorating all functions in-source or maintaining a manual list of
internal function names used for tests:
https://learn.microsoft.com/cpp/build/reference/wholearchive-include-all-library-object-files
Viktor Szakats [Fri, 25 Oct 2024 20:31:29 +0000 (22:31 +0200)]
build: disable warning `-Wunreachable-code-break`
This warning remains silent in unity builds. Since we're using unity
in CI for most jobs, warnings remain undetected there.
Disable them for all builds to avoid a surprise warning outside our CI.
The issue caught by the warning is useful for a tidy codebase, but
doesn't affect executed code. It was enabled in 84338c4de2d7c798e3c270c9610d51a4ad18a90b #12331 (2023-11-15).
Daniel Stenberg [Sat, 26 Oct 2024 20:12:41 +0000 (22:12 +0200)]
multi: split multi_runsingle into sub functions
Introduce five functions named after the state they serve:
- state_connect for MSTATE_CONNECT
- state_do for MSTATE_DO
- state_performing for MSTATE_PERFORMING
- state_ratelimiting for MSTATE_RATELIMITING
- state_resolving for MSTATE_RESOLVING
Viktor Szakats [Fri, 25 Oct 2024 01:01:58 +0000 (03:01 +0200)]
cmake: clear package version after `pkg-config` detection
`pkg_check_modules()` seems to leave `<PACKAGE>_VERSION` defined with an
empty value, if the package is not found.
When the package is also not found in the fallback branch,
`find_package_handle_standard_args()` logs and error message. In this
message it includes the bogus empty value as: `(found version "")`:
```
Could NOT find Libssh2 (missing: LIBSSH2_INCLUDE_DIR LIBSSH2_LIBRARY) (found version "")
```
https://github.com/curl/curl/actions/runs/11509727553/job/32040378958?pr=15408#step:31:99
Clear the version number to avoid the confusion:
```
Could NOT find Libssh2 (missing: LIBSSH2_INCLUDE_DIR LIBSSH2_LIBRARY)
```
https://github.com/curl/curl/actions/runs/11510022503/job/32041149129?pr=15408#step:31:99
Viktor Szakats [Thu, 24 Oct 2024 00:04:22 +0000 (02:04 +0200)]
cmake: document `-D` and env build options
Extend `INSTALL-CMAKE` document with the list of available options,
a short description and default values.
The list may not be 100% complete.
There are no component boundaries in CMake, so the line is blurry
between curl options, CMake options, CMake Find modules options.
I included certain CMake options that seemed useful, and/or have
dedicated use withing curl's CMake source. But, all CMake built-in
options are usable, as documented upstream in CMake.
The naming of the options has a heritage and the inconsistencies with
it, including a lack of clear namespace. This may be subject to future
updates, also after figuring out which name has special meaning within
CMake and/or CMake projects out of unwritten convention or something
more tangible.
CMake allows to initialize any internal variable via `-D`. This may be
useful to pre-initialize/override feature check results. The list
doesn't contain these, and they remain officially undocumented.
Also:
- make adjustments to keep the spellchecker happy.
- retrofit description changes to the cmake sources.
- stop documenting deprecated `Find*` variables.
Reported-by: Daniel Stenberg
Fixes https://github.com/curl/curl/discussions/14885
Closes #15388
Viktor Szakats [Thu, 24 Oct 2024 10:15:44 +0000 (12:15 +0200)]
cmake: tidy up and shorten symbol hiding initialization
- drop redundant local variables.
- drop extra Intel C feature check that always returned success due
to not passing the tested compiler option. Nobody reported an issue
with this since 2016, suggesting the version check alone is enough.
Daniel Stenberg [Thu, 24 Oct 2024 10:48:07 +0000 (12:48 +0200)]
url.md: clarify
- the specified URL can also get data sent to it
- rephrase the scheme guessing part
- mention target options for each URL for saving data
- mention --remote-name-all
- remove "warning" and make it into normal text
Daniel Stenberg [Thu, 24 Oct 2024 10:27:21 +0000 (12:27 +0200)]
version: minor cleanups
- remove typecasts and parentheses in zstd_version()
- create and use oldap_version() for OpenLDAP
- create and use psl_version() for libpsl
- reduce the size of the 40 byte buffers to 30 bytes
- use the brotil/zstd like the others (add the lib name in the functions)
- create and use idn_version for IDN builds
- handle (unlikely) error from ldap_get_option
- Create a new macro SCH_DEV() to manage verbose debug messages that are
only useful for debugging Schannel recv decryption.
schannel_recv contains a lot of useful debug messages to help debug the
function, however in practice they are not otherwise useful and showing
them in debug builds adds a lot of noise.
To show these messages curl must now be built with
CURL_SCHANNEL_DEV_DEBUG defined.
Prior to this change many, but not all, extra-verbose messages were
wrapped in DEBUGF() so they were only shown in debug builds.
Stefan Eissing [Wed, 23 Oct 2024 09:12:47 +0000 (11:12 +0200)]
schannel: ignore error on recv beyond close notify
When receiving data, schannel does a recv from the lower filters, e.g.
the socket, *before* it decrypts and analyses the buffered data it
already has. When that buffer contains a close-notify, e.g. the end of
the TLS stream, any error on the previous receive from the socket are
not applicable to its return codes.
Example from #153345: a server sends a close notify and closes its
connection. The encrypted data, including the close notify is received.
Another receive on the sockets gets a CONNABORTED which curl reports as
CURLE_RECV_ERROR. Schannel analyses its bufferi, sees the close notify
and early returns to the caller. On this return, the error on the
attempted receive does not apply.
renovate[bot] [Sat, 19 Oct 2024 23:17:04 +0000 (23:17 +0000)]
GHA: update five dependencies
- rojopolis/spellcheck-github-actions digest to ab8ac45
- nghttp2/nghttp2 to v1.64.0
- actions/cache digest to 6849a64
- github/codeql-action digest to 6624720
- Update actions/checkout digest to 11bd719
Viktor Szakats [Tue, 22 Oct 2024 11:22:39 +0000 (13:22 +0200)]
cmake: avoid setting `BUILD_TESTING`
`BUILD_TESTING` variable is used by other projects and CMake internally.
Replace `cmake_dependent_option()` with `option()` and introduce an
internal variable to track if want and can do testing.
Viktor Szakats [Wed, 23 Oct 2024 13:40:18 +0000 (15:40 +0200)]
libssh2: delete duplicate `break`
```
lib/vssh/libssh2.c:2495:7: warning: 'break' will never be executed [-Wunreachable-code-break]
break;
^~~~~
```
CI did not catch it due to llvm skipping this check for all #included
files. It's designed this way to avoid performance issues and false
positive when checking headers:
https://github.com/llvm/llvm-project/issues/71046
Viktor Szakats [Wed, 23 Oct 2024 09:06:27 +0000 (11:06 +0200)]
GHA/windows: work around Git for Windows perf regression
Fix the significant perf regression for vcpkg jobs by switching to the
MSYS2 shell environment from Git for Windows. This env is already used
for old-mingw-w64 job that remained unaffected by this issue.
The issue began with the windows-runner update 20241015.1.0. It bumped
Git for Windows from Git 2.46.2.windows.1 to Git 2.47.0.windows.1. GfW
bumped its MSYS2 components, including `msys-2.0.dll`. That's Cygwin
code, which may have contributed to this. Pipes were involved and
`runtests.pl` relies on pipes heavily in parallel mode. (The issue was
not seen with parallel tests disabled, in retrospect.)
This is useful as a permanent solution too. It drop GfW as a dependency
and makes Windows jobs use one less shell/env flavour.
Long term it might help to use native Windows Perl to avoid the MSYS
layer completely, if there is a way to make that work.
Viktor Szakats [Wed, 23 Oct 2024 08:54:14 +0000 (10:54 +0200)]
GHA/linux: drop patch from openssl3 thread sanitizer
The patch is now part of the 3.4.0 stable release.
(Turns out it was part of 3.3.2 already.)
Also:
- rename this local build to match the scheme used with wolfssl.
- drop '3' from local openssl build name.
- sync job name with others.
- quote step names where missing.
Viktor Szakats [Tue, 17 Sep 2024 16:43:17 +0000 (18:43 +0200)]
runtests: use deterministic sort for `TESTINFO` lines
Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.
Max Dymond [Tue, 22 Oct 2024 12:31:22 +0000 (13:31 +0100)]
ci: fix renovate's matching for OpenSSL and quictls
Renovate only matches on the raw version numbers of a package, but
OpenSSL includes `openssl-` as a prefix in the version number. This
change means that the match string now expects the `openssl-` prefix
and will just update the version portion.
This also updates quictls so that renovate can detect and update the
version correctly.
Viktor Szakats [Tue, 22 Oct 2024 15:02:14 +0000 (17:02 +0200)]
cmake/FindCares: fix version detection for c-ares 1.34.1
Due to a regression in c-ares 1.34.1, the non-pkg-config version
detection method broke for this version. c-ares 1.34.2 fixes it,
but update our detection code anyway to also work with 1.34.1.
Viktor Szakats [Tue, 22 Oct 2024 15:32:28 +0000 (17:32 +0200)]
cmake: use the `BSD` variable
- use `BSD` in addition to backwards-compatible method.
- add `BSD` to the configuration log and `buildinfo.txt` if detected.
- add `BSD` tag to `buildinfo.txt` also via `./configure`.
The `BSD` variable is supported by CMake 3.25.0 and upper.