Daniel Engberg [Sun, 10 Nov 2024 21:23:26 +0000 (22:23 +0100)]
GHA/non-native: streamline installed packages on FreeBSD
Instead of installing the cmake package which is a meta-port (includes
documentation and manpages etc) install cmake-core which is cmake itself
to save a few cpu cycles.
Also drop nghttp2 in favour of the slimmer libnghttp2.
The script's previous treatment of this meta-data was a
misunderstanding. (Added in 1ebc53df25181908) The mistrust is not for
the root cert at this date (it would simply be removed from the bundle
then instead) but for created server certificates:
If a builtin certificate has a CKA_NSS_SERVER_DISTRUST_AFTER
timestamp before the SCT or NotBefore date of a certificate that
builtin issued, then clients can elect not to trust it.
That is however information that cannot be provided in the generated PEM
output.
Fixes #15547 Reported-by: Andrew Ayer
Closes #15552
Viktor Szakats [Mon, 11 Nov 2024 12:59:08 +0000 (13:59 +0100)]
curl-rustls.m4: keep existing `CPPFLAGS`/`LDFLAGS` when detected
Different variable names were used in flag save and restore operations,
which could cause existing `CPPFLAGS` and `LDFLAGS` be accidentally lost
when detecting Rustls.
Viktor Szakats [Fri, 8 Nov 2024 17:51:45 +0000 (18:51 +0100)]
build: use `_fseeki64()` on Windows, drop detections
A recent update caused CMake builds to mis-detect this symbol on iOS.
Auto-detection also seems redundant given that it's a Windows-only
function and most Windows builds were already opted-in.
Drop detections and use it in all Windows builds with large file support
enabled.
Ethan Everett [Fri, 8 Nov 2024 19:49:32 +0000 (11:49 -0800)]
nghttp2: use custom memory functions
Provide libcurl's memory functions as callbacks to replace nghttp2's own memory
functions. This allows custom memory callbacks provided by users of libcurl to
be used by nghttp2 as well.
Viktor Szakats [Thu, 7 Nov 2024 07:09:30 +0000 (08:09 +0100)]
macos: disable gcc `availability` workaround as needed
Homebrew gcc 14.2.0_1 fixed the issue, and the workaround is no longer
needed. Not only not needed, but the workaround is breaking builds with
the fixed gcc.
Auto-detect the upstream fix and stop applying the local workaround if
detected.
Viktor Szakats [Fri, 8 Nov 2024 00:29:50 +0000 (01:29 +0100)]
GHA/macos: let gcc dictate the configured Apple SDK
As discovered earlier, Homebrew gcc is built against a specific Apple
SDK version and doesn't work when matched up with a different version,
e.g. the one advertised as default by the macos runner image.
Before this patch this was resolved with brute force by zapping the
hack-layer gcc component to avoid the bad interference. This worked
for us, but it's fragile, accidental and doesn't translate to
real-world build environments. Thus, impractical.
Avoid this by explicitly selecting the SDK version gcc was built for and
meant to be used with, as shown by `gcc --print-sysroot`.
It assumes that the gcc binaries preinstalled on the runner images
always ship with the SDK version they reference. It also assumes
this works with and without `brew update`.
Also:
- add 4 quick build-only jobs to test all gcc/macos combos.
- list SDKs offered via CommandLineTools.
Suggested-by: Bo Anderson
Ref: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2462764619
Viktor Szakats [Fri, 8 Nov 2024 02:17:44 +0000 (03:17 +0100)]
GHA: add `apt update` where missing
To mitigate this kind of (repeat) errors:
```
Err:14 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 python3-werkzeug all 3.0.1-3ubuntu0.1
404 Not Found [IP: 40.81.13.82 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/p/python-werkzeug/python3-werkzeug_3.0.1-3ubuntu0.1_all.deb
Fetched 10.4 MB in 4s (2593 kB/s)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
```
https://github.com/curl/curl/actions/runs/11732257460/job/32684111508#step:3:74
Daniel Stenberg [Tue, 5 Nov 2024 10:52:38 +0000 (11:52 +0100)]
mbedtls: remove failf() use from mbedtls_random
Since data can be NULL in here, calling failf() can be bad. This should
also be a terribly rare error so the lack of error message for this
should be manageable.
Reported-by: wxiaoguang on github
Fixes #15485
Closes #15486
Daniel Stenberg [Fri, 1 Nov 2024 16:05:15 +0000 (17:05 +0100)]
openssl: extend the OpenSSL error messages
Previously there were three error situations that only added the (unhelpful)
OpenSSL error strings, now we prefix those with a short explanation for the
error situation.
curl_addrinfo: support operating systems with only getaddrinfo(3)
The gethostbyname(3) family was removed in POSIX-1.2008 in favor of
getaddrinfo(3) introduced in POSIX-1.2001. Modern POSIX systems such as
Sortix does not have gethostbyname nor the related definitions and
structures.
curl already only uses getaddrinfo(3) if available and thread safe,
although there is mild breakage if the related gethostbyname definitions
are missing.
This change attempts to fix that breakage:
Remove an unnecessary configure error if gethostbyname is missing since
getaddrinfo is enough as a fallback.
Rewrite Curl_ip2addr to not use struct hostent as it no longer is
standardized and create the struct Curl_addrinfo directly.
Only define the Curl_he2ai function on non-getaddrinfo systems where it
is going to be used with struct hoestent.
Revoke the fallback logic for when it's unknown whether getaddrinfo is
thread safe. It doesn't appear to make any sense since h_errno is
unrelated to getaddrinfo. The logic prevents new POSIX.1-2024 systems
from passing the thread safety test since h_errno does not exist anymore
and POSIX already requires getaddrinfo to be thread safe. There's
already a denylist in place for operating systems with known buggy
implementations.
Stefan Eissing [Thu, 31 Oct 2024 11:47:05 +0000 (12:47 +0100)]
ngtcp2: do not loop on recv
The vquic_recv_packets() function already loops when not all requested
packets can be received (until EAGAIN) and there is not need to do that
again in ngtcp2.
Viktor Szakats [Tue, 29 Oct 2024 15:13:45 +0000 (16:13 +0100)]
build: fix clang-cl builds, add CI job
- appveyor: add build-only job for clang-cl.
- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
automatically, which makes `__int64` detection fail.
Explictly disable this compiler warning for clang-cl to make the
feature detection work and to accept `__int64` in the source code.
- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
to fix these when encountering `__int64`:
```
lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
```
- make `__GNUC__` warning suppressions apply to `__clang__` too.
Necessary for clang-cl, which defines the latter, but not the former.
(Regular clang defines both.)
- examples: fix clang-cl compiler warning in `http2-upload.c`.
```
docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
```
Marwan Yassini [Tue, 29 Oct 2024 17:22:03 +0000 (17:22 +0000)]
mqtt: fix mqtt.md wording and add clearer explanation
Some of the wording in the mqtt.md confused me as to how the commands
were used and what they did, so I cleared up some of the wording to
better explain what each command does.
- Remove reference to 'Developer Command Prompt for Visual Studio'
shortcut since it opens in x86 mode.
That prompt may confuse users since it is not easily switched to x64.
Our instruction says vcvarsall can be used to change the platform but it
is not in the path in any version that I checked (VS 2010, 2013, 2022).
Instead users will now only see the remaining instruction to use a
platform specific command prompt to build curl, like "x64 Native Tools".
There's several links as well to Microsoft documentation for users that
have more complicated requirements, such as using vcvarsall.
Daniel Stenberg [Tue, 29 Oct 2024 08:21:37 +0000 (09:21 +0100)]
tool_operate: url_proto improvements
- renamed to url_proto_and_rewrite to better reveal what it does
- clarify the functionality in the top comment
- make it return CURLE_OUT_OF_MEMORY appropriately
- remove check for URL being set, use assert instead
Daniel Stenberg [Tue, 29 Oct 2024 07:01:47 +0000 (08:01 +0100)]
setopt_cptr: make overflow check only done when needed
An overflow check for if the value of a curl_off_t is larger than a
size_t can hold, is only necessary if the two types are actually
differently sized, now checked by the preprocessor. To avoid
"Unreachable Conditional".
Daniel Stenberg [Mon, 28 Oct 2024 12:27:58 +0000 (13:27 +0100)]
tls: avoid abusing CURLE_SSL_ENGINE_INITFAILED
That error code was introduced and has been used for OpenSSL ENGINE
things and not others, so switch the other use cases over to other TLS
related error codes.
Stefan Eissing [Mon, 28 Oct 2024 10:59:06 +0000 (11:59 +0100)]
tests/http: add --insecure tests
Add two test cases that connection using a hostname the server has no
certificate for. First, verify that the peer verification fail, as
expected. Second, provide '--insecure' to test that the connection
succeeded and returned some data.
Daniel Stenberg [Sun, 27 Oct 2024 10:50:48 +0000 (11:50 +0100)]
tool_getparam: drop unused time() call
The second argument to curl_getdate() once took a time argument, but
that feature has been gone for decades, thus passing in a date there
makes no difference.
Viktor Szakats [Fri, 25 Oct 2024 08:47:25 +0000 (10:47 +0200)]
cmake: tweaks around debug mode and hidden symbols
- emit warning for `ENABLE_DEBUG` builds.
- add words to clarify that `ENABLE_DEBUG` is meant for developing curl
itself.
- add comment saying `CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON`
CMake option may export extra, non-curl symbols.
Ref: https://github.com/microsoft/vcpkg/issues/41761
Unexplained exports seen also in curl CI:
```
[ 742] _tcschr
[ 743] _tcsncmp
[ 744] _tcsncpy
[ 745] _tcspbrk
```
https://ci.appveyor.com/project/curlorg/curl/builds/50864041/job/lolledrg4h7hu6e4?fullLog=true#L2160
CMake extracts these symbols from `.obj` files:
https://gitlab.kitware.com/cmake/cmake/-/issues/22092#note_943718
I have not found any new MSVC option that helps fixing this without
decorating all functions in-source or maintaining a manual list of
internal function names used for tests:
https://learn.microsoft.com/cpp/build/reference/wholearchive-include-all-library-object-files
Viktor Szakats [Fri, 25 Oct 2024 20:31:29 +0000 (22:31 +0200)]
build: disable warning `-Wunreachable-code-break`
This warning remains silent in unity builds. Since we're using unity
in CI for most jobs, warnings remain undetected there.
Disable them for all builds to avoid a surprise warning outside our CI.
The issue caught by the warning is useful for a tidy codebase, but
doesn't affect executed code. It was enabled in 84338c4de2d7c798e3c270c9610d51a4ad18a90b #12331 (2023-11-15).
Daniel Stenberg [Sat, 26 Oct 2024 20:12:41 +0000 (22:12 +0200)]
multi: split multi_runsingle into sub functions
Introduce five functions named after the state they serve:
- state_connect for MSTATE_CONNECT
- state_do for MSTATE_DO
- state_performing for MSTATE_PERFORMING
- state_ratelimiting for MSTATE_RATELIMITING
- state_resolving for MSTATE_RESOLVING
Viktor Szakats [Fri, 25 Oct 2024 01:01:58 +0000 (03:01 +0200)]
cmake: clear package version after `pkg-config` detection
`pkg_check_modules()` seems to leave `<PACKAGE>_VERSION` defined with an
empty value, if the package is not found.
When the package is also not found in the fallback branch,
`find_package_handle_standard_args()` logs and error message. In this
message it includes the bogus empty value as: `(found version "")`:
```
Could NOT find Libssh2 (missing: LIBSSH2_INCLUDE_DIR LIBSSH2_LIBRARY) (found version "")
```
https://github.com/curl/curl/actions/runs/11509727553/job/32040378958?pr=15408#step:31:99
Clear the version number to avoid the confusion:
```
Could NOT find Libssh2 (missing: LIBSSH2_INCLUDE_DIR LIBSSH2_LIBRARY)
```
https://github.com/curl/curl/actions/runs/11510022503/job/32041149129?pr=15408#step:31:99
Viktor Szakats [Thu, 24 Oct 2024 00:04:22 +0000 (02:04 +0200)]
cmake: document `-D` and env build options
Extend `INSTALL-CMAKE` document with the list of available options,
a short description and default values.
The list may not be 100% complete.
There are no component boundaries in CMake, so the line is blurry
between curl options, CMake options, CMake Find modules options.
I included certain CMake options that seemed useful, and/or have
dedicated use withing curl's CMake source. But, all CMake built-in
options are usable, as documented upstream in CMake.
The naming of the options has a heritage and the inconsistencies with
it, including a lack of clear namespace. This may be subject to future
updates, also after figuring out which name has special meaning within
CMake and/or CMake projects out of unwritten convention or something
more tangible.
CMake allows to initialize any internal variable via `-D`. This may be
useful to pre-initialize/override feature check results. The list
doesn't contain these, and they remain officially undocumented.
Also:
- make adjustments to keep the spellchecker happy.
- retrofit description changes to the cmake sources.
- stop documenting deprecated `Find*` variables.
Reported-by: Daniel Stenberg
Fixes https://github.com/curl/curl/discussions/14885
Closes #15388
Viktor Szakats [Thu, 24 Oct 2024 10:15:44 +0000 (12:15 +0200)]
cmake: tidy up and shorten symbol hiding initialization
- drop redundant local variables.
- drop extra Intel C feature check that always returned success due
to not passing the tested compiler option. Nobody reported an issue
with this since 2016, suggesting the version check alone is enough.
Daniel Stenberg [Thu, 24 Oct 2024 10:48:07 +0000 (12:48 +0200)]
url.md: clarify
- the specified URL can also get data sent to it
- rephrase the scheme guessing part
- mention target options for each URL for saving data
- mention --remote-name-all
- remove "warning" and make it into normal text