Viktor Szakats [Thu, 16 Jan 2025 19:39:13 +0000 (20:39 +0100)]
build: stop detecting `sched_yield()` on Windows
On Windows a successful `sched_yield()` detection requires mingw-w64
built with POSIX threads (not Win32 threads) and GCC (not llvm/clang).
(linking to `winpthread` via custom options may also work.)
In CMake builds, it was pre-cached as unavailable before this patch.
When detected (via autotools), it got only used for Windows XP or older
targets combined with a non-GCC, non-clang compiler that doesn't support
`__builtin_ia32_pause()`, or with the Intel C compiler. According to
`lib/easy_lock.h`.
mingw-w64 only supports GCC and clang, leaving a very narrow chance when
`shed_yield()` gets called on Windows. Even then, `sched_yield()` is
implemented in `winpthread` as `Sleep(0)`, which may or not be a useful.
It's also trivial to implement locally if it is, and such rare build
combination is also deemed useful.
Thus, this patch marks `sched_yields()` permanently unavailable on the
Windows platform also with autotools, and instead of pre-caching, skip
this feature check with CMake.
This syncs `HAVE_SCHED_YIELDS` between builds methods on Windows.
Viktor Szakats [Thu, 16 Jan 2025 14:21:23 +0000 (15:21 +0100)]
cmake: pre-fill `HAVE_STDATOMIC_H`, `HAVE_ATOMIC` for mingw-w64
`stdatomic.h` and `_Atomic` were first available in gcc 4.9.0 and
llvm/clang 3.6. Set detection values accordingly and save these two
detections on configure runs.
Stefan Eissing [Fri, 17 Jan 2025 10:57:00 +0000 (11:57 +0100)]
TLS: check connection for SSL use, not handler
Protocol handler option PROTOPT_SSL is used to setup a connection
filters. Once that is done, used `Curl_conn_is_ssl()` to check if
a connection uses SSL.
There may be other reasons to add SSL to a connection, e.g. starttls.
Viktor Szakats [Wed, 15 Jan 2025 14:46:32 +0000 (15:46 +0100)]
android: add CI jobs, buildinfo, cmake docs, disable `CURL_USE_PKGCONFIG` by default
- GHA/non-native: add Android builds, both cmake and autotools,
both NDK 21 (oldest available) and 35 (newest available)
https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md
It comes with a maintenance burden to bump the oldest/latest values
with CI runner updates.
- cmake: disable `CURL_USE_PKGCONFIG` by default for Android.
To avoid picking up system package by default.
- build: add `ANDROID-<NDK-LEVEL>` flag to `buildinfo.txt`.
Also detect NDK level with the CMake built-in build method:
https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-android
- INSTALL.md: add CMake build instructions for Android.
- INSTALL.md: make NDK levels consistent in `./configure` example.
Viktor Szakats [Thu, 16 Jan 2025 18:12:02 +0000 (19:12 +0100)]
GHA: start using ARM Linux runners
- GHA/linux: allow per-job runner image override.
- GHA/linux: add an arm version of an existing job.
Add workaround for broken `PATH` in the arm runner image.
- GHA/non-native: add CPU arch to job name where missing.
- GHA/checkdocs: switch a linter job to arm.
Performance looks a little bit better than Intel
(presumably with lower power consumption).
Viktor Szakats [Thu, 16 Jan 2025 21:19:51 +0000 (22:19 +0100)]
GHA/linux: enable wolfSSH in a wolfSSL job
With tests. (Most SFTP tests fail with wolfSSH.)
Also:
- disable pseudo-terminal in wolfSSH builds (also in CircleCI).
Not needed for curl.
- test642: add "compressed" to the description.
To avoid being the same as test600.
Follow-up to b7b4dc0d49543175ab0d9bb1cdc257a2d7f7cf0a #1735
Stefan Eissing [Thu, 2 Jan 2025 15:34:52 +0000 (16:34 +0100)]
websocket: fix message send corruption
- Fix a bug in EAGAIN handling when sending frames that led to a
corrupted last byte of the frame sent.
- Restore sanity to curl_ws_send() behaviour:
- Partial writes are reported as OK with the actual number of
payload bytes sent.
- CURLE_AGAIN is only returned when none of the payload bytes
(or for 0-length frames, not all of the frame header bytes)
could be sent.
- curl_ws_send() now behaves like a common send() call.
- Change 'ws-data' test client to allow concurrent send/recv
operations and vary frame sizes and repeat count.
- Add DEBUG env var CURL_WS_CHUNK_EAGAIN to simulate blocking
after a chunk of an encoded websocket frame has been sent.
- Add tests.
Prior to this change data corruption may occur when sending websocket
messages due to two bugs:
1) 3e64569a (precedes 8.10.0) caused a data corruption bug in the last
byte of frame of large messages.
2) curl_ws_send had non-traditional send behavior and could return
CURLE_AGAIN with bytes sent and expect the caller to adjust buffer
and buflen in a subsequent call. That behavior was not documented.
Viktor Szakats [Thu, 16 Jan 2025 15:47:15 +0000 (16:47 +0100)]
autotools: add support for mingw UWP builds
To match cmake builds.
- GHA/windows: allow autotools UWP builds.
- detect UWP and add to `buildinfo.txt`.
Consider it enabled if `CPPFLAGS` contains `-DWINSTORECOMPAT`.
- disable telnet with UWP.
- enable Unicode with UWP.
- do not use `wldap32` with UWP.
- do not enable `USE_WIN32_CRYPTO` with UWP.
- make sure to link to `ws2_32` in UWP builds.
To fix `undefined reference to `in6addr_any'` when linking
`tests/server` programs. More in the comment.
Stefan Eissing [Wed, 15 Jan 2025 15:45:25 +0000 (16:45 +0100)]
cf-https-connect: look into httpsrr alpns when available
Improved the filter implementation to be flexible in which order h3 and
h2/h1 are attempted. When HTTPSRR is enabled, look at the ALPNs it found
and use the order given for connecting in default setups.
- GHA/windows: drop the word "old" from standalone mingw-w64 jobs to not
conflate it with "old mingw" we no longer support (while also keeping
it short).
Jay Satiro [Mon, 13 Jan 2025 08:57:45 +0000 (03:57 -0500)]
transfer: fix CURLOPT_CURLU override logic
- Change setopt and pretransfer to always reset URL related variables
for a CURLU handle set CURLOPT_CURLU.
This change is to ensure we are in compliance with the doc which says
CURLU handles must be able to override a URL set via CURLOPT_URL and
that if the contents of the CURLU handle changes between transfers then
the updated contents must be used.
Prior to this change, although subsequent transfers appear to be
performed correctly in those cases, the work URL `data->state.url` was
not updated. CURLINFO_EFFECTIVE_URL returns data->state.url to the user
so it would return the URL from the initial transfer which was the wrong
URL. It's likely there are other cases as well.
Ref: https://curl.se/libcurl/c/CURLOPT_CURLU.html
Reported-by: Nicolás San Martín
Fixes https://github.com/curl/curl/issues/15984
Closes https://github.com/curl/curl/pull/15985
Jay Satiro [Tue, 14 Jan 2025 09:21:38 +0000 (04:21 -0500)]
mprintf: terminate snprintf output on windows
- Null terminate the end of the snprintf output buffer on Windows.
Old versions of the Windows CRT (which are often found on later versions
of Windows) do not terminate the snprintf output buffer if the output
reaches the max size.
This is a follow-up to parent 7e32f656 which made the same change but
limited it to mingw, however it is a CRT version issue irrespective of
compiler.
Daniel Stenberg [Mon, 13 Jan 2025 12:24:31 +0000 (13:24 +0100)]
mprintf: fix integer handling in float precision
In the double output function when an extremely large width and
precision is set that reaches the libcurl maximum (325), the handling of
the precision part would do wrong which could lead to bad output.
Also: work-around for single-byte buffer snprintf overflow with mingw.
Daniel Stenberg [Mon, 13 Jan 2025 11:16:19 +0000 (12:16 +0100)]
telnet: handle single-byte input option
Coverity CID 1638753 correctly identies this code misbehaved if the
passed in suboption is exactly one byte long by substracting two from
the unsigned size_t variable.
Daniel Stenberg [Sun, 12 Jan 2025 16:35:39 +0000 (17:35 +0100)]
multihandle: add an ssl_scache here
The TLS session cache is now held by the multi handle unless it is
shared, so that all easy handles within a multi handle get the benefit
of sharing the same, larger, cache.
The multi handle session cache size is set to 25, unless it is the
internal one used for the easy interface - which still uses only 3.
Viktor Szakats [Sun, 12 Jan 2025 10:57:24 +0000 (11:57 +0100)]
config: drop unused code and variables
- cmake, config-*: drop unused `PACKAGE*`, `VERSION` variables.
- config-win32: indentation
- config-win32ce: drop mingw-specific code.
This header is not used with MinGW.
- config-win32ce: `_WIN64` is never true for Windows CE, drop.
Viktor Szakats [Sun, 12 Jan 2025 02:25:36 +0000 (03:25 +0100)]
cmake: drop VS2010 "Dialog Hell" workaround added in 2013
Delete the workaround added via a94a68a3c1d04ccb53e46baa69753bbf6354ee14
(2013-02-04). The commit message has no details. The comment mentions
"Dialog Hell", and seems to fix CMake missing to regenerate `CURL.sln`
with VS2010. It also added a FIXME saying the workaround can be deleted
with future versions of CMake.
At the time CMake's latest version was v2.8.10.
curl now requires v3.7 (2018) minimum, and v3.24 (2022) was the
latest CMake natively supporting VS2010. Assume this has since been
fixed.
Also: format an MSVC version reference in comment.
Viktor Szakats [Sat, 11 Jan 2025 23:55:32 +0000 (00:55 +0100)]
configure: streamline Windows large file feature check
Before this patch the `CURL_CHECK_WIN32_LARGEFILE` feature check was
running an `AC_COMPILE` snippet that always succeeded. (except for
Windows CE, which isn't supported in other parts of `./configure` yet.)
The only Windows toolchain autotools supports is mingw. Of them, curl
only supports mingw-w64. All mingw-w64 versions support large files.
This allows to drop the check and assume it supported on Windows. To not
lose Windows CE support, rework that too, without using `AC_COMPILE`.
Drop the feature check altogether for non-Windows targets.
Viktor Szakats [Fri, 10 Jan 2025 15:58:48 +0000 (16:58 +0100)]
curl_setup: fix missing `ADDRESS_FAMILY` type in rare build cases
Build failed when both `ADDRESS_FAMILY` and `sockaddr_un` stuct were
missing from the Windows SDK, with UnixSockets enabled.
Seen with GNU 4.4.0 in CeGCC 0.59.1:
```
lib/curl_setup.h:983: error: expected specifier-qualifier-list before 'ADDRESS_FAMILY'
lib/curl_setup.h:985: warning: struct has no members
```
Also reported with VS2003:
https://datagirl.xyz/posts/wolfssl_curl_w2k.html
Viktor Szakats [Sat, 11 Jan 2025 02:22:10 +0000 (03:22 +0100)]
windows: drop redundant `USE_WIN32_SMALL_FILES` macro
In effect it meant `_WIN32 && !USE_WIN32_LARGE_FILES`.
Replace it with these macros.
Also:
- configure: delete tautological check for small file support.
- configure: delete stray `_MSC_VER` reference. autotools does not
support MSVC.
- drop tautological checks for WinCE in `config-win32*.h` when setting
`USE_WIN32_LARGE_FILES`.
- merge related PP logic.
- prefer `#ifdef`, fix whitespace.
- drop compatibility error for `CURL_WANTS_CA_BUNDLE_ENV`.
This macro was once set by `Makefile.mk` and Watcom makefiles.
They are no longer supported, making the compatibility message moot.
Viktor Szakats [Thu, 9 Jan 2025 17:19:35 +0000 (18:19 +0100)]
msvc: tidy up `_CRT_*_NO_DEPRECATE` definitions
Dedupe and migrate MSVC-specific warning suppressions to `curl_setup.h`.
Make cmake set `_CRT_SECURE_NO_DEPRECATE` for examples and standalone
tests, and stop setting `_CRT_NONSTDC_NO_DEPRECATE` for them.
Details:
- drop version guards. On ancient MSVC version these macro are a no-op.
- move to `curl_setup.h` from `config-win32*.h`.
- sync macro values with CMake.
- cmake: stop setting them globally in favour of `curl_setup.h`.
- cmake: re-add these macros to `docs/examples` and `tests/http/clients`,
which do not use `curl_setup.h`.
- cmake: drop `_CRT_NONSTDC_NO_DEPRECATE` for examples and tests.
They build fine without.
- update comments.
Viktor Szakats [Mon, 6 Jan 2025 22:34:19 +0000 (23:34 +0100)]
cmake: deprecate winbuild, add migration guide from legacy build methods
We recommend migrating to CMake from winbuild and Visual Studio project
files. winbuild is deprecated and will be dropped in September 2025.
CMake supports all the features and options, with new ones added
promptly. It supports out-of-tree, unity and documentation builds.
- deprecate winbuild method in favour of CMake by September 2025.
- add migration guide from winbuild to CMake.
- add migration guide from Visual Studio Project Files to CMake.
- add deprecation message to winbuild.
Need to ack with `WINBUILD_ACKNOWLEDGE_DEPRECATED=yes` Authored-by: Jay Satiro
- mention `CMAKE_BUILD_TYPE` option in `INSTALL-CMAKE`.
- document missing `SSH_PATH` winbuild option.
Assume the MSVC compiler offers the necessary support.
It makes curl require Visual Studio .NET 2003, v7.1 (`_MSC_VER = 1310`).
With the possibility that 1300 (Visual Studio .NET, v7.0, 2002), or 1200
(Visual C++, 32-bit, v6.0, 1998) may also work.
Daniel Stenberg [Wed, 8 Jan 2025 09:19:26 +0000 (10:19 +0100)]
cookie: cap expire times to 400 days
The pending cookie RFC update (currently known as 6265bis draft-19) says
Let cookie-age-limit be the maximum age of the cookie (which name of
Max-Age and an attribute-value of expiry-time. SHOULD be 400 days or
less.
This change makes received cookies over the wire get capped to 400 days.
It does not cap the expiry date of cookies loaded from file.
It does this by rounding the expire time to a even minute. This, to
allow the test suite to do the same and have a chance to get the same
number for stable testing without requiring a debug build.
The test script generates TWO numbers in the output file for each
%days[] used in the input test file, and the function that subsequently
compares and verifies output is fine with *either* of the two numbers.
This is done so that if the test case is generated the second
immediately before curl runs, that updated expiry number is also deemed
okay. It still checks for an exact match of either number.
Viktor Szakats [Thu, 9 Jan 2025 02:33:03 +0000 (03:33 +0100)]
msvc: drop checks for ancient versions
- drop version guard for `__inline`.
Supported since `_MSC_VER` 1000.
Visual C++, 32-bit, version 4.0 (1996)
- drop version guard for `__declspec(noreturn)` and `__forceinline`.
Supported since `_MSC_VER` 1200.
Visual C++, 32-bit, version 6.0 (1998)
For ancient versions, it's possible to override the default behaviour
by setting these macros via `CPPFLAGS`: `CURL_NORETURN`, `CURL_INLINE`,
`CURL_FORCEINLINE`
Viktor Szakats [Wed, 8 Jan 2025 11:19:11 +0000 (12:19 +0100)]
build: delete `-Wsign-conversion` related FIXMEs
We decided last year not to pursue avoiding this warning, because it
adds noise and friction, while in most cases not revealing actual code
issues. We fixed the interesting portion of them throughout mid-2024.
Conclude this effort by deleting related FIXMEs and temporary comments.
Daniel Stenberg [Tue, 7 Jan 2025 17:06:24 +0000 (18:06 +0100)]
DEPRECATE: remove msh3 in six months
The msh3 backed for QUIC and HTTP/3 was introduced in April 2022 but has
never been made to work properly. It has seen no visible traction or
developer activity from the msh3 main author (or anyone else seemingly
interested) in two years. As a non-functional backend, it only adds
friction and "weight" to the development and maintenance.
Meanwhile, we have a fully working backend in the ngtcp2 one and we have
two fully working backends in OpenSSL-QUIC and quiche well on their way
of ending their experimental status in a future.
We remove msh3 support from the curl source tree in July 2025.
Neil Horman [Fri, 3 Jan 2025 15:17:56 +0000 (10:17 -0500)]
osslq: use SSL_poll to determine writeability of QUIC streams
This discussion:
https://github.com/openssl/openssl/discussions/23339#discussion-6094341
Specifically item number 2 (Send Blocking) was raised by the curl team,
noting that SSL_want_write returning false was not a good indicator of
when a stream is writeable. The suggestion in that discussion was to use
SSL_poll with an SSL_POLL_EVENT_W flag instead, as that is a proper
indication of when an SSL_object will allow writing without blocking.
While ssl_want_write updates its state based on the last error
encountered (implying a need to retry an operation to update the
last_error state again), SSL_poll checks stream buffer status during the
call, giving it more up to date information on request. This is the
method used by our guide demos (quic-hq-interop specifically), and it
works well.
This change has been run through the curl test suite, and shown to pass
all tests. However, given the initial problem description I'm not sure
if there is a test case that explicitly checks for blocking and
unblocking of streams. As such some additional testing may be warranted.
Stefan Eissing [Wed, 8 Jan 2025 15:34:38 +0000 (16:34 +0100)]
HTTP/2: strip TE request header
The TE request header field is invalid in HTTP/2. Since clients may not
know in advance if a connection negotiates HTTP/2, automatically strip
such a header when h2 is in play.
Add test_01_10 to verify.
Reported-by: Jiri Stary
Fixes #15941
Closes #15943
Stefan Eissing [Tue, 7 Jan 2025 11:41:26 +0000 (12:41 +0100)]
vtls: feature ssls-export for SSL session im-/export
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Viktor Szakats [Tue, 7 Jan 2025 22:44:32 +0000 (23:44 +0100)]
appveyor: bump VS2008 jobs to VS2010
VS2008 has been partly broken for a while with its shared-debug builds
crashing on startup. Its compiler output (UTF-16 HTML) was also barely
readable even after conversion. It's also the only platform in CI
missing `stdint.h`.
This patch migrates a VS2008 job to VS2010 and drops another that
already had a VS2010 equivalent.
We recommend switching to VS2010 or newer when using MSVC to build curl.
Viktor Szakats [Tue, 7 Jan 2025 23:04:20 +0000 (00:04 +0100)]
appveyor: always use cmake `-A` option to select x64
The `Win64` generator suffix alternative was required by old CMake
versions (<3.1) only:
https://cmake.org/cmake/help/v3.22/generator/Visual%20Studio%2010%202010.html
As seen on Linux with 0.7.0:
```
/home/runner/msh3/include/msh3.h:377:18: error: width of ‘RESERVED’ exceeds its type
377 | bool RESERVED : 5;
| ^~~~~~~~
/home/runner/msh3/include/msh3.h:490:18: error: width of ‘RESERVED’ exceeds its type
490 | bool RESERVED : 7;
| ^~~~~~~~
```
https://github.com/curl/curl/actions/runs/12655717818/job/35266716846#step:35:195