Stefan Eissing [Fri, 21 Nov 2025 12:06:00 +0000 (13:06 +0100)]
apple-sectrust: always ask when `native_ca_store` is in use
When OpenSSL fails to verify the peer certificate, we checked for
one specific reason code and did not ask Apple SecTrust for any
other failure.
Always ask Apple SecTrust after OpenSSL fails when the `native_ca_store`
is enabled. If the user configures a CAfile or CApath, the native store
is disabled, so this does not affect use cases where users asks curl
to use a specific set of trust anchors.
Do the same for GnuTLS
Fixes #19636 Reported-by: ffath-vo on github
Closes #19638
To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.
Other than these cases, we recommend leaving type checking enabled.
Viktor Szakats [Mon, 17 Nov 2025 22:49:15 +0000 (23:49 +0100)]
lib: replace `_tcsncpy`/`wcsncpy`/`wcscpy` with `_s` counterparts (Windows)
Replace:
- curl_sspi: macro `_tcsncpy()` with `_tcsncpy_s()`.
- curlx/fopen: `wcsncpy()` with `wcsncpy_s()`.
- curlx/fopen: `wcscpy()` with `wcscpy_s()`.
Use of the pre-existing functions were safe. This patch aims to use the
recommended Windows CRT functions. Handle errors returned by them. Also
to avoid the compiler warnings silenced via `_CRT_SECURE_NO_WARNINGS`:
```
lib/curl_sspi.c(152): warning C4996: 'wcsncpy': This function or variable may be unsafe. Consider using wcsncpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS.
lib/curlx/fopen.c(161): warning C4996: 'wcsncpy': This function or variable may be unsafe. Consider using wcsncpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS.
lib/curlx/fopen.c(162): warning C4996: 'wcscpy': This function or variable may be unsafe. Consider using wcscpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS.
lib/curlx/fopen.c(174): warning C4996: 'wcsncpy': This function or variable may be unsafe. Consider using wcsncpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS.
lib/curlx/fopen.c(175): warning C4996: 'wcscpy': This function or variable may be unsafe. Consider using wcscpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS.
```
Viktor Szakats [Thu, 20 Nov 2025 16:39:53 +0000 (17:39 +0100)]
src: move `memdebug.h` to be the last include
`memdebug.h` must be included last within each source. This breaks when
including it in a header, which ends up being included in the middle of
other headers, and `memdebug.h` also ending up in the middle of
includes.
Viktor Szakats [Fri, 21 Nov 2025 02:52:04 +0000 (03:52 +0100)]
GHA/checksrc: switch xmllint job to Linux (from macOS)
macOS was chosen because xmllint comes preinstalled, saving the prereq
install step. But, macOS's xmllint jobs sometimes doesn't finish in 1m
(instead of under 1 second) and gets cancelled, causing flaky failures.
Go with Linux and an install phase (of 15s) instead.
Viktor Szakats [Fri, 21 Nov 2025 01:44:08 +0000 (02:44 +0100)]
GHA/codeql: add tweak to successfully build libtests for CodeQL
Turns out the cause of CodeQL hangs (or probably just extreme long
compile) is the header `curl/typecheck-gcc.h`. By accident I noticed
that the preprocessed output of libtests.c is 75 MB (megabytes). This
is much higher than the amounf of source code hinted, also compared to
e.g. units.c or other build targets. The reason for the extreme size
is each easy option call pulling in the large checker logic defined
in this header.
By compiling with `-DCURL_DISABLE_TYPECHECK`, preprocessed output drops
to 2.2 MB (34x), and the libtests target builds without issues.
Also build all tests and examples with the Linux HTTP/3 config, covering
3 more files.
With these, CodeQL C coverage is 893 out of 930 (96%) (was: 645 69%)
Viktor Szakats [Wed, 19 Nov 2025 22:33:52 +0000 (23:33 +0100)]
cmake: honor `CURL_DISABLE_INSTALL` and `CURL_ENABLE_EXPORT_TARGET` in lib and src
Based on existing code and commit history it appears
`CURL_DISABLE_INSTALL` means to prevent calling `install()`;
`CURL_ENABLE_EXPORT_TARGET` means to prevent calling `export()` and
`install()`s with `EXPORT` in them.
Fix them to also apply to the lib and src directories in that vain:
- lib: honor `CURL_DISABLE_INSTALL`
- src: honor `CURL_DISABLE_INSTALL`
- src: honor `CURL_ENABLE_EXPORT_TARGET`
Viktor Szakats [Wed, 19 Nov 2025 00:10:48 +0000 (01:10 +0100)]
curlx: add and use `curlx_freopen()`
To complement the existing `curlx_fopen()` internal API.
It's used by the curl's `--stderr` option.
`curlx_freopen()` adds two features to the bare `freopen()`:
- tracing for debug-enabled builds.
- Unicode and long-filename support for Windows builds.
In effect this adds long-filename and enables Unicode support for
the `--stderr <filename>` curl command-line option on Windows.
Viktor Szakats [Sat, 1 Nov 2025 02:07:36 +0000 (03:07 +0100)]
runtests: add `-m=seconds` to override test curl command timeout
To override the curl default of 5 minutes (300000 ms).
Sometimes a simple test data change can result in a stuck test, this
option makes it exit with an error early. Possible future use in CI
or fast machines to prevent a single test taking 5 minutes and failing
the whole job.
Viktor Szakats [Wed, 19 Nov 2025 03:53:55 +0000 (04:53 +0100)]
GHA: set `--buildinfo` for `test-torture` jobs
Only the `test-ci` build target sets `--buildinfo` automatically,
since 985f39c0ce78b546e832c250588c14023123edfb. It needs to be set
manually for other targets used in CI, such as `test-torture`,
to enable the `buildinfo.txt` dump in the runtests step.
For Test Clutch. In an attempt to re-sync `targetarch` with the rest of
macOS jobs on the feature matrix page:
https://testclutch.curl.se/static/reports/feature-matrix.html
Before this patch and possibly since the breaking update It's `aarch64e`
for torture jobs and `aarch64` for the rest
(stricly speaking `aarch64e` is the correct value for all macOS jobs, but
autotools and cmake report arm64/aarch64 without the `e`.)
Viktor Szakats [Wed, 19 Nov 2025 00:36:28 +0000 (01:36 +0100)]
curl_setup.h: document more funcs flagged by `_CRT_SECURE_NO_WARNINGS`
Based on these logs (non-Unicode, Unicode Schannel):
https://github.com/curl/curl/actions/runs/19446115443/job/55640968722?pr=19175
https://github.com/curl/curl/actions/runs/19446115443/job/55640968764?pr=19175
Viktor Szakats [Tue, 11 Nov 2025 19:21:44 +0000 (20:21 +0100)]
tests/data: more XML-compliance via `%LT` and `%GT` macros in email addresses
Reduce number of files failing `xmllint --format` from 133 to 57 (-76)
(3% of 1894), by replacing `<` and `>` with new macro `%LT` and `%GT`,
in most places, which is in email addresses (192 lines).
- curl_setup.h: replace `_CRT_SECURE_NO_DEPRECATE` with
`_CRT_SECURE_NO_WARNINGS`, which seems to be the preferred,
more recent macro for this. Also syncing with libssh2.
They are equivalent for curl sources with the supported compilers.
- cmake: stop setting `_CRT_SECURE_NO_DEPRECATE` globally for examples.
- examples: suppress CRT deprecation warnings on a per-file basis.
To make it work when compiling examples out of curl's build systems.
Use `_CRT_SECURE_NO_WARNINGS`.
- examples: document the functions requiring `_CRT_SECURE_NO_WARNINGS`.
- examples/block_ip: delete superfluous `_CRT_SECURE_NO_WARNINGS`.
- examples/block_ip: limit `_CRT_NONSTDC_NO_DEPRECATE` to MSVC.
- examples/log_failed_transfers: fix to set `_CRT_SECURE_NO_WARNINGS`
before headers and limit to MSVC.
- curl_setup.h: document which SDKs support `_CRT_NONSTDC_NO_DEPRECATE`.
Marcel Raad [Mon, 17 Nov 2025 08:25:22 +0000 (09:25 +0100)]
build: exclude clang prereleases from compiler warning options
Starting with clang 18, stable clang releases start with minor version 1.
Exclude pre-releases with minor version 0 from the compiler warning
options for that major version.
This fixes the build with Android NDK r29, which uses a prerelease
version of clang 21 that doesn't know the new options yet.
Viktor Szakats [Mon, 17 Nov 2025 00:16:07 +0000 (01:16 +0100)]
cf-socket: limit use of `TCP_KEEP*` to Windows 10.0.16299+ at runtime
Before this patch `TCP_KEEP*` socket options were unconditionally used
if the build-time SDK supported them. This caused curl logging errors
(or trace messages since #19527) on Windows versions missing support
for them. After this patch, use them only when the runtime environment
supports it and fall back to the alternate method (`SIO_KEEPALIVE_VALS`)
dynamically.
Also:
- log a trace message when using the Win10 method.
- document which SDK versions offer `TCP_KEEP*` macros.
Stefan Eissing [Sat, 15 Nov 2025 11:45:54 +0000 (12:45 +0100)]
pytest fixes and improvements
- fix test_17_20 flakiness: the test case did not have `nghttpx` in
its parameters, causing it to no check if a reload was necessary.
When that test ran behind one that gave nghttpx another certificate,
eg. in parallel mode, it used the wrong pinned pubkey.
- Have `env` provide lists of HTTP protocol versions available for
testing. Replace parameterized tests on a fixed protocol list with
the dynamic one from env. This makes checks for protocol availability
in the test function bodies superfluous.
Viktor Szakats [Sun, 16 Nov 2025 18:56:39 +0000 (19:56 +0100)]
runtests: detect bad libssh differently for test 1459 (fixing CircleCI libssh job)
test 1459 "SFTP with corrupted known_hosts" was seen failing in the past.
To fix it, the test was automatically disabled when detecting libssh
0.9.3 or older, as in the curl CircleCI job, running on Ubuntu 20.04.
This work for a long time, until bumping the CircleCI runner to Ubuntu
22.04 (to have OpenSSL 3), where the test was running again, and failing
with the isssue seen in the past.
- Test skipped with Ubuntu 20.04 (libssh 0.9.3):
https://app.circleci.com/pipelines/github/curl/curl/16445/workflows/7f198763-e0b0-4037-9245-4c4b40ab8726/jobs/155164
- Failure seen with Ubuntu 22.04 (libssh 0.9.6):
https://app.circleci.com/pipelines/github/curl/curl/16452/workflows/b817a808-0fd4-40b0-8eb0-d064926efe12/jobs/155206?invite=true#step-107-211709_45
- Failure seen with Ubuntu 24.04 (libssh 0.10.6):
https://app.circleci.com/pipelines/github/curl/curl/16455/workflows/86c631f1-3c5f-4438-b398-3df2bdab5d20/jobs/155218
Turns out the issue issue isn't libssh 0.9.3 itself, but
a CircleCI-specific default configuration in `/etc/ssh/ssh_config`:
```
# BEGIN ANSIBLE MANAGED BLOCK
Host *
StrictHostKeyChecking no <------ this particular line
HashKnownHosts no
SendEnv LANG LC_*
# END ANSIBLE MANAGED BLOCK
```
libssh will consult configuration files on hard-coded default system
locations and alter its behavior based on settings found in them.
This libssh behavior is present in all supported versions:
https://gitlab.com/libssh/libssh-mirror/-/commit/5a2abd34ce9ad97c69906c5fb7b07e26e96fceaa
https://gitlab.com/libssh/libssh-mirror/-/tags/libssh-0.9.0
It means the existing disable logic based on libssh version worked by
coincidence, and what needs to be checked is these configurations
to decide if it's safe to run the test. Another, simpler option is
to also accept the result code 67, though in that case the test
wouldn't actually test what we want, but would pass anyway.
With the old `oldlibssh` workaround deleted, and the problematic setting
manually overridden (`StrictHostKeyChecking yes`):
- CircleCI Ubuntu 20.04 passes with 1459 enabled:
https://app.circleci.com/pipelines/github/curl/curl/16483/workflows/87a9f389-76a2-4a32-acde-c0b411a4c842/jobs/155302
- CircleCI Ubuntu 22.04 does too:
https://app.circleci.com/pipelines/github/curl/curl/16483/workflows/87a9f389-76a2-4a32-acde-c0b411a4c842/jobs/155303
To fix, replace the `runtests` `oldlibssh` detection logic to parse
libssh config files (instead of checking for libssh version) and disable
test 1459 based on that. Notice the detection is making a light attempt
to parse these files, and does not implement most config file features
(such as includes, quoted values and `=` operator.)
The new runtests workaround tests OK with the:
- default CircleCI configuration, disabling 1459 automatically.
- a sudoless configuration fix, with 1459 run successfully.
Also keep setting this option in CircleCI jobs.
- a sudo configuration fix, with 1459 run successfully.
Ref: https://app.circleci.com/pipelines/github/curl/curl/16492/workflows/56f39335-97ba-412c-9a9b-3d662694375a
GHA jobs are not affected and they work fine, with 1459 running successfully
before and after this patch.
It's possible the libssh API offers ways to control config file use
and/or set the strict host checking option programatically. Maybe
to enable in debug mode (albeit CircleCI job are not debug-enabled),
or offer an option for them. It may be something for a future patch.
Viktor Szakats [Sun, 16 Nov 2025 00:06:24 +0000 (01:06 +0100)]
CI: avoid restart prompt on libssh-dev install in CircleCI
By setting `DEBIAN_FRONTEND=noninteractive`.
Also:
- add `curl -V` step to CircleCI jobs.
- drop duplicate `libpsl` from `apt install`.
- replace sudo pip with venv, fixing a warning and syncing with GHA.
- Note that test 1459 was disabled on Ubuntu 20.04 due to past issues.
When running on newer CircleCI Ubuntu runners (22.04 or 24.04), the
test is not disabled, and also fails with the issue seen in the past.
I've identified the root cause and will fix it in a separate PR.
Viktor Szakats [Mon, 14 Jul 2025 22:36:08 +0000 (00:36 +0200)]
build: drop Windows CE / CeGCC support
Windows CE support was limited to successful builds with ming32ce
(a toolchain that hasn't seen an update since 2009, using an ancient gcc
version and "old mingw"-style SDK headers, that curl deprecated earlier).
Builds with MSVC were broken for a long time. mingw32ce builds were never
actually tested and runtime and unlikely to work due to missing stubs.
Windows CE toolchains also miss to comply with C89. Paired with lack of
demand and support for the platform, curl deprecated it earlier.
This patch removes support from the codebase to ease maintaining Windows
codepaths.
Viktor Szakats [Fri, 14 Nov 2025 16:55:33 +0000 (17:55 +0100)]
badwords: check indented lines in source code, fix fallouts
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.