]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
12 months agotool_operate: split up the huge single_transfer into sub functions 15385/head
Daniel Stenberg [Wed, 23 Oct 2024 09:56:29 +0000 (11:56 +0200)] 
tool_operate: split up the huge single_transfer into sub functions

- split up in a few smaller and easier to read functions
- simplify several sections
- avoid superfluous extra allocations
- remove unused debug code

Closes #15385

12 months agosetopt: split Curl_vsetopt() into several sub functions
Daniel Stenberg [Tue, 22 Oct 2024 14:48:05 +0000 (16:48 +0200)] 
setopt: split Curl_vsetopt() into several sub functions

Reduce the ~3000 line super function into smaller pieces, easier to read and
manage.

Extract the option's argument earlier and use a fixed type instead of using
va_arg() everywhere.

Closes #15376

12 months agocmake: avoid setting `BUILD_TESTING`
Viktor Szakats [Tue, 22 Oct 2024 11:22:39 +0000 (13:22 +0200)] 
cmake: avoid setting `BUILD_TESTING`

`BUILD_TESTING` variable is used by other projects and CMake internally.
Replace `cmake_dependent_option()` with `option()` and introduce an
internal variable to track if want and can do testing.

Follow-up to #6036
Follow-up to 3a1e798009799be1e9fad30666351b66f250befb #6072

Reported-by: Robert Maynard
Fixes #15351
Closes #15355

12 months agolibssh2: delete duplicate `break`
Viktor Szakats [Wed, 23 Oct 2024 13:40:18 +0000 (15:40 +0200)] 
libssh2: delete duplicate `break`

```
lib/vssh/libssh2.c:2495:7: warning: 'break' will never be executed [-Wunreachable-code-break]
      break;
      ^~~~~
```

CI did not catch it due to llvm skipping this check for all #included
files. It's designed this way to avoid performance issues and false
positive when checking headers:
https://github.com/llvm/llvm-project/issues/71046

Closes #15384

12 months agoGHA: drop "3" from openssl names and keys
Viktor Szakats [Wed, 23 Oct 2024 12:50:26 +0000 (14:50 +0200)] 
GHA: drop "3" from openssl names and keys

Also:
- drop patch suffix from cache key for thread-sanitizer local build
  Follow-up to 73d2779196f5b4d5b45945e06b4bbdec11b6d921 #15379

Closes #15383

12 months agocmake: tidy up line order [ci skip]
Viktor Szakats [Wed, 23 Oct 2024 11:13:01 +0000 (13:13 +0200)] 
cmake: tidy up line order [ci skip]

12 months agoGHA/windows: work around Git for Windows perf regression
Viktor Szakats [Wed, 23 Oct 2024 09:06:27 +0000 (11:06 +0200)] 
GHA/windows: work around Git for Windows perf regression

Fix the significant perf regression for vcpkg jobs by switching to the
MSYS2 shell environment from Git for Windows. This env is already used
for old-mingw-w64 job that remained unaffected by this issue.

The issue began with the windows-runner update 20241015.1.0. It bumped
Git for Windows from Git 2.46.2.windows.1 to Git 2.47.0.windows.1. GfW
bumped its MSYS2 components, including `msys-2.0.dll`. That's Cygwin
code, which may have contributed to this. Pipes were involved and
`runtests.pl` relies on pipes heavily in parallel mode. (The issue was
not seen with parallel tests disabled, in retrospect.)

This is useful as a permanent solution too. It drop GfW as a dependency
and makes Windows jobs use one less shell/env flavour.

Long term it might help to use native Windows Perl to avoid the MSYS
layer completely, if there is a way to make that work.

Assortment of possibly related links:
https://cygwin.com/pipermail/cygwin/2024-August/256398.html
https://github.com/cygwin/cygwin/commit/f78009cb1ccf84cc343cf2441c76196461d87532
https://github.com/cygwin/cygwin/commit/7f3c22532577ae0a926e8eb8ad63787c9841abbf

https://github.com/actions/runner-images/issues/10843
https://github.com/git-for-windows/git/issues/5199
https://github.com/git-for-windows/msys2-runtime/pull/75
https://github.com/git-for-windows/msys2-runtime/commit/7913a41703dbc476ad3cf1b85e6939ebbe524251
https://github.com/git-for-windows/msys2-runtime/commit/555afcb2f3a6638084912ce1011bd6acef59ea79
https://github.com/cygwin/cygwin/commit/1c5f4dcdc5ec3344e3fd741c43fa359d0e1323c0

Follow-up to c33174d42fc8a4a0625b46f1d09f5e79eb2abbf1 #15364
Follow-up to 1e0305973c22b1d84036fe0c4eee34aea5cd40cc #15356

Closes #15380

12 months agoGHA/linux: drop patch from openssl3 thread sanitizer
Viktor Szakats [Wed, 23 Oct 2024 08:54:14 +0000 (10:54 +0200)] 
GHA/linux: drop patch from openssl3 thread sanitizer

The patch is now part of the 3.4.0 stable release.
(Turns out it was part of 3.3.2 already.)

Also:
- rename this local build to match the scheme used with wolfssl.
- drop '3' from local openssl build name.
- sync job name with others.
- quote step names where missing.

Follow-up to a2bcec0ee0895c23b98aea8e72ad4e9278fa67c8 #14751
Closes #15379

12 months agoCI: update dependency openssl/openssl to v3.4.0
renovate[bot] [Wed, 23 Oct 2024 08:41:58 +0000 (08:41 +0000)] 
CI: update dependency openssl/openssl to v3.4.0

Closes #15377

12 months agoruntests: use deterministic sort for `TESTINFO` lines
Viktor Szakats [Tue, 17 Sep 2024 16:43:17 +0000 (18:43 +0200)] 
runtests: use deterministic sort for `TESTINFO` lines

Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.

Closes #15374

12 months agoci: fix renovate's matching for OpenSSL and quictls
Max Dymond [Tue, 22 Oct 2024 12:31:22 +0000 (13:31 +0100)] 
ci: fix renovate's matching for OpenSSL and quictls

Renovate only matches on the raw version numbers of a package, but
OpenSSL includes `openssl-` as a prefix in the version number. This
change means that the match string now expects the `openssl-` prefix
and will just update the version portion.

This also updates quictls so that renovate can detect and update the
version correctly.

Closes #15359

12 months agoGHA: use `--no-install-suggests --no-install-recommends` where missing
Viktor Szakats [Tue, 22 Oct 2024 20:44:44 +0000 (22:44 +0200)] 
GHA: use `--no-install-suggests --no-install-recommends` where missing

It prevents `apt-get install` commands installing unnecessary packages.

Makes the 8 HTTP/3 jobs around 30 seconds faster each.

before: https://github.com/curl/curl/actions/runs/11466168597
after:  https://github.com/curl/curl/actions/runs/11469013245?pr=15373

Closes #15373

12 months agomk-lib1521: fix the long return code check
Daniel Stenberg [Tue, 22 Oct 2024 22:11:12 +0000 (00:11 +0200)] 
mk-lib1521: fix the long return code check

It worked mostly by accident since it checked the variable from the
previous setopt invoke.

Closes #15372

12 months agoGHA/linux: merge 32-bit Linux workflow
Viktor Szakats [Tue, 22 Oct 2024 19:08:12 +0000 (21:08 +0200)] 
GHA/linux: merge 32-bit Linux workflow

Also:
- tidy up a step condition.

Closes #15370

12 months agotests: Fix FILEFORMAT <file name=""> directive
Yedaya Katsman [Tue, 22 Oct 2024 19:52:21 +0000 (22:52 +0300)] 
tests: Fix FILEFORMAT <file name=""> directive

Follow-up to dcc52095

Closes #15371

12 months agoGHA/linux: merge torture jobs into the main workflow
Viktor Szakats [Tue, 22 Oct 2024 12:41:55 +0000 (14:41 +0200)] 
GHA/linux: merge torture jobs into the main workflow

They complete in 4 and 7 minutes, and do not hold back the main Linux
workflow.

Also:
- bump default parallelism for `test-torture` target to `-j20`
  (was: `-j2`).
- drop redundant package install from `rustls` jobs.

Closes #15360

12 months agoGHA/macos: use `test-torture` target for torture tests
Viktor Szakats [Tue, 22 Oct 2024 12:40:26 +0000 (14:40 +0200)] 
GHA/macos: use `test-torture` target for torture tests

They used `test-ci` before this patch.

Closes #15369

12 months agocmake/FindCares: fix version detection for c-ares 1.34.1
Viktor Szakats [Tue, 22 Oct 2024 15:02:14 +0000 (17:02 +0200)] 
cmake/FindCares: fix version detection for c-ares 1.34.1

Due to a regression in c-ares 1.34.1, the non-pkg-config version
detection method broke for this version. c-ares 1.34.2 fixes it,
but update our detection code anyway to also work with 1.34.1.

Ref:
https://github.com/c-ares/c-ares/commit/126e2741592ce3f88bce8b85e0c405a12a014253
https://github.com/c-ares/c-ares/pull/903

Closes #15368

12 months agocmake: use the `BSD` variable
Viktor Szakats [Tue, 22 Oct 2024 15:32:28 +0000 (17:32 +0200)] 
cmake: use the `BSD` variable

- use `BSD` in addition to backwards-compatible method.
- add `BSD` to the configuration log and `buildinfo.txt` if detected.
- add `BSD` tag to `buildinfo.txt` also via `./configure`.

The `BSD` variable is supported by CMake 3.25.0 and upper.

Ref: https://cmake.org/cmake/help/latest/variable/BSD.html

Closes #15367

12 months agocmake: replace `CURL_*_DIR` with `{PROJECT,CMAKE_CURRENT}_*_DIR`
Viktor Szakats [Mon, 21 Oct 2024 10:58:40 +0000 (12:58 +0200)] 
cmake: replace `CURL_*_DIR` with `{PROJECT,CMAKE_CURRENT}_*_DIR`

It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.

- replace `CURL_SOURCE_DIR`
  with `PROJECT_SOURCE_DIR`.

- replace `CURL_BINARY_DIR`
  with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.

- replace a single use of `CMAKE_BINARY_DIR`
  with `PROJECT_BINARY_DIR`.

- replace `CMAKE_CURRENT_*_DIR`
  with `PROJECT_*_DIR` where it makes the code more uniform.

- quote an argument (formatting).

Closes #15331

12 months agoGHA/windows: increase timeout for vcpkg jobs due to slowness
Viktor Szakats [Tue, 22 Oct 2024 14:25:15 +0000 (16:25 +0200)] 
GHA/windows: increase timeout for vcpkg jobs due to slowness

The openssl job no longer fits into 10 minutes since the 20241015.1.0
GHA windows-latest image update. This caused all runs to fail.

The `run tests` step takes ~10 minutes now, up from ~4. This is
6 minutes more than before these updates. It's seen with other vcpkg
jobs too, tests run slower than half speed since.

Bump the timeout to make it, though the headroom is now less than it
was.

Before:
https://github.com/curl/curl/actions/runs/11386748199/job/31679733295
https://github.com/curl/curl/actions/runs/11347976608/job/31560690219

After:
https://github.com/curl/curl/actions/runs/11462332743/job/31893491625?pr=15364

Ref: https://github.com/actions/runner-images/commit/fcc4cdb1d095af1317859c4809364538953b3497
Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953
Follow-up to 1e0305973c22b1d84036fe0c4eee34aea5cd40cc #15356
Closes #15364

12 months agoGHA: fix the msh3 renovate thing
Daniel Stenberg [Tue, 22 Oct 2024 13:36:08 +0000 (15:36 +0200)] 
GHA: fix the msh3 renovate thing

Follow-up to 943df95ae7

Closes #15363

12 months agoCI: run with standard mod_http2
Stefan Eissing [Tue, 22 Oct 2024 08:39:44 +0000 (10:39 +0200)] 
CI: run with standard mod_http2

We used to include a special mod_h2 in our CI that supports the
directive H2MaxDataFrameLen for test_02_20. Since then, ubuntu-lastest
includes a more recent apache httpd. Let's see if we can live without
the special.

Closes #15353

12 months agoGHA/windows: add http3 to libressl vcpkg job
Tal Regev [Sat, 19 Oct 2024 12:10:44 +0000 (15:10 +0300)] 
GHA/windows: add http3 to libressl vcpkg job

Also drop a libressl workaround no longer necessary.

Closes #15338

12 months agoGHA/windows: ignore results for test 987
Viktor Szakats [Tue, 22 Oct 2024 11:51:32 +0000 (13:51 +0200)] 
GHA/windows: ignore results for test 987

987 is `SMTPS with redundant explicit SSL request`.

Root cause undiscovered.

Started failing after GHA bumping the windows image to `20241015.1.0`:
https://github.com/actions/runner-images/commit/fcc4cdb1d095af1317859c4809364538953b3497
https://github.com/actions/runner-images/pull/10803

vcpkg packages also got bumped as a result. They seem unrelated:
c-ares  1.33.1 -> 1.34.1
nghttp2 1.62.1 -> 1.63.0
nghttp3 1.5.0  -> 1.6.0
(there may be more)

Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953

Closes #15356

12 months agoGHA/linux: tidy up and performance
Viktor Szakats [Mon, 21 Oct 2024 17:06:50 +0000 (19:06 +0200)] 
GHA/linux: tidy up and performance

- replace openssl3 default local build with packaged one.
- drop valgrind from IntelC job.
- drop IntelC no-ssl job.
- bump local openssl to 3.3.2.
- disable tests in the cmake variant of a job.
- add comment to the remaining local openssl3 build.
  We can drop the patch after upgrading to upcoming 3.4.0.
- drop gcc-11 from jobs. packaged gcc is now newer at 13.2.0.
  (saves more than 1m install time for each of the 5 jobs.)
  Follow-up to 9cc9a6472c5d2e13a117ca02f432443db2d1be57 #9454

Closes #15349

12 months agocmake: fix compile warnings for clang-cl
zjyhjqs [Wed, 16 Oct 2024 16:17:38 +0000 (00:17 +0800)] 
cmake: fix compile warnings for clang-cl

clang-cl is an alternative command-line interface to Clang, designed
for compatibility with the Visual C++ compiler, `cl.exe`:
https://clang.llvm.org/docs/UsersManual.html#clang-cl

The way to test clang-cl in CMake:
- `CMAKE_<LANGUAGE>_COMPILER_ID`: "Clang"
- `CMAKE_<LANGUAGE>_COMPILER_FRONTEND_VARIANT`: "MSVC"

Note: `CMAKE_<LANGUAGE>_COMPILER_FRONTEND_VARIANT` was introduced since
CMake 3.14, but the variable `MSVC` works fine here.

https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_FRONTEND_VARIANT.html
https://cmake.org/cmake/help/latest/variable/MSVC.html

Closes #15337

12 months agoversion: say quictls in MSH3 builds
Viktor Szakats [Fri, 18 Oct 2024 22:44:36 +0000 (00:44 +0200)] 
version: say quictls in MSH3 builds

Before: `curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV OpenSSL/3.3.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2 msh3/0.6.0.0`
After:  `curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV quictls/3.3.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2 msh3/0.6.0.0`

Closes #15335

12 months agochecksrc: add check for spaces around logical AND operators
Gabriel Marin [Thu, 3 Oct 2024 16:20:09 +0000 (19:20 +0300)] 
checksrc: add check for spaces around logical AND operators

Closes #15144

12 months agocurl_ws_recv.md: the 'meta' pointer is only returned on success
Daniel Stenberg [Sun, 20 Oct 2024 20:34:10 +0000 (22:34 +0200)] 
curl_ws_recv.md: the 'meta' pointer is only returned on success

Reported-by: Dylam De La Torre
Fixes #15340
Closes #15343

12 months agocurl_ws_recv: return recv 0 and point meta to NULL on all errors
Daniel Stenberg [Sun, 20 Oct 2024 20:29:08 +0000 (22:29 +0200)] 
curl_ws_recv: return recv 0 and point meta to NULL on all errors

Previously it could accidentally return some errors without having reset
the values.

Closes #15342

12 months agoGHA/linux: bump to quictls 3.3.0
Viktor Szakats [Fri, 18 Oct 2024 22:21:43 +0000 (00:21 +0200)] 
GHA/linux: bump to quictls 3.3.0

Closes #15334

12 months agocurl_multi_perform.md: fix typo
Jay Satiro [Sat, 19 Oct 2024 16:34:27 +0000 (12:34 -0400)] 
curl_multi_perform.md: fix typo

Reported-by: Dylam De La Torre
Fixes https://github.com/curl/curl/issues/15339

12 months agodocs: fix a typo in some cipher options
Dan Fandrich [Fri, 18 Oct 2024 21:42:34 +0000 (14:42 -0700)] 
docs: fix a typo in some cipher options

12 months agoGHA: update ngtcp2/ngtcp2 and awslabs/aws-lc
renovate[bot] [Thu, 17 Oct 2024 14:28:21 +0000 (14:28 +0000)] 
GHA: update ngtcp2/ngtcp2 and awslabs/aws-lc

- update ngtcp2/ngtcp2 to v1.8.1
- update awslabs/aws-lc to v1.37.0

Closes #15318
Closes #15329

12 months agoDockerfile: update Docker digest to d830561
renovate[bot] [Thu, 17 Oct 2024 06:35:12 +0000 (06:35 +0000)] 
Dockerfile: update Docker digest to d830561

Closes #15315

12 months agowinbuild: add initial wolfSSL support
Viktor Szakats [Fri, 11 Oct 2024 09:13:40 +0000 (11:13 +0200)] 
winbuild: add initial wolfSSL support

Ref: https://datagirl.xyz/posts/wolfssl_curl_w2k.html

Closes #15264

12 months agoKNOWN_BUGS: LDFLAGS passed too late
Daniel Stenberg [Wed, 16 Oct 2024 07:28:46 +0000 (09:28 +0200)] 
KNOWN_BUGS: LDFLAGS passed too late

Makes linking fail on some (ancient) platforms.

Closes #14893
Closes #15306

12 months agohsts: support "implied LWS" properly around max-age
Daniel Stenberg [Fri, 18 Oct 2024 07:38:13 +0000 (09:38 +0200)] 
hsts: support "implied LWS" properly around max-age

Adjust test 780 to verify.

Reported-by: newfunction
Closes #15330

12 months agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 18 Oct 2024 08:13:38 +0000 (10:13 +0200)] 
RELEASE-NOTES: synced

12 months agocmake: set version for `project()` and add CPack support
zjyhjqs [Sun, 13 Oct 2024 08:09:48 +0000 (16:09 +0800)] 
cmake: set version for `project()` and add CPack support

Note: the version like `8.11.0-DEV` is not a valid version for
`project()`, so need to extract the major, minor and patch parts.

Previous, manual, `CURL_VERSION` macro is defined by `project()`
after this patch, so rename existing `CURL_VERSION*` variables to
`_curl_version*`.

Closes #15281

12 months agotool_operate: reuse the schannel backend check
Daniel Stenberg [Thu, 17 Oct 2024 14:01:08 +0000 (16:01 +0200)] 
tool_operate: reuse the schannel backend check

The transfer_per_config is called once per new transfer. It now saves
the result of the first TLS backend check done so that subsequent
invokes are more efficient and reuses the existing knowledge.

This change also splits the logic into several smaller functions.

Closes #15323

12 months agolibcurl/opts: improve phrasing for connection cap related options
Daniel Stenberg [Wed, 16 Oct 2024 21:37:05 +0000 (23:37 +0200)] 
libcurl/opts: improve phrasing for connection cap related options

Unify, clarify.

Closes #15324

12 months agohttp2: auto reset stream on server eos
Stefan Eissing [Thu, 17 Oct 2024 15:00:41 +0000 (17:00 +0200)] 
http2: auto reset stream on server eos

When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.

Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.

Reported-by: jkamp-aws on github
Fixes #15316
Closes #15325

12 months agolibtests: generate the lib1521 atomically
Daniel Stenberg [Thu, 17 Oct 2024 15:50:02 +0000 (17:50 +0200)] 
libtests: generate the lib1521 atomically

By renaming from a temporary file name to the .c once completed. This
avoids the risk that the checksrc job tries to verify the file before it
is complete, in parallel build setups.

Reported-by: Dan Frandrich
Fixes #15258
Closes #15327

12 months agoGHA: drop the hyper job
Daniel Stenberg [Thu, 17 Oct 2024 15:11:53 +0000 (17:11 +0200)] 
GHA: drop the hyper job

Hyper support is being removed in 2025. No one works on it. Getting
flaky test runs with this job adds nothing to the project.

Closes #15326

12 months agoopenssl: improve retries on shutdown
Stefan Eissing [Thu, 17 Oct 2024 11:53:06 +0000 (13:53 +0200)] 
openssl: improve retries on shutdown

Once SSL_shutdown() has been called, OpenSSL does not really seem to
like it when it is called again and the other side has some finally data
to deliver.

Instead SSL_read() needs to be used solely, once the close notify has
been sent from curl's side.

Closes #15321

12 months agotool_operate: break out of loop on error
Daniel Stenberg [Thu, 17 Oct 2024 13:32:48 +0000 (15:32 +0200)] 
tool_operate: break out of loop on error

Follow-up to 69bf530dfd2a

The loop could get stuck there in torture tests/OOM.

Closes #15322

12 months agoGHA: switch off proselint
Daniel Stenberg [Thu, 17 Oct 2024 06:33:04 +0000 (08:33 +0200)] 
GHA: switch off proselint

Because we cannot disable the individual warnings we do not care about,
making this tool almost unusable for our purposes. See
https://github.com/amperser/proselint/issues/1367

Instead, make 'very' a banned word (as recently that has been what
proselint most commonly points out for us).

Closes #15314

12 months agosource: avoid use of 'very' in comments
Daniel Stenberg [Thu, 17 Oct 2024 07:51:26 +0000 (09:51 +0200)] 
source: avoid use of 'very' in comments

12 months agoDISTROS: avoid use of "very"
Daniel Stenberg [Thu, 17 Oct 2024 07:42:00 +0000 (09:42 +0200)] 
DISTROS: avoid use of "very"

12 months agoDISABLED: disable test 1060 with hyper
Daniel Stenberg [Thu, 17 Oct 2024 10:57:33 +0000 (12:57 +0200)] 
DISABLED: disable test 1060 with hyper

... as it has started to fail and nobody wants to debug this.

Closes #15319

12 months agotests/http: fix ubuntu GnuTLS CI failures
Stefan Eissing [Wed, 16 Oct 2024 14:21:03 +0000 (16:21 +0200)] 
tests/http: fix ubuntu GnuTLS CI failures

Override the system default config in test_17_09, since we want to check
all TLS versions. Provide own, empty config file to gnutls, so that any
system wide file has no effect.

The latest ubunu image in GH CI disables TLS 1.0 and 1.1
system wide for GnuTLS. Good intentions.

Closes #15310

12 months agotests: update some HTTP/2 over HTTPS tests
Daniel Stenberg [Thu, 17 Oct 2024 09:58:28 +0000 (11:58 +0200)] 
tests: update some HTTP/2 over HTTPS tests

- improve descriptions
- require http/2, not h2c, since they are done over HTTPS

Closes #15317

12 months agowinbuild/README: document how to clean a build
Jay Satiro [Mon, 14 Oct 2024 08:41:33 +0000 (04:41 -0400)] 
winbuild/README: document how to clean a build

- Add a new section explaining that a build can be cleaned by adding the
  keyword "clean" to the build command.

- Add an example of using the "x64 Native Tools" prompt to the VS
  command prompt section.

- Update the Legacy Windows section's lack-of-cipher support warning to
  say "Windows 8 and earlier" instead of "Windows XP and earlier".

Ref: https://github.com/curl/curl/discussions/15277

Closes https://github.com/curl/curl/pull/15291

12 months agoGHA/macos: merge autotools and cmake jobs
Viktor Szakats [Tue, 15 Oct 2024 00:58:44 +0000 (02:58 +0200)] 
GHA/macos: merge autotools and cmake jobs

To match other workflows and to avoid repetition in rules.

Also:
- fix build example step for cmake. update a job to use it.
- use `cmake` to invoke the builds (instead of ninja directly).
- extend test 2100 exclusion to more jobs.
  It fails with all `!debug gcc-12` jobs with autotools.
  With cmake this only happened for gcc-12 Secure Transport jobs
  for some reason.

Closes #15312

12 months agoCI: explicitly specify the OS version when necessary
Dan Fandrich [Wed, 16 Oct 2024 16:52:36 +0000 (09:52 -0700)] 
CI: explicitly specify the OS version when necessary

Commit 8ea120f6 added --break-system-packages which works in Ubuntu
24.04 but not 22.04, so explicitly specify that version in the runner
instead of relying on ubuntu-latest to provide it.  Some runners have
regressed back to 22.04 for ubuntu-latest, resulting in build failures.

12 months agotests: capture stdin to get the vsftpd version number
Dan Fandrich [Sat, 12 Oct 2024 17:38:40 +0000 (10:38 -0700)] 
tests: capture stdin to get the vsftpd version number

vsftpd 3.0 at least writes its version number to stdin (!) instead of
stderr. This works due for backwards compatibility reasons in UNIX, so
we must check stdin for anything written there to reliably parse the
version string.

Closes #15278

12 months agosrc: guard for double declaration of `curl_ca_embed` in unity builds
Viktor Szakats [Wed, 16 Oct 2024 10:14:52 +0000 (12:14 +0200)] 
src: guard for double declaration of `curl_ca_embed` in unity builds

Seen with curl-for-win linux-musl-from-mac build with gcc 9.2.0.

```
n file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:136:
/Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/tool_ca_embed.c:4:28: warning: redundant redeclaration of 'curl_ca_embed' [-Wredundant-decls]
    4 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
In file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
/Users/runner/work/curl-for-win/curl-for-win/curl/src/tool_operate.c:107:28: note: previous declaration of 'curl_ca_embed' was here
  107 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
```
https://github.com/curl/curl-for-win/actions/runs/11192203640/job/31116070669#step:3:4894

Follow-up to 8a3740bc8e558b9a9d4a652b74cf27a0961d7010 #14059
Closes #15307

12 months agolibssh: use CURL_PATH_MAX instead of PATH_MAX
Daniel Stenberg [Wed, 16 Oct 2024 14:07:04 +0000 (16:07 +0200)] 
libssh: use CURL_PATH_MAX instead of PATH_MAX

Follow-up to facf59c30e9a6a10c4

Reported-by: Viktor Szakats
Bug: https://github.com/curl/curl/pull/15285#issuecomment-2416947731
Closes #15309

12 months agovquic: fix compiler warning with gcc + MUSL
Viktor Szakats [Wed, 16 Oct 2024 00:27:24 +0000 (02:27 +0200)] 
vquic: fix compiler warning with gcc + MUSL

```
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c: In function 'msghdr_get_udp_gro':
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:344: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  344 | #pragma clang diagnostic push
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:345: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  345 | #pragma clang diagnostic ignored "-Wsign-compare"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:346: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  346 | #pragma clang diagnostic ignored "-Wcast-align"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:350: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  350 | #pragma clang diagnostic pop
      |
```
https://github.com/curl/curl-for-win/actions/runs/11356281008/job/31587180874#step:3:9534

Follow-up to a571afc02e11c1ab9a9f59c2150e11acca423fcc #14012
Closes #15303

12 months agolibssh2: use the filename buffer when getting the homedir
Daniel Stenberg [Tue, 15 Oct 2024 22:17:24 +0000 (00:17 +0200)] 
libssh2: use the filename buffer when getting the homedir

Avoids having to use a big stack buffer for this.

Closes #15285

12 months agolibssh2: put the readdir buffers into struct
Daniel Stenberg [Sun, 13 Oct 2024 21:50:11 +0000 (23:50 +0200)] 
libssh2: put the readdir buffers into struct

... instead of separate malloc() calls:

- removes two mallocs (and associated error handling paths)
- makes cleanup easier

Also reduce maximum SFTP file path lengths to 1024 bytes universally
everywhere. Using the system's own MAX_PATH did not make sense since
this is mostly about getting a remote file name.

Closes #15285

12 months agoCI: update GHA dependencies
renovate[bot] [Tue, 15 Oct 2024 14:16:14 +0000 (14:16 +0000)] 
CI: update GHA dependencies

- update Mbed-TLS/mbedtls to v3.6.2
- update libressl-portable/portable to v4
- update github/codeql-action digest to f779452

Closes #15301
Closes #15299
Closes #15188

12 months agoGHA: silence proselint warnings and an error
Viktor Szakats [Mon, 14 Oct 2024 09:27:54 +0000 (11:27 +0200)] 
GHA: silence proselint warnings and an error

Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293

12 months agotests: fix callback signatures to please UndefinedBehaviorSanitizer 15289/head
Daniel Stenberg [Mon, 14 Oct 2024 11:01:19 +0000 (13:01 +0200)] 
tests: fix callback signatures to please UndefinedBehaviorSanitizer

Make test applications use the correct prototypes for callbacks.

Closes #15289

12 months agocurl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs
Daniel Stenberg [Mon, 14 Oct 2024 12:09:59 +0000 (14:09 +0200)] 
curl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs

It makes the callbacks get different signnatures when used from within
libcurl vs outside of it by libcurl-using applications (such as the
libtests) and this triggers UndefinedBehaviorSanitizer errors.

Closes #15289

12 months agolib: remove function pointer typecasts for hmac/sha256/md5
Daniel Stenberg [Mon, 14 Oct 2024 07:47:03 +0000 (09:47 +0200)] 
lib: remove function pointer typecasts for hmac/sha256/md5

Make sure we use functions with the correct prototype.

Closes #15289

12 months agoconncache: More efficient implementation of cpool_remove_bundle
Michael Kaufmann [Mon, 14 Oct 2024 08:45:12 +0000 (10:45 +0200)] 
conncache: More efficient implementation of cpool_remove_bundle

Closes #15292

12 months agoGHA/linux: add cmake job for system mbedTLS with pkg-config
Viktor Szakats [Mon, 14 Oct 2024 00:54:03 +0000 (02:54 +0200)] 
GHA/linux: add cmake job for system mbedTLS with pkg-config

Add a build-only cmake job with system mbedTLS package and `pkg-config`
enabled. Ubuntu 24.04 comes with mbedTLS 2.28.8 which supports
`pkg-config`.

Follow-up to 7bab201abe3915a0167c002f9308950cb8a06e4b #15193
Closes #15286

12 months agoserver/mqttd: fix two memory leaks
Daniel Stenberg [Mon, 14 Oct 2024 08:21:10 +0000 (10:21 +0200)] 
server/mqttd: fix two memory leaks

Closes #15290

12 months agoGHA/linux: fixup pip for Ubuntu 24.04
Viktor Szakats [Mon, 14 Oct 2024 01:01:41 +0000 (03:01 +0200)] 
GHA/linux: fixup pip for Ubuntu 24.04

`ubuntu-latest` became `ubuntu-24.04` today.

Closes #15287

12 months agotool_operate: make --skip-existing work for --parallel
Daniel Stenberg [Sun, 13 Oct 2024 15:33:38 +0000 (17:33 +0200)] 
tool_operate: make --skip-existing work for --parallel

Reported-by: Tobias Wendorff
Fixes #15261
Closes #15283

12 months agourl: use same credentials on redirect
Daniel Stenberg [Sat, 12 Oct 2024 21:54:39 +0000 (23:54 +0200)] 
url: use same credentials on redirect

Previously it could lose the username and only use the password.

Added test 998 and 999 to verify.

Reported-by: Tobias Bora
Fixes #15262
Closes #15282

12 months agolib: move curl_path.[ch] into vssh/
Daniel Stenberg [Sun, 13 Oct 2024 15:56:23 +0000 (17:56 +0200)] 
lib: move curl_path.[ch] into vssh/

As this contains code only used by SSH backends.

Closes #15284

12 months agoftp: move listen handling to socket filter
Stefan Eissing [Thu, 5 Sep 2024 14:41:53 +0000 (16:41 +0200)] 
ftp: move listen handling to socket filter

Move the listen/accept handling of the FTP active data connection
into the socket filter and monitor 'connected' status of that as
with passive connections - more or less.

The advantage is that the socket filter now reports being connected
only when the server has actually called and accept() has been done.
This enables to bootstrap the filter chain on the data connection
just like any other. A require SSL filter can then be added right
at the start and does not need to be patched in later.

Still, the active connection keeps on needing special handling in
ftp.c as the control connection needs to be monitored while waiting
as the server might send error responses this way. So, things did
not turn out quite as squeaky clean as hoped for, but still seems
better to do that way.

Closes #14798

12 months agombedTLS: fix handling of TLSv1.3 sessions
Stefan Eissing [Thu, 10 Oct 2024 10:47:41 +0000 (12:47 +0200)] 
mbedTLS: fix handling of TLSv1.3 sessions

For TLSv1.3, if supported, observer special return code to retrieve
newly arrived session from mbedTLS.

Adjust test expectations now that TLSv1.3 session resumption works in
mbedTLS >= 3.6.0.

Based on #14135 by @ad-chaos
Closes #15245

12 months agowolfSSL: fix handling of TLSv1.3 sessions
Stefan Eissing [Thu, 10 Oct 2024 09:44:39 +0000 (11:44 +0200)] 
wolfSSL: fix handling of TLSv1.3 sessions

Register a callback to get notified of new SSL sessions by wolfSSL.
Remove the explicit session retrieval after handshake, since this does
not work for TLSv1.3.

Adjust test expectations now that TLSv1.3 session resumption works
in wolfSSL.

Closes #15243

12 months agocurl-rustls.m4: set linker flags to allow rustls build on macos
Stefan Eissing [Mon, 7 Oct 2024 13:58:26 +0000 (15:58 +0200)] 
curl-rustls.m4: set linker flags to allow rustls build on macos

Assisted-by: Viktor Szakats
Closes #15175

12 months agosmb: do not redefine `getpid` on Windows
Viktor Szakats [Fri, 11 Oct 2024 08:53:32 +0000 (10:53 +0200)] 
smb: do not redefine `getpid` on Windows

Replace with namespaced local macro `Curl_getpid()`.

Redefining symbols can backfire if that symbol is used in system
headers, especially with unity build. We haven't seen a fallout in CI
or supported envs, but do it anyway for good measure.

Bug report: https://datagirl.xyz/posts/wolfssl_curl_w2k.html
Closes #15263

12 months agoGHA: optimize test prereq steps
Viktor Szakats [Sat, 12 Oct 2024 08:45:31 +0000 (10:45 +0200)] 
GHA: optimize test prereq steps

- Linux: move test and pytest prereqs right before test run.
  - returns build phase results faster.
  - allows skipping steps for jobs that don't need them.
  - makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.

Closes #15275

12 months agopytest: include `buildinfo.txt` in the output
Viktor Szakats [Sat, 12 Oct 2024 21:56:38 +0000 (23:56 +0200)] 
pytest: include `buildinfo.txt` in the output

If present.

It aims to provide TextClutch the same build information that
`runtests.pl` already is providing.

Ref: https://testclutch.curl.se/static/reports/feature-matrix.html
Ref: #15256
Follow-up to 1fdea1684602a1ae2870c67b5f3e8fd34f63da95 #14802
Closes #15279

12 months agoGHA/windows: drop vcpkg workaround
Tal Regev [Sat, 5 Oct 2024 02:22:05 +0000 (05:22 +0300)] 
GHA/windows: drop vcpkg workaround

Reverts 841f42150ae7b9f903cc5d266cec60a8090c0f27 #15133
Closes #15217

12 months agocmake: tidy-ups and rebase fixups
Viktor Szakats [Fri, 11 Oct 2024 16:36:43 +0000 (18:36 +0200)] 
cmake: tidy-ups and rebase fixups

- limit `SIZEOF_SA_FAMILY_T` detection to non-Windows.
- make sure `sys/socket.h` exists before detecting `SIZEOF_SA_FAMILY_T`.
- limit `mach_absolute_time()` detection to `APPLE`. Drop from Windows
  pre-cache.
- skip `HAVE_LIBSOCKET` detection for Windows, drop pre-cached value.
- drop redundant pre-cached `HAVE_LIBZ` for Windows.
- `curl_required_libpaths()`: stop accepting multiple arguments.
  To prepare for `CMAKE_REQUIRED_LINK_DIRECTORIES` support.
  Follow-up to 7bab201abe3915a0167c002f9308950cb8a06e4b #15193
- GSS: fix recent rebase mistakes:
  - fix variable name.
  - do not add a header twice.
  Follow-up to 91d451b48809f20415ba8627786f5d4f5aaf8bfe #15157
- GSS: quote a variable.

Closes #15271

12 months agotests: allow pytests to run in out-of-tree builds
Dan Fandrich [Thu, 10 Oct 2024 17:44:56 +0000 (10:44 -0700)] 
tests: allow pytests to run in out-of-tree builds

Some of the files it needs are in the build directory but the code did
not make a distinction.

Closes #15257

12 months agoGHA/linux: mbedTLS 3.6.1
Viktor Szakats [Sat, 12 Oct 2024 06:27:34 +0000 (08:27 +0200)] 
GHA/linux: mbedTLS 3.6.1

Closes #15274

12 months agoCI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact
renovate[bot] [Tue, 8 Oct 2024 20:20:34 +0000 (20:20 +0000)] 
CI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact

- update rojopolis/spellcheck-github-actions digest to 7ff888c
- update actions/checkout digest to eef6144
- update actions/upload-artifact digest to b4b15b8

Closes #15207
Closes #15184
Closes #15187

12 months agoCI: bump github/codeql-action, vmactions/omnios-vm and actions/cache
dependabot[bot] [Mon, 7 Oct 2024 14:54:53 +0000 (14:54 +0000)] 
CI: bump github/codeql-action, vmactions/omnios-vm and actions/cache

- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1

Closes #15178
Closes #15179
Closes #15244

12 months agognutls: use session cache for QUIC
Stefan Eissing [Fri, 11 Oct 2024 11:09:51 +0000 (13:09 +0200)] 
gnutls: use session cache for QUIC

Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.

Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.

Closes #15265

12 months agotool_xattr: create the user.creator xattr attribute
Dan Fandrich [Wed, 9 Oct 2024 23:47:20 +0000 (16:47 -0700)] 
tool_xattr: create the user.creator xattr attribute

This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.

Closes #15237

12 months agocmake: apply `WIN32_LEAN_AND_MEAN` to all feature checks
Viktor Szakats [Thu, 10 Oct 2024 22:38:43 +0000 (00:38 +0200)] 
cmake: apply `WIN32_LEAN_AND_MEAN` to all feature checks

Enable `WIN32_LEAN_AND_MEAN` for all feature detections on Windows.

(Also drop it from individual detections.)

Cherry-picked from #15164

12 months agocmake: untangle feature detection interdependencies
Viktor Szakats [Sat, 5 Oct 2024 23:01:22 +0000 (01:01 +0200)] 
cmake: untangle feature detection interdependencies

- reduce `check_include_file_concat()` use to those headers that either
  depend on a previously detected header, or another header or symbol
  detection depend on it.

- replace `check_symbol_exists()` with `check_function_exists()` for
  functions that are detected with `AC_CHECK_FUNCS()` in `./configure`.
  This makes `setmode()` no longer be detected with MSYS, syncing
  this with `./configure`. Instead `_setmode()` is used now also in
  CMake MSYS builds. This is consistent with Cygwin builds also.

- add comment about which header/symbol detection depends on what
  header. Based on `./configure` mainly.

- form `CURL_TEST_DEFINES` manually, and include only those macros which
  are actually used in `CMake/CurlTests.c`.

- change `curl_internal_test()` to use `CMAKE_REQUIRED_DEFINITIONS`,
  instead of `CMAKE_REQUIRED_FLAGS` to simplify the logic, and to allow
  dropping the latter macro completely.

- drop `windows.h` from header and symbol checks.

- `./configure`: add comment about whether `netinet/in6.h`, `sys/un.h`
  are indeed meant to be included for all detections. There is a chance
  they were added there by accident.

Detection resuls were cross-checked between
436bbbe7abebf0ee3a2b0bfb3ec5db7ce8c8db4c (master) and
48ff4694e608ccfdedf7ce5bab2b96d6b2c23cda (this PR), for CI GHA Linux,
Linux HTTP/3, non-native, macOS and Windows jobs.

Closes #15164

12 months agoci: dump `curl_config.h` to log in all jobs
Viktor Szakats [Fri, 11 Oct 2024 11:40:48 +0000 (13:40 +0200)] 
ci: dump `curl_config.h` to log in all jobs

Also:
- GHA/windows: merge full and brief dump into a single job step.
- fix shellcheck warning 'useless cat'.

Closes #15266

12 months agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 11 Oct 2024 12:52:41 +0000 (14:52 +0200)] 
RELEASE-NOTES: synced

12 months agolibssh2: split the statemachine function into smaller sub functions
Daniel Stenberg [Thu, 10 Oct 2024 12:22:09 +0000 (14:22 +0200)] 
libssh2: split the statemachine function into smaller sub functions

A giant function is hard to read.

Closes #15250

12 months agonetrc: cache the netrc file in memory
Daniel Stenberg [Thu, 10 Oct 2024 16:08:07 +0000 (18:08 +0200)] 
netrc: cache the netrc file in memory

So that on redirects etc it does not reread the file but just parses it
again.

Reported-by: Pierre-Etienne Meunier
Fixes #15248
Closes #15259

12 months agoTLS: TLSv1.3 earlydata support for curl
Stefan Eissing [Wed, 9 Oct 2024 12:46:32 +0000 (14:46 +0200)] 
TLS: TLSv1.3 earlydata support for curl

Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.

- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
  bytes sent and accepted/rejected by the server.

Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
  that ALPN protocol for negoptiation with the server. When the
  sessions ALPN does not match the connections ALPN, earlydata
  will not be enabled.
- Check that the server selected the correct ALPN protocol for
  an earlydata connect. If the server does not confirm or reports
  something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
  if not connect-only.

Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature

Assisted-by: ad-chaos on github
Closes #15211

12 months agomulti: convert Curl_follow to static multi_follow
Daniel Stenberg [Fri, 11 Oct 2024 07:20:37 +0000 (09:20 +0200)] 
multi: convert Curl_follow to static multi_follow

Moved over from transfer.c because it is only used in multi.c

Closes #15260

12 months agocookie: overhaul and cleanup
Daniel Stenberg [Thu, 10 Oct 2024 08:08:15 +0000 (10:08 +0200)] 
cookie: overhaul and cleanup

- split the huge Curl_cookie_add() into several smaller static functions

- switch to using the common llist instead of custom linked list

- use less memory for *getlist()

- use bitfields for flags in the Cookie struct

- avoid the copy for date parsing

- more consistent variable naming

Closes #15247

12 months agocmake: replace `check_include_file_concat()` for LDAP and GSS detection
Viktor Szakats [Sat, 5 Oct 2024 00:12:13 +0000 (02:12 +0200)] 
cmake: replace `check_include_file_concat()` for LDAP and GSS detection

Replace `check_include_file_concat()` with `check_include_file()` in
GSS/LDAP detection to avoid these headers spilling into subsequent
feature checks.

- For LDAP, reverse detection order to match with `./configure`.
  Though, in current LDAP packages `ldap.h` does include `lber.h`.

- For GSS, align header detection logic with `./configure`, where
  `gssapi/gssapi_generic.h` might require `gssapi/gssapi.h`, and
  `gssapi/gssapi_krb5.h` might require both.

Ref: #436
Closes #15157