]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
2 years agodist: add missing `lib/optiontable.pl`
Viktor Szakats [Thu, 8 Aug 2024 21:23:22 +0000 (23:23 +0200)] 
dist: add missing `lib/optiontable.pl`

Closes #14467

2 years agoconfigure: fixup copy-paste mistake
Viktor Szakats [Fri, 9 Aug 2024 00:42:55 +0000 (02:42 +0200)] 
configure: fixup copy-paste mistake

Delete duplicate call to `CURL_DARWIN_CFLAGS`.

Follow-up to ada8ebe18c795cc50a1ee3c56af410f7b8094675 #14419
Closes #14468

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 9 Aug 2024 07:43:28 +0000 (09:43 +0200)] 
RELEASE-NOTES: synced

2 years agotest677: improve robustness
Stefan Eissing [Thu, 8 Aug 2024 11:29:12 +0000 (13:29 +0200)] 
test677: improve robustness

Do not treat CURLE_AGAIN as error.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14455

2 years agotest579: improve robustness
Stefan Eissing [Thu, 8 Aug 2024 11:26:10 +0000 (13:26 +0200)] 
test579: improve robustness

Log progress only at start and end of transfer to give normalized
output when upload data is only partially sent or temporarily blocked.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14454

2 years agotest556: improve robustness
Stefan Eissing [Thu, 8 Aug 2024 11:23:32 +0000 (13:23 +0200)] 
test556: improve robustness

Fix handling of partial/blocked sends.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14453

2 years agomk-ca-bundle.pl: include a link to the caextract webpage
Daniel Stenberg [Thu, 8 Aug 2024 21:00:47 +0000 (23:00 +0200)] 
mk-ca-bundle.pl: include a link to the caextract webpage

Makes it easier for users who find the file to find its origin.

Fixes https://github.com/curl/curl-www/issues/374

Closes #14465

2 years agoHISTORY: fill in some events from recent years
Daniel Stenberg [Thu, 8 Aug 2024 22:17:42 +0000 (00:17 +0200)] 
HISTORY: fill in some events from recent years

Closes #14466

2 years agoftp: flush pingpong before response
Stefan Eissing [Thu, 8 Aug 2024 11:12:53 +0000 (13:12 +0200)] 
ftp: flush pingpong before response

Fix FTP protocol to flush the pingpong's send buffer before receiving a
response from the server, as it may never come otherwise.

Fixes FTP/FTPS tests with `CURL_DBG_SOCK_WBLOCK=90` set.

Closes #14452

2 years agomanpage: ensure a maximum width for the text version
Daniel Stenberg [Tue, 6 Aug 2024 15:11:20 +0000 (17:11 +0200)] 
manpage: ensure a maximum width for the text version

... using the new script 'maxline' to which we specify the maximum
number of columns we allow any single line to be, or it will cause an
error.

Starting out with a max width at 100 columns.

Bonus: shorten the long line in the --ipfs-gateway section.

Closes #14423

2 years agocmake: more small tidy-ups and fixes
Viktor Szakats [Wed, 7 Aug 2024 22:53:38 +0000 (00:53 +0200)] 
cmake: more small tidy-ups and fixes

- tidy up two `MATCHES` expression by avoiding macros expansion and
  adding quotes. Then convert then to `STREQUAL` to match other places
  in the code doing the same checks.

- fix setting `_ALL_SOURCE` for AIX to match what autotools does.

- delete stray `_ALL_SOURCE` reference from `lib/config_riscos.h`

- simplify/fix two `STREQUAL ""` checks.
  The one in the `openssl_check_symbol_exists()` macro succeeded
  regardless of the value. The other could return TRUE when
  `CMAKE_OSX_SYSROOT` was undefined.

- delete code for CMake versions (<3.7) we no longer support.

- prefer `LIST(APPEND ...)` to extend `CURL_LIBS`.

- use `CURL_LIBS` to add the `network` lib for Haiku.
  Before this patch it was done via raw C flags. I could not test this.

- move `_WIN32_WINNT`-related code next to each other.
  It also moves detection to the top, allowing more code to use
  the result.

- merge two `WIN32` blocks.

- rename internal variables to underscore + lowercase.

- unwrap a line, indent, whitespace.

Closes #14450

2 years agokrb5: add Linux/macOS CI tests, fix cmake GSS detection
Viktor Szakats [Wed, 7 Aug 2024 18:39:34 +0000 (20:39 +0200)] 
krb5: add Linux/macOS CI tests, fix cmake GSS detection

- GHA/macos: enable GSS krb5 in a cmake job.
  Uses CMake-native detection.

- GHA/linux: enable GSS krb5 in autotools job and add a cmake job to
  match.
  CMake uses `pkg-config`-based detection.

- GHA/linux: add step to dump configure logs.

- fix and simplify logic digesting FindGSS output.

- cmake: add `heimdal-gssapi` to `libcurl.pc`.

Closes #14447

2 years agocmake: detect and show VCPKG in platform flags
Viktor Szakats [Thu, 8 Aug 2024 10:18:37 +0000 (12:18 +0200)] 
cmake: detect and show VCPKG in platform flags

Use `VCPKG_TOOLCHAIN` to detect a vcpkg build environment.

Closes #14451

2 years agoGHA/non-native: ignore FTP results in OpenBSD job
Viktor Szakats [Thu, 8 Aug 2024 11:19:54 +0000 (13:19 +0200)] 
GHA/non-native: ignore FTP results in OpenBSD job

They are flaky.

2 years agocmake: tidy up more value comparisons
Viktor Szakats [Wed, 7 Aug 2024 21:40:05 +0000 (23:40 +0200)] 
cmake: tidy up more value comparisons

- unquote numeric value.

- `NOT LESS` -> `GREATER_EQUAL`.

- replace macro with variable name.
  It also avoids this error when the variable is undefined:
  ```
  CMake Error at CMakeLists.txt:1529 (if):
  if given arguments:
    "GREATER" "4"
  Unknown arguments specified
  ```
  https://github.com/curl/curl/actions/runs/10289921657/job/28478722584#step:30:356

Follow-up to 72ae0d86a42fea83612d8baf59cff2f945aca22a #14409
Follow-up to acbc6b703f6b0ee568d053f6f2565fbc107b5fd3 #14197

Closes #14449

2 years agocmake: fix version variable references in FindGSS
Viktor Szakats [Wed, 7 Aug 2024 21:38:15 +0000 (23:38 +0200)] 
cmake: fix version variable references in FindGSS

Assign the value of the variable instead of the name of the variable
when detecting GSS version via `pkg-config` on old (?) CMake.

(On recent CMake, there is an empty value in these variables.)

Closes #14445

2 years agocmake: more syntax tidy-up
Viktor Szakats [Sun, 4 Aug 2024 18:35:13 +0000 (20:35 +0200)] 
cmake: more syntax tidy-up

- quote string literals.
  In the hope it improves syntax-highlighting and readability.

- use lowercase, underscore-prefixed local var names.
  As a hint for scope, to help readability.

- prefer `pkg_search_module` (over `pkg_check_modules`).
  They are the same, but `pkg_search_module` stops searching
  at the first hit.

- more `IN LISTS` in `foreach()`.

- OtherTests.cmake: clear `CMAKE_EXTRA_INCLUDE_FILES` after use.

- add `PROJECT_LABEL` for http/client and unit test targets.

- sync `Find*` module comments and formatting.

- drop a few local variables.

- drop bogus `CARES_LIBRARIES` from comment.

- unquote numeric literal.

Follow-up to acbc6b703f6b0ee568d053f6f2565fbc107b5fd3 #14197
Closes #14388

2 years agowolfssl: avoid taking cached x509 store ref if sslctx already using it
Alex Snast [Wed, 7 Aug 2024 14:22:22 +0000 (17:22 +0300)] 
wolfssl: avoid taking cached x509 store ref if sslctx already using it

Closes #14442

2 years agotracing: allow CURL_DEBUG override
Stefan Eissing [Wed, 7 Aug 2024 10:54:58 +0000 (12:54 +0200)] 
tracing: allow CURL_DEBUG override

On debug builds, allow environment variable CURL_DEBUG to override any
setting done via '-v' or '--no-verbose'.

Closes #14436

2 years agohttp/2: simplify eos/blocked handling
Stefan Eissing [Wed, 7 Aug 2024 10:42:27 +0000 (12:42 +0200)] 
http/2: simplify eos/blocked handling

- rely on the new flush to handle blocked sends. No longer
  do simulated EAGAIN on (partially) blocked sends with their
  need to handle repeats.
- fix some debug handling CURL_SMALLREQSEND env var
- add some assertings in request.c for affirming we do it right
- enhance assertion output in test_16 for easier analysis

Closes #14435

2 years agocurl: fix --proxy-pinnedpubkey
Jan Venekamp [Wed, 7 Aug 2024 12:09:04 +0000 (14:09 +0200)] 
curl: fix --proxy-pinnedpubkey

This option was added in #2268 but never connected in
tool_operate.c.

Closes #14438

2 years agoverbose.md: polish, mostly remove back-ticks
Daniel Stenberg [Wed, 7 Aug 2024 13:44:15 +0000 (15:44 +0200)] 
verbose.md: polish, mostly remove back-ticks

To make the page render nicer as manpage and text-only.

Closes #14441

2 years agomax-filesize.md: mention zero disables the limit
Daniel Stenberg [Wed, 7 Aug 2024 15:01:43 +0000 (17:01 +0200)] 
max-filesize.md: mention zero disables the limit

Reported-by: MasterInQuestion on github
Fixes #14440
Closes #14443

2 years agocmake: fix `pkg-config`-based detection in `FindGSS.cmake`
Viktor Szakats [Tue, 6 Aug 2024 23:19:44 +0000 (01:19 +0200)] 
cmake: fix `pkg-config`-based detection in `FindGSS.cmake`

Before this patch `pkg-config`-based detection was ignored, and used
solely as a path hint for native detection.

- fix `pkg_search_module()` result prefix to match what code expects:
  `_GSS` (was: `_GSS_PKG`). Update variable that were in sync with old
  prefix.

- update the pkg-config codepath to use `_GSS_MODULE_NAME` to detect
  GSS flavour. This requires CMake 3.16.
  Otherwise fall back to the old method. (The old method doesn't seem to
  work anymore (?) as of CMake 3.30.1. Documented
  `<prefix>_<modulename>_VERSION` variable is defined, but empty.)

- update the pkg-config codepath to use `_GSS_VERSION` set by CMake.
  Resort to the old code when this variable is empty. (The old code
  doesn't seem to work anymore (?) as of CMake 3.30.1)

- fix pkg-config codepath to set the documented result variables.

- align native detection variable names with those generated by
  `pkg_search_module()` in the pkg-config codepath.

- GHA/macos: enable GSS Heimdal in a cmake job.
  Uses the native detection.

- GHA/linux: enable GSS Heimdal in cmake and autotools jobs.
  CMake uses `pkg-config`-based detection.

- suppress test 2077 and 2078 results on Linux + Heimdal.
  ```
  FAIL-IGNORED 2077: 'curl --fail --negotiate to unauthenticated service fails' HTTP, HTTP GET, GSS-API
  FAIL-IGNORED 2078: 'curl --negotiate should not send empty POST request only' HTTP, HTTP GET, GSS-API
  ```
  Failing with valgrind errors in both autotools and cmake builds:
  https://github.com/curl/curl/actions/runs/10282222581/job/28453472068?pr=14430#step:38:3638
  https://github.com/curl/curl/actions/runs/10282222581/job/28453473398?pr=14430#step:38:7831

Closes #14430

2 years agokrb5: fix `-Wcast-align`
Viktor Szakats [Wed, 7 Aug 2024 02:27:49 +0000 (04:27 +0200)] 
krb5: fix `-Wcast-align`

```
lib/krb5.c:343:39: warning: cast from 'void **' to 'unsigned char **' increases required alignment from 2 to 8 [-Wcast-align]
                               (unsigned char **)&_gssresp.value,
                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Seen on macOS Intel with Apple clang and brew heimdal 7.8.0_1.

Closes #14433

2 years agocmake: add debug function to dump all variables
Viktor Szakats [Wed, 7 Aug 2024 12:46:17 +0000 (14:46 +0200)] 
cmake: add debug function to dump all variables

Place a `curl_dumpvars()` call anywhere in the CMake code to use it.

Closes #14439

2 years agoGHA/macos: tweak toolchain dump steps
Viktor Szakats [Wed, 7 Aug 2024 01:15:04 +0000 (03:15 +0200)] 
GHA/macos: tweak toolchain dump steps

- use documented flavour of `xcrun` option.

- show SDK version with a dedicated command.
  (Sometimes the SDK path is a symlink and doesn't tell the version.
  This is not at the moment the case in CI, but handle it anyway.)

- align group header with reality.
  Preinstalled vs. installed Homebrew packages can be recognized
  by their directory timestamps. Installed ones have a current date.

Closes #14434

2 years agoidn: more strictly check AppleIDN errors
Bo Anderson [Wed, 7 Aug 2024 02:18:34 +0000 (03:18 +0100)] 
idn: more strictly check AppleIDN errors

UIDNA API returns two error values but we were only checking one.
Checking both better aligns the behaviour with that of libidn2.

Closes #14431

2 years agoidn: support non-UTF-8 input under AppleIDN
Bo Anderson [Wed, 7 Aug 2024 02:14:57 +0000 (03:14 +0100)] 
idn: support non-UTF-8 input under AppleIDN

This aligns the behaviour with libidn2 and the curl documentation.

Closes #14431

2 years agoBINDINGS: add zig binding
Jiacai Liu [Wed, 7 Aug 2024 11:46:01 +0000 (19:46 +0800)] 
BINDINGS: add zig binding

Closes #14437

2 years agocmake: delete MSVC warning suppression for tests/server
Viktor Szakats [Tue, 6 Aug 2024 22:56:39 +0000 (00:56 +0200)] 
cmake: delete MSVC warning suppression for tests/server

Server code no longer produces this warning.

Closes #14428

2 years agodist: add missing `test_*.py` scripts
Viktor Szakats [Tue, 6 Aug 2024 21:14:53 +0000 (23:14 +0200)] 
dist: add missing `test_*.py` scripts

Closes #14427

2 years agotests: show snapshot commit in testcurl
Dan Fandrich [Tue, 6 Aug 2024 22:00:05 +0000 (15:00 -0700)] 
tests: show snapshot commit in testcurl

This disambiguates the source code being tested. The output format is
the same as when testing out of a git repo, but with no description and
a long hash.

Ref: #14363
Closes #14429

2 years agows: flags to opcodes should ignore CURLWS_CONT flag
Pete Cordell [Mon, 5 Aug 2024 14:30:15 +0000 (15:30 +0100)] 
ws: flags to opcodes should ignore CURLWS_CONT flag

When converting WebSocket flags such as CURLWS_TEXT | CURLWS_CONT we
want to exclude CURLWS_CONT from the lookup.

Closes #14397

2 years agocurl: fix --test-event --parallel
Stefan Eissing [Tue, 6 Aug 2024 09:23:04 +0000 (11:23 +0200)] 
curl: fix --test-event --parallel

(in debug-builds)

Fix implementation in curl using libuv to process parallel transfers.
Add pytest capabilities to run test cases with --test-event.

- fix uv_timer handling to carry correct 'data' pointing to uv context.
- fix uv_loop handling to reap and add transfers when possible
- fix return code when a transfer errored

Closes #14413

2 years agocurl: warn on unsupported SSL options
Jan Venekamp [Mon, 5 Aug 2024 22:09:36 +0000 (00:09 +0200)] 
curl: warn on unsupported SSL options

Closes #14406

2 years agovtls: add SSLSUPP_CIPHER_LIST
Jan Venekamp [Mon, 5 Aug 2024 21:52:33 +0000 (23:52 +0200)] 
vtls: add SSLSUPP_CIPHER_LIST

Added SSLSUPP_CIPHER_LIST so be able to differniate SSL Backends
that support CURLOPT_SSL_CIPHER_LIST.

Closes #14406

2 years agotests: ignore QUIT from FTP protocol comparisons
Daniel Stenberg [Mon, 5 Aug 2024 21:54:13 +0000 (23:54 +0200)] 
tests: ignore QUIT from FTP protocol comparisons

- QUIT is not an important FTP command
- curl only sends it "best effort", meaning it might not be sent
- it is a known "flaky" thing in test output because of this

Closes #14404

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 7 Aug 2024 06:25:43 +0000 (08:25 +0200)] 
RELEASE-NOTES: synced

2 years agocurl: support repeated use of the verbose option; -vv etc
Stefan Eissing [Tue, 6 Aug 2024 09:44:23 +0000 (11:44 +0200)] 
curl: support repeated use of the verbose option; -vv etc

- make mentioning `-v` on the curl command line increase the
  verbosity of the trace output
- related discussion https://github.com/curl/curl/discussions/13810
- make a single -v revert all previous -v+ changes
- make --no-verbose also reset all trace configs

Closes #13977

2 years agotool_help: handle longer lines, exit on too long
Daniel Stenberg [Tue, 6 Aug 2024 14:58:39 +0000 (16:58 +0200)] 
tool_help: handle longer lines, exit on too long

Follow-up to 9a0cf56471c1a

- increase the buffer to handle 160 characters manpage lines
- add another assert
- if the line buffer gets full, abort

Ideally, we add another step in the build process that makes the build
fail if this long lines are used.

Closes #14422

2 years agotests/runner: only allow [!A-Za-z0-9_-] in %if feature names
Daniel Stenberg [Tue, 6 Aug 2024 12:34:33 +0000 (14:34 +0200)] 
tests/runner: only allow [!A-Za-z0-9_-] in %if feature names

... in the %if preprocessor parser.

This guards against accidentally getting a carriage-return part of the
feature name.

Fixes #14403
Closes #14411

2 years agoruntests: if DISABLED cannot be read, error out
Daniel Stenberg [Tue, 6 Aug 2024 07:46:55 +0000 (09:46 +0200)] 
runtests: if DISABLED cannot be read, error out

Because then runtests cannot disable the right set of tests and things
will be annoying.

Closes #14411

2 years agocmake: cleanup header paths
Viktor Szakats [Tue, 6 Aug 2024 09:45:39 +0000 (11:45 +0200)] 
cmake: cleanup header paths

- sync build-dir/source-dir header path order with autotools, by
  including build-dir first, then source-dir.
  This prevents out-of-tree builds breaking due to leftover generated
  headers in the source tree.

- tests/unit: move `src` ahead of `libtest` in header path, syncing with
  autotools.

- stop adding non-existing generated `include` dir to header path.
  There are no generated `include` headers and this directory is either
  missing in out-of-tree builds or the same as the one already added
  globally via the root `CMakeLists.txt`.

- lib: stop adding a duplicate source include directory to the header
  path.
  It's already added globally via the root `CMakeLists.txt`.

- lib: stop adding the project root to the header path.

- docs/examples: drop internal header paths.
  Examples do not and should not use internal headers.

- replace `curl_setup_once.h` in comments with `curl_setup.h`,
  the header actually used, and also referred to in autotools comments.

- add comment why we need `src` in include path for `tests/server`.

- add quotes around header directories.

Closes #14416

2 years agoGHA/macos: enable AppleIDN in autotools job
Viktor Szakats [Tue, 6 Aug 2024 12:54:30 +0000 (14:54 +0200)] 
GHA/macos: enable AppleIDN in autotools job

- make `--with-apple-idn` override libidn2, in sync with cmake and
  `lib/curl_setup.h`.

- sync detection function name with cmake.

- limit AppleIDN feature check to Darwin.
  (also drop !WinIDN precondition check.)

Follow-up to 8de8fe8c98309c8b6183b22cc2e209ce4648173b #14401
Closes #14419

2 years agoMakefile.mk: fixup enabling libidn2
Viktor Szakats [Tue, 6 Aug 2024 14:24:59 +0000 (16:24 +0200)] 
Makefile.mk: fixup enabling libidn2

Replicate the method used by autotools and cmake to enable libidn2.
This way `lib/curl_setup.h` sets `USE_LIBIDN2` automatically.

Before this patch, `USE_LIBIDN2` was enabled directly, shortcutting
internal logic prioritizing IDN backends.

(This is academic now because `Makefile.mk` no longer supports
other IDN backends. But still useful for clarity.)

Closes #14421

2 years agocmake: drop unused `HAVE_IDNA_STRERROR`
Viktor Szakats [Tue, 6 Aug 2024 13:47:24 +0000 (15:47 +0200)] 
cmake: drop unused `HAVE_IDNA_STRERROR`

Unused since 9c91ec778104ae3b744b39444d544e82d5ee9ece

Closes #14420

2 years agocmake: show CMake platform/compiler flags
Viktor Szakats [Tue, 6 Aug 2024 10:49:36 +0000 (12:49 +0200)] 
cmake: show CMake platform/compiler flags

To better understand which is active in which build job.

Closes #14417

2 years agoGHA: run badwords check on tests/*.md too
Daniel Stenberg [Tue, 6 Aug 2024 08:53:25 +0000 (10:53 +0200)] 
GHA: run badwords check on tests/*.md too

With several wording updates accordingly.

Closes #14412

2 years agodist: drop buildconf
Daniel Stenberg [Tue, 6 Aug 2024 08:01:17 +0000 (10:01 +0200)] 
dist: drop buildconf

The documented and mandated step has been to not use buildconf but to
invoke 'autoreconf -fi' for four years already.

This change only drops buildconf from the release tarball, it remains
present in git for now.

Follow-up to 85868537d6d5b8

Closes #14412

2 years agocmake: add math library when using wolfssl and ngtcp2
Tal Regev [Thu, 1 Aug 2024 18:55:12 +0000 (21:55 +0300)] 
cmake: add math library when using wolfssl and ngtcp2

Closes #14343

2 years agodocs: mention "@-" in more places
Yedaya Katsman [Mon, 5 Aug 2024 20:08:25 +0000 (23:08 +0300)] 
docs: mention "@-" in more places

Closes #14402

2 years agocmake: replace an `MSVC_VERSION` with `MSVC`
Viktor Szakats [Tue, 6 Aug 2024 01:36:55 +0000 (03:36 +0200)] 
cmake: replace an `MSVC_VERSION` with `MSVC`

Where the actual version is not relevant.

Follow-up to ce81aeb877c242a173f7eecf02dd2723a0cbce42
Closes #14410

2 years agocmake: use numeric comparison for `HAVE_WIN32_WINNT`
Viktor Szakats [Mon, 5 Aug 2024 23:38:32 +0000 (01:38 +0200)] 
cmake: use numeric comparison for `HAVE_WIN32_WINNT`

Turns out CMake supports numeric comparison with hexadecimal values.
Confirmed in GHA/linux-old with CMake 3.7.2. I could not find
documentation about this, but our CMakeLists.txt already used it before
this patch.

Extend that method to two more comparisons.

Also pad the value in the existing one to 4 digits.

The padding/lowercasing logic when setting `HAVE_WIN32_WINNT` is no
longer required, but keep it anyway for uniform log output.

Follow-up to 2100d9fde267eea68f8097ff0a8ba7b3c9742c7f #12044
Closes #14409

2 years agoconfigure: detect AppleIDN
Daniel Stenberg [Mon, 5 Aug 2024 18:29:48 +0000 (20:29 +0200)] 
configure: detect AppleIDN

Follow-up to add22feeef07858307

Closes #14401

2 years agocmake: add Linux CI job, fix pytest with cmake
Viktor Szakats [Sun, 4 Aug 2024 15:18:39 +0000 (17:18 +0200)] 
cmake: add Linux CI job, fix pytest with cmake

- extend existing Linux workflow with CMake support.
  Including running pytest the first time with CMake.

- cmake: generate `tests/config` and `tests/http/config.ini`.
  Required for pytest tests.
  Uses basic detection logic. Feel free to take it from here.
  Also dump config files in a CI step for debugging purposes.

- cmake: build `tests/http/clients` programs.

- fix portability issues with `tests/http/clients` programs.
  Some of them use `getopt()`, which is not supported by MSVC.
  Fix the rest to compile in CI (old-mingw-w64, MSVC, Windows).

- GHA/linux: add CMake job matching an existing autotools one.

- GHA/linux: test `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`
  in the new CMake job.

- reorder testdeps to build server, client tests first and then
  libtests and units, to catch errors in the more complex/unique
  sources earlier.

- sort list in `tests/http/clients/Makefile.inc`.

Closes #14382

2 years agocmake: add support for `CURL_USE_LIBUV` option
Viktor Szakats [Mon, 5 Aug 2024 16:44:37 +0000 (18:44 +0200)] 
cmake: add support for `CURL_USE_LIBUV` option

Also use an `#undef` hack for CMake Unity builds to avoid the previously
included `memdebug.h` header messing up the declarations pulled in by
`uv.h`:
```
In file included from ~/curl/bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
In file included from ~/curl/src/tool_operate.c:54:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv.h:71:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv/unix.h:34:
/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/netdb.h:269:7: error: expected parameter declarator
void            freeaddrinfo(struct addrinfo *);
                ^
~/curl/lib/memdebug.h:167:31: note: expanded from macro 'freeaddrinfo'
  curl_dbg_freeaddrinfo(data, __LINE__, __FILE__)
                              ^
```

Follow-up to 38d334e3e17f16107921299a6d7b6654f6ec553b #14298
Closes #14399

2 years agoGHA/windows: bump msys2/setup-msys2 from 2.24.0 to 2.24.1
dependabot[bot] [Mon, 5 Aug 2024 14:27:53 +0000 (14:27 +0000)] 
GHA/windows: bump msys2/setup-msys2 from 2.24.0 to 2.24.1

Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.24.0 to 2.24.1.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/msys2/setup-msys2/compare/5df0ca6cbf14efcd08f8d5bd5e049a3cc8e07fd2...ddf331adaebd714795f1042345e6ca57bd66cea8)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14396

2 years agoaws_sigv4: fix canon order for headers with same prefix
Austin Moore [Sun, 4 Aug 2024 03:43:45 +0000 (23:43 -0400)] 
aws_sigv4: fix canon order for headers with same prefix

If a request containing two headers that have equivalent prefixes (ex.
"x-amz-meta-test:test" and "x-amz-meta-test-two:test2") AWS expects the
header with the shorter name to come first. The previous implementation
used `strcmp` on the full header. Using the example, this would result
in a comparison between the ':' and '-' chars and sort
"x-amz-meta-test-two" before "x-amz-meta-test", which produces a
different "StringToSign" than the one calculated by AWS.

Test 1976 verifies

Closes #14370

2 years agodocs: wolfssl and mbedtls add CURLOPT_TLS13_CIPHERS support
Jan Venekamp [Sun, 4 Aug 2024 18:30:37 +0000 (20:30 +0200)] 
docs: wolfssl and mbedtls add CURLOPT_TLS13_CIPHERS support

Documentation for #14384 and #14385

Closes #14386

2 years agowolfssl: add CURLOPT_TLS13_CIPHERS support
Jan Venekamp [Sun, 4 Aug 2024 18:07:17 +0000 (20:07 +0200)] 
wolfssl: add CURLOPT_TLS13_CIPHERS support

Bring setting ciphers with WolfSSL in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for WolfSSL when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14385

2 years agoVULN-DISCLOSURE-POLICY.md: small typo fix
Daniel Stenberg [Mon, 5 Aug 2024 15:15:31 +0000 (17:15 +0200)] 
VULN-DISCLOSURE-POLICY.md: small typo fix

2 years agocmake: fix `GSS_VERSION` for Heimdal found via pkg-config
Viktor Szakats [Mon, 5 Aug 2024 08:48:50 +0000 (10:48 +0200)] 
cmake: fix `GSS_VERSION` for Heimdal found via pkg-config

Previously used source variable was never defined, possibly due
to a copy-paste-edit typo.

Closes #14393

2 years agombedtls: add CURLOPT_TLS13_CIPHERS support
Jan Venekamp [Sun, 4 Aug 2024 18:06:27 +0000 (20:06 +0200)] 
mbedtls: add CURLOPT_TLS13_CIPHERS support

Bring setting ciphers with mbedTLS in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for mbedTLS when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14384

2 years agongtcp2: use NGHTTP3 prefix instead of NGTCP2 for errors in h3 callbacks
Moritz Buhl [Mon, 5 Aug 2024 11:38:07 +0000 (13:38 +0200)] 
ngtcp2: use NGHTTP3 prefix instead of NGTCP2 for errors in h3 callbacks

Closes #14394

2 years agotool_help: fix a NULL deref in the --help option code
Daniel Stenberg [Mon, 5 Aug 2024 06:39:08 +0000 (08:39 +0200)] 
tool_help: fix a NULL deref in the --help option code

Follow-up to 9a0cf56471c1a

Pointed out by CodeSonar

Closes #14391

2 years agoKNOWN_BUGS: "special characers" in URL works with aws-sigv4
Daniel Stenberg [Mon, 5 Aug 2024 07:39:58 +0000 (09:39 +0200)] 
KNOWN_BUGS: "special characers" in URL works with aws-sigv4

Since #13754
Pointed-out-by: Slaven Rezić
Bug: https://github.com/curl/curl/issues/13754#issuecomment-2268374427
Closes #14392

2 years agocurl: use libuv for parallel transfers with --test-event
Daniel Stenberg [Mon, 5 Aug 2024 07:12:04 +0000 (09:12 +0200)] 
curl: use libuv for parallel transfers with --test-event

add --with-libuv to configure to (optionally) use it in debug-builds to
drive the event-based API

Use curl_multi_socket_action() and friends to drive parallel transfers.

tests/README has brief documentation for this

Closes #14298

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 5 Aug 2024 07:03:11 +0000 (09:03 +0200)] 
RELEASE-NOTES: synced

2 years agohttp2: improved upload eos handling
Stefan Eissing [Sun, 4 Aug 2024 07:51:26 +0000 (09:51 +0200)] 
http2: improved upload eos handling

- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
  is not needed
- this PR is based on #14220

http2, cf-h2-proxy: fix EAGAINed out buffer
- in adjust pollset and shutdown handling, a non-empty `ctx->outbufq`
  must trigger send polling, irregardless of http/2 flow control
- in http2, fix retry handling of blocked GOAWAY frame

test case improvement:
- let client 'upload-pausing' handle http versions

Closes #14253

2 years agowolfssl: improve shutdown handling
Stefan Eissing [Sun, 4 Aug 2024 11:22:03 +0000 (13:22 +0200)] 
wolfssl: improve shutdown handling

Improve handling of shutdown when sending gets blocked.

Add workaround for <https://github.com/wolfSSL/wolfssl/issues/7784>
where wolfSSL keeps on adding close notify messages to its outgoing
buffer on ever attempt.

Closes #14376

2 years agoopenssl: improve shutdown handling
Stefan Eissing [Sun, 4 Aug 2024 11:08:51 +0000 (13:08 +0200)] 
openssl: improve shutdown handling

Make sure that `io_need` is cleared and set at the filter operations.
Add some more tracing for shutdown situations.

Improve shutdown handling for blocked sends. OpenSSL is a bit tricksy
here that it only reports WANT_WRITE on SSL_shutdown(), but never on
SSL_read() on blocked sends. So we need to use both.

At last, set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER when available since we
are not always retrying sends from the very same address, as testing
showed.

Closes #14375

2 years agobearssl: improve shutdown handling
Stefan Eissing [Sun, 4 Aug 2024 10:56:59 +0000 (12:56 +0200)] 
bearssl: improve shutdown handling

- replace own "adjust_pollset()" with vtls' standard one
- set io_need always accordingly
- handle EAGAIN on shutdown

Closes #14374

2 years agoconfigure: fail if PSL is not disabled but not found
Daniel Stenberg [Sun, 4 Aug 2024 14:38:11 +0000 (16:38 +0200)] 
configure: fail if PSL is not disabled but not found

Regression since 9b3f67e (shipped in 8.7.0)
Reported-by: Ryan Carsten Schmidt
Fixes #14373
Assisted-by: Viktor Szakats
Closes #14379

2 years agoKNOWN_BUGS: mention AppleIDN and WinIDN test problems
Daniel Stenberg [Sun, 4 Aug 2024 21:20:07 +0000 (23:20 +0200)] 
KNOWN_BUGS: mention AppleIDN and WinIDN test problems

Closes #14176
Closes #14387

2 years agotool_operhlp: fix "potentially uninitialized local variable 'pc' used"
Tal Regev [Sun, 4 Aug 2024 22:03:09 +0000 (01:03 +0300)] 
tool_operhlp: fix  "potentially uninitialized local variable 'pc' used"

A false positive by MSVC.

Closes #14389

2 years agosigpipe: init the struct so that first apply ignores
Daniel Stenberg [Sun, 4 Aug 2024 22:17:17 +0000 (00:17 +0200)] 
sigpipe: init the struct so that first apply ignores

Initializes 'no_signal' to TRUE, so that a call to sigpipe_apply() after
init ignores the signal (unless CURLOPT_NOSIGNAL) is set.

I have read the existing code multiple times now and I think it gets the
initial state reversed this missing to ignore.

Regression from 17e6f06ea37136c36d27

Reported-by: Rasmus Thomsen
Fixes #14344
Closes #14390

2 years agowolfssl: add support for ssl cert blob / ssl key blob options
Alex Snast [Tue, 25 Jun 2024 16:34:24 +0000 (09:34 -0700)] 
wolfssl: add support for ssl cert blob / ssl key blob options

wolfSSL supports setting certificates/private keys from memory blobs
which allow us to implement both CURLOPT_SSLCERT_BLOB and
CURLOPT_SSLKEY_BLOB options.

Closes #14018

2 years agocmake: add support for versioned symbols option
Viktor Szakats [Sun, 4 Aug 2024 11:52:44 +0000 (13:52 +0200)] 
cmake: add support for versioned symbols option

Implement the `--enable-versioned-symbols` feature available in
`./configure` for CMake.

Enable with `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`. Customize the version
prefix with `-DCURL_LIBCURL_VERSIONED_SYMBOLS_PREFIX=MYPREFIX_`. By
default the prefix matches what autotools uses.

When enabled, the feature is detected and a warning shown if
unavailable. (E.g. on Apple and Windows, it isn't.)

Included `HIDDEN {};` to match autotools, though I don't know if it's
necessary, useful or making any difference.

Differences from the autotools implementation:
- soversion is dynamic instead of hard-coded.
- omits referencing non-curl symbols.
- allows prefix/flavour override.
- more universal feature detection.
- doesn't rely on the in-repo `lib/libcurl.vers.in` file.

Also:
- add mbedTLS and BearSSL versioned symbol prefix support to autotools.
- enable this option in an old-linux job.

Follow-up to 7cc2e8b349df28d55d5f40bfae323485df9f0cf2
Fixes #14349
Closes #14378

2 years agoeasy: fix curl_easy_upkeep for shared connection caches
Jay Satiro [Wed, 10 Jan 2024 08:53:23 +0000 (03:53 -0500)] 
easy: fix curl_easy_upkeep for shared connection caches

- Determine which connection cache is or will be used by the easy handle
  and perform connection upkeep on that cache.

This change allows curl_easy_upkeep to be effective on easy handles that
are using a shared connection cache, either from a user created shared
connection cache or a user created multi which has its own shared
connection cache.

Prior to this change curl_easy_upkeep would upkeep the connection cache
for the easy handle only if that cache was from the multi owned by the
easy handle (ie curl_easy_perform was previously called and there's a
connection cache exclusive to the easy handle in
data->multi_easy->conn_cache).

Ref: https://curl.se/mail/lib-2024-01/0016.html

Closes https://github.com/curl/curl/pull/12677

2 years agovtls: stop offering alpn http/1.1 for http2-prior-knowledge
Jay Satiro [Tue, 23 Jul 2024 21:34:41 +0000 (17:34 -0400)] 
vtls: stop offering alpn http/1.1 for http2-prior-knowledge

- For HTTPS if http2-prior-knowledge is set then only offer h2 (HTTP/2)
  alpn to the server for protocol negotiation.

Prior to this change both HTTP/2 ("h2") and HTTP/1.1 ("http/1.1") were
offered for ALPN when http2-prior-knowledge was set.

CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE (tool: --http2-prior-knowledge) is
meant to send non-TLS HTTP requests HTTP/2 when it is known the server
supports them. However when HTTPS is used then it attempts to first
negotiate the connection with ALPN. In that case the user likely does
not want to offer http/1.1 to the server as an acceptable protocol.

Reported-by: kit-ty-kate@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/9963
Closes https://github.com/curl/curl/pull/14266

2 years agocurl: add --skip-existing
Daniel Stenberg [Sun, 4 Aug 2024 14:14:24 +0000 (16:14 +0200)] 
curl: add --skip-existing

With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.

Test 994, 995 and 996 verify.

Ref: #11012
Closes #13993

2 years agorevert "tests/http: configure test httpd to honor client cipher order"
Jan Venekamp [Sun, 4 Aug 2024 15:00:30 +0000 (17:00 +0200)] 
revert "tests/http: configure test httpd to honor client cipher order"

revert f6cb3c63 #14338

Setting SSLHonorCipherOrder to on means it honors the server cipher
order. From the documentation: "When choosing a cipher during an SSLv3
or TLSv1 handshake, normally the client's preference is used. If this
directive is enabled, the server's preference will be used instead."

Also the commit inhibits test_17_07_ssl_ciphers. The test tries to
tests if all the ciphers specified, and only those, are properly set
in curl. For that to work we need have cases where some or all ciphers
do no intersect with the cipher-set of the server. We need to be able
to assert a failed connection based on a cipher set mismatch.

That is why a restricted set of ciphers is used on the server. This
set is so chosen that it contains the well known most secure ciphers.
Except with the slower aes256 variant intentionally left out, to be
able to test above described.

As test_17_07_ssl_ciphers is currently the only test that tests the
functioning of the --ciphers and --tls13-ciphers options, it is
important that its coverage is as good as possible.

Closes #14381

2 years agoGHA/windows: add mbedTLS MSVC job
Tal Regev [Tue, 16 Jul 2024 18:43:55 +0000 (21:43 +0300)] 
GHA/windows: add mbedTLS MSVC job

Ref: 92e28f2897dc4adf0014af197e88fd8e1ccc6aeb #14228
Closes #14203

2 years agotidy-up: OS names
Viktor Szakats [Fri, 2 Aug 2024 23:09:57 +0000 (01:09 +0200)] 
tidy-up: OS names

Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360

2 years agodist: add missing `docs/examples/CMakeLists.txt`
Viktor Szakats [Sun, 4 Aug 2024 14:56:58 +0000 (16:56 +0200)] 
dist: add missing `docs/examples/CMakeLists.txt`

Follow-up to dfdd978f7c60224dffe2aac25b436dc0a5cd0186 #13491
Closes #14380

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Sun, 4 Aug 2024 14:26:21 +0000 (16:26 +0200)] 
RELEASE-NOTES: synced

2 years agomaketgz: accept option to include latest commit hash
Daniel Stenberg [Sat, 3 Aug 2024 22:03:41 +0000 (00:03 +0200)] 
maketgz: accept option to include latest commit hash

If the second argument to the script is "commit", then this will
generate a file named `docs/tarball-commit.txt` that contains the latest
commit hash (git rev-parse HEAD) at the time the script runs.

Doing this breaks the reproducibility so it will not be used for "real"
releases but is meant for automated daily snapshots and similar.

Reported-by: Dan Fandrich
Fixes #14363
Closes #14369

2 years agocurl: --help [option] displays documentation for given cmdline option
Daniel Stenberg [Sat, 3 Aug 2024 18:24:12 +0000 (20:24 +0200)] 
curl: --help [option] displays documentation for given cmdline option

Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It then first searches for
"\nALL OPTIONS".

Then, it looks for the start to display at "\n    -[option]" and stops
again at "\n    -". Except for the last option in the man page, which
ends at "\nFILES" - the subtitle for the section following all options
in the manpage.

Test 1707 to 1710 verify

Closes #13997

2 years agotool_operate: support --dump-header % to direct to stderr
Daniel Stenberg [Sun, 4 Aug 2024 14:01:41 +0000 (16:01 +0200)] 
tool_operate: support --dump-header % to direct to stderr

Similar to how --trace and --trace-ascii already do it.

Added test 1489 to verify

Closes #13992

2 years agotool_operate: for -O, use "default" as filename when the URL has none
Daniel Stenberg [Sat, 3 Aug 2024 18:08:27 +0000 (20:08 +0200)] 
tool_operate: for -O, use "default" as filename when the URL has none

... or pick the last directory part from the path if available.

Instead of returning error.

Add test 690 and 691 to verify. Test 76 and 2036 no longer apply.

Closes #13988

2 years agodoh-url.md: point out DOH server IP pinning
David Sardari [Sun, 4 Aug 2024 11:35:06 +0000 (13:35 +0200)] 
doh-url.md: point out DOH server IP pinning

Closes #14377
Signed-off-by: David Sardari <d@duxsco.de>
2 years agotests: fixup `tests/data/Makefile.am` references
Viktor Szakats [Sun, 4 Aug 2024 10:25:45 +0000 (12:25 +0200)] 
tests: fixup `tests/data/Makefile.am` references

Follow-up to f5b826532f2c564ef240df0ba2f3287d521df711 #14357
Reported-by: Stefan Eissing
Fixes #14371
Closes #14372

2 years agoGHA/non-native: ignore FreeBSD FTP test results
Viktor Szakats [Sun, 4 Aug 2024 00:08:06 +0000 (02:08 +0200)] 
GHA/non-native: ignore FreeBSD FTP test results

They are flaky.

2 years agopytests: add tests for HEAD requests in all HTTP versions
Stefan Eissing [Sat, 3 Aug 2024 10:46:28 +0000 (12:46 +0200)] 
pytests: add tests for HEAD requests in all HTTP versions

Closes #14367

2 years agocmake: tidy-ups
Viktor Szakats [Fri, 12 Jul 2024 19:10:57 +0000 (21:10 +0200)] 
cmake: tidy-ups

- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.

Closes #14197

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 3 Aug 2024 18:38:54 +0000 (20:38 +0200)] 
RELEASE-NOTES: synced

2 years agodocs/cmdline-opts: update see-also to use show-headers
Daniel Stenberg [Sat, 3 Aug 2024 18:29:30 +0000 (20:29 +0200)] 
docs/cmdline-opts: update see-also to use show-headers

Since --include is now deprecated since 82c53f821fa3fd4

2 years agogetparam: make --rate accept "number of units"
Daniel Stenberg [Sat, 20 Jul 2024 15:54:44 +0000 (17:54 +0200)] 
getparam: make --rate accept "number of units"

Do no more than 5 transfers per 15 seconds with "5/15s" or limit it to 3
transfers per 4 hours with "3/4h" etc.

Previously it would always only work with a single time unit.

Ref: #14242
Closes #14245

2 years agoGHA/windows: move Cygwin into its own workflow
Viktor Szakats [Sat, 3 Aug 2024 10:01:28 +0000 (12:01 +0200)] 
GHA/windows: move Cygwin into its own workflow

Cygwin runs stable but slow, while native Windows in the same workflow
runs fast but unreliable (hangs). GHA requires all jobs to finish before
allowing to re-run failed ones. Before this patch this meant waiting for
the slow Cygwin jobs to re-run hung native jobs. After this patch Cygwin
jobs run in their own workflow, allowing to re-run broken Windows jobs
earlier.

Follow-up to 0d125287296857fc4a931a0c20d0ffcb1f731807 #14236
Closes #14366