]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
15 months agotool_getparam: make --show-headers the same as --include
Daniel Stenberg [Wed, 10 Jul 2024 14:30:17 +0000 (16:30 +0200)] 
tool_getparam: make --show-headers the same as --include

Simply a name alias that better explains what the option does.

Closes #13987

15 months agocfilters: send flush
Stefan Eissing [Thu, 25 Jul 2024 11:10:01 +0000 (13:10 +0200)] 
cfilters: send flush

Since data can be held in connection filter buffers when sending gives
EAGAIN, add methods to query this and perform flushing of those buffers.

The transfer loop will continue sending until all upload data is
processed and the connection is flushed.

- add `CF_QUERY_SEND_PENDING` to query filters
- add `CF_CTRL_DATA_SEND_FLUSH` to flush filters
- change `Curl_req_want_send()` to query the connection
  if it needs flushing
- use `Curl_req_want_send()` to determine the POLLOUT
  in the PERFORMING multi state
- implement flush handling in the HTTP/2 connection filter

Closes #14271

15 months agolib: add eos flag to send methods
Stefan Eissing [Thu, 18 Jul 2024 09:29:37 +0000 (11:29 +0200)] 
lib: add eos flag to send methods

Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.

Closes #14220

15 months agovtls: init ssl peer only once
Stefan Eissing [Thu, 11 Jul 2024 10:44:15 +0000 (12:44 +0200)] 
vtls: init ssl peer only once

- check that `struct ssl_peer` is only initialized once
- fix vtls peer init to run only once
- check in peer init that hostname is not empty, fail otherwise

Closes #14152

15 months agourl: dns_entry related improvements
Stefan Eissing [Fri, 12 Jul 2024 10:46:50 +0000 (12:46 +0200)] 
url: dns_entry related improvements

Replace Curl_resolv_unlock() with Curl_resolv_unlink():

-replace inuse member with refcount in Curl_dns_entry

- pass Curl_dns_entry ** to unlink, so it gets always cleared

- solve potential (but unlikley) UAF in FTP's handling of looked up
  Curl_dns_entry. Esp. do not use addr information after unlinking an entry.
  In reality, the unlink will not free memory, as the dns entry is still
  referenced by the hostcache. But this is not safe and relying on no other
  code pruning the cache in the meantime.

- pass permanent flag when adding a dns entry instead of fixing timestamp
  afterwards.

url.c: fold several static *resolve_* functions into one.

Closes #14195

15 months agoCurl_rand_bytes to control env override
Stefan Eissing [Tue, 23 Jul 2024 10:21:51 +0000 (12:21 +0200)] 
Curl_rand_bytes to control env override

- in DEBUGBUILD, all specifying if true random numbers
  are desired or simulated ones via CURL_ENTROPY
- allows to use randoms in other DEBUG checks to not
  interfere with the CURL_ENTROPY
- without this change, any Curl_rand() use will alter
  results of some AUTHENTICATION methods like DIGEST

Closes #14264

15 months agoCI: enable parallel testing in CI builds
Dan Fandrich [Thu, 22 Jun 2023 17:03:16 +0000 (10:03 -0700)] 
CI: enable parallel testing in CI builds

The test-ci target now uses 2 processes by default, but the amount of
parallelism is tuned for each CI service and build environment based on
results of a number of test runs.  Some CI services use super-
oversubscribed build machines that can barely run the curl tests
already with no parallelism without frequently failing with
timing-induced failures. These continue to be run without parallelism.
Other services provide two fast, unloaded cores and these run with 14
processes, which is a good default for this kind of environment.

Here's a summary of the number of test processes by CI service:

  Appveyor - 2 (Windows MSVC), 1 (others)
  Azure - 2
  Circle CI - 14
  Cirrus - 28 (macOS), 14 (Linux), 7 (FreeBSD), 5 (macOS torture), 2 (Windows)
  GitHub Actions - 3 (macOS), 2 (Linux)

Some of these are a bit conservative to keep timing-induced flakiness down.

The net result is that the first test results should arrive only
3 minutes after a commit submission.

Changes merged via separate commits:
2a7c8b27fdd266894fe24d6d0f5d853ea843dff4 #14171
72341068a2d8f1b94a26add16830c725cc4054e6
efce544418971b064d9fc8183a6caa582c54559b #14244
c6cf411bacf97f230db160e543d0cc3bbe5b9aba

Ref: #10818
Closes #11510

15 months agoCI: realign cmake build settings (for nghttp2, libidn2)
Viktor Szakats [Sat, 3 Aug 2024 08:01:49 +0000 (10:01 +0200)] 
CI: realign cmake build settings (for nghttp2, libidn2)

- nghttp2 is now on by default, adjust build configuration accordingly.
  Follow-up to 87aa4ebd821ebae0023df8658360c724efcf5e00 #14136

- GHA/macos: disable libidn2 in combination builds, syncing with
  autotools. Otherwise it's now auto-detected via pkg-config by default.
  Follow-up to f43adc2c4978f7f82a359e89186e58a31d17b0ad #14137

Closes #14364

15 months agocurl: support embedding a CA bundle
Viktor Szakats [Sat, 29 Jun 2024 01:30:14 +0000 (03:30 +0200)] 
curl: support embedding a CA bundle

Add the ability to embed a CA bundle into the curl binary. It is used
when no other runtime or build-time option set one.

This helps curl-for-win macOS and Linux builds to run standalone, and
also helps Windows builds to avoid picking up the CA bundle from an
arbitrary (possibly world-writable) location (though this behaviour is
not currently disablable).

Usage:
- cmake: `-DCURL_CA_EMBED=/path/to/curl-ca-bundle.crt`
- autotools: `--with-ca-embed=/path/to/curl-ca-bundle.crt`
- Makefile.mk: `CURL_CA_EMBED=/path/to/curl-ca-bundle.crt`

Also add new command-line option `--dump-ca-embed` to dump the embedded
CA bundle to standard output.

Closes #14059

15 months agocmake: detect `nghttp2` via `pkg-config`, enable by default
Viktor Szakats [Tue, 9 Jul 2024 09:39:48 +0000 (11:39 +0200)] 
cmake: detect `nghttp2` via `pkg-config`, enable by default

- also detect nghttp2 via `pkg-config` to match nghttp3 detection
  and autotools.

- enable nghttp2 by default to match autotools.

Cherry-picked from #14097
Closes #14136

15 months agocmake: drop unused internal variable
Viktor Szakats [Fri, 2 Aug 2024 22:23:11 +0000 (00:23 +0200)] 
cmake: drop unused internal variable

Closes #14361

15 months agovtls: fix MSVC 'cast truncates constant value' warning
Tal Regev [Thu, 1 Aug 2024 21:07:12 +0000 (00:07 +0300)] 
vtls: fix MSVC 'cast truncates constant value' warning

Fixes:
```
curl\lib\vtls\cipher_suite.c(193,3): error C2220: the following warning is treated as an error
curl\lib\vtls\cipher_suite.c(193,3): warning C4310: cast truncates constant value
```

Closes #14341

15 months agoci: Update actions/upload-artifact digest to 89ef406
renovate[bot] [Fri, 2 Aug 2024 21:06:28 +0000 (21:06 +0000)] 
ci: Update actions/upload-artifact digest to 89ef406

Closes #14359

15 months agocmake: drop reference to undefined variable
Viktor Szakats [Fri, 2 Aug 2024 13:05:35 +0000 (15:05 +0200)] 
cmake: drop reference to undefined variable

Undefined since the initial commit adding CMake support.

Closes #14358

15 months agocmake: drop no-op `tests/data/CMakeLists.txt`
Viktor Szakats [Fri, 2 Aug 2024 11:41:36 +0000 (13:41 +0200)] 
cmake: drop no-op `tests/data/CMakeLists.txt`

It's a no-op since
d162fca69a22d199c0369c5ce3d66a9531752ba2 #9333 (2022-08-18).

Also revert 476499c75c83ffa3d65da398b5759a5f2fdb980d that is
no longer necessary: move `Makefile.inc` back into `Makefile.am`.

Closes #14357

15 months agocmake: drop custom `CMakeOutput.log`/`CMakeError.log` logs
Viktor Szakats [Fri, 2 Aug 2024 11:38:14 +0000 (13:38 +0200)] 
cmake: drop custom `CMakeOutput.log`/`CMakeError.log` logs

CMake writes this information into `CMakeConfigureLog.yaml`
automatically.

Closes #14356

15 months agox509asn1: raise size limit for x509 certification information
Stefan Eissing [Fri, 2 Aug 2024 10:50:07 +0000 (12:50 +0200)] 
x509asn1: raise size limit for x509 certification information

Raise the limit for certification information from 10 thousand to 100
thousand bytes. Certificates can be larger than 10k.

Change the infof() debug output to add '...' at the end when the max
limit it can handle is exceeded.

Reported-by: Sergio Durigan Junior
Fixes #14352
Closes #14354

15 months agoGHA/distcheck: add a reproducible release check
Daniel Stenberg [Fri, 2 Aug 2024 06:58:05 +0000 (08:58 +0200)] 
GHA/distcheck: add a reproducible release check

15 months agoverify-release: shell script that verifies a release tarball
Daniel Stenberg [Fri, 2 Aug 2024 06:46:06 +0000 (08:46 +0200)] 
verify-release: shell script that verifies a release tarball

This script remakes a provided curl release and verifies that the newly
built version is identical to the original file.

Due to bugs in releases up to and including curl 8.9.1, it does not work
on tarballs generated before commit 754acd1a9dc6.

Closes #14350

15 months agoMakefile: remove 'scripts' duplicate from DIST_SUBDIRS
Daniel Stenberg [Fri, 2 Aug 2024 07:49:31 +0000 (09:49 +0200)] 
Makefile: remove 'scripts' duplicate from DIST_SUBDIRS

Also fix the .dist replacing by avoiding all Makefiles because it
otherwise also went into the temporary release folder and got confused
about the Makefile.dist in there.

15 months agodmaketgz: only run 'make distclean' if Makefile exists
Daniel Stenberg [Fri, 2 Aug 2024 07:16:31 +0000 (09:16 +0200)] 
dmaketgz: only run 'make distclean' if Makefile exists

... so that we can avoid the build failure if we run this in a clean
checkout.

Also remove -it from the docker invoke since it is not interactive and
it needs no TTY. They made the job fail in the CI.

15 months agoautotools: fix typo in tests/data target
Viktor Szakats [Fri, 2 Aug 2024 11:33:59 +0000 (13:33 +0200)] 
autotools: fix typo in tests/data target

Ref: c1b369fd4c2d538c89d6f9342092bead2628ff5c (2002-12-12)
Closes #14355

15 months agoGHA/non-native: reduce FreeBSD test parallelism to -j8 [ci skip]
Viktor Szakats [Fri, 2 Aug 2024 12:32:25 +0000 (14:32 +0200)] 
GHA/non-native: reduce FreeBSD test parallelism to -j8 [ci skip]

To reduce flakiness.

15 months agotests: gitignore newly generated files
Stefan Eissing [Fri, 2 Aug 2024 07:58:52 +0000 (09:58 +0200)] 
tests: gitignore newly generated files

Closes #14351

15 months agoprogress: ratelimit/progress tweaks
Stefan Eissing [Thu, 1 Aug 2024 11:05:49 +0000 (13:05 +0200)] 
progress: ratelimit/progress tweaks

- multi.c: when ratelimiting a transfer stops (MSTATE_RATELIMITING ->
  MSTATE_PERFORMING), run the MSTATE_PERFORMING state right away

- urldata.h: factor out upload and download progress counters into a
  struct, use that for passing these to progress update functions

- progress.c/getinfo.c: change names of moved progress counters

- progress.c: use new structs and a helper struct to factor repeated
  calculation into static helpers

Closes #14335

15 months agohttp2: improve rate limiting of downloads
Stefan Eissing [Thu, 1 Aug 2024 07:53:50 +0000 (09:53 +0200)] 
http2: improve rate limiting of downloads

Set the initial stream window size to 64KB and increase that to the 10MB
we used to start with on the first server reply, unless a rate limit is
in effect.

Continously monitory changes to the transfers rate limit and adjust the
stream window size accordingly. `max_recv_speed` is a transfer propert
that can be changed during processing by a callback.

Closes #14326

15 months agoGHA: update awslabs/aws-lc to v1.33.0
renovate[bot] [Fri, 2 Aug 2024 02:32:54 +0000 (02:32 +0000)] 
GHA: update awslabs/aws-lc to v1.33.0

Closes #14347

15 months agotests/http: configure test httpd to honor client cipher order
Stefan Eissing [Thu, 1 Aug 2024 14:16:55 +0000 (16:16 +0200)] 
tests/http: configure test httpd to honor client cipher order

Let the client, e.g. curl, influence the cipher selected in a TLS
handshake. TLS backends have different preferences and honor that
in httpd the same as Caddy does.

Also makes for a more fair compare of different TLS backends.

Closes #14338

15 months agodist: fix reproducible build from release tarball
Daniel Stenberg [Thu, 1 Aug 2024 11:51:43 +0000 (13:51 +0200)] 
dist: fix reproducible build from release tarball

Make it possible to rebuild an identical copy from a release tarball. It
was previously only possible from a checked out git repository.

- add release-tools.sh to dist
- keep Makefile.dist around to include it in dist
- regenerate tool_huge.c with the new version in dist
- fix the dist CI job to not do make clean like before

Closes #14336

15 months agocmake: add gnutls to multissl feature
Tal Regev [Thu, 1 Aug 2024 20:13:42 +0000 (23:13 +0300)] 
cmake: add gnutls to multissl feature

Closes #14345

15 months agocurl: allow 500MB data URL encode strings
Daniel Stenberg [Thu, 1 Aug 2024 14:45:50 +0000 (16:45 +0200)] 
curl: allow 500MB data URL encode strings

Previously it would bail out of the generated data reached 8MB in
memory.

Reported-by: Antoine du Hamel
Fixes #14337
Closes #14340

15 months agoescape: allow curl_easy_escape to generate 3*input length output
Daniel Stenberg [Thu, 1 Aug 2024 14:42:58 +0000 (16:42 +0200)] 
escape: allow curl_easy_escape to generate 3*input length output

Instead of capping it to the 3 * CURL_MAX_INPUT_LENGTH. To allow users
to URL encode larger chunks of data.

Closes #14339

15 months agoCHANGES: rename to CHANGES.md, no longer generated
Daniel Stenberg [Thu, 1 Aug 2024 07:58:31 +0000 (09:58 +0200)] 
CHANGES: rename to CHANGES.md, no longer generated

- scripts/log2changes.pl was not included in release tarballs, which broke
  reproducible builds

- since log2changes uses git to generate the contents, it makes it difficult
  to generate the same contents later (it would need to be fixed)

- the CHANGES file has outlived its purpose. the main changes are in the
  RELEASE-NOTES, the rest are better tracked directly using git or on GitHub

- put a fixed CHANGES.md in there instead pointing out where the info lives
  now

Closes #14331

15 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 1 Aug 2024 11:36:28 +0000 (13:36 +0200)] 
RELEASE-NOTES: synced

Plus add a link to https://curl.se/changes.html

15 months agoGHA: scan git repository and detect unvetted binary files
Daniel Stenberg [Thu, 1 Aug 2024 09:16:21 +0000 (11:16 +0200)] 
GHA: scan git repository and detect unvetted binary files

The only binary-looking files that are accepted in the git repository
need to match the checksums in the sha256sum file
".github/scripts/binarycheck.sums".

This is done to make sure that no one has planted any hidden (encrypted)
potentially dangerous payload in the tree.

Closes #14333

15 months agoGHA/windows: drop FTP tests
Viktor Szakats [Wed, 31 Jul 2024 19:53:15 +0000 (21:53 +0200)] 
GHA/windows: drop FTP tests

Try reducing hangs.

Closes #14330

15 months agoGHA/windows: remove vcpkg bin path in MSVC jobs
Tal Regev [Wed, 31 Jul 2024 18:49:51 +0000 (21:49 +0300)] 
GHA/windows: remove vcpkg bin path in MSVC jobs

- the path is wrong, because we compile on debug, and we are using
  the release bin path.
- the path is not needed, cmake curl copy the needed dlls to the
  compilation cmake folder where the curl exe is found.

Closes #14329

15 months agoGHA/windows: timeout earlier with hung tests
Viktor Szakats [Fri, 19 Jul 2024 20:42:42 +0000 (22:42 +0200)] 
GHA/windows: timeout earlier with hung tests

Finishing tests takes on average 10 or less minutes depending on
platform. Reduce job step timeouts to reflect that. It helps
concluding hung/failed tests earlier, which allows to retry them
earlier.

This makes it more difficult to tell from a job if it hung or not,
because we lose the long runtime as a telltale sign. Let's see how it
works out and adjust as necessary.

Also fix a comment while here.

Closes #14236

15 months agoINSTALL.md: MultiSSL and QUIC are mutually exclusive
Daniel Stenberg [Wed, 31 Jul 2024 16:40:44 +0000 (18:40 +0200)] 
INSTALL.md: MultiSSL and QUIC are mutually exclusive

Reported-by: Matt Jolly
Fixes #14308
Closes #14328

15 months agolib: convert some debugf()s into traces
Stefan Eissing [Wed, 31 Jul 2024 09:36:08 +0000 (11:36 +0200)] 
lib: convert some debugf()s into traces

Use CURL_TRC_CF() for some useful tracing information instead of
DEBUGF().

Closes #14322

15 months agocmake: distcheck for files in CMake subdir
Viktor Szakats [Wed, 31 Jul 2024 09:53:31 +0000 (11:53 +0200)] 
cmake: distcheck for files in CMake subdir

- add CMake option to verify if the `CMake/*.cmake`, `CMake/*.in` files
  are listed as distributable in autotools' `EXTRA_DIST`. The check can
  be enabled with `-DENABLE_DIST_TEST=ON` CMake option.

- add CI job to that effect.

Ref: #14320
Closes #14323

15 months agolibcurl.pc: add `Cflags.private`
Viktor Szakats [Wed, 31 Jul 2024 08:49:38 +0000 (10:49 +0200)] 
libcurl.pc: add `Cflags.private`

Ref: https://manpages.debian.org/bookworm/pkgconf/pc.5.en.html#PROPERTY_KEYWORDS
Ref: https://www.msys2.org/docs/pkgconfig/#cflagsprivate-static-libraries

Closes #14321

15 months agodist: add missing `FindNettle.cmake`
Viktor Szakats [Wed, 31 Jul 2024 08:02:13 +0000 (10:02 +0200)] 
dist: add missing `FindNettle.cmake`

Follow-up to 669ce42275635dc1f881dab3dfc9a55c9ab49b21 #14285
Reported-by: Christoph Reiter
Bug: https://github.com/curl/curl/pull/14285#issuecomment-2259880050
Closes #14320

15 months agotests: provide docs a as curldown, not nroff
Daniel Stenberg [Wed, 31 Jul 2024 10:56:33 +0000 (12:56 +0200)] 
tests: provide docs a as curldown, not nroff

As runtests.md and testcurl.md. Very few people actually need these as
manpages anyway.

With this, we have no more nroff formatted documents in git.

Closes #14324

15 months agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 31 Jul 2024 08:48:48 +0000 (10:48 +0200)] 
RELEASE-NOTES: synced

Bump to 8.10.0

15 months agorustls: make all tests pass
Joe Birr-Pixton [Tue, 30 Jul 2024 13:23:50 +0000 (14:23 +0100)] 
rustls: make all tests pass

- supports IP addresses in certs since 0.21
- Remove workaround, and re-enable tests

Closes #14317

15 months agoGHA/windows: enable MulitSSL in an MSVC job
Tal Regev [Wed, 31 Jul 2024 05:18:18 +0000 (08:18 +0300)] 
GHA/windows: enable MulitSSL in an MSVC job

Ref: 98da147b18fa4963efb425df8400ca40e7526646 #14305
Closes #14276

15 months agoRELEASE-NOTES: synced curl-8_9_1
Daniel Stenberg [Wed, 31 Jul 2024 06:50:04 +0000 (08:50 +0200)] 
RELEASE-NOTES: synced

15 months agoTHANKS: add names from the 8.9.1 release notes
Daniel Stenberg [Wed, 31 Jul 2024 06:50:04 +0000 (08:50 +0200)] 
THANKS: add names from the 8.9.1 release notes

Also remove duplicates

15 months agox509asn1: unittests and fixes for gtime2str
Stefan Eissing [Tue, 30 Jul 2024 14:40:48 +0000 (16:40 +0200)] 
x509asn1: unittests and fixes for gtime2str

Fix issues in GTime2str() and add unit test cases to verify correct
behaviour.

Follow-up to 3c914bc6801

Closes #14316

15 months agovtls: avoid forward declaration in MultiSSL builds
Tal Regev [Tue, 30 Jul 2024 03:07:50 +0000 (06:07 +0300)] 
vtls: avoid forward declaration in MultiSSL builds

The MSVC compiler cannot have forward declaration with const and static
variable, causing this error:
```
curl\lib\vtls\vtls.c(417,44): warning C4132: 'Curl_ssl_multi': const object should be initialized
```

Ref: #14276
Closes #14305

15 months agotidy-up: URL updates (one more)
Viktor Szakats [Tue, 30 Jul 2024 19:35:59 +0000 (21:35 +0200)] 
tidy-up: URL updates (one more)

Follow-up to 767d5811b5c783b42cea999dd42ecf0453085d17 #14318

15 months agotidy-up: URL updates
Viktor Szakats [Tue, 30 Jul 2024 16:05:02 +0000 (18:05 +0200)] 
tidy-up: URL updates

Closes #14318

15 months agocmake: drop `if(PKG_CONFIG_FOUND)` guard for `pkg_check_modules()`
Viktor Szakats [Tue, 30 Jul 2024 08:52:32 +0000 (10:52 +0200)] 
cmake: drop `if(PKG_CONFIG_FOUND)` guard for `pkg_check_modules()`

The oldest cmake supported by curl is v3.7.0, which already has such
guard (using `PKG_CONFIG_EXECUTABLE`) inside `pkg_check_modules()`. The
advantage of leaving that guard to CMake is that it will define/reset
all output variables, while the manual guard doesn't do this and also
leaves for example `NETTLE_FOUND` undefined.

Delete the single use of this guard from the recently added `nettle`
detection, where I included it by accident. Then possibly re-introduce
it universally if we find it useful after more evaluation.

Follow-up to 669ce42275635dc1f881dab3dfc9a55c9ab49b21 #14285
Closes #14309

15 months agomailmap: dedupe an author showing up twice in shortlog -s
Daniel Stenberg [Tue, 30 Jul 2024 12:25:30 +0000 (14:25 +0200)] 
mailmap: dedupe an author showing up twice in shortlog -s

15 months agomisc: cleanup after removing years from copyright
Ivan Kuchin [Tue, 30 Jul 2024 10:39:50 +0000 (12:39 +0200)] 
misc: cleanup after removing years from copyright

- remove leftover copyright years from few test files
- fix email in copyright lines
- consistent format of copyright lines

Closes #14312

15 months agowolfssl: avoid calling get_cached_x509_store if store is uncachable
Alex Snast [Tue, 30 Jul 2024 05:12:26 +0000 (08:12 +0300)] 
wolfssl: avoid calling get_cached_x509_store if store is uncachable

There's no need for get_cached_x509_store call if the return value won't
be used for caching anyway.

Closes #14306

15 months agocontrithanks.sh: use -F with -v to match lines as strings
Daniel Stenberg [Tue, 30 Jul 2024 10:14:48 +0000 (12:14 +0200)] 
contrithanks.sh: use -F with -v to match lines as strings

Makes names involving [brackets] work.

15 months agoGHA/non-native: bump FreeBSD/arm64 python modules
Viktor Szakats [Tue, 30 Jul 2024 09:52:27 +0000 (11:52 +0200)] 
GHA/non-native: bump FreeBSD/arm64 python modules

FreeBSD seems to upgrade their Python separately for arm64
and Intel. Today, arm64 caught up with the Intel packages.
Update our CI to reflect it.

Closes #14310

15 months agoGHA: bump github/codeql-action and msys2/setup-msys2
dependabot[bot] [Mon, 29 Jul 2024 14:45:58 +0000 (14:45 +0000)] 
GHA: bump github/codeql-action and msys2/setup-msys2

- bump github/codeql-action from 3.25.13 to 3.25.15

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.13 to 3.25.15.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/2d790406f505036ef40ecba973cc774a50395aac...afb54ba388a7dca6ecae48f608c4ff05ff4cc77a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14300

- bump msys2/setup-msys2 from 2.23.0 to 2.24.0

Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/msys2/setup-msys2/compare/d0e80f58dffbc64f6a3a1f43527d469b4fc7b6c8...5df0ca6cbf14efcd08f8d5bd5e049a3cc8e07fd2)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14301

15 months agox509asn1: clean up GTime2str
Daniel Stenberg [Tue, 30 Jul 2024 08:05:17 +0000 (10:05 +0200)] 
x509asn1: clean up GTime2str

Co-authored-by: Stefan Eissing
Reported-by: Dov Murik
Closes #14307

15 months agotool_operate: more defensive socket code
Daniel Stenberg [Mon, 29 Jul 2024 17:49:32 +0000 (19:49 +0200)] 
tool_operate: more defensive socket code

- use 'struct sockaddr' to getsockname() and its sa_family member

- use 'curl_socklen_t' instead of 'socklen_t'

- check for AF_INET6 to exist instead assuming it does

Should be generally more portable.

Reported-by: Harry Sintonen
Closes #14304

15 months agoconfigure: limit `__builtin_available` test to Darwin
Viktor Szakats [Sun, 14 Jul 2024 19:49:35 +0000 (21:49 +0200)] 
configure: limit `__builtin_available` test to Darwin

This feature test always fails on non-Apple systems. (For Apple targets
it's supported by llvm and Apple clang.)

Syncs behaviour with CMake.

Follow-up to cfd6f43d6ca7e57670b422bab7bbf10221a2cf3e #14127
Cherry-picked from #14097
Closes #14196

15 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 29 Jul 2024 19:24:32 +0000 (21:24 +0200)] 
RELEASE-NOTES: synced

15 months agoCURLOPT_SSL_CTX_FUNCTION.md: mention CA caching
Daniel Stenberg [Mon, 29 Jul 2024 16:58:40 +0000 (18:58 +0200)] 
CURLOPT_SSL_CTX_FUNCTION.md: mention CA caching

and add a few more see-also

Closes #14302

15 months agocmake: detect `libssh` via `pkg-config`
Viktor Szakats [Tue, 16 Jul 2024 10:48:13 +0000 (12:48 +0200)] 
cmake: detect `libssh` via `pkg-config`

Also:
- fix broken libssh `pkg-config` behaviour on old Linux.
  (when found, `LIBSSH_LINK_LIBRARIES` remains undefined.)

- delete manual libssh config from Old Linux CI job,
  it's no longer necessary.

Closes #14199

15 months agoGHA/non-native: improve, migrate x86_64 FreeBSD with tests from Cirrus CI
Viktor Szakats [Sat, 20 Jul 2024 11:31:45 +0000 (13:31 +0200)] 
GHA/non-native: improve, migrate x86_64 FreeBSD with tests from Cirrus CI

- run tests via `make test-ci` instead of `make check` with autotools.
- add `x86_64` job for FreeBSD, with tests.
  It matches the existing Cirrus CI job, with these differences:
  - finishes 3x faster (thanks to parallel tests enabled).
  - librtmp is not enabled because it's slated for removal by FreeBSD.
    (already past the removal deadline, thought the package still
    installs.)
  - DICT and TELNET servers fail to start. Couldn't figure out why.
    It means skipping test 1450 and 1452.
  - it runs more tests, e.g. websockets and ip6-localhost.
  - no `pkg update -f`.
  - it misses the `CRYPTOGRAPHY_DONT_BUILD_RUST=1`, `pkg delete curl`,
    `chmod 777`, `sudo -u nobody` and `sysctl net.inet.tcp.blackhole`
    tricks. The latter is the default in these runners, the others did
    not affect results.
- set `-j0` for tests in the NetBSD job. Flaky otherwise.

Closes #14244

15 months agocmake: detect `nettle` when building with GnuTLS
Viktor Szakats [Sat, 27 Jul 2024 09:47:56 +0000 (11:47 +0200)] 
cmake: detect `nettle` when building with GnuTLS

`nettle` is a direct dependency of curl, when building with GnuTLS.
Add a new `Find` module to detect it.

Also:
- GHA/macos: drop `nettle` hack no longer necessary.
- add `nettle` to `libcurl.pc`.
- also add `nettle` to `libcurl.pc` in autotools builds.

Follow-up to 781242ffa44a9f9b95b6da5ac5a1bf6372ec6257 #11967
Closes #14285

15 months agomacos: fix Apple SDK bug workaround for non-macOS targets
Viktor Szakats [Thu, 25 Jul 2024 22:10:21 +0000 (00:10 +0200)] 
macos: fix Apple SDK bug workaround for non-macOS targets

Turns out that MAC != OSX, despite what these names otherwise mean and
what's suggested by source code comments. "MAC" in fact means Darwin
(aka Apple), not macOS. "OSX" means macOS.

GitHub bumped the macos-14 runner default to Xcode 15.4, hitting the
llvm@15 incompatibility bug by default. Meaning the previous workaround
for the SDK bug is necessary.

This patch extend the workaround to not apply to mobile OS variants.

Follow-up to ff784af461175584c73e7e2b65af00b1a5a6f67f #14159
Reported-by: Sergey
Confirmed-by: Marcel Raad
Fixes #14269
Closes #14275

15 months agowolfssl: CA store share fix
Stefan Eissing [Fri, 26 Jul 2024 13:07:52 +0000 (15:07 +0200)] 
wolfssl: CA store share fix

When sharing the x509 store in wolfSSL, always use an explicitly
constructed one, as the SSLCTX might have "only" an internal one which
is not obeying reference count lifetimes.

Fixes #14278
Reported-by: Alex Snast
Closes #14279

15 months agocurl: support __ss_family use on NonStop platforms
Randall S. Becker [Thu, 25 Jul 2024 21:18:46 +0000 (22:18 +0100)] 
curl: support __ss_family use on NonStop platforms

The definition of sockaddr_storage incorrectly specifies the ss_family
field as __ss_family. This fix conditionally allows builds to succeed on
all NonStop platforms.

Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
Closes #14273

15 months agotest993: list 1000 messages over POP3
Daniel Stenberg [Mon, 29 Jul 2024 08:56:48 +0000 (10:56 +0200)] 
test993: list 1000 messages over POP3

Attempt to verify issue #14201

Closes #14297

15 months agoconnect: fix connection shutdown for event based processing
Stefan Eissing [Mon, 29 Jul 2024 08:23:20 +0000 (10:23 +0200)] 
connect: fix connection shutdown for event based processing

connections being shutdown would register sockets for events, but then
never remove these sockets again. Nor would the shutdown effectively
been performed.

- If a socket event involves a transfer, check if that is the
  connection cache internal handle and run its multi_perform()
  instead (the internal handle is used for all shutdowns).
- When a timer triggers for a transfer, check also if it is
  about the connection cache internal handle.
- During processing shutdowns in the connection cache, assess
  the shutdown timeouts. Register a Curl_expire() of the lowest
  value for the cache's internal handle.

Reported-by: Gordon Parke
Fixes #14280
Closes #14296

15 months agotests: provide FTP directory contents in the test file
Daniel Stenberg [Mon, 29 Jul 2024 08:17:08 +0000 (10:17 +0200)] 
tests: provide FTP directory contents in the test file

Instead of providing a fixed single synthetic response in the test
server itself. To allow us to better use *different* directory listings
in different test cases. In this change, most listings remain the same
as before.

The wildcard match tests still use synthetic responses but we should fix
that as well.

Updated numerous test cases to use this.

Closes #14295

15 months agoftpserver.pl: make POP3 LIST serve content from the test file
Daniel Stenberg [Mon, 29 Jul 2024 07:11:47 +0000 (09:11 +0200)] 
ftpserver.pl: make POP3 LIST serve content from the test file

instead of a fixed list in the test server.

Adjust test 853 accordingly.

Closes #14293

15 months agoTODO: thread-safe sharing
Daniel Stenberg [Sun, 28 Jul 2024 14:06:47 +0000 (16:06 +0200)] 
TODO: thread-safe sharing

15 months agoCURLSHOPT_SHARE.md: mention sessions/cookies as not thread-safe
Daniel Stenberg [Sun, 28 Jul 2024 14:03:16 +0000 (16:03 +0200)] 
CURLSHOPT_SHARE.md: mention sessions/cookies as not thread-safe

Sharing of these between threads are apparently also not done safely.

Ref: #14290
Reported-by: Aki Sakurai
Closes #14292

15 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sun, 28 Jul 2024 14:11:29 +0000 (16:11 +0200)] 
RELEASE-NOTES: synced

15 months agoos400: build cli manual.
Patrick Monnerat [Sat, 27 Jul 2024 22:36:09 +0000 (00:36 +0200)] 
os400: build cli manual.

Use PASE perl to run manual generation scripts.

As PASE perl is not aware of all possible input file encoding, convert
all files to UTF-8 upon build start (this might be lengthy).

OS/400 terminal emulation may only offer 76 columns, thus a new -c
parameter has been added to the managen program, defining the allowed
width.

If perl is not available, omit generation and disable online manual.

Closes #14289

15 months agoexample/multi-uv: remove the use of globals
Daniel Stenberg [Sat, 27 Jul 2024 16:11:55 +0000 (18:11 +0200)] 
example/multi-uv: remove the use of globals

- shows how to pass on local variables (better)

- start the transfers nicer (with curl_multi_socket_action)

- consistent and helpful function naming - to better show what functions
  and callbacks that are used for what

- build warning-free with gcc -W -Wall -pedantic

Closes #14287

15 months agoruntests: fold timing details with GHA, sync `-r` tflags
Viktor Szakats [Sat, 20 Jul 2024 16:20:06 +0000 (18:20 +0200)] 
runtests: fold timing details with GHA, sync `-r` tflags

- move timing details into a foldable group when run in GitHub Actions.
  Spec:
  https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#grouping-log-lines

- enable `-r` (run time stats) option in autotools' `test-ci` target,
  syncing it with cmake.

Closes #14284

15 months agoGHA/windows: increase timeout for vcpkg build step [ci skip]
Viktor Szakats [Sat, 27 Jul 2024 17:08:33 +0000 (19:08 +0200)] 
GHA/windows: increase timeout for vcpkg build step [ci skip]

Examples:
https://github.com/curl/curl/actions/runs/10102112253/job/27937088909?pr=14274
https://github.com/curl/curl/actions/runs/10102112253/job/27937082353?pr=14274
https://github.com/curl/curl/actions/runs/10102112253/job/27937088478?pr=14274

15 months agoGHA/macos: update comment about default Xcode on macos-14 runner [ci skip]
Viktor Szakats [Sat, 27 Jul 2024 08:17:20 +0000 (10:17 +0200)] 
GHA/macos: update comment about default Xcode on macos-14 runner [ci skip]

New default since:
https://github.com/actions/runner-images/blob/releases/macos-14-arm64/20240722/images/macos/macos-14-arm64-Readme.md

15 months agoos400: workaround an IBM ASCII run-time library bug
Patrick Monnerat [Fri, 26 Jul 2024 15:15:27 +0000 (17:15 +0200)] 
os400: workaround an IBM ASCII run-time library bug

IBM-provided ASCII function puts() does not output an expected trailing
newline: emulate the correct behavior using other functions.

Closes #14281

15 months agotransfer: speed limiting fix for 32bit systems
Stefan Eissing [Fri, 26 Jul 2024 08:38:45 +0000 (10:38 +0200)] 
transfer: speed limiting fix for 32bit systems

When checking if a speed limit on receives applies, compare the receive
sizes using the large int type to prevent an overflow on systems where
size_t is 32bit.

Fixes #14272
Reported-by: Mamoru Tasaka
Closes #14277

15 months agowolfSSL: allow wolfSSL's implementation of kyber to be used
Anthony Hu [Wed, 24 Jul 2024 22:23:09 +0000 (18:23 -0400)] 
wolfSSL: allow wolfSSL's implementation of kyber to be used

Closes #14268

15 months agolib: survive some NULL input args
Daniel Stenberg [Tue, 23 Jul 2024 09:06:56 +0000 (11:06 +0200)] 
lib: survive some NULL input args

The input string pointer to:

curl_escape
curl_easy_escape
curl_unescape
curl_easy_unescape

The running_handles pointer to:

curl_multi_perform
curl_multi_socket_action
curl_multi_socket_all
curl_multi_socket

Reported-by: icy17 on github
Fixes #14247
Closes #14262

15 months agoRELEASE-PROCEDURE.md: restore next release date
Daniel Stenberg [Wed, 24 Jul 2024 07:10:35 +0000 (09:10 +0200)] 
RELEASE-PROCEDURE.md: restore next release date

Pointed-out-by: extrimexxx on github
Bug: https://github.com/curl/curl/pull/14267#issuecomment-2247062235

15 months agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 24 Jul 2024 06:48:18 +0000 (08:48 +0200)] 
RELEASE-NOTES: synced

bumped to 8.9.1

15 months agoRELEASE-PROCEDURE.md: remove the initial build step
Daniel Stenberg [Tue, 23 Jul 2024 21:40:13 +0000 (23:40 +0200)] 
RELEASE-PROCEDURE.md: remove the initial build step

Because it is no longer needed to be done by a person as the dmaketgz
script does it by itself.

Removed two past release dates, added two new future ones

Closes #14267

15 months agoRELEASE-NOTES: synced curl-8_9_0
Daniel Stenberg [Sat, 20 Jul 2024 22:02:38 +0000 (00:02 +0200)] 
RELEASE-NOTES: synced

15 months agoTHANKS: updates from the 8.9.0 release
Daniel Stenberg [Tue, 23 Jul 2024 21:36:30 +0000 (23:36 +0200)] 
THANKS: updates from the 8.9.0 release

15 months agocurl_easy_escape.md: move historic details to HISTORY
Daniel Stenberg [Tue, 23 Jul 2024 09:03:15 +0000 (11:03 +0200)] 
curl_easy_escape.md: move historic details to HISTORY

Closes #14261

15 months agodocs/libcurl: add to cleanup docs that their inputs go invalid
Daniel Stenberg [Mon, 22 Jul 2024 17:04:12 +0000 (19:04 +0200)] 
docs/libcurl: add to cleanup docs that their inputs go invalid

Reported-by: icy17 on github
Fixes #14248
Closes #14258

15 months agoGHA: bump github/codeql-action from 3.25.11 to 3.25.13
dependabot[bot] [Mon, 22 Jul 2024 14:17:30 +0000 (14:17 +0000)] 
GHA: bump github/codeql-action from 3.25.11 to 3.25.13

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.11 to 3.25.13.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b611370bb5703a7efb587f9d136a52ea24c5c38c...2d790406f505036ef40ecba973cc774a50395aac)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14255

15 months agoconncache: connection shutdown, multi_socket handling
Stefan Eissing [Mon, 22 Jul 2024 15:04:30 +0000 (17:04 +0200)] 
conncache: connection shutdown, multi_socket handling

- implement the socket hash user/reader/writer processing also
  for connections that are being shut down by the connection cache.
- split out handling of current vs. last pollset socket event handling
  into a function available in other code parts
- add `shutdown_poll` pollset to `connectdata` struct so that changes
  in the pollset can be recorded during shutdown. (The internal handle
  cannot keep it since it might be used for many connections)

Reported-by: calvin2021y on github
Fixes #14252
Closes #14257

15 months agotool_cb_prg: output "flying saucers" with leading carriage return
Daniel Stenberg [Sat, 20 Jul 2024 21:21:16 +0000 (23:21 +0200)] 
tool_cb_prg: output "flying saucers" with leading carriage return

Because that is how the progress-bar is output, so when the progress-bar
has been shown at least once and the information is reset, like for a
redirect, there might be a moment where the size goes from known to
unknown and then the flying saucerts are shown after a brief display of
the progress-bar.

It could previously cause accidental character leftovers on the right
side of the bar when using a narrow display.

Reported-by: Chris Webb
Fixes #14213
Closes #14246

15 months agolib: Curl_posttransfer => multi_posttransfer
Daniel Stenberg [Fri, 19 Jul 2024 22:53:24 +0000 (00:53 +0200)] 
lib: Curl_posttransfer => multi_posttransfer

Moved from transfer.c to multi.c as it was only used within multi.c

Made a void, as it returned a fixed return code nothing checked.

Closes #14240

15 months agoCURLOPT_SSL_VERIFYHOST.md: refresh
Daniel Stenberg [Fri, 19 Jul 2024 23:11:13 +0000 (01:11 +0200)] 
CURLOPT_SSL_VERIFYHOST.md: refresh

Move mentions of old behavior to the HISTORY section to make it easier
to read about modern behavior.

Added a MATCHING section.

Closes #14241

15 months agomulti: do a final progress update on connect failure
Daniel Stenberg [Fri, 19 Jul 2024 22:44:27 +0000 (00:44 +0200)] 
multi: do a final progress update on connect failure

To fix timing metric etc

Co-authored-by: Justin Maggard
Fixes #14204
Closes #14239