]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
12 months agocurl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs
Daniel Stenberg [Mon, 14 Oct 2024 12:09:59 +0000 (14:09 +0200)] 
curl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs

It makes the callbacks get different signnatures when used from within
libcurl vs outside of it by libcurl-using applications (such as the
libtests) and this triggers UndefinedBehaviorSanitizer errors.

Closes #15289

12 months agolib: remove function pointer typecasts for hmac/sha256/md5
Daniel Stenberg [Mon, 14 Oct 2024 07:47:03 +0000 (09:47 +0200)] 
lib: remove function pointer typecasts for hmac/sha256/md5

Make sure we use functions with the correct prototype.

Closes #15289

12 months agoconncache: More efficient implementation of cpool_remove_bundle
Michael Kaufmann [Mon, 14 Oct 2024 08:45:12 +0000 (10:45 +0200)] 
conncache: More efficient implementation of cpool_remove_bundle

Closes #15292

12 months agoGHA/linux: add cmake job for system mbedTLS with pkg-config
Viktor Szakats [Mon, 14 Oct 2024 00:54:03 +0000 (02:54 +0200)] 
GHA/linux: add cmake job for system mbedTLS with pkg-config

Add a build-only cmake job with system mbedTLS package and `pkg-config`
enabled. Ubuntu 24.04 comes with mbedTLS 2.28.8 which supports
`pkg-config`.

Follow-up to 7bab201abe3915a0167c002f9308950cb8a06e4b #15193
Closes #15286

12 months agoserver/mqttd: fix two memory leaks
Daniel Stenberg [Mon, 14 Oct 2024 08:21:10 +0000 (10:21 +0200)] 
server/mqttd: fix two memory leaks

Closes #15290

12 months agoGHA/linux: fixup pip for Ubuntu 24.04
Viktor Szakats [Mon, 14 Oct 2024 01:01:41 +0000 (03:01 +0200)] 
GHA/linux: fixup pip for Ubuntu 24.04

`ubuntu-latest` became `ubuntu-24.04` today.

Closes #15287

12 months agotool_operate: make --skip-existing work for --parallel
Daniel Stenberg [Sun, 13 Oct 2024 15:33:38 +0000 (17:33 +0200)] 
tool_operate: make --skip-existing work for --parallel

Reported-by: Tobias Wendorff
Fixes #15261
Closes #15283

12 months agourl: use same credentials on redirect
Daniel Stenberg [Sat, 12 Oct 2024 21:54:39 +0000 (23:54 +0200)] 
url: use same credentials on redirect

Previously it could lose the username and only use the password.

Added test 998 and 999 to verify.

Reported-by: Tobias Bora
Fixes #15262
Closes #15282

12 months agolib: move curl_path.[ch] into vssh/
Daniel Stenberg [Sun, 13 Oct 2024 15:56:23 +0000 (17:56 +0200)] 
lib: move curl_path.[ch] into vssh/

As this contains code only used by SSH backends.

Closes #15284

12 months agoftp: move listen handling to socket filter
Stefan Eissing [Thu, 5 Sep 2024 14:41:53 +0000 (16:41 +0200)] 
ftp: move listen handling to socket filter

Move the listen/accept handling of the FTP active data connection
into the socket filter and monitor 'connected' status of that as
with passive connections - more or less.

The advantage is that the socket filter now reports being connected
only when the server has actually called and accept() has been done.
This enables to bootstrap the filter chain on the data connection
just like any other. A require SSL filter can then be added right
at the start and does not need to be patched in later.

Still, the active connection keeps on needing special handling in
ftp.c as the control connection needs to be monitored while waiting
as the server might send error responses this way. So, things did
not turn out quite as squeaky clean as hoped for, but still seems
better to do that way.

Closes #14798

12 months agombedTLS: fix handling of TLSv1.3 sessions
Stefan Eissing [Thu, 10 Oct 2024 10:47:41 +0000 (12:47 +0200)] 
mbedTLS: fix handling of TLSv1.3 sessions

For TLSv1.3, if supported, observer special return code to retrieve
newly arrived session from mbedTLS.

Adjust test expectations now that TLSv1.3 session resumption works in
mbedTLS >= 3.6.0.

Based on #14135 by @ad-chaos
Closes #15245

12 months agowolfSSL: fix handling of TLSv1.3 sessions
Stefan Eissing [Thu, 10 Oct 2024 09:44:39 +0000 (11:44 +0200)] 
wolfSSL: fix handling of TLSv1.3 sessions

Register a callback to get notified of new SSL sessions by wolfSSL.
Remove the explicit session retrieval after handshake, since this does
not work for TLSv1.3.

Adjust test expectations now that TLSv1.3 session resumption works
in wolfSSL.

Closes #15243

12 months agocurl-rustls.m4: set linker flags to allow rustls build on macos
Stefan Eissing [Mon, 7 Oct 2024 13:58:26 +0000 (15:58 +0200)] 
curl-rustls.m4: set linker flags to allow rustls build on macos

Assisted-by: Viktor Szakats
Closes #15175

12 months agosmb: do not redefine `getpid` on Windows
Viktor Szakats [Fri, 11 Oct 2024 08:53:32 +0000 (10:53 +0200)] 
smb: do not redefine `getpid` on Windows

Replace with namespaced local macro `Curl_getpid()`.

Redefining symbols can backfire if that symbol is used in system
headers, especially with unity build. We haven't seen a fallout in CI
or supported envs, but do it anyway for good measure.

Bug report: https://datagirl.xyz/posts/wolfssl_curl_w2k.html
Closes #15263

12 months agoGHA: optimize test prereq steps
Viktor Szakats [Sat, 12 Oct 2024 08:45:31 +0000 (10:45 +0200)] 
GHA: optimize test prereq steps

- Linux: move test and pytest prereqs right before test run.
  - returns build phase results faster.
  - allows skipping steps for jobs that don't need them.
  - makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.

Closes #15275

12 months agopytest: include `buildinfo.txt` in the output
Viktor Szakats [Sat, 12 Oct 2024 21:56:38 +0000 (23:56 +0200)] 
pytest: include `buildinfo.txt` in the output

If present.

It aims to provide TextClutch the same build information that
`runtests.pl` already is providing.

Ref: https://testclutch.curl.se/static/reports/feature-matrix.html
Ref: #15256
Follow-up to 1fdea1684602a1ae2870c67b5f3e8fd34f63da95 #14802
Closes #15279

12 months agoGHA/windows: drop vcpkg workaround
Tal Regev [Sat, 5 Oct 2024 02:22:05 +0000 (05:22 +0300)] 
GHA/windows: drop vcpkg workaround

Reverts 841f42150ae7b9f903cc5d266cec60a8090c0f27 #15133
Closes #15217

12 months agocmake: tidy-ups and rebase fixups
Viktor Szakats [Fri, 11 Oct 2024 16:36:43 +0000 (18:36 +0200)] 
cmake: tidy-ups and rebase fixups

- limit `SIZEOF_SA_FAMILY_T` detection to non-Windows.
- make sure `sys/socket.h` exists before detecting `SIZEOF_SA_FAMILY_T`.
- limit `mach_absolute_time()` detection to `APPLE`. Drop from Windows
  pre-cache.
- skip `HAVE_LIBSOCKET` detection for Windows, drop pre-cached value.
- drop redundant pre-cached `HAVE_LIBZ` for Windows.
- `curl_required_libpaths()`: stop accepting multiple arguments.
  To prepare for `CMAKE_REQUIRED_LINK_DIRECTORIES` support.
  Follow-up to 7bab201abe3915a0167c002f9308950cb8a06e4b #15193
- GSS: fix recent rebase mistakes:
  - fix variable name.
  - do not add a header twice.
  Follow-up to 91d451b48809f20415ba8627786f5d4f5aaf8bfe #15157
- GSS: quote a variable.

Closes #15271

12 months agotests: allow pytests to run in out-of-tree builds
Dan Fandrich [Thu, 10 Oct 2024 17:44:56 +0000 (10:44 -0700)] 
tests: allow pytests to run in out-of-tree builds

Some of the files it needs are in the build directory but the code did
not make a distinction.

Closes #15257

12 months agoGHA/linux: mbedTLS 3.6.1
Viktor Szakats [Sat, 12 Oct 2024 06:27:34 +0000 (08:27 +0200)] 
GHA/linux: mbedTLS 3.6.1

Closes #15274

12 months agoCI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact
renovate[bot] [Tue, 8 Oct 2024 20:20:34 +0000 (20:20 +0000)] 
CI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact

- update rojopolis/spellcheck-github-actions digest to 7ff888c
- update actions/checkout digest to eef6144
- update actions/upload-artifact digest to b4b15b8

Closes #15207
Closes #15184
Closes #15187

12 months agoCI: bump github/codeql-action, vmactions/omnios-vm and actions/cache
dependabot[bot] [Mon, 7 Oct 2024 14:54:53 +0000 (14:54 +0000)] 
CI: bump github/codeql-action, vmactions/omnios-vm and actions/cache

- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1

Closes #15178
Closes #15179
Closes #15244

12 months agognutls: use session cache for QUIC
Stefan Eissing [Fri, 11 Oct 2024 11:09:51 +0000 (13:09 +0200)] 
gnutls: use session cache for QUIC

Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.

Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.

Closes #15265

12 months agotool_xattr: create the user.creator xattr attribute
Dan Fandrich [Wed, 9 Oct 2024 23:47:20 +0000 (16:47 -0700)] 
tool_xattr: create the user.creator xattr attribute

This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.

Closes #15237

12 months agocmake: apply `WIN32_LEAN_AND_MEAN` to all feature checks
Viktor Szakats [Thu, 10 Oct 2024 22:38:43 +0000 (00:38 +0200)] 
cmake: apply `WIN32_LEAN_AND_MEAN` to all feature checks

Enable `WIN32_LEAN_AND_MEAN` for all feature detections on Windows.

(Also drop it from individual detections.)

Cherry-picked from #15164

12 months agocmake: untangle feature detection interdependencies
Viktor Szakats [Sat, 5 Oct 2024 23:01:22 +0000 (01:01 +0200)] 
cmake: untangle feature detection interdependencies

- reduce `check_include_file_concat()` use to those headers that either
  depend on a previously detected header, or another header or symbol
  detection depend on it.

- replace `check_symbol_exists()` with `check_function_exists()` for
  functions that are detected with `AC_CHECK_FUNCS()` in `./configure`.
  This makes `setmode()` no longer be detected with MSYS, syncing
  this with `./configure`. Instead `_setmode()` is used now also in
  CMake MSYS builds. This is consistent with Cygwin builds also.

- add comment about which header/symbol detection depends on what
  header. Based on `./configure` mainly.

- form `CURL_TEST_DEFINES` manually, and include only those macros which
  are actually used in `CMake/CurlTests.c`.

- change `curl_internal_test()` to use `CMAKE_REQUIRED_DEFINITIONS`,
  instead of `CMAKE_REQUIRED_FLAGS` to simplify the logic, and to allow
  dropping the latter macro completely.

- drop `windows.h` from header and symbol checks.

- `./configure`: add comment about whether `netinet/in6.h`, `sys/un.h`
  are indeed meant to be included for all detections. There is a chance
  they were added there by accident.

Detection resuls were cross-checked between
436bbbe7abebf0ee3a2b0bfb3ec5db7ce8c8db4c (master) and
48ff4694e608ccfdedf7ce5bab2b96d6b2c23cda (this PR), for CI GHA Linux,
Linux HTTP/3, non-native, macOS and Windows jobs.

Closes #15164

12 months agoci: dump `curl_config.h` to log in all jobs
Viktor Szakats [Fri, 11 Oct 2024 11:40:48 +0000 (13:40 +0200)] 
ci: dump `curl_config.h` to log in all jobs

Also:
- GHA/windows: merge full and brief dump into a single job step.
- fix shellcheck warning 'useless cat'.

Closes #15266

12 months agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 11 Oct 2024 12:52:41 +0000 (14:52 +0200)] 
RELEASE-NOTES: synced

12 months agolibssh2: split the statemachine function into smaller sub functions
Daniel Stenberg [Thu, 10 Oct 2024 12:22:09 +0000 (14:22 +0200)] 
libssh2: split the statemachine function into smaller sub functions

A giant function is hard to read.

Closes #15250

12 months agonetrc: cache the netrc file in memory
Daniel Stenberg [Thu, 10 Oct 2024 16:08:07 +0000 (18:08 +0200)] 
netrc: cache the netrc file in memory

So that on redirects etc it does not reread the file but just parses it
again.

Reported-by: Pierre-Etienne Meunier
Fixes #15248
Closes #15259

12 months agoTLS: TLSv1.3 earlydata support for curl
Stefan Eissing [Wed, 9 Oct 2024 12:46:32 +0000 (14:46 +0200)] 
TLS: TLSv1.3 earlydata support for curl

Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.

- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
  bytes sent and accepted/rejected by the server.

Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
  that ALPN protocol for negoptiation with the server. When the
  sessions ALPN does not match the connections ALPN, earlydata
  will not be enabled.
- Check that the server selected the correct ALPN protocol for
  an earlydata connect. If the server does not confirm or reports
  something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
  if not connect-only.

Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature

Assisted-by: ad-chaos on github
Closes #15211

12 months agomulti: convert Curl_follow to static multi_follow
Daniel Stenberg [Fri, 11 Oct 2024 07:20:37 +0000 (09:20 +0200)] 
multi: convert Curl_follow to static multi_follow

Moved over from transfer.c because it is only used in multi.c

Closes #15260

12 months agocookie: overhaul and cleanup
Daniel Stenberg [Thu, 10 Oct 2024 08:08:15 +0000 (10:08 +0200)] 
cookie: overhaul and cleanup

- split the huge Curl_cookie_add() into several smaller static functions

- switch to using the common llist instead of custom linked list

- use less memory for *getlist()

- use bitfields for flags in the Cookie struct

- avoid the copy for date parsing

- more consistent variable naming

Closes #15247

12 months agocmake: replace `check_include_file_concat()` for LDAP and GSS detection
Viktor Szakats [Sat, 5 Oct 2024 00:12:13 +0000 (02:12 +0200)] 
cmake: replace `check_include_file_concat()` for LDAP and GSS detection

Replace `check_include_file_concat()` with `check_include_file()` in
GSS/LDAP detection to avoid these headers spilling into subsequent
feature checks.

- For LDAP, reverse detection order to match with `./configure`.
  Though, in current LDAP packages `ldap.h` does include `lber.h`.

- For GSS, align header detection logic with `./configure`, where
  `gssapi/gssapi_generic.h` might require `gssapi/gssapi.h`, and
  `gssapi/gssapi_krb5.h` might require both.

Ref: #436
Closes #15157

12 months agocmake: allow manual configuration for LDAP
Viktor Szakats [Sat, 5 Oct 2024 00:01:21 +0000 (02:01 +0200)] 
cmake: allow manual configuration for LDAP

Via these configuration values:
- `LDAP_LIBRARY`
- `LDAP_LBER_LIBRARY`
- `LDAP_INCLUDE_DIR`

Following the naming scheme used in `Find` modules.

Cherry-picked from #15157
Closes #15255

12 months agocmake: add comments to feature check options applied globally
Viktor Szakats [Sat, 5 Oct 2024 11:59:28 +0000 (13:59 +0200)] 
cmake: add comments to feature check options applied globally

Add comments saying when we want values set in feature check option
variables to apply to all feature checks, globally. These are currently:
`ws2_32` and `socket` libraries, and `-D_WIN32_WINNT=` macro.

Also use `list(APPEND ...)` for the libraries to avoid overwriting
potentially existing values.

Cherry-picked from #15157
Closes #15253

12 months agocmake: stop adding dependency headers to global `CMAKE_REQUIRED_INCLUDES`
Viktor Szakats [Sat, 5 Oct 2024 12:01:49 +0000 (14:01 +0200)] 
cmake: stop adding dependency headers to global `CMAKE_REQUIRED_INCLUDES`

It was done for `zlib`, `brotli`, `libpsl`, `libssh2`, `wolfssh`
(a copy-paste case for `wolfssh`).

Feature detections should not rely by default on dependency headers.
There is no evidence they do now. If it becomes necessary, headers
should added for the duration of the feature check.

Ref: 118977f19d2d7e842bdf4424c7cfbc5991781412
Cherry-picked from #15157
Closes #15252

12 months agocmake: use `cmake_push_check_state()` around feature checks
Viktor Szakats [Fri, 4 Oct 2024 23:12:44 +0000 (01:12 +0200)] 
cmake: use `cmake_push_check_state()` around feature checks

Enclose
`CMAKE_EXTRA_INCLUDE_FILES`,
`CMAKE_REQUIRED_DEFINITIONS`,
`CMAKE_REQUIRED_FLAGS`,
`CMAKE_REQUIRED_INCLUDES`,
`CMAKE_REQUIRED_LIBRARIES`,
`CMAKE_REQUIRED_LINK_OPTIONS`,
settings within `cmake_push_check_state()`/`cmake_pop_check_state()`
calls. It prevents spilling them into other feature checks. It also
replaces manual resets found in some places (which can have
the undesired side-effect of destroying values meant for global use.)

Cherry-picked from #15157
Closes #15251

12 months agoGHA: drop `--parallel` option for CMake + Ninja jobs
Viktor Szakats [Thu, 10 Oct 2024 12:53:50 +0000 (14:53 +0200)] 
GHA: drop `--parallel` option for CMake + Ninja jobs

It's managed by Ninja automatically.

Closes #15249

12 months agocmake: add native `pkg-config` detection for mbedTLS, MSH3, Quiche, Rustls, wolfSSL
Viktor Szakats [Tue, 8 Oct 2024 10:32:52 +0000 (12:32 +0200)] 
cmake: add native `pkg-config` detection for mbedTLS, MSH3, Quiche, Rustls, wolfSSL

Also:
- detect and add required system libraries for Rustls on macOS and
  non-Windows.
- add Linux CMake jobs for the touched dependencies.
  Caveats:
  - MSH3 generates a broken `libmsh3.pc`, so needs manual config.
    Upstream PR: https://github.com/nibanks/msh3/pull/225
  - Rustls `.pc` file missing, so needs manual config.

An internal change worthy of mention is that we are using the lib path
and name information returned by `pkg-config` as-is. Meaning the libname
doesn't include the full path, like it's usual with native cmake
detection. The path comes separately and needs to be rolled separately.
For this we add it to targets via `link_directories()`. We also keep tab
of them in `CURL_LIBDIRS` and use that in `libcurl.pc`. Feature checks
also need to receive these paths. CMake doesn't offer
a `CMAKE_REQUIRED_*` variable for this purpose, only
a `CMAKE_REQUIRED_LINK_OPTIONS` accepting raw linker flags. Add a macro
to convert a list of paths to linker options to solve it. wolfSSL
requires this for now.

Closes #15193

12 months agocmake: tidy up detection C code
Viktor Szakats [Tue, 8 Oct 2024 11:16:16 +0000 (13:16 +0200)] 
cmake: tidy up detection C code

Closes #15195

12 months agoGHA/linux: skip installing rust if rustls is in cache
Viktor Szakats [Thu, 10 Oct 2024 10:48:53 +0000 (12:48 +0200)] 
GHA/linux: skip installing rust if rustls is in cache

Rust remains installed for the Hyper job, because the rustls cache flag
is always false in that case.

Closes #15246

12 months agoGHA/linux, http3-linux: add CMake support, sync steps, other improvements
Viktor Szakats [Thu, 10 Oct 2024 09:21:09 +0000 (11:21 +0200)] 
GHA/linux, http3-linux: add CMake support, sync steps, other improvements

- use shallow clone for submodules.
- reduce total job timeout from 90/60 -> 45 minutes.
- use `$HOME` instead of literal.
- http3-linux: sync step yaml order with linux.yml.
- http3-linux: add cmake + ninja support like in linux.yml.
- http3-linux: dump confgure log, test config, curl -V like in linux.yml.
- http3-linux: skip restoring gnutls and wolfssl when not used.
- dump `curl_config.h`.
- fold a long line.

Closes #15242

12 months agoGHA/mac: simplify detecting SDK version bound to GCC
Viktor Szakats [Thu, 10 Oct 2024 09:01:35 +0000 (11:01 +0200)] 
GHA/mac: simplify detecting SDK version bound to GCC

Previous method extracted this information from the "hack layer"
headers' copyright comments.

It turns out that the same information can be obtained by querying
the default SDK root via a GCC option.

According to this test run, they do match:
https://github.com/curl/curl/actions/runs/11264283159/job/31323963221?pr=15193

Closes #15241

12 months agoGHA/linux: fix mbedTLS cmake build
Viktor Szakats [Thu, 10 Oct 2024 00:22:29 +0000 (02:22 +0200)] 
GHA/linux: fix mbedTLS cmake build

CMake builds mbedTLS in Debug mode by default, which was the reason
for these consistent test failures:
```
FAIL 1631: 'FTP through HTTPS-proxy' FTP, HTTPS-proxy
FAIL 1632: 'FTP through HTTPS-proxy, with connection reuse' FTP, HTTPS-proxy
```
Sometimes also:
```
FAIL 303: 'HTTPS with 8 secs timeout' HTTPS, HTTP GET, timeout, FAILURE
```
https://github.com/curl/curl/actions/runs/11260616621/job/31313234198

Fix it by building in `RelWithDebInfo` mode, matching the bare
`Makefile` builds used earlier. (`Release` mode also works.)

Cache sizes:
- Makefile: 10MB
- CMake Release: 1MB
- CMake RelWithDebInfo: 2.5MB

Ref: #15215
Follow-up to e377c917664241d8cccf67316b96d59a280ad8e4 #15208

Closes #15238

12 months agopackages/OS400/curlmain: remove the strncpy calls
Daniel Stenberg [Wed, 9 Oct 2024 13:59:38 +0000 (15:59 +0200)] 
packages/OS400/curlmain: remove the strncpy calls

Closes #15214

12 months agotests/server/util.c: remove use of strncpy
Daniel Stenberg [Wed, 9 Oct 2024 13:56:28 +0000 (15:56 +0200)] 
tests/server/util.c: remove use of strncpy

... and ban the function in code in this directory.

Closes #15213

12 months agotool_getparam: replace two uses of strncpy(), ban strncpy
Daniel Stenberg [Wed, 9 Oct 2024 13:45:32 +0000 (15:45 +0200)] 
tool_getparam: replace two uses of strncpy(), ban strncpy

strncpy() is rarely the right choice

Closes #15212

12 months agotests: 780 - 783, new HSTS tests
Daniel Stenberg [Wed, 9 Oct 2024 09:27:29 +0000 (11:27 +0200)] 
tests: 780 - 783, new HSTS tests

test780: verify updated HSTS data in response header

test781: HSTS update expiry, with parent includeSubDomains domain
present

test782: HSTS update expiry, with two includeSubDomains domains present

test783: HSTS update expiry, removing includesubdomains in update

12 months agohsts: improve subdomain handling
Daniel Stenberg [Wed, 9 Oct 2024 08:04:35 +0000 (10:04 +0200)] 
hsts: improve subdomain handling

- on load, only replace existing HSTS entries if there is a full host
  match

- on matching, prefer a full host match and secondary the longest tail
  subdomain match

Closes #15210

12 months agomulti: make curl_multi_cleanup invalidate magic latter
Emanuel Komínek [Tue, 8 Oct 2024 16:35:13 +0000 (18:35 +0200)] 
multi: make curl_multi_cleanup invalidate magic latter

When a multi handle is being cleaned up, it can still cause user
callbacks to be fired. Notably Curl_cpool_destroy calls socket_callback
on all pooled connections. It's still possible for the callback to call
curl_multi_assign leading to an assert.

This commit moves clearing of a multi handle magic to a point where the
multi handle stops being a valid object.

Fixes #15201
Closes #15206

12 months agowolfssl: use old version API without openssl extra
Colton Willey [Tue, 8 Oct 2024 19:24:42 +0000 (12:24 -0700)] 
wolfssl: use old version API without openssl extra

Closes #15205

13 months agoGHA: add Linux and macOS mbedTLS jobs, fix issue
Viktor Szakats [Tue, 8 Oct 2024 20:41:18 +0000 (22:41 +0200)] 
GHA: add Linux and macOS mbedTLS jobs, fix issue

- update mbedTLS repo URL.
- switch local mbedTLS build to use CMake, and Ninja.
  CMake build is required to create and install mbedTLS `pkg-config`
  files. (as of v3.6.1)
  `-DCMAKE_POSITION_INDEPENDENT_CODE=ON` required to avoid this error
  when linking mbedtls to `libcurl.so`:
  ```
  /usr/bin/ld: /home/runner/mbedtls/lib/libmbedcrypto.a(cipher.c.o): warning: relocation against `mbedtls_cipher_base_lookup_table' in read-only section `.text'
  /usr/bin/ld: /home/runner/mbedtls/lib/libmbedtls.a(ssl_tls.c.o): relocation R_X86_64_PC32 against symbol `mbedtls_x509_crt_profile_suiteb' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: final link failed: bad value
  ```
  Ref: https://github.com/curl/curl/actions/runs/11245069259/job/31264386723#step:40:43
- make local mbedTLS build 10x smaller by omitting programs and tests.
- GHA/linux: fix cmake warning by adding `-B .` option.
- GHA/linux: add build-only cmake job for packaged mbedTLS (2.x).
- fix compiler warning when building with mbedTLS 2.x:
  ```
  /home/runner/work/curl/curl/lib/vtls/mbedtls.c:344:1: error: ‘mbed_cipher_suite_get_str’ defined but not used [-Werror=unused-function]
    344 | mbed_cipher_suite_get_str(uint16_t id, char *buf, size_t buf_size,
        | ^~~~~~~~~~~~~~~~~~~~~~~~~
  ```
  Ref: https://github.com/curl/curl/actions/runs/11244999065/job/31264168295#step:40:50

Also in preparation for #15193
Closes #15208

13 months agoGHA/windows: drop vcpkg shiftmedia-gnutls, replace with mbedtls
Viktor Szakats [Tue, 8 Oct 2024 18:18:56 +0000 (20:18 +0200)] 
GHA/windows: drop vcpkg shiftmedia-gnutls, replace with mbedtls

GnuTLS vcpkg package broken again with the latest runner image update:
https://github.com/curl/curl/actions/runs/11240011311/job/31248406051?pr=15203#step:5:137

Previous breakage: 09b21e4755b4cc8ff31e9126aa9caf464988551e #14864
Closes #15204

13 months agoINSTALL.md: fix a typo that slipped in to RISC OS
Dan Fandrich [Tue, 8 Oct 2024 16:55:49 +0000 (09:55 -0700)] 
INSTALL.md: fix a typo that slipped in to RISC OS

Also, illumos is spelled with lower case.

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 8 Oct 2024 14:50:57 +0000 (16:50 +0200)] 
RELEASE-NOTES: synced

13 months agojson.md: cli-option `--json` is an alias of `--data-binary`
Sebastian Walz [Tue, 8 Oct 2024 12:19:36 +0000 (14:19 +0200)] 
json.md: cli-option `--json` is an alias of `--data-binary`

Closes #15197

13 months agohttp_aws_sigv4: avoid local buffer and strcpy
Daniel Stenberg [Tue, 8 Oct 2024 13:28:01 +0000 (15:28 +0200)] 
http_aws_sigv4: avoid local buffer and strcpy

This avoids the buffer, the copy and the previous host name length
limit.

Closes #15200

13 months agotftp: avoid two memcpy/strcpy
Daniel Stenberg [Tue, 8 Oct 2024 13:19:31 +0000 (15:19 +0200)] 
tftp: avoid two memcpy/strcpy

Closes #15199

13 months agotelnet: avoid two strcpy() by pointing to the strings instead
Daniel Stenberg [Tue, 8 Oct 2024 13:18:02 +0000 (15:18 +0200)] 
telnet: avoid two strcpy() by pointing to the strings instead

Closes #15198

13 months agosetopt: avoid superfluous length checks before strcmp()
Daniel Stenberg [Tue, 8 Oct 2024 10:42:05 +0000 (12:42 +0200)] 
setopt: avoid superfluous length checks before strcmp()

Simplify the checks

Closes #15192

13 months agobearssl: avoid strpcy() when generating TLS version log message
Daniel Stenberg [Tue, 8 Oct 2024 11:04:43 +0000 (13:04 +0200)] 
bearssl: avoid strpcy() when generating TLS version log message

Closes #15194

13 months agosmb: replace use of strcpy() with snprintf()
Daniel Stenberg [Tue, 8 Oct 2024 11:25:42 +0000 (13:25 +0200)] 
smb: replace use of strcpy() with snprintf()

Closes #15196

13 months agoaltsvc: avoid using local buffer and memcpy
Daniel Stenberg [Tue, 8 Oct 2024 09:37:05 +0000 (11:37 +0200)] 
altsvc: avoid using local buffer and memcpy

When parsing alt-svc headers. Store and use the length of the name
separately.

Closes #15191

13 months agohsts: avoid the local buffer and memcpy on lookup
Daniel Stenberg [Tue, 8 Oct 2024 09:20:40 +0000 (11:20 +0200)] 
hsts: avoid the local buffer and memcpy on lookup

Closes #15190

13 months agoconfigure: add GSS to `libcurl.pc` `Depends:`
Viktor Szakats [Mon, 7 Oct 2024 14:13:04 +0000 (16:13 +0200)] 
configure: add GSS to `libcurl.pc` `Depends:`

GSS was the last (known) missing dependency missing from `libcurl.pc`.

Closes #15177

13 months agocmake: detect GNU GSS
Viktor Szakats [Mon, 7 Oct 2024 14:13:32 +0000 (16:13 +0200)] 
cmake: detect GNU GSS

Fix to set `HAVE_GSSGNU` when GNU GSS is detected.

Also set the appropriate `pkg-config` dependency and do version
detection for the GNU GSS flavour.

Tested with `pkg-config` and partly tested without. The latter case
picks up everything else but, in my env. This is likely not the last
word to implement this detection correctly for all build-cases.

GNU GSS doesn't seem to have a Homebrew formula and building
it locally needs manual tweaks to make finish successfully.

Also move a MIT-specific header detection into to MIT-specific `if`
branch.

Closes #15176

13 months agoCURLOPT_APPEND.md: goes for SFTP as well
Daniel Stenberg [Mon, 7 Oct 2024 12:30:13 +0000 (14:30 +0200)] 
CURLOPT_APPEND.md: goes for SFTP as well

Closes #15181

13 months agoconncache: find bundle again in case it is removed
Daniel Stenberg [Tue, 8 Oct 2024 06:00:55 +0000 (08:00 +0200)] 
conncache: find bundle again in case it is removed

When the pool is cleaned up due to host limits, the bundle may be
cleaned up as well making the old pointer invalid.

Fixes #15185
Reported-by: Moritz Knüsel
Closes #15186

13 months agotest1915: remove wrong comment
Daniel Stenberg [Tue, 8 Oct 2024 07:35:56 +0000 (09:35 +0200)] 
test1915: remove wrong comment

13 months agosetopt: use a single function for HTTPAUTH and PROXYAUTH
Daniel Stenberg [Mon, 7 Oct 2024 22:35:28 +0000 (00:35 +0200)] 
setopt: use a single function for HTTPAUTH and PROXYAUTH

Avoid duplicated almost-the-same code.

Closes #15182

13 months agocmake: do not propagate unused `HAVE_GSSAPI_GSSAPI_KRB5_H` to C
Viktor Szakats [Mon, 7 Oct 2024 10:56:04 +0000 (12:56 +0200)] 
cmake: do not propagate unused `HAVE_GSSAPI_GSSAPI_KRB5_H` to C

Closes #15174

13 months agocmake: detect `HAVE_NETINET_IN6_H`, `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`
Viktor Szakats [Sun, 6 Oct 2024 00:54:25 +0000 (02:54 +0200)] 
cmake: detect `HAVE_NETINET_IN6_H`, `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`

To sync with `./configure`.

- `HAVE_NETINET_IN6_H` is
  needed by HPE NonStop NSE and NSX systems.
  Follow-up to 76ebd54175bad02b29769d797adf72fdf3df119f #2155

- `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`
  are for AmigaOS.
  (Note: `./configure` tries to detect these for all targets, cmake does
  it only for AmigaOS, to not inflate configure time.)

Closes #15172

13 months agoconfigure: drop unused bare `socket.h` detection
Viktor Szakats [Mon, 7 Oct 2024 02:07:30 +0000 (04:07 +0200)] 
configure: drop unused bare `socket.h` detection

Added in 37eba37019388f767f5de67b4071641044d7b026 (2009-06-17) to help
detecting socket functions.

But, this `socket.h` isn't used in the source code since
90dd1fc66401d5bb7814f4edeb16a06c925b1f1e #8288 (2022-01-16).

Closes #15173

13 months agosws: fix unused static function with `TCP_NODELAY` undefined
Viktor Szakats [Sun, 6 Oct 2024 09:37:38 +0000 (11:37 +0200)] 
sws: fix unused static function with `TCP_NODELAY` undefined

Closes #15171

13 months agoconfigure: drop duplicate feature checks for `poll()`, `if_nametoindex()`
Viktor Szakats [Sun, 6 Oct 2024 00:26:48 +0000 (02:26 +0200)] 
configure: drop duplicate feature checks for `poll()`, `if_nametoindex()`

Before this patch they were detected via manual methods, then with
`AC_CHECK_FUNCS()`.

Delete the manual checks and keep the latter.

Also delete `CURL_INCLUDES_POLL()` which is no longer used after
the above.

Closes #15170

13 months agobuild: detect and use `_setmode()` with Cygwin/MSYS, also use on Windows
Viktor Szakats [Sun, 6 Oct 2024 13:41:48 +0000 (15:41 +0200)] 
build: detect and use `_setmode()` with Cygwin/MSYS, also use on Windows

Before this patch `setmode()` was not detected with Cygwin/MSYS, because
it's a macro, not a function, and detection is looking for a function.

Switching to symbol detection doesn't work because it mis-detects it on
BSD systems which features a function with the same name but different
functionality and arguments.

Fix it by looking for a `_setmode()` function on Cygwin/MSYS, and use it
if available.

`_setmode()` is recommended over `setmode()` by Windows documentation so
use that on Windows too. It seems to be available on all supported
compilers, so omit detection.

https://learn.microsoft.com/cpp/c-runtime-library/reference/posix-setmode
https://learn.microsoft.com/cpp/c-runtime-library/reference/setmode

Officially Windows requires argument `_O_BINARY` with an underscore.
`O_BINARY` is also supported but bound to conditions. Continue to use it
for simplicity. Cygwin supports `O_BINARY` (no underscore).

Closes #15169

13 months agoech: spelling, whitespace, say `--ech` default config
Viktor Szakats [Sat, 5 Oct 2024 21:29:22 +0000 (23:29 +0200)] 
ech: spelling, whitespace, say `--ech` default config

Closes #15167

13 months agoGHA/macos: comment spelling and clarity
Viktor Szakats [Sun, 6 Oct 2024 11:02:12 +0000 (13:02 +0200)] 
GHA/macos: comment spelling and clarity

Closes #15166

13 months agobuild: add `ldap` to `libcurl.pc` `Requires:`
Viktor Szakats [Fri, 4 Oct 2024 23:57:33 +0000 (01:57 +0200)] 
build: add `ldap` to `libcurl.pc` `Requires:`

Closes #15163

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 7 Oct 2024 06:33:32 +0000 (08:33 +0200)] 
RELEASE-NOTES: synced

13 months agoINSTALL-CMAKE.md: mention focus on shared libraries
Daniel Stenberg [Sun, 6 Oct 2024 21:13:29 +0000 (23:13 +0200)] 
INSTALL-CMAKE.md: mention focus on shared libraries

Match what the INSTALL.md document says for configure builds.

Reported-by: Nemos2024 on github
Fixes #14845
Closes #15168

13 months agoci: update dependency ngtcp2/nghttp3 to v1.6.0
renovate[bot] [Sat, 5 Oct 2024 18:38:57 +0000 (18:38 +0000)] 
ci: update dependency ngtcp2/nghttp3 to v1.6.0

Closes #15161

13 months agoci: update dependency ngtcp2/ngtcp2 to v1.8.0
renovate[bot] [Sat, 5 Oct 2024 18:39:02 +0000 (18:39 +0000)] 
ci: update dependency ngtcp2/ngtcp2 to v1.8.0

Closes #15162

13 months agoGHA/non-native: fix OmniOS job to fail on tests
Viktor Szakats [Sat, 5 Oct 2024 15:23:17 +0000 (17:23 +0200)] 
GHA/non-native: fix OmniOS job to fail on tests

Before this patch a failed test did not result in a failed CI job.

Closes #15160

13 months agocmake: use OpenSSL for LDAP detection only if available
Viktor Szakats [Fri, 4 Oct 2024 19:24:54 +0000 (21:24 +0200)] 
cmake: use OpenSSL for LDAP detection only if available

Also unset `CMAKE_REQUIRED_LIBRARIES` in two error branches.

Reported-by: Vollstrecker on github
Fixes #15077
Closes #15152

13 months agowarnless: remove curlx_sktosi and curlx_sitosk
Daniel Stenberg [Fri, 4 Oct 2024 20:32:04 +0000 (22:32 +0200)] 
warnless: remove curlx_sktosi and curlx_sitosk

They were only used by test server code. Use plain old typecasts there.

Closes #15153

13 months agotests: enable additional ruff Python lint options
Dan Fandrich [Thu, 26 Sep 2024 21:31:39 +0000 (14:31 -0700)] 
tests: enable additional ruff Python lint options

These all seem reasonable to enable for this code.

13 months agoCI: run pytype and ruff on Python code
Dan Fandrich [Thu, 26 Sep 2024 18:33:58 +0000 (11:33 -0700)] 
CI: run pytype and ruff on Python code

These ensure a more consistent style and can find some errors statically
that would otherwise only be seen at run-time. Also, bump the Ubuntu
version of some other checks to get newer versions of some linters.

Closes #15067

13 months agotests: change Python code style to pass ruff checks
Dan Fandrich [Thu, 26 Sep 2024 19:40:43 +0000 (12:40 -0700)] 
tests: change Python code style to pass ruff checks

Most of the changes consisted of removing unused imports and unnecessary
f-strings.

13 months agotests: fix some Python typing issues
Dan Fandrich [Thu, 26 Sep 2024 18:58:57 +0000 (11:58 -0700)] 
tests: fix some Python typing issues

These otherwise raise errors in pytype. A few problematic methods
weren't being used and are deleted.

13 months agoCURLOPT_HEADERFUNCTION.md: do not modify the passed in buffer
Daniel Stenberg [Fri, 4 Oct 2024 11:53:20 +0000 (13:53 +0200)] 
CURLOPT_HEADERFUNCTION.md: do not modify the passed in buffer

Closes #15148

13 months agoasyn-ares: remove typecast, fix expire
Daniel Stenberg [Thu, 3 Oct 2024 21:53:10 +0000 (23:53 +0200)] 
asyn-ares: remove typecast, fix expire

- Use the appropriate variable type for the curlx_tvtoms() return code:
  timediff_t and remove the typecast.

- Simplify the function and avoid the odd expire adjustment that
  probably is a rest from ancient days when the expire function did not
  handle zero millisecond timeouts.

Closes #15145

13 months agocmake: add missed variable to comment [ci skip]
Viktor Szakats [Fri, 4 Oct 2024 00:10:00 +0000 (02:10 +0200)] 
cmake: add missed variable to comment [ci skip]

Follow-up to 9f56bb608ecfbb8978c6cb72a04d9e8b23162d82 #14681

13 months agotest1915: add tracing and connect timeout
Stefan Eissing [Tue, 1 Oct 2024 10:48:46 +0000 (12:48 +0200)] 
test1915: add tracing and connect timeout

Since we see Windows fails of 1915, add tracing and a connect timeout.
The test uses a port no one is supposed to listen on, but Windows has
this weird wait logic. So, set a short timeout.

Closes #15107

13 months agourlapi: normalize the IPv6 address
Daniel Stenberg [Thu, 3 Oct 2024 12:37:00 +0000 (14:37 +0200)] 
urlapi: normalize the IPv6 address

As the parsing and address "regeneration" are done anyway, we might as
well use the updated version in the result and thereby A) get a
normalized (and lower cased) version of the address and B) avoid a
strcpy().

Updated test 1560 to verify.

Closes #15143

13 months agotests/valgrind.supp: remove a travis suppression, add a Debian
Daniel Stenberg [Thu, 3 Oct 2024 12:32:54 +0000 (14:32 +0200)] 
tests/valgrind.supp: remove a travis suppression, add a Debian

We have not used Travis for years. The Debian one appears on my dev
machine since a while back.

Closes #15142

13 months agoopenssl quic: populate x509 store before handshake
Stefan Eissing [Thu, 3 Oct 2024 08:51:26 +0000 (10:51 +0200)] 
openssl quic: populate x509 store before handshake

Since OpenSSL does its own send/recv internally, we may miss the moment
to populate the x509 store right before the server response. Do it
instead before we start the handshake, at the loss of the time to set
this up.

Closes #15137

13 months agopytest: improve pytest_07_42a reliability
Stefan Eissing [Thu, 3 Oct 2024 09:10:29 +0000 (11:10 +0200)] 
pytest: improve pytest_07_42a reliability

Due to timings on paused response receive, the error code may vary due
to the location where it is detected that the server closed the transfer
prematurely.

Be more lenient in always allowing PARTIAL_FILE as ok.

Closes #15138

13 months agotest1515: add tracing and more debug info
Stefan Eissing [Thu, 3 Oct 2024 11:09:17 +0000 (13:09 +0200)] 
test1515: add tracing and more debug info

Test failed on some Windows CI and log files did not really
tell why. Add more output.

Closes #15140