]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
12 months agocookie: overhaul and cleanup
Daniel Stenberg [Thu, 10 Oct 2024 08:08:15 +0000 (10:08 +0200)] 
cookie: overhaul and cleanup

- split the huge Curl_cookie_add() into several smaller static functions

- switch to using the common llist instead of custom linked list

- use less memory for *getlist()

- use bitfields for flags in the Cookie struct

- avoid the copy for date parsing

- more consistent variable naming

Closes #15247

12 months agocmake: replace `check_include_file_concat()` for LDAP and GSS detection
Viktor Szakats [Sat, 5 Oct 2024 00:12:13 +0000 (02:12 +0200)] 
cmake: replace `check_include_file_concat()` for LDAP and GSS detection

Replace `check_include_file_concat()` with `check_include_file()` in
GSS/LDAP detection to avoid these headers spilling into subsequent
feature checks.

- For LDAP, reverse detection order to match with `./configure`.
  Though, in current LDAP packages `ldap.h` does include `lber.h`.

- For GSS, align header detection logic with `./configure`, where
  `gssapi/gssapi_generic.h` might require `gssapi/gssapi.h`, and
  `gssapi/gssapi_krb5.h` might require both.

Ref: #436
Closes #15157

12 months agocmake: allow manual configuration for LDAP
Viktor Szakats [Sat, 5 Oct 2024 00:01:21 +0000 (02:01 +0200)] 
cmake: allow manual configuration for LDAP

Via these configuration values:
- `LDAP_LIBRARY`
- `LDAP_LBER_LIBRARY`
- `LDAP_INCLUDE_DIR`

Following the naming scheme used in `Find` modules.

Cherry-picked from #15157
Closes #15255

12 months agocmake: add comments to feature check options applied globally
Viktor Szakats [Sat, 5 Oct 2024 11:59:28 +0000 (13:59 +0200)] 
cmake: add comments to feature check options applied globally

Add comments saying when we want values set in feature check option
variables to apply to all feature checks, globally. These are currently:
`ws2_32` and `socket` libraries, and `-D_WIN32_WINNT=` macro.

Also use `list(APPEND ...)` for the libraries to avoid overwriting
potentially existing values.

Cherry-picked from #15157
Closes #15253

12 months agocmake: stop adding dependency headers to global `CMAKE_REQUIRED_INCLUDES`
Viktor Szakats [Sat, 5 Oct 2024 12:01:49 +0000 (14:01 +0200)] 
cmake: stop adding dependency headers to global `CMAKE_REQUIRED_INCLUDES`

It was done for `zlib`, `brotli`, `libpsl`, `libssh2`, `wolfssh`
(a copy-paste case for `wolfssh`).

Feature detections should not rely by default on dependency headers.
There is no evidence they do now. If it becomes necessary, headers
should added for the duration of the feature check.

Ref: 118977f19d2d7e842bdf4424c7cfbc5991781412
Cherry-picked from #15157
Closes #15252

12 months agocmake: use `cmake_push_check_state()` around feature checks
Viktor Szakats [Fri, 4 Oct 2024 23:12:44 +0000 (01:12 +0200)] 
cmake: use `cmake_push_check_state()` around feature checks

Enclose
`CMAKE_EXTRA_INCLUDE_FILES`,
`CMAKE_REQUIRED_DEFINITIONS`,
`CMAKE_REQUIRED_FLAGS`,
`CMAKE_REQUIRED_INCLUDES`,
`CMAKE_REQUIRED_LIBRARIES`,
`CMAKE_REQUIRED_LINK_OPTIONS`,
settings within `cmake_push_check_state()`/`cmake_pop_check_state()`
calls. It prevents spilling them into other feature checks. It also
replaces manual resets found in some places (which can have
the undesired side-effect of destroying values meant for global use.)

Cherry-picked from #15157
Closes #15251

12 months agoGHA: drop `--parallel` option for CMake + Ninja jobs
Viktor Szakats [Thu, 10 Oct 2024 12:53:50 +0000 (14:53 +0200)] 
GHA: drop `--parallel` option for CMake + Ninja jobs

It's managed by Ninja automatically.

Closes #15249

12 months agocmake: add native `pkg-config` detection for mbedTLS, MSH3, Quiche, Rustls, wolfSSL
Viktor Szakats [Tue, 8 Oct 2024 10:32:52 +0000 (12:32 +0200)] 
cmake: add native `pkg-config` detection for mbedTLS, MSH3, Quiche, Rustls, wolfSSL

Also:
- detect and add required system libraries for Rustls on macOS and
  non-Windows.
- add Linux CMake jobs for the touched dependencies.
  Caveats:
  - MSH3 generates a broken `libmsh3.pc`, so needs manual config.
    Upstream PR: https://github.com/nibanks/msh3/pull/225
  - Rustls `.pc` file missing, so needs manual config.

An internal change worthy of mention is that we are using the lib path
and name information returned by `pkg-config` as-is. Meaning the libname
doesn't include the full path, like it's usual with native cmake
detection. The path comes separately and needs to be rolled separately.
For this we add it to targets via `link_directories()`. We also keep tab
of them in `CURL_LIBDIRS` and use that in `libcurl.pc`. Feature checks
also need to receive these paths. CMake doesn't offer
a `CMAKE_REQUIRED_*` variable for this purpose, only
a `CMAKE_REQUIRED_LINK_OPTIONS` accepting raw linker flags. Add a macro
to convert a list of paths to linker options to solve it. wolfSSL
requires this for now.

Closes #15193

12 months agocmake: tidy up detection C code
Viktor Szakats [Tue, 8 Oct 2024 11:16:16 +0000 (13:16 +0200)] 
cmake: tidy up detection C code

Closes #15195

12 months agoGHA/linux: skip installing rust if rustls is in cache
Viktor Szakats [Thu, 10 Oct 2024 10:48:53 +0000 (12:48 +0200)] 
GHA/linux: skip installing rust if rustls is in cache

Rust remains installed for the Hyper job, because the rustls cache flag
is always false in that case.

Closes #15246

12 months agoGHA/linux, http3-linux: add CMake support, sync steps, other improvements
Viktor Szakats [Thu, 10 Oct 2024 09:21:09 +0000 (11:21 +0200)] 
GHA/linux, http3-linux: add CMake support, sync steps, other improvements

- use shallow clone for submodules.
- reduce total job timeout from 90/60 -> 45 minutes.
- use `$HOME` instead of literal.
- http3-linux: sync step yaml order with linux.yml.
- http3-linux: add cmake + ninja support like in linux.yml.
- http3-linux: dump confgure log, test config, curl -V like in linux.yml.
- http3-linux: skip restoring gnutls and wolfssl when not used.
- dump `curl_config.h`.
- fold a long line.

Closes #15242

12 months agoGHA/mac: simplify detecting SDK version bound to GCC
Viktor Szakats [Thu, 10 Oct 2024 09:01:35 +0000 (11:01 +0200)] 
GHA/mac: simplify detecting SDK version bound to GCC

Previous method extracted this information from the "hack layer"
headers' copyright comments.

It turns out that the same information can be obtained by querying
the default SDK root via a GCC option.

According to this test run, they do match:
https://github.com/curl/curl/actions/runs/11264283159/job/31323963221?pr=15193

Closes #15241

12 months agoGHA/linux: fix mbedTLS cmake build
Viktor Szakats [Thu, 10 Oct 2024 00:22:29 +0000 (02:22 +0200)] 
GHA/linux: fix mbedTLS cmake build

CMake builds mbedTLS in Debug mode by default, which was the reason
for these consistent test failures:
```
FAIL 1631: 'FTP through HTTPS-proxy' FTP, HTTPS-proxy
FAIL 1632: 'FTP through HTTPS-proxy, with connection reuse' FTP, HTTPS-proxy
```
Sometimes also:
```
FAIL 303: 'HTTPS with 8 secs timeout' HTTPS, HTTP GET, timeout, FAILURE
```
https://github.com/curl/curl/actions/runs/11260616621/job/31313234198

Fix it by building in `RelWithDebInfo` mode, matching the bare
`Makefile` builds used earlier. (`Release` mode also works.)

Cache sizes:
- Makefile: 10MB
- CMake Release: 1MB
- CMake RelWithDebInfo: 2.5MB

Ref: #15215
Follow-up to e377c917664241d8cccf67316b96d59a280ad8e4 #15208

Closes #15238

12 months agopackages/OS400/curlmain: remove the strncpy calls
Daniel Stenberg [Wed, 9 Oct 2024 13:59:38 +0000 (15:59 +0200)] 
packages/OS400/curlmain: remove the strncpy calls

Closes #15214

12 months agotests/server/util.c: remove use of strncpy
Daniel Stenberg [Wed, 9 Oct 2024 13:56:28 +0000 (15:56 +0200)] 
tests/server/util.c: remove use of strncpy

... and ban the function in code in this directory.

Closes #15213

12 months agotool_getparam: replace two uses of strncpy(), ban strncpy
Daniel Stenberg [Wed, 9 Oct 2024 13:45:32 +0000 (15:45 +0200)] 
tool_getparam: replace two uses of strncpy(), ban strncpy

strncpy() is rarely the right choice

Closes #15212

12 months agotests: 780 - 783, new HSTS tests
Daniel Stenberg [Wed, 9 Oct 2024 09:27:29 +0000 (11:27 +0200)] 
tests: 780 - 783, new HSTS tests

test780: verify updated HSTS data in response header

test781: HSTS update expiry, with parent includeSubDomains domain
present

test782: HSTS update expiry, with two includeSubDomains domains present

test783: HSTS update expiry, removing includesubdomains in update

12 months agohsts: improve subdomain handling
Daniel Stenberg [Wed, 9 Oct 2024 08:04:35 +0000 (10:04 +0200)] 
hsts: improve subdomain handling

- on load, only replace existing HSTS entries if there is a full host
  match

- on matching, prefer a full host match and secondary the longest tail
  subdomain match

Closes #15210

12 months agomulti: make curl_multi_cleanup invalidate magic latter
Emanuel Komínek [Tue, 8 Oct 2024 16:35:13 +0000 (18:35 +0200)] 
multi: make curl_multi_cleanup invalidate magic latter

When a multi handle is being cleaned up, it can still cause user
callbacks to be fired. Notably Curl_cpool_destroy calls socket_callback
on all pooled connections. It's still possible for the callback to call
curl_multi_assign leading to an assert.

This commit moves clearing of a multi handle magic to a point where the
multi handle stops being a valid object.

Fixes #15201
Closes #15206

12 months agowolfssl: use old version API without openssl extra
Colton Willey [Tue, 8 Oct 2024 19:24:42 +0000 (12:24 -0700)] 
wolfssl: use old version API without openssl extra

Closes #15205

13 months agoGHA: add Linux and macOS mbedTLS jobs, fix issue
Viktor Szakats [Tue, 8 Oct 2024 20:41:18 +0000 (22:41 +0200)] 
GHA: add Linux and macOS mbedTLS jobs, fix issue

- update mbedTLS repo URL.
- switch local mbedTLS build to use CMake, and Ninja.
  CMake build is required to create and install mbedTLS `pkg-config`
  files. (as of v3.6.1)
  `-DCMAKE_POSITION_INDEPENDENT_CODE=ON` required to avoid this error
  when linking mbedtls to `libcurl.so`:
  ```
  /usr/bin/ld: /home/runner/mbedtls/lib/libmbedcrypto.a(cipher.c.o): warning: relocation against `mbedtls_cipher_base_lookup_table' in read-only section `.text'
  /usr/bin/ld: /home/runner/mbedtls/lib/libmbedtls.a(ssl_tls.c.o): relocation R_X86_64_PC32 against symbol `mbedtls_x509_crt_profile_suiteb' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: final link failed: bad value
  ```
  Ref: https://github.com/curl/curl/actions/runs/11245069259/job/31264386723#step:40:43
- make local mbedTLS build 10x smaller by omitting programs and tests.
- GHA/linux: fix cmake warning by adding `-B .` option.
- GHA/linux: add build-only cmake job for packaged mbedTLS (2.x).
- fix compiler warning when building with mbedTLS 2.x:
  ```
  /home/runner/work/curl/curl/lib/vtls/mbedtls.c:344:1: error: ‘mbed_cipher_suite_get_str’ defined but not used [-Werror=unused-function]
    344 | mbed_cipher_suite_get_str(uint16_t id, char *buf, size_t buf_size,
        | ^~~~~~~~~~~~~~~~~~~~~~~~~
  ```
  Ref: https://github.com/curl/curl/actions/runs/11244999065/job/31264168295#step:40:50

Also in preparation for #15193
Closes #15208

13 months agoGHA/windows: drop vcpkg shiftmedia-gnutls, replace with mbedtls
Viktor Szakats [Tue, 8 Oct 2024 18:18:56 +0000 (20:18 +0200)] 
GHA/windows: drop vcpkg shiftmedia-gnutls, replace with mbedtls

GnuTLS vcpkg package broken again with the latest runner image update:
https://github.com/curl/curl/actions/runs/11240011311/job/31248406051?pr=15203#step:5:137

Previous breakage: 09b21e4755b4cc8ff31e9126aa9caf464988551e #14864
Closes #15204

13 months agoINSTALL.md: fix a typo that slipped in to RISC OS
Dan Fandrich [Tue, 8 Oct 2024 16:55:49 +0000 (09:55 -0700)] 
INSTALL.md: fix a typo that slipped in to RISC OS

Also, illumos is spelled with lower case.

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 8 Oct 2024 14:50:57 +0000 (16:50 +0200)] 
RELEASE-NOTES: synced

13 months agojson.md: cli-option `--json` is an alias of `--data-binary`
Sebastian Walz [Tue, 8 Oct 2024 12:19:36 +0000 (14:19 +0200)] 
json.md: cli-option `--json` is an alias of `--data-binary`

Closes #15197

13 months agohttp_aws_sigv4: avoid local buffer and strcpy
Daniel Stenberg [Tue, 8 Oct 2024 13:28:01 +0000 (15:28 +0200)] 
http_aws_sigv4: avoid local buffer and strcpy

This avoids the buffer, the copy and the previous host name length
limit.

Closes #15200

13 months agotftp: avoid two memcpy/strcpy
Daniel Stenberg [Tue, 8 Oct 2024 13:19:31 +0000 (15:19 +0200)] 
tftp: avoid two memcpy/strcpy

Closes #15199

13 months agotelnet: avoid two strcpy() by pointing to the strings instead
Daniel Stenberg [Tue, 8 Oct 2024 13:18:02 +0000 (15:18 +0200)] 
telnet: avoid two strcpy() by pointing to the strings instead

Closes #15198

13 months agosetopt: avoid superfluous length checks before strcmp()
Daniel Stenberg [Tue, 8 Oct 2024 10:42:05 +0000 (12:42 +0200)] 
setopt: avoid superfluous length checks before strcmp()

Simplify the checks

Closes #15192

13 months agobearssl: avoid strpcy() when generating TLS version log message
Daniel Stenberg [Tue, 8 Oct 2024 11:04:43 +0000 (13:04 +0200)] 
bearssl: avoid strpcy() when generating TLS version log message

Closes #15194

13 months agosmb: replace use of strcpy() with snprintf()
Daniel Stenberg [Tue, 8 Oct 2024 11:25:42 +0000 (13:25 +0200)] 
smb: replace use of strcpy() with snprintf()

Closes #15196

13 months agoaltsvc: avoid using local buffer and memcpy
Daniel Stenberg [Tue, 8 Oct 2024 09:37:05 +0000 (11:37 +0200)] 
altsvc: avoid using local buffer and memcpy

When parsing alt-svc headers. Store and use the length of the name
separately.

Closes #15191

13 months agohsts: avoid the local buffer and memcpy on lookup
Daniel Stenberg [Tue, 8 Oct 2024 09:20:40 +0000 (11:20 +0200)] 
hsts: avoid the local buffer and memcpy on lookup

Closes #15190

13 months agoconfigure: add GSS to `libcurl.pc` `Depends:`
Viktor Szakats [Mon, 7 Oct 2024 14:13:04 +0000 (16:13 +0200)] 
configure: add GSS to `libcurl.pc` `Depends:`

GSS was the last (known) missing dependency missing from `libcurl.pc`.

Closes #15177

13 months agocmake: detect GNU GSS
Viktor Szakats [Mon, 7 Oct 2024 14:13:32 +0000 (16:13 +0200)] 
cmake: detect GNU GSS

Fix to set `HAVE_GSSGNU` when GNU GSS is detected.

Also set the appropriate `pkg-config` dependency and do version
detection for the GNU GSS flavour.

Tested with `pkg-config` and partly tested without. The latter case
picks up everything else but, in my env. This is likely not the last
word to implement this detection correctly for all build-cases.

GNU GSS doesn't seem to have a Homebrew formula and building
it locally needs manual tweaks to make finish successfully.

Also move a MIT-specific header detection into to MIT-specific `if`
branch.

Closes #15176

13 months agoCURLOPT_APPEND.md: goes for SFTP as well
Daniel Stenberg [Mon, 7 Oct 2024 12:30:13 +0000 (14:30 +0200)] 
CURLOPT_APPEND.md: goes for SFTP as well

Closes #15181

13 months agoconncache: find bundle again in case it is removed
Daniel Stenberg [Tue, 8 Oct 2024 06:00:55 +0000 (08:00 +0200)] 
conncache: find bundle again in case it is removed

When the pool is cleaned up due to host limits, the bundle may be
cleaned up as well making the old pointer invalid.

Fixes #15185
Reported-by: Moritz Knüsel
Closes #15186

13 months agotest1915: remove wrong comment
Daniel Stenberg [Tue, 8 Oct 2024 07:35:56 +0000 (09:35 +0200)] 
test1915: remove wrong comment

13 months agosetopt: use a single function for HTTPAUTH and PROXYAUTH
Daniel Stenberg [Mon, 7 Oct 2024 22:35:28 +0000 (00:35 +0200)] 
setopt: use a single function for HTTPAUTH and PROXYAUTH

Avoid duplicated almost-the-same code.

Closes #15182

13 months agocmake: do not propagate unused `HAVE_GSSAPI_GSSAPI_KRB5_H` to C
Viktor Szakats [Mon, 7 Oct 2024 10:56:04 +0000 (12:56 +0200)] 
cmake: do not propagate unused `HAVE_GSSAPI_GSSAPI_KRB5_H` to C

Closes #15174

13 months agocmake: detect `HAVE_NETINET_IN6_H`, `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`
Viktor Szakats [Sun, 6 Oct 2024 00:54:25 +0000 (02:54 +0200)] 
cmake: detect `HAVE_NETINET_IN6_H`, `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`

To sync with `./configure`.

- `HAVE_NETINET_IN6_H` is
  needed by HPE NonStop NSE and NSX systems.
  Follow-up to 76ebd54175bad02b29769d797adf72fdf3df119f #2155

- `HAVE_CLOSESOCKET_CAMEL`, `HAVE_PROTO_BSDSOCKET_H`
  are for AmigaOS.
  (Note: `./configure` tries to detect these for all targets, cmake does
  it only for AmigaOS, to not inflate configure time.)

Closes #15172

13 months agoconfigure: drop unused bare `socket.h` detection
Viktor Szakats [Mon, 7 Oct 2024 02:07:30 +0000 (04:07 +0200)] 
configure: drop unused bare `socket.h` detection

Added in 37eba37019388f767f5de67b4071641044d7b026 (2009-06-17) to help
detecting socket functions.

But, this `socket.h` isn't used in the source code since
90dd1fc66401d5bb7814f4edeb16a06c925b1f1e #8288 (2022-01-16).

Closes #15173

13 months agosws: fix unused static function with `TCP_NODELAY` undefined
Viktor Szakats [Sun, 6 Oct 2024 09:37:38 +0000 (11:37 +0200)] 
sws: fix unused static function with `TCP_NODELAY` undefined

Closes #15171

13 months agoconfigure: drop duplicate feature checks for `poll()`, `if_nametoindex()`
Viktor Szakats [Sun, 6 Oct 2024 00:26:48 +0000 (02:26 +0200)] 
configure: drop duplicate feature checks for `poll()`, `if_nametoindex()`

Before this patch they were detected via manual methods, then with
`AC_CHECK_FUNCS()`.

Delete the manual checks and keep the latter.

Also delete `CURL_INCLUDES_POLL()` which is no longer used after
the above.

Closes #15170

13 months agobuild: detect and use `_setmode()` with Cygwin/MSYS, also use on Windows
Viktor Szakats [Sun, 6 Oct 2024 13:41:48 +0000 (15:41 +0200)] 
build: detect and use `_setmode()` with Cygwin/MSYS, also use on Windows

Before this patch `setmode()` was not detected with Cygwin/MSYS, because
it's a macro, not a function, and detection is looking for a function.

Switching to symbol detection doesn't work because it mis-detects it on
BSD systems which features a function with the same name but different
functionality and arguments.

Fix it by looking for a `_setmode()` function on Cygwin/MSYS, and use it
if available.

`_setmode()` is recommended over `setmode()` by Windows documentation so
use that on Windows too. It seems to be available on all supported
compilers, so omit detection.

https://learn.microsoft.com/cpp/c-runtime-library/reference/posix-setmode
https://learn.microsoft.com/cpp/c-runtime-library/reference/setmode

Officially Windows requires argument `_O_BINARY` with an underscore.
`O_BINARY` is also supported but bound to conditions. Continue to use it
for simplicity. Cygwin supports `O_BINARY` (no underscore).

Closes #15169

13 months agoech: spelling, whitespace, say `--ech` default config
Viktor Szakats [Sat, 5 Oct 2024 21:29:22 +0000 (23:29 +0200)] 
ech: spelling, whitespace, say `--ech` default config

Closes #15167

13 months agoGHA/macos: comment spelling and clarity
Viktor Szakats [Sun, 6 Oct 2024 11:02:12 +0000 (13:02 +0200)] 
GHA/macos: comment spelling and clarity

Closes #15166

13 months agobuild: add `ldap` to `libcurl.pc` `Requires:`
Viktor Szakats [Fri, 4 Oct 2024 23:57:33 +0000 (01:57 +0200)] 
build: add `ldap` to `libcurl.pc` `Requires:`

Closes #15163

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 7 Oct 2024 06:33:32 +0000 (08:33 +0200)] 
RELEASE-NOTES: synced

13 months agoINSTALL-CMAKE.md: mention focus on shared libraries
Daniel Stenberg [Sun, 6 Oct 2024 21:13:29 +0000 (23:13 +0200)] 
INSTALL-CMAKE.md: mention focus on shared libraries

Match what the INSTALL.md document says for configure builds.

Reported-by: Nemos2024 on github
Fixes #14845
Closes #15168

13 months agoci: update dependency ngtcp2/nghttp3 to v1.6.0
renovate[bot] [Sat, 5 Oct 2024 18:38:57 +0000 (18:38 +0000)] 
ci: update dependency ngtcp2/nghttp3 to v1.6.0

Closes #15161

13 months agoci: update dependency ngtcp2/ngtcp2 to v1.8.0
renovate[bot] [Sat, 5 Oct 2024 18:39:02 +0000 (18:39 +0000)] 
ci: update dependency ngtcp2/ngtcp2 to v1.8.0

Closes #15162

13 months agoGHA/non-native: fix OmniOS job to fail on tests
Viktor Szakats [Sat, 5 Oct 2024 15:23:17 +0000 (17:23 +0200)] 
GHA/non-native: fix OmniOS job to fail on tests

Before this patch a failed test did not result in a failed CI job.

Closes #15160

13 months agocmake: use OpenSSL for LDAP detection only if available
Viktor Szakats [Fri, 4 Oct 2024 19:24:54 +0000 (21:24 +0200)] 
cmake: use OpenSSL for LDAP detection only if available

Also unset `CMAKE_REQUIRED_LIBRARIES` in two error branches.

Reported-by: Vollstrecker on github
Fixes #15077
Closes #15152

13 months agowarnless: remove curlx_sktosi and curlx_sitosk
Daniel Stenberg [Fri, 4 Oct 2024 20:32:04 +0000 (22:32 +0200)] 
warnless: remove curlx_sktosi and curlx_sitosk

They were only used by test server code. Use plain old typecasts there.

Closes #15153

13 months agotests: enable additional ruff Python lint options
Dan Fandrich [Thu, 26 Sep 2024 21:31:39 +0000 (14:31 -0700)] 
tests: enable additional ruff Python lint options

These all seem reasonable to enable for this code.

13 months agoCI: run pytype and ruff on Python code
Dan Fandrich [Thu, 26 Sep 2024 18:33:58 +0000 (11:33 -0700)] 
CI: run pytype and ruff on Python code

These ensure a more consistent style and can find some errors statically
that would otherwise only be seen at run-time. Also, bump the Ubuntu
version of some other checks to get newer versions of some linters.

Closes #15067

13 months agotests: change Python code style to pass ruff checks
Dan Fandrich [Thu, 26 Sep 2024 19:40:43 +0000 (12:40 -0700)] 
tests: change Python code style to pass ruff checks

Most of the changes consisted of removing unused imports and unnecessary
f-strings.

13 months agotests: fix some Python typing issues
Dan Fandrich [Thu, 26 Sep 2024 18:58:57 +0000 (11:58 -0700)] 
tests: fix some Python typing issues

These otherwise raise errors in pytype. A few problematic methods
weren't being used and are deleted.

13 months agoCURLOPT_HEADERFUNCTION.md: do not modify the passed in buffer
Daniel Stenberg [Fri, 4 Oct 2024 11:53:20 +0000 (13:53 +0200)] 
CURLOPT_HEADERFUNCTION.md: do not modify the passed in buffer

Closes #15148

13 months agoasyn-ares: remove typecast, fix expire
Daniel Stenberg [Thu, 3 Oct 2024 21:53:10 +0000 (23:53 +0200)] 
asyn-ares: remove typecast, fix expire

- Use the appropriate variable type for the curlx_tvtoms() return code:
  timediff_t and remove the typecast.

- Simplify the function and avoid the odd expire adjustment that
  probably is a rest from ancient days when the expire function did not
  handle zero millisecond timeouts.

Closes #15145

13 months agocmake: add missed variable to comment [ci skip]
Viktor Szakats [Fri, 4 Oct 2024 00:10:00 +0000 (02:10 +0200)] 
cmake: add missed variable to comment [ci skip]

Follow-up to 9f56bb608ecfbb8978c6cb72a04d9e8b23162d82 #14681

13 months agotest1915: add tracing and connect timeout
Stefan Eissing [Tue, 1 Oct 2024 10:48:46 +0000 (12:48 +0200)] 
test1915: add tracing and connect timeout

Since we see Windows fails of 1915, add tracing and a connect timeout.
The test uses a port no one is supposed to listen on, but Windows has
this weird wait logic. So, set a short timeout.

Closes #15107

13 months agourlapi: normalize the IPv6 address
Daniel Stenberg [Thu, 3 Oct 2024 12:37:00 +0000 (14:37 +0200)] 
urlapi: normalize the IPv6 address

As the parsing and address "regeneration" are done anyway, we might as
well use the updated version in the result and thereby A) get a
normalized (and lower cased) version of the address and B) avoid a
strcpy().

Updated test 1560 to verify.

Closes #15143

13 months agotests/valgrind.supp: remove a travis suppression, add a Debian
Daniel Stenberg [Thu, 3 Oct 2024 12:32:54 +0000 (14:32 +0200)] 
tests/valgrind.supp: remove a travis suppression, add a Debian

We have not used Travis for years. The Debian one appears on my dev
machine since a while back.

Closes #15142

13 months agoopenssl quic: populate x509 store before handshake
Stefan Eissing [Thu, 3 Oct 2024 08:51:26 +0000 (10:51 +0200)] 
openssl quic: populate x509 store before handshake

Since OpenSSL does its own send/recv internally, we may miss the moment
to populate the x509 store right before the server response. Do it
instead before we start the handshake, at the loss of the time to set
this up.

Closes #15137

13 months agopytest: improve pytest_07_42a reliability
Stefan Eissing [Thu, 3 Oct 2024 09:10:29 +0000 (11:10 +0200)] 
pytest: improve pytest_07_42a reliability

Due to timings on paused response receive, the error code may vary due
to the location where it is detected that the server closed the transfer
prematurely.

Be more lenient in always allowing PARTIAL_FILE as ok.

Closes #15138

13 months agotest1515: add tracing and more debug info
Stefan Eissing [Thu, 3 Oct 2024 11:09:17 +0000 (13:09 +0200)] 
test1515: add tracing and more debug info

Test failed on some Windows CI and log files did not really
tell why. Add more output.

Closes #15140

13 months agoGHA/curl-for-win: tidy up `DOCKER_CONTENT_TRUST`
Viktor Szakats [Thu, 3 Oct 2024 11:19:00 +0000 (13:19 +0200)] 
GHA/curl-for-win: tidy up `DOCKER_CONTENT_TRUST`

Follow-up to 08d13c0e46ab535b04c4099d6c51d51f7c288b6c
Closes #15141

13 months agobufq: unwrite fix
Stefan Eissing [Thu, 3 Oct 2024 08:15:07 +0000 (10:15 +0200)] 
bufq: unwrite fix

`Curl_bufq_unwrite()` used the head instead of the tail chunk to shrink
the bufq's content. Fix this and add test case that checks correct
behaviour.

Amended test 2601 accordingly.

Reported-by: Chris Stubbs
Closes #15136

13 months agoGHA/curl-for-win: re-enable image verification for debian:bookworm-slim
Viktor Szakats [Thu, 3 Oct 2024 10:51:05 +0000 (12:51 +0200)] 
GHA/curl-for-win: re-enable image verification for debian:bookworm-slim

Follow-up to a35f223cd8b29739dab7e8490674093c4024191d

13 months agoGHA/windows: add workaround for upstream vcpkg issue
Tal Regev [Thu, 3 Oct 2024 04:15:01 +0000 (07:15 +0300)] 
GHA/windows: add workaround for upstream vcpkg issue

Fixes:
```
error: https://github.com/google/brotli/archive/v1.1.0.tar.gz: WinHttpSendRequest failed with exit code 10106
```

Apply workaround:
https://github.com/microsoft/vcpkg/issues/41199#issuecomment-2378255699

Bug commit: https://github.com/microsoft/vcpkg-tool/commit/d3fc35774f86782f356ef0f9352ba857fa57302e#diff-33cf7c311a76d4a838f91b078c2f8cbc984557379f7b345a268ec6deb665a91eR666-R675
Fix commit: https://github.com/microsoft/vcpkg-tool/commit/3122da72b1e4bf98bfc3d51937e6ad6cc4292cdf
Fix PR: https://github.com/microsoft/vcpkg-tool/pull/1501

Fixes #15128
Reported-by: Viktor Szakats
Closes #15133

13 months agoGHA/curl-for-win: disable `DOCKER_CONTENT_TRUST`
Viktor Szakats [Thu, 3 Oct 2024 09:03:05 +0000 (11:03 +0200)] 
GHA/curl-for-win: disable `DOCKER_CONTENT_TRUST`

As a workaround for the latest Debian image missing the necessary keys:
```
No signatures for debian:testing-20240926-slim

Administrative keys for debian:testing-20240926-slim

  Repository Key:       5717dcd81d9fb5b73aa15f2d887a6a0de543829ab9b2d411acce9219c2f8ba3a
  Root Key:     575d013f89e3cbbb19e0fb06aa33566c22718318e0c9ffb1ab5cc4291e07bf84
No valid trust data for testing-20240926-slim
```
https://github.com/curl/curl/actions/runs/11067644862/job/30752034560?pr=15065

Ref: https://github.com/curl/curl-for-win/commit/0ad1ad511bcbfbad8935685f7c130b281e66f7f4

13 months agoDEPRECATE: remove hyper in January 2025
Daniel Stenberg [Thu, 3 Oct 2024 07:21:45 +0000 (09:21 +0200)] 
DEPRECATE: remove hyper in January 2025

Previously this document stated we would do it after February, but now
it will be done already for the first January 2025 release.

The reason being that since we decided to deprecate hyper, the
degradation speed has increased as now no one bothers to fix issues in
the hyper side of things. Also: not a single soul has yet spoken up in
favor of keeping the support.

Closes #15135

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 3 Oct 2024 07:34:53 +0000 (09:34 +0200)] 
RELEASE-NOTES: synced

13 months agolib: use bool/TRUE/FALSE properly
Daniel Stenberg [Wed, 2 Oct 2024 09:53:20 +0000 (11:53 +0200)] 
lib: use bool/TRUE/FALSE properly

booleans should use the type 'bool' and set the value to TRUE/FALSE

non-booleans should not be 'bool' and should not set the value to
TRUE/FALSE

Closes #15123

13 months agowolfssl: add proper colon separator
Daniel Stenberg [Thu, 3 Oct 2024 06:26:44 +0000 (08:26 +0200)] 
wolfssl: add proper colon separator

Follow-up to 6fd5a9777acb720e1ac872478151e8b1

Fixes #15132
Reported-by: Viktor Szakats
Closes #15134

13 months agovtls: convert Curl_pin_peer_pubkey to use dynbuf
Daniel Stenberg [Wed, 2 Oct 2024 12:26:57 +0000 (14:26 +0200)] 
vtls: convert Curl_pin_peer_pubkey to use dynbuf

Closes #15126

13 months agovtls: convert pubkey_pem_to_der to use dynbuf
Daniel Stenberg [Wed, 2 Oct 2024 12:14:18 +0000 (14:14 +0200)] 
vtls: convert pubkey_pem_to_der to use dynbuf

... instead of malloc and "manual" buffer stuffing

Closes #15126

13 months agotests: let openssl generate random cert serials
Viktor Szakats [Wed, 2 Oct 2024 21:20:00 +0000 (23:20 +0200)] 
tests: let openssl generate random cert serials

Generate the certificate serial numbers automatically instead of doing
from shell (or Perl earlier).

Fixes intermittent CI failures due to the shell-based random generator
generating the same serial number twice:
```
$ openssl ca -config EdelCurlRoot-ca.cnf -revoke Server-localhost0h-sv.crt
Using configuration from EdelCurlRoot-ca.cnf
ERROR:Already revoked, serial number 66FDB23A
make: *** [../../../tests/certs/Server-localhost0h-sv.pem] Error 1
```
https://github.com/curl/curl/actions/runs/11151401083/job/30994755798?pr=15128#step:10:498

Follow-up to fa461b4eff52b413f88debf543b5350a6cef4724 #14486
Follow-up to fa69b41c7790fab86fd363242c81d8ef2e89e183 #13307
Closes #15129

13 months agoGHA/linux: fix wolfSSL version in cache key
Viktor Szakats [Wed, 2 Oct 2024 21:46:29 +0000 (23:46 +0200)] 
GHA/linux: fix wolfSSL version in cache key

This needs the version in the GHA context, so move those back
from external file `VERSIONS`.

Also move back `VERSIONS` content in it previous place to
`.circleci/config.yml`. Update renovate config.

Renovate bot should keep updating the wolfSSL version, but from now on
not in one, but two files.

Follow-up to 820afa2b7cb33a0daaca5b61e8755546b46136d5 #15030
Follow-up to 73a36021207284ad2b4340ffde34a51b0ba4d47a
Closes #15130

13 months agoGHA/linux: drop duplicate names from cache keys
Viktor Szakats [Wed, 2 Oct 2024 22:03:54 +0000 (00:03 +0200)] 
GHA/linux: drop duplicate names from cache keys

Closes #15131

13 months agotests: simplify `pathhelp.pm`, avoid using external tools
Viktor Szakats [Sat, 28 Sep 2024 18:26:44 +0000 (20:26 +0200)] 
tests: simplify `pathhelp.pm`, avoid using external tools

Instead of calling the shell and external tools, rely on Perl functions
like `Cwd::getcwd()`, `Cwd::abs_path()`, `Cygwin::posix_to_win_path()`,
`Cygwin::win_to_posix_path()` to retrieve the current directory and
convert between POSIX and Windows formats.

This adds native Windows Perl support, avoids most failure modes and
makes format guessing and other internal functions unnecessary.

Also:
- delete unused `sys_native_path()`.
- delete redundant `normalize_path()` because Perl `abs_path()` already
  does it.

Cherry-picked from #14949
Closes #15111

13 months agowolfssl: convert malloc + memcpys to dynbuf for cipher string
Daniel Stenberg [Wed, 2 Oct 2024 12:00:56 +0000 (14:00 +0200)] 
wolfssl: convert malloc + memcpys to dynbuf for cipher string

Closes #15124

13 months agolib: avoid assigning 'result' temporarily
Daniel Stenberg [Wed, 2 Oct 2024 09:45:19 +0000 (11:45 +0200)] 
lib: avoid assigning 'result' temporarily

Closes #15122

13 months agomulti: make multi_handle_timeout use the connect timeout
Daniel Stenberg [Wed, 2 Oct 2024 07:01:56 +0000 (09:01 +0200)] 
multi: make multi_handle_timeout use the connect timeout

For all states before MSTATE_DO the connect timeout needs to be
considered.

Regression since #13371 (be659030ba078d6) shipped in 8.8.0

Reported-by: Deniz Sökmen
Fixes #15100
Closes #15119

13 months agoGHA/labeler: adjust some docs patterns
Daniel Stenberg [Wed, 2 Oct 2024 08:50:22 +0000 (10:50 +0200)] 
GHA/labeler: adjust some docs patterns

Follow-up to 20aa8d8f31527 and 0e06603b23183 that moved and removed some
docs.

Closes #15121

13 months agotests: remove debug requirement on 38 tests
Daniel Stenberg [Tue, 1 Oct 2024 22:01:27 +0000 (00:01 +0200)] 
tests: remove debug requirement on 38 tests

For all tests using -O that were previously relying on a debug build and
the CURL_TESTDIR environment variable, use the plain --output-dir option
instead so that they can run proper in non-debug builds.

Closes #15114

13 months agovtls: skip a "useless assignment"
Daniel Stenberg [Wed, 2 Oct 2024 05:41:47 +0000 (07:41 +0200)] 
vtls: skip a "useless assignment"

Pointed out by CodeSonar

Closes #15117

13 months agotool: support --show-headers AND --remote-header-name
Daniel Stenberg [Mon, 30 Sep 2024 13:38:56 +0000 (15:38 +0200)] 
tool: support --show-headers AND --remote-header-name

By keeping the headers in memory until we know the target file name,
then output them all.

Previously this option combination would cause an error.

Add test 1310 and 1492 to verify. Adjusted test 1460 to work in the new
conditions.

Closes #15110

13 months agoGHA/macos: update comment with new Xcode default for macos-13 [ci skip]
Viktor Szakats [Wed, 2 Oct 2024 00:16:59 +0000 (02:16 +0200)] 
GHA/macos: update comment with new Xcode default for macos-13 [ci skip]

13 months agoGHA/macos: drop unsupported Xcode version references
Viktor Szakats [Tue, 1 Oct 2024 23:31:53 +0000 (01:31 +0200)] 
GHA/macos: drop unsupported Xcode version references

Closes #15115

13 months agoGHA/macos: delete `macos-12` jobs, update matrix for `macos-14`
Viktor Szakats [Tue, 1 Oct 2024 20:16:25 +0000 (22:16 +0200)] 
GHA/macos: delete `macos-12` jobs, update matrix for `macos-14`

- Days of `macos-12` are numbered:
  https://github.com/actions/runner-images/issues/10721
  Drop 5 affected jobs, replaced by `macos-15` ones added earlier today.

- Drop Xcode 14 and 16 from `macos-14` jobs.
  Following up upstream announcement:
  https://github.com/actions/runner-images/issues/10703
  (No active jobs are affected by this.)

Follow-up to 4b4ff444dd68308e8938a1e5c33e0c8b9390d1da #15108
Closes #15113

13 months agoGHA/macos: Sequoia chores, fixes for llvm 18
Viktor Szakats [Tue, 1 Oct 2024 11:20:09 +0000 (13:20 +0200)] 
GHA/macos: Sequoia chores, fixes for llvm 18

- add `macos-15` Sequoia to combinations jobs.

- add `llvm@18` to combinations jobs for `macos-15`.

- fix cmake, autotools configurations for llvm 18.
  It requires `CMAKE_OSX_SYSROOT` for cmake, and `--sysroot` within `CC`
  for both cmake and autotools.
  (Also add `--target` to `CC` for all jobs.)

- stop `brew update` on `macos-12`.
  `macos-12` is Monterey. Homebrew stopped providing binaries for it
  after the latest macOS was out. It means newer versions of packages
  have to be built from source, which is slow and undesired. Try to
  avoid it using this trick. If it doesn't work, `macos-12` jobs will
  have to be dropped.

- update Xcode support matrix.

Ref: https://github.com/actions/runner-images/blob/main/images/macos/macos-15-arm64-Readme.md
Closes #15108

13 months agotests: fixup `checkcmd` `PATH` on non-unixy platforms
Viktor Szakats [Tue, 1 Oct 2024 10:25:21 +0000 (12:25 +0200)] 
tests: fixup `checkcmd` `PATH` on non-unixy platforms

Do not add unixy system directories to `PATH` when looking up commands
on Windows, MS-DOS or OS/2.

Cherry-picked from #14949
Closes #15106

13 months agotests: fix shell quoting on native Windows Perl
Viktor Szakats [Fri, 20 Sep 2024 13:10:42 +0000 (15:10 +0200)] 
tests: fix shell quoting on native Windows Perl

Cherry-picked from #14949
Closes #15105

13 months agotests: fix `%POSIX_PWD` on native Windows Perl
Viktor Szakats [Tue, 1 Oct 2024 10:16:50 +0000 (12:16 +0200)] 
tests: fix `%POSIX_PWD` on native Windows Perl

Though this variable isn't used in any of the current tests.

Cherry-picked from #14949
Closes #15104

13 months agotests: replace `%PWD` with `%SSH_PWD` in SCP/SFTP tests
Viktor Szakats [Sun, 29 Sep 2024 11:20:51 +0000 (13:20 +0200)] 
tests: replace `%PWD` with `%SSH_PWD` in SCP/SFTP tests

Cherry-picked from #14949
Closes #15103

13 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 1 Oct 2024 13:48:16 +0000 (15:48 +0200)] 
RELEASE-NOTES: synced

13 months agoCI: bump actions/checkout from 4.1.7 to 4.2.0
dependabot[bot] [Mon, 30 Sep 2024 14:45:57 +0000 (14:45 +0000)] 
CI: bump actions/checkout from 4.1.7 to 4.2.0

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.7 to 4.2.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/692973e3d937129bcbf40652eb9f2f61becf3332...d632683dd7b4114ad314bca15554477dd762a938)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #15095