]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
2 years agotests: Fix Windows test helper tool search & use it for handle64 origin/master
Dan Fandrich [Fri, 13 Oct 2023 18:46:39 +0000 (11:46 -0700)] 
tests: Fix Windows test helper tool search & use it for handle64

The checkcmd() and checktestcmd() functions would not have worked on
Windows due to hard-coding the UNIX PATH separator character and not
adding .exe file extension. This meant that tools like stunnel, valgrind
and nghttpx would not have been found and used on Windows, and
inspection of previous test runs show none of those being found in pure
Windows CI builds.

With this fixed, they can be used to detect the handle64.exe program
before attempting to use it. When handle64.exe was called
unconditionally without it existing, it caused perl to abort the test
run with the error

    The running command stopped because the preference variable
    "ErrorActionPreference" or common parameter is set to Stop:
    sh: handle64.exe: command not found

Closes #12115

2 years agomulti: use pipe instead of socketpair to *wakeup()
Daniel Stenberg [Tue, 17 Oct 2023 06:28:55 +0000 (08:28 +0200)] 
multi: use pipe instead of socketpair to *wakeup()

If pipe() is present. Less overhead.

Closes #12142

2 years agobuild: fix 'threadsafe' feature detection for older gcc
Jay Satiro [Sun, 15 Oct 2023 01:55:42 +0000 (21:55 -0400)] 
build: fix 'threadsafe' feature detection for older gcc

- Add 'threadsafe' to the feature list shown during build if POSIX
  threads are being used.

This is a follow-up to 5adb6000 which added support for building a
thread-safe libcurl with older versions of gcc where atomic is not
available but pthread is.

Reported-by: Dan Fandrich
Co-authored-by: Dan Fandrich
Fixes https://github.com/curl/curl/issues/12125
Closes https://github.com/curl/curl/pull/12127

2 years agotest729: verify socks4a with excessive proxy user name length
Daniel Stenberg [Fri, 13 Oct 2023 12:16:47 +0000 (14:16 +0200)] 
test729: verify socks4a with excessive proxy user name length

2 years agosocks: better buffer size checks for socks4a user and hostname
Daniel Stenberg [Thu, 12 Oct 2023 22:15:29 +0000 (00:15 +0200)] 
socks: better buffer size checks for socks4a user and hostname

Also limit the proxy user name to 255 bytes, which is the same limit as
in SOCKS5.

Reported-by: sd0 on hackerone
Closes #12139

2 years agocurl.h: on FreeBSD include sys/param.h instead of osreldate.h
Daniel Stenberg [Sat, 14 Oct 2023 20:47:49 +0000 (22:47 +0200)] 
curl.h: on FreeBSD include sys/param.h instead of osreldate.h

Should things build on Playstation as well

Fixes #12107
Reported-by: Faraz Fallahi
Closes #12123

2 years agotool_operate: fix links in ipfs errors
Marcin Rataj [Sun, 15 Oct 2023 23:34:54 +0000 (01:34 +0200)] 
tool_operate: fix links in ipfs errors

URL fragment links generated from headers in
https://curl.se/docs/ipfs.html are lowercase.

Closes #12133

2 years agocmake: replace `check_library_exists_concat()`
Viktor Szakats [Mon, 9 Oct 2023 08:31:04 +0000 (08:31 +0000)] 
cmake: replace `check_library_exists_concat()`

The idea of `check_library_exists_concat()` is that it detects an
optional component and adds it to the list of libs that we also use in
subsequent component checks. This caused problems when detecting
components with unnecessary dependencies that were not yet built.

CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used
for component checks, which we already use in most cases. That left 4
uses of `check_library_exists_concat()`. Only one of these actually
needed the 'concat' feature (ldap/lber).

Delete this function and replace it with standard
`check_library_exists()` and manual management of our `CURL_LIBS`
list we use when linking build targets. And special logic to handle the
ldap/lber case.

(We have a similar function for headers: `check_include_file_concat()`.
It works, but problematic for performance reasons and because it hides
the actual headers required in `check_symbol_exists()` calls.)

Ref: #11537 #11558
Fixes #11285
Fixes #11648
Closes #12070

2 years agotool_cb_wrt: fix write output for very old Windows versions
LoRd_MuldeR [Sun, 15 Oct 2023 14:55:43 +0000 (16:55 +0200)] 
tool_cb_wrt: fix write output for very old Windows versions

- Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW()
  function.

Apparently this parameter was *not* optional on older Windows versions.

Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1.
So at some point between those two Microsoft changed the behavior.

Prior to this change, on those versions if parameter is NULL then the
function call fails with error ERROR_INVALID_ACCESS.

Regression since af3f4e41.

Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299

Fixes https://github.com/curl/curl/issues/12131
Closes https://github.com/curl/curl/pull/12130

2 years agotool_urlglob: fix build for old gcc versions
Jay Satiro [Sun, 15 Oct 2023 02:28:17 +0000 (22:28 -0400)] 
tool_urlglob: fix build for old gcc versions

- Don't use __builtin_mul_overflow for GCC 4 and earlier.

The function was added in GCC 5.

Ref: https://gcc.gnu.org/gcc-5/changes.html

Reported-by: Dan Fandrich
Fixes https://github.com/curl/curl/issues/12124
Closes https://github.com/curl/curl/pull/12128

2 years agodocs/libcurl: fix three minor man page format mistakes
Carlos Henrique Lima Melara [Sun, 15 Oct 2023 01:36:04 +0000 (22:36 -0300)] 
docs/libcurl: fix three minor man page format mistakes

Reported-by: Samuel Henrique
Closes https://github.com/curl/curl/pull/12126

2 years agotests/server: add more SOCKS5 handshake error checking
Jay Satiro [Sat, 14 Oct 2023 05:45:28 +0000 (01:45 -0400)] 
tests/server: add more SOCKS5 handshake error checking

- Add additional checking for missing and too-short SOCKS5 handshake
  messages.

Prior to this change the SOCKS5 test server did not check that all parts
of the handshake were received successfully. If those parts were missing
or too short then the server would access uninitialized memory.

This issue was discovered in CI job 'memory-sanitizer' test results.
Test 2055 was failing due to the SOCKS5 test server not running. It was
not running because either it crashed or memory sanitizer aborted it
during Test 728. Test 728 connects to the SOCKS5 test server on a
redirect but does not send any data on purpose. The test server was not
prepared for that.

Reported-by: Dan Fandrich
Fixes https://github.com/curl/curl/issues/12117
Closes https://github.com/curl/curl/pull/12118

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 14 Oct 2023 21:10:36 +0000 (23:10 +0200)] 
RELEASE-NOTES: synced

2 years agotool_getparam: limit --rate to be smaller than number of ms
Sohom Datta [Fri, 13 Oct 2023 21:01:16 +0000 (23:01 +0200)] 
tool_getparam: limit --rate to be smaller than number of ms

Currently, curl allows users to specify absurd request rates that might
be higher than the number of milliseconds in the unit (ex: curl --rate
3600050/h http://localhost:8080 does not error out despite there being
only 3600000ms in a hour).

This change adds a conditional check before the millisecond calculation
making sure that the number is not higher than the numerator (the unit)
If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE

Closes #12116

2 years agoopts: fix two minor man page format mistakes
Daniel Stenberg [Sat, 14 Oct 2023 16:22:03 +0000 (18:22 +0200)] 
opts: fix two minor man page format mistakes

2 years agocurl_trc: remove a bad assertion
Jay Satiro [Fri, 13 Oct 2023 00:50:45 +0000 (20:50 -0400)] 
curl_trc: remove a bad assertion

- Remove DEBUGASSERT that an internal handle must not have user
  private_data set before calling the user's debug callback.

This is a follow-up to 0dc40b2a. The user can distinguish their easy
handle from an internal easy handle by setting CURLOPT_PRIVATE on their
easy handle. I had wrongly assumed that meant the user couldn't then
set CURLOPT_PRIVATE on an internal handle as well.

Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697
Reported-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/12104

2 years agotest613: stop showing an error on missing output file
Dan Fandrich [Sat, 14 Oct 2023 03:24:08 +0000 (20:24 -0700)] 
test613: stop showing an error on missing output file

This test would show an error message if the output was missing during
the log post-processing step, but the message was not captured by the
test harness and wasn't useful since the normal golden log file
comparison would the problem more clearly.

2 years agoquic: manage connection idle timeouts
Stefan Eissing [Tue, 10 Oct 2023 10:51:25 +0000 (12:51 +0200)] 
quic: manage connection idle timeouts

- configure a 120s idle timeout on our side of the connection
- track the timestamp when actual socket IO happens
- check IO timestamp to our *and* the peer's idle timeouts
  in "is this connection alive" checks

Reported-by: calvin2021y on github
Fixes #12064
Closes #12077

2 years agoCI: ignore test 286 on Appveyor gcc 9 build
Dan Fandrich [Fri, 13 Oct 2023 02:33:19 +0000 (19:33 -0700)] 
CI: ignore test 286 on Appveyor gcc 9 build

This test fails sometimes with a super fast retry loop due to what may
just be a compiler bug. The test results are ignored on the one CI job
where it occurs because there seems to be nothing we can do to fix it.

Fixes #12040
Closes #12106

2 years agolib: fix gcc warning in printf call
Viktor Szakats [Wed, 11 Oct 2023 04:02:45 +0000 (04:02 +0000)] 
lib: fix gcc warning in printf call

Do not pass NULL to printf %s.

Seen with gcc 13.2.0 on Debian:
```
.../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wformat-overflow=]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/17584426483#step:3:11104

Ref: #10284
Co-authored-by: Jay Satiro
Closes #12082

2 years agohttp2: safer invocation of populate_binsettings
Alex Klyubin [Thu, 12 Oct 2023 18:27:47 +0000 (12:27 -0600)] 
http2: safer invocation of populate_binsettings

populate_binsettings now returns a negative value on error, instead of a
huge positive value. Both places which call this function have been
updated to handle this change in its contract.

The way populate_binsettings had been used prior to this change the huge
positive values -- due to signed->unsigned conversion of the potentially
negative result of nghttp2_pack_settings_payload which returns negative
values on error -- are not possible. But only because http2.c currently
always provides a large enough output buffer and provides H2 SETTINGS
IVs which pass the verification logic inside nghttp2.  If the
verification logic were to change or if http2.c started passing in more
IVs without increasing the output buffer size, the overflow could become
reachable, and libcurl/curl might start leaking memory contents to
servers/proxies...

Closes #12101

2 years agoopenssl: avoid BN_num_bits() NULL pointer derefs
Daniel Stenberg [Thu, 12 Oct 2023 14:00:38 +0000 (16:00 +0200)] 
openssl: avoid BN_num_bits() NULL pointer derefs

Reported-by: icy17 on github
Fixes #12099
Closes #12100

2 years agowolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA
Daniel Stenberg [Fri, 13 Oct 2023 07:00:50 +0000 (09:00 +0200)] 
wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA

This define is set in wolfssl's options.h file when this function and
feature is present. Handles both builds with the feature explicitly
disabled and wolfSSL versions before 5.5.2 - which introduced this API
call.

Closes #12108

2 years agotool_urlglob: make multiply() bail out on negative values
Daniel Stenberg [Thu, 12 Oct 2023 22:13:23 +0000 (00:13 +0200)] 
tool_urlglob: make multiply() bail out on negative values

- Does not work correctly with negative values
- use __builtin_mul_overflow() on gcc

Reported-by: Torben Dury
Closes #12102

2 years agocmake: fix CURL_DISABLE_GETOPTIONS
Loïc Yhuel [Wed, 11 Oct 2023 11:13:50 +0000 (13:13 +0200)] 
cmake: fix CURL_DISABLE_GETOPTIONS

- Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake.

Prior to this change the option had no effect because it was missing
from that file.

Closes https://github.com/curl/curl/pull/12091

2 years agoeasy_lock: add a pthread_mutex_t fallback
Loïc Yhuel [Wed, 11 Oct 2023 11:07:48 +0000 (13:07 +0200)] 
easy_lock: add a pthread_mutex_t fallback

This allows to keep the init threadsafe with gcc < 4.9.0 (no C11
atomics).

Closes https://github.com/curl/curl/pull/12090

2 years agoCI: add autotools, out-of-tree, debug build to distro check job
Viktor Szakats [Wed, 11 Oct 2023 09:07:03 +0000 (09:07 +0000)] 
CI: add autotools, out-of-tree, debug build to distro check job

Add a job that builds curl from a generated source tarball sample, with
autotools, out-of-tree, in debug mode.

Ref: #12085
Closes #12088

2 years agohttp: avoid Expect: 100-continue if Upgrade: is used
Daniel Stenberg [Sun, 8 Oct 2023 10:04:59 +0000 (12:04 +0200)] 
http: avoid Expect: 100-continue if Upgrade: is used

Reported-by: Daniel Jelinski
Fixes #12022
Closes #12062

2 years agodocs: use SOURCE_DATE_EPOCH for generated manpages
Jan Alexander Steffens (heftig) [Sun, 8 Oct 2023 10:31:57 +0000 (12:31 +0200)] 
docs: use SOURCE_DATE_EPOCH for generated manpages

This should make builds from Git reproducible.

Closes #12092

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 12 Oct 2023 07:09:49 +0000 (09:09 +0200)] 
RELEASE-NOTES: synced

Bumped to 8.4.1

2 years agocmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection
Viktor Szakats [Wed, 11 Oct 2023 16:06:12 +0000 (16:06 +0000)] 
cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection

Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet,
aligning this with autotools. This fixes an error when doing
cross-builds and also actually detects this feature. It affected systems
not allowlisted into this, e.g. SerenityOS.

We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`.

Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979
Ref: #12095 (closed in favour of this patch)
Ref: #11964 (effort to sync cmake detections with autotools)

Reported-by: Kartatz on Github
Assisted-by: Kartatz on Github
Fixes #12093
Closes #12094

2 years agobuild: add `src/.checksrc` to source tarball
Viktor Szakats [Wed, 11 Oct 2023 08:28:31 +0000 (08:28 +0000)] 
build: add `src/.checksrc` to source tarball

Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958

Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071
Reported-by: Romain Geissler
Fixes #12084
Closes #12085

2 years agoRELEASE-NOTES: synced curl-8_4_0
Daniel Stenberg [Wed, 11 Oct 2023 05:34:19 +0000 (07:34 +0200)] 
RELEASE-NOTES: synced

2 years agoTHANKS: add contributors from 8.4.0
Daniel Stenberg [Wed, 11 Oct 2023 05:34:19 +0000 (07:34 +0200)] 
THANKS: add contributors from 8.4.0

2 years agosocks: return error if hostname too long for remote resolve
Jay Satiro [Wed, 11 Oct 2023 05:34:19 +0000 (07:34 +0200)] 
socks: return error if hostname too long for remote resolve

Prior to this change the state machine attempted to change the remote
resolve to a local resolve if the hostname was longer than 255
characters. Unfortunately that did not work as intended and caused a
security issue.

Bug: https://curl.se/docs/CVE-2023-38545.html

2 years agoCI: remove slowed-network tests
Stefan Eissing [Tue, 10 Oct 2023 08:50:17 +0000 (10:50 +0200)] 
CI: remove slowed-network tests

- remove these tests as they are currently not reliable in our CI
  setups.

curl handles the test cases, but CI sometimes fails on these due to
additional conditions. Rather than mix them in, an additional CI job
will be added in the future that is specific to them.

Closes https://github.com/curl/curl/pull/12075

2 years agolibcurl-env-dbg.3: move debug variables from libcurl-env.3
Jay Satiro [Thu, 7 Sep 2023 07:04:16 +0000 (03:04 -0400)] 
libcurl-env-dbg.3: move debug variables from libcurl-env.3

- Move documentation of libcurl environment variables used only in debug
  builds from libcurl-env into a separate document libcurl-env-dbg.

- Document more debug environment variables.

Previously undocumented or missing a description:

CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL,
CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME,
CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS,
CURL_TIME.

Closes https://github.com/curl/curl/pull/11811

2 years agotest670: increase the test timeout
Dan Fandrich [Mon, 9 Oct 2023 21:15:29 +0000 (14:15 -0700)] 
test670: increase the test timeout

This should make it more immune to loaded servers.

Ref: #11328

2 years agoMQTT: improve receive of ACKs
Stefan Eissing [Mon, 9 Oct 2023 09:36:37 +0000 (11:36 +0200)] 
MQTT: improve receive of ACKs

- add `mq->recvbuf` to provide buffering of incomplete
  ACK responses
- continue ACK reading until sufficient bytes available
- fixes test failures on low network receives

Closes #12071

2 years agoquic: fix BoringSSL build
Viktor Szakats [Sun, 8 Oct 2023 23:02:07 +0000 (23:02 +0000)] 
quic: fix BoringSSL build

Add guard around `SSL_CTX_set_ciphersuites()` use.

Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885

Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Closes #12067

2 years agotest1540: improve reliability
Stefan Eissing [Mon, 9 Oct 2023 08:30:33 +0000 (10:30 +0200)] 
test1540: improve reliability

- print that bytes have been received on pausing, but not how many

Closes #12069

2 years agotest2302: improve reliability
Stefan Eissing [Mon, 9 Oct 2023 08:18:20 +0000 (10:18 +0200)] 
test2302: improve reliability

- make result print collected write data, unless
  change in meta flags is detected
- will show same result even when data arrives via
  several writecb invocations

Closes #12068

2 years agocurl_easy_pause: set "in callback" true on exit if true
Daniel Stenberg [Sun, 8 Oct 2023 08:39:39 +0000 (10:39 +0200)] 
curl_easy_pause: set "in callback" true on exit if true

Because it might have called another callback in the mean time that then
set the bit FALSE on exit.

Reported-by: Jay Satiro
Fixes #12059
Closes #12061

2 years agoh3: add support for ngtcp2 with AWS-LC builds
Viktor Szakats [Sun, 8 Oct 2023 17:43:10 +0000 (17:43 +0000)] 
h3: add support for ngtcp2 with AWS-LC builds

```
curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS-LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile MultiSSL NTLM SSL threadsafe UnixSockets
```

Also delete an obsolete GnuTLS TODO and update the header comment in
`FindNGTCP2.cmake`.

Reviewed-by: Daniel Stenberg
Closes #12066

2 years agobuild: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros
Viktor Szakats [Sun, 8 Oct 2023 15:37:41 +0000 (15:37 +0000)] 
build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros

Syncing this up with CMake.

Source code uses the built-in `OPENSSL_IS_AWSLC` and
`OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is
necessary from the build tools.

The one use of `HAVE_BORINGSSL` in the source turned out to be no longer
necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634

autotools detects this anyway for display purposes.
CMake detects this to decide whether to use the BoringSSL-specific
crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection
result just yet (planned in #12066).

Ref: #11964

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #12065

2 years agoCI: move distcheck job from Azure Pipelines to GitHub Actions
Marc Hoersken [Sat, 17 Sep 2022 21:13:27 +0000 (23:13 +0200)] 
CI: move distcheck job from Azure Pipelines to GitHub Actions

This will allow for more trigger excludes within Azure Pipelines.

Also fixes seemingly broken check with scripts/installcheck.sh.
Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5

Assisted-by: Philip Heiduck
Closes #9532

2 years agourl: fall back to http/https proxy env-variable if ws/wss not set
Daniel Stenberg [Sat, 7 Oct 2023 19:13:04 +0000 (21:13 +0200)] 
url: fall back to http/https proxy env-variable if ws/wss not set

Reported-by: Craig Andrews
Fixes #12031
Closes #12058

2 years agocf-socket: simulate slow/blocked receives in debug
Stefan Eissing [Thu, 5 Oct 2023 08:05:12 +0000 (10:05 +0200)] 
cf-socket: simulate slow/blocked receives in debug

add 2 env variables for non-UDP sockets:
1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly
   should return EAGAIN
2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket

Closes #12035

2 years agohttp2: refused stream handling for retry
Stefan Eissing [Sat, 7 Oct 2023 14:29:58 +0000 (16:29 +0200)] 
http2: refused stream handling for retry

- answer HTTP/2 streams refused via a GOAWAY from the server to
  respond with CURLE_RECV_ERROR in order to trigger a retry
  on another connection

Reported-by: black-desk on github
Ref #11859
Closes #12054

2 years agoCURLOPT_DEBUGFUNCTION.3: warn about internal handles
Jay Satiro [Thu, 5 Oct 2023 07:19:47 +0000 (03:19 -0400)] 
CURLOPT_DEBUGFUNCTION.3: warn about internal handles

- Warn that the user's debug callback may be called with the handle
  parameter set to an internal handle.

Without this warning the user may assume that the only handles their
debug callback receives are the easy handles on which they set
CURLOPT_DEBUGFUNCTION.

This is a follow-up to f8cee8cc which changed DoH handles to inherit
the debug callback function set in the user's easy handle. As a result
those handles are now passed to the user's debug callback function.

Closes https://github.com/curl/curl/pull/12034

2 years agourl: fix typo
Jay Satiro [Sat, 7 Oct 2023 22:43:14 +0000 (18:43 -0400)] 
url: fix typo

2 years agotest458: verify --expand-output, expanding a file name accepting option
Daniel Stenberg [Sat, 7 Oct 2023 18:37:38 +0000 (20:37 +0200)] 
test458: verify --expand-output, expanding a file name accepting option

Verifies the fix in #12055 (commit f2c8086ff15e6e995e1)

2 years agotool_getparam: accept variable expansion on file names too
Daniel Stenberg [Sat, 7 Oct 2023 17:36:36 +0000 (19:36 +0200)] 
tool_getparam: accept variable expansion on file names too

Reported-by: PBudmark on github
Fixes #12048
Closes #12055

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 7 Oct 2023 21:01:06 +0000 (23:01 +0200)] 
RELEASE-NOTES: synced

2 years agomulti: do CURLM_CALL_MULTI_PERFORM at two more places
Daniel Stenberg [Sat, 7 Oct 2023 18:10:36 +0000 (20:10 +0200)] 
multi: do CURLM_CALL_MULTI_PERFORM at two more places

... when it does a state transition but there is no particular socket or
timer activity. This was made apparent when commit b5bb84c removed a
superfluous timer expiry.

Reported-by: Dan Fandrich.
Fixes #12033
Closes #12056

2 years agoGHA/linux: mbedtls 3.5.0 + minor dep bumps
Viktor Szakats [Sat, 7 Oct 2023 19:09:13 +0000 (19:09 +0000)] 
GHA/linux: mbedtls 3.5.0 + minor dep bumps

Closes #12057

2 years agoCI: bump OpenLDAP package version on FreeBSD
Dan Fandrich [Sat, 7 Oct 2023 19:37:14 +0000 (12:37 -0700)] 
CI: bump OpenLDAP package version on FreeBSD

The old one is no longer available.

2 years agodocs/libcurl/opts/Makefile.inc: add missing manpage files
Marc Hoersken [Tue, 19 Sep 2023 19:08:05 +0000 (21:08 +0200)] 
docs/libcurl/opts/Makefile.inc: add missing manpage files

Detected with #9532

2 years agotests: fix a race condition in ftp server disconnect
Dan Fandrich [Sat, 7 Oct 2023 01:18:49 +0000 (18:18 -0700)] 
tests: fix a race condition in ftp server disconnect

If a client disconnected and reconnected quickly, before the ftp server
had a chance to respond, the protocol message/ack (ping/pong) sequence
got out of sync, causing messages sent to the old client to be delivered
to the new.  A disconnect must now be acknowledged and intermediate
requests thrown out until it is, which ensures that such synchronization
problems can't occur. This problem could affect ftp, pop3, imap and smtp
tests.

Fixes #12002
Closes #12049

2 years agoappveyor: bump mingw-w64 job to gcc 13 (was: 8)
Viktor Szakats [Sat, 7 Oct 2023 10:29:39 +0000 (10:29 +0000)] 
appveyor: bump mingw-w64 job to gcc 13 (was: 8)

This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9).
Adding a modern gcc version to the tests.

(The gcc 8 job used to take around 50 minutes. The new image with gcc 13
finished in 32, 35, 34 minutes in the 3 test runs so far.)

It also adds a modern CMake version and OS env to our mingw-w64 builds.

Closes #12051

2 years agoopenssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR
David Benjamin [Thu, 5 Oct 2023 16:50:55 +0000 (12:50 -0400)] 
openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR

While the struct is still public in OpenSSL, there is a (somewhat
inconvenient) accessor. Use it to remain compatible if it becomes opaque
in the future.

Closes #12038

2 years agocurl_easy_pause.3: mention it works within callbacks
Daniel Stenberg [Fri, 6 Oct 2023 07:22:26 +0000 (09:22 +0200)] 
curl_easy_pause.3: mention it works within callbacks

Reported-by: Maxim Dzhura
Bug: https://curl.se/mail/lib-2023-10/0010.html
Closes #12046

2 years agocurl_easy_pause.3: mention h2/h3 buffering
Daniel Stenberg [Fri, 6 Oct 2023 07:11:57 +0000 (09:11 +0200)] 
curl_easy_pause.3: mention h2/h3 buffering

Asked-by: Maxim Dzhura
Ref: https://curl.se/mail/lib-2023-10/0011.html

Closes #12045

2 years agocmake: re-add missed C89 headers for specific detections
Viktor Szakats [Thu, 5 Oct 2023 23:04:40 +0000 (23:04 +0000)] 
cmake: re-add missed C89 headers for specific detections

We removed C89 `setjmp.h` and `signal.h` detections and excluded them
from the global header list we use when detecting functions [1]. Then
missed to re-add these headers to the specific functions which need
them to be detected [2]. Fix this omission in this patch.

[1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951
[2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940

Closes #12043

2 years agomulti: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE
Daniel Stenberg [Thu, 5 Oct 2023 21:29:41 +0000 (23:29 +0200)] 
multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE

Since there is nothing to wait for there. Avoids the test 1233 hang
reported in #12033.

Reported-by: Dan Fandrich
Closes #12042

2 years agotest1903: actually verify the cookies after the test 12041/head
Dan Fandrich [Thu, 5 Oct 2023 20:11:08 +0000 (13:11 -0700)] 
test1903: actually verify the cookies after the test

The test otherwise could do just about anything (except leak memory in
debug mode) and its bad behaviour wouldn't be detected. Now, check the
resulting cookie file to ensure the cookies are still there.

Closes #12041

2 years agotest: add missing <feature>s
Dan Fandrich [Thu, 5 Oct 2023 19:38:17 +0000 (12:38 -0700)] 
test: add missing <feature>s

The tests will otherwise fail if curl has them disabled.

2 years agotest1906: set a lower timeout since it's hit on Windows
Dan Fandrich [Thu, 5 Oct 2023 08:58:41 +0000 (01:58 -0700)] 
test1906: set a lower timeout since it's hit on Windows

msys2 builds actually hit the connect timeout in normal operation, so
lower the timeout from 5 minutes to 5 seconds to reduce test time.

Ref: #11328
Closes #12036

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 5 Oct 2023 07:20:27 +0000 (09:20 +0200)] 
RELEASE-NOTES: synced

2 years agoidn: fix WinIDN null ptr deref on bad host
Jay Satiro [Fri, 29 Sep 2023 07:15:19 +0000 (03:15 -0400)] 
idn: fix WinIDN null ptr deref on bad host

- Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from
  UTF-8 to UTF-16.

Prior to this change a failed conversion erroneously returned CURLE_OK
which meant 'decoded' pointer (what would normally point to the
punycode) would not be written to, remain NULL and be dereferenced
causing an access violation.

Closes https://github.com/curl/curl/pull/11983

2 years agotests: close the shell used to start sshd
Dan Fandrich [Wed, 4 Oct 2023 22:14:05 +0000 (15:14 -0700)] 
tests: close the shell used to start sshd

This shell isn't needed once sshd starts, so use "exec" so it doesn't
stick around.

Closes #12032

2 years agobase64: also build for curl
Daniel Stenberg [Mon, 2 Oct 2023 13:10:55 +0000 (15:10 +0200)] 
base64: also build for curl

Since the tool itself now uses the base64 code using the curlx way, it
needs to build also when the tool needs it. Starting now, the tool build
defines BULDING_CURL to allow lib-side code to use it.

Follow-up to 2e160c9c6525

Closes #12010

2 years agotests: Fix zombie processes left behind by FTP tests.
Eduard Strehlau [Tue, 3 Oct 2023 15:10:46 +0000 (11:10 -0400)] 
tests: Fix zombie processes left behind by FTP tests.

ftpserver.pl correctly cleans up spawned server processes,
but forgets to wait for the shell used to spawn them.
This is barely noticeable during a normal testrun,
but causes process exhaustion and test failure
during a complete torture run of the FTP tests.

Fixes #12018
Closes #12020

2 years agogithub/labeler: improve labeler matches
Dan Fandrich [Fri, 22 Sep 2023 17:30:34 +0000 (10:30 -0700)] 
github/labeler: improve labeler matches

2 years agotest574: add a timeout to the test
Dan Fandrich [Fri, 29 Sep 2023 19:28:57 +0000 (12:28 -0700)] 
test574: add a timeout to the test

This one hangs occasionally, so this will speed up a test run and allow
logs to be seen when it does.

Closes #12025

2 years agotests: propagate errors in libtests
Dan Fandrich [Fri, 29 Sep 2023 23:32:48 +0000 (16:32 -0700)] 
tests: propagate errors in libtests

Use the test macros to automatically propagate some errors, and check
and log others while running the tests. This can help in debugging
exactly why a test has failed.

2 years agotests: set --expect100-timeout to improve test reliability
Dan Fandrich [Fri, 29 Sep 2023 22:27:21 +0000 (15:27 -0700)] 
tests: set --expect100-timeout to improve test reliability

On an overloaded server, the default 1 second timeout can go by without
the test server having a chance to respond with the expected headers,
causing tests to fail. Increase the 1 second timeout to 99 seconds so
this failure mode is no longer a problem on test 1129. Some other tests
already set a high value, but make them consistently 99 seconds so if
something goes wrong the test is stalled for less time.

Ref: #11328

2 years agoCI: ignore the "flaky" and "timing-dependent" test results in CMake
Dan Fandrich [Fri, 29 Sep 2023 19:06:30 +0000 (12:06 -0700)] 
CI: ignore the "flaky" and "timing-dependent" test results in CMake

This was already done for automake builds but CMake builds were missed.
Test 1086 actually causes the test harness to crash with:

Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projects/curl/tests/ftpserver.pl line 527

Rather than fix it now, this change leaves test 1086 entirely skipped on
those builds that show this problem.

Follow-up to 589dca761

Ref: #11865

2 years agocmake: improve OpenLDAP builds
Viktor Szakats [Tue, 3 Oct 2023 18:17:37 +0000 (18:17 +0000)] 
cmake: improve OpenLDAP builds

- cmake: detect OpenLDAP based on function `ldap_init_fd`.
  autotools does this. autotools also publishes this detection result
  in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source
  doesn't use this value. (it might need to be remove-listed in
  `scripts/cmp-config.pl` for future OpenLDAP test builds.)
  This also deletes existing self-declaration method via the
  CMake-specific `CURL_USE_OPENLDAP` configuration.

- cmake: define `LDAP_DEPRECATED=1` for OpenLDAP.
  Like autotools does. This fixes a long list of these warnings:
  ```
  /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' is not defined, evaluates to 0 [-Wundef]
  ```

- cmake: delete LDAP TODO comment no longer relevant.

Also:

- autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature
  detection functions.

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #12024

2 years agocmake: fix unity builds for more build combinations
Viktor Szakats [Tue, 3 Oct 2023 21:18:01 +0000 (21:18 +0000)] 
cmake: fix unity builds for more build combinations

By using unique static function/variable names in source files
implementing these interfaces.

- OpenLDAP combined with any SSH backend.

- MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport.

Closes #12027

2 years agotests: remove leading spaces from some tags
Daniel Stenberg [Wed, 4 Oct 2023 09:35:54 +0000 (11:35 +0200)] 
tests: remove leading spaces from some tags

The threee tags `<name>`, `</name>` and `<command>` were frequently used
with a leading space that this removes. The reason this habbit is so
widespread in testcases is probably that they have been copy and pasted.

Hence, fixing them all now might curb this practice from now on.

Closes #12028

2 years agoGHA: bump actions/checkout
Viktor Szakats [Tue, 3 Oct 2023 21:58:26 +0000 (21:58 +0000)] 
GHA: bump actions/checkout

Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964
Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000

Closes #12023

2 years agospelling: fix codespell 2.2.6 typos
Viktor Szakats [Tue, 3 Oct 2023 14:44:13 +0000 (14:44 +0000)] 
spelling: fix codespell 2.2.6 typos

Closes #12019

2 years agoGHA: add workflow to compare configure vs cmake outputs
Daniel Stenberg [Wed, 27 Sep 2023 11:35:03 +0000 (13:35 +0200)] 
GHA: add workflow to compare configure vs cmake outputs

Uses scripts/cmp-config.pl two compare two curl_config.h files,
presumbly generated with configure and cmake. It displays the
differences and filters out a lot of known lines we ignore.

The script also shows the matches that were *not* used. Possibly
subjects for removal.

Closes #11964

2 years agoappveyor: enable test 571
Daniel Stenberg [Tue, 3 Oct 2023 12:25:52 +0000 (14:25 +0200)] 
appveyor: enable test 571

Follow-up from 8a940fd55c175f7 / #12013

Closes #12017

2 years agobuild: alpha-sort source files for lib and src
Viktor Szakats [Tue, 3 Oct 2023 09:38:02 +0000 (09:38 +0000)] 
build: alpha-sort source files for lib and src

Closes #12014

2 years agocmake: delete old `HAVE_LDAP_URL_PARSE` logic
Viktor Szakats [Tue, 3 Oct 2023 10:36:27 +0000 (10:36 +0000)] 
cmake: delete old `HAVE_LDAP_URL_PARSE` logic

Left there by accident after adding proper detection for this.

Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #12015

2 years agotests: increase lib571 timeout from 3s to 30s
Stefan Eissing [Tue, 3 Oct 2023 09:06:27 +0000 (11:06 +0200)] 
tests: increase lib571 timeout from 3s to 30s

- 3s is too short for our CI, making this test fail occasionally
- test usually experiences no delay run locally, so 30s wont hurt

Closes #12013

2 years agocmake: fix unity with Windows Unicode + TrackMemory
Viktor Szakats [Tue, 3 Oct 2023 02:27:05 +0000 (02:27 +0000)] 
cmake: fix unity with Windows Unicode + TrackMemory

Found the root cause of the startup crash in unity builds with Unicode
and TrackMemory enabled at the same time.

We must make sure that the `memdebug.h` header doesn't apply to
`lib/curl_multibyte.c` (as even noted in a comment there.) In unity
builds all headers apply to all sources, including `curl_multibyte.c`.
This probably resulted in an infinite loop on startup.

Exclude this source from unity compilation with TrackMemory enabled,
in both libcurl and curl tool. Enable unity mode for a debug Unicode
CI job to keep it tested. Also delete the earlier workaround that
fully disabled unity for affected builds.

Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005
Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095

Closes #11928

2 years agocmake: disable unity mode with Windows Unicode + TrackMemory
Viktor Szakats [Mon, 2 Oct 2023 01:03:43 +0000 (01:03 +0000)] 
cmake: disable unity mode with Windows Unicode + TrackMemory

"TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`,
aka `-DCURLDEBUG`).

There is an issue with memory tracking and Unicode when built in "unity"
mode, which results in the curl tool crashing right on startup, even
without any command-line option. Interestingly this doesn't happen under
WINE (at least on the system I tested this on), but consistenly happens
on real Windows machines. Crash is 0xC0000374 heap corruption. Both
shared and static curl executables are affected.

This limitation probably won't hit too many people, but it remains
a TODO to find and fix the root cause and drop this workaround.

Example builds and runs:
https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd7iwj#L313 (static)
https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tbyqu9c#L317 (shared)

Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095

Ref: #11928
Closes #12005

2 years agocmake: tidy-up `NOT_NEED_LBER_H` detection
Viktor Szakats [Mon, 2 Oct 2023 22:56:02 +0000 (22:56 +0000)] 
cmake: tidy-up `NOT_NEED_LBER_H` detection

Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006

2 years agoappveyor: rewrite batch in PowerShell + CI improvements
Viktor Szakats [Sun, 1 Oct 2023 17:32:59 +0000 (17:32 +0000)] 
appveyor: rewrite batch in PowerShell + CI improvements

1. Rewrite in PowerShell:

- rewrite MS-DOS batch build script in PowerShell.
- move some bash operations into native PowerShell.
- fixups for PowerShell insisting on failure when a command outputs
  something to stderr.
- fix to actually run `curl -V` after every build.
  (and exclude ARM64 builds.)
- also say why we skipped `curl -V` if we had to skip.
- fix CMake warnings about unused configuration variables, by adapting
  these dynamically for build cases.
- dedupe OpenSSL path into a variable.
- disable `test1451` failing with a warning anyway due to missing python
  impacket. (after trying and failing to install impacket)
  PowerShell promotes these warnings to errors by PowerShell. We can also
  suppress they wholesale if they start causing issues in the future,
  like we already to with `autoreconf` and `./configure`.

PowerShell is better than MS-DOS batches, so the hope is this makes it
easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't
supported inline by AppVeyor on Windows build machines, but we are okay
to keep it in an external script, so it's also an option.

2. CI improvements:

- enable tests for a "unity" build job.
- speed-up CI initialization by using shallow clones of the curl repo.
- speed-up CMake MSVC jobs with `TrackFileAccess=false`.
- enable parallelism in `VisualStudioSolution` builds.
- display CMake version before builds.
- always show the CPU in job names.
- tell which jobs are build-only in job names.
- move `TESTING:` value next to `DISABLED_TESTS:` in two jobs.
- add `config.log` (autotools) to dumped logs (need to enable manually).

3. Style:

- use single-quotes in YAML like we do in other CI YAML files.
  It also allows to drop quoting characters and lighter to write/read.
  (keep double quotes for PowerShell strings needing expansion.)

Closes #11999

2 years agocmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows
Viktor Szakats [Mon, 2 Oct 2023 09:57:14 +0000 (09:57 +0000)] 
cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows

- set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists.
  Before this patch we set it based it on the presence of `stricmp`,
  which correctly enabled it on e.g. Windows, but was inaccurate for
  other platforms.

- always set `HAVE_LDAP_SSL` if an LDAP backend is detected and
  LDAPS is not explicitly disabled. This mimics autotools behaviour.
  Previously we set it only for Windows LDAP. After this fix, LDAPS is
  correctly enabled in default macOS builds.

- enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks)
  to avoid deprecation warnings for LDAP API.

- always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled.
  This doesn't make much sense, but let's do it to sync behaviour with
  autotools.

- fix benign typo in variable name.

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #12006

2 years agoautotools: restore `HAVE_IOCTL_*` detections
Viktor Szakats [Mon, 2 Oct 2023 11:24:14 +0000 (11:24 +0000)] 
autotools: restore `HAVE_IOCTL_*` detections

This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in
4d73854462f30948acab12984b611e9e33ee41e6 and
c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the
`HAVE_IOCTL` result it generated was unused in the source. But,
I did miss the fact that this had two dependent checks:
`CURL_CHECK_FUNC_IOCTL_FIONBIO`,
`CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need:
`HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`.

Regression from 4d73854462f30948acab12984b611e9e33ee41e6

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #12008

2 years agoRELEASE-PROCEDURE.md: updated coming release dates
Daniel Stenberg [Mon, 2 Oct 2023 11:31:51 +0000 (13:31 +0200)] 
RELEASE-PROCEDURE.md: updated coming release dates

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 2 Oct 2023 07:41:33 +0000 (09:41 +0200)] 
RELEASE-NOTES: synced

2 years agocmake: pre-cache `HAVE_POLL_FINE` on Windows
Viktor Szakats [Sun, 1 Oct 2023 13:13:28 +0000 (13:13 +0000)] 
cmake: pre-cache `HAVE_POLL_FINE` on Windows

Windows doesn't support `poll()`, so we can safely skip checking for
fine poll.

Closes #12003

2 years agogha: bump actions to latest versions
Viktor Szakats [Sat, 30 Sep 2023 15:34:53 +0000 (15:34 +0000)] 
gha: bump actions to latest versions

- actions@checkout@v4 (from v3 and v2)

- fsfe/reuse-action@v2 (from v1)

Closes #12000

2 years agoh2: testcase and fix for pausing h2 streams
Stefan Eissing [Fri, 29 Sep 2023 12:17:08 +0000 (14:17 +0200)] 
h2: testcase and fix for pausing h2 streams

- refs #11982 where it was noted that paused transfers may
  close successfully without delivering the complete data
- made sample poc into tests/http/client/h2-pausing.c and
  added test_02_27 to reproduce

Closes #11989
Fixes #11982
Reported-by: Harry Sintonen
2 years agocmake: validate `CURL_DEFAULT_SSL_BACKEND` config value
Viktor Szakats [Sat, 30 Sep 2023 11:13:27 +0000 (11:13 +0000)] 
cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value

Before this patch CMake builds accepted any value and it was used at
runtime as-is. This patch make sure that the selected default backend
is also enabled in the build. It also enforces a full lowercase value.

This improves reproducibility and brings CMake in sync with autotools
which already worked like described above.

Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774

Closes #11998