]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
17 months agotransfer: do not use EXPIRE_NOW while blocked
Stefan Eissing [Fri, 7 Jun 2024 12:38:51 +0000 (14:38 +0200)] 
transfer: do not use EXPIRE_NOW while blocked

- When a transfer sets `data->state.select_bits`, it is
  scheduled for rerun with EXPIRE_NOW. If such a transfer
  is blocked (due to PAUSE, for example), this will lead to
  a busy loop.
- multi.c: check for transfer block
- sendf.*: add Curl_xfer_is_blocked()
- sendf.*: add client reader `is_paused()` callback
- implement is_paused()` callback where needed

Closes #13908

17 months agoci: update dependency ngtcp2/ngtcp2 to v1.6.0
renovate[bot] [Thu, 13 Jun 2024 11:39:21 +0000 (11:39 +0000)] 
ci: update dependency ngtcp2/ngtcp2 to v1.6.0

Closes #13939

17 months agoci: update ngtcp2/nghttp3 to v1.4.0
renovate[bot] [Thu, 13 Jun 2024 11:39:16 +0000 (11:39 +0000)] 
ci: update ngtcp2/nghttp3 to v1.4.0

Closes #13938

17 months agocmake: stop setting SOVERSION for the static lib target
Viktor Szakats [Thu, 13 Jun 2024 09:34:28 +0000 (11:34 +0200)] 
cmake: stop setting SOVERSION for the static lib target

Also move the logic closer to its use and related tidy-ups.

Cherry-picked from #13898
Closes #13936

17 months agoos400: make it compilable again
Patrick Monnerat [Wed, 12 Jun 2024 12:18:16 +0000 (14:18 +0200)] 
os400: make it compilable again

A newly introduced use of getsockname() in the cli tool makes it require
the ascii wrapper module, which is not available outside of the library:
as the tool only uses the address family field (binary), disable
wrappers outside of libcurl.

Fix setsockopt() parameter type mismatch using a (void *) cast.

Sync ILE/RPG binding.

Closes #13930

17 months agolibcurl.pc: add `Requires.private`, `Requires` for static linking
Viktor Szakats [Fri, 7 Jun 2024 22:41:24 +0000 (00:41 +0200)] 
libcurl.pc: add `Requires.private`, `Requires` for static linking

- cmake: populate for dependencies.
- autotools: populate for dependencies.
  (including mbedtls, though the script does not detect
  mbedtls through pkgconfig. mbedtls 3.6.0 now supports it.)

Skip dealing with gssapi in this patch.

Fixes #864
Closes #13911

17 months agocmake: bring `curl-config.cmake` closer to `FindCURL`
Viktor Szakats [Wed, 5 Jun 2024 20:35:48 +0000 (22:35 +0200)] 
cmake: bring `curl-config.cmake` closer to `FindCURL`

Set `CURL_LIBRARIES` and `CURL_INCLUDE_DIRS` variables
for compatibility with CMake's `FindCURL.cmake`:
https://github.com/Kitware/CMake/blob/b411d0146c2e06acfb0c823bb039e99f0191b611/Modules/FindCURL.cmake#L209

For dependent projects, CMake's suggestion is to replace
`CURL_LIBRARIES` with `CURL::libcurl`, and drop `CURL_INCLUDE_DIRS`.

Reported-by: Aurélien Pierre
Ref: https://curl.se/mail/lib-2024-06/0014.html
Ref: https://gitlab.kitware.com/cmake/cmake/-/issues/24580
Closes #13897

17 months agotool_getparam: fix the bsearch call for ip-tos names
Daniel Stenberg [Wed, 12 Jun 2024 22:36:40 +0000 (00:36 +0200)] 
tool_getparam: fix the bsearch call for ip-tos names

Follow-up to 3c20ae08b9591
Reported-by: Samuel Chiang
Fixes #13932
Closes #13933

17 months agorequest: change the struct field bodywrites to a bool, only for hyper
Daniel Stenberg [Wed, 12 Jun 2024 09:20:00 +0000 (11:20 +0200)] 
request: change the struct field bodywrites to a bool, only for hyper

Only hyper needs to know this, and it can use it as a boolean.

Closes #13928

17 months agotest: fix CURLOPT_TCP_KEEPCNT typo
Andy Pan [Wed, 12 Jun 2024 13:23:27 +0000 (21:23 +0800)] 
test: fix CURLOPT_TCP_KEEPCNT typo

Follow up to b77d627d242

Closes #13931

17 months agohttp: remove "struct HTTP"
Daniel Stenberg [Wed, 12 Jun 2024 09:05:44 +0000 (11:05 +0200)] 
http: remove "struct HTTP"

It is not actually used anymore and only contained a dummy struct field.
Remove all traces and uses of it.

Closes #13927

17 months agocd2nroff: convert two warnings to errors
Daniel Stenberg [Wed, 12 Jun 2024 09:38:13 +0000 (11:38 +0200)] 
cd2nroff: convert two warnings to errors

Since the warnings tend to get missed too easily and these are problems
we rather want addressed than letting slide.

Closes #13929

17 months agourlapi: use a correct value for CURLU_NO_GUESS_SCHEME
Daniel Stenberg [Wed, 12 Jun 2024 09:06:34 +0000 (11:06 +0200)] 
urlapi: use a correct value for CURLU_NO_GUESS_SCHEME

It was mistakenly set to the same value as CURLU_GET_EMPTY uses.

Reported-by: Patrick Monnerat
Bug: https://github.com/curl/curl/commit/655d44d139489625e77cf6790d36
Closes #13926

17 months agofile: separate fake headers and body with a stand-alone CRLF
Daniel Stenberg [Wed, 12 Jun 2024 06:55:07 +0000 (08:55 +0200)] 
file: separate fake headers and body with a stand-alone CRLF

Instead of bolting on the extra CRLF to the final header - as that makes
the behavior inconsistent and not as documented. The final CRLF is now
also made unconditional, just like it is for HTTP.

Reported-by: dogma
Bug: https://curl.se/mail/lib-2024-06/0033.html
Closes #13925

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 12 Jun 2024 07:37:36 +0000 (09:37 +0200)] 
RELEASE-NOTES: synced

17 months agotcpkeepalive: add CURLOPT_TCP_KEEPCNT and --keepalive-cnt
Andy Pan [Wed, 5 Jun 2024 03:30:16 +0000 (11:30 +0800)] 
tcpkeepalive: add CURLOPT_TCP_KEEPCNT and --keepalive-cnt

Closes #13885

17 months agoTODO: make it "Add missing features to TLS backends"
Daniel Stenberg [Tue, 11 Jun 2024 21:26:42 +0000 (23:26 +0200)] 
TODO: make it "Add missing features to TLS backends"

... instead of just mentioning CA caching.

Closes #13924

17 months agocurl: support VLAN Priority: --vlan-priority
Orgad Shaneh [Tue, 14 May 2024 06:54:26 +0000 (09:54 +0300)] 
curl: support VLAN Priority: --vlan-priority

Add --vlan-priority option to the command line tool for setting VLAN
priority.

Closes #13907

17 months agomisc: fix typos
RainRat [Tue, 11 Jun 2024 17:16:50 +0000 (10:16 -0700)] 
misc: fix typos

Closes #13923

17 months agoCURLOPT_ECH.md: remove repeated 'if'
Daniel Stenberg [Tue, 11 Jun 2024 12:19:52 +0000 (14:19 +0200)] 
CURLOPT_ECH.md: remove repeated 'if'

Closes #13922

17 months agovms: fixed language in comment
Daniel Stenberg [Tue, 11 Jun 2024 12:01:35 +0000 (14:01 +0200)] 
vms: fixed language in comment

It started with me fixing a repeated "are are" but the wording was
incomprehensible so I tried to untangle it.

Closes #13921

17 months agolib: xfer_setup and non-blocking shutdown
Stefan Eissing [Mon, 10 Jun 2024 11:32:13 +0000 (13:32 +0200)] 
lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
  which socket they operate on. Add a shutdown flag for secondary
  sockets
- change Curl_xfer_setup() calls to new functions
- implement non-blocking connection shutdown at the end of receiving or
  sending a transfer

Closes #13913

17 months agotest1486: verify that write-out.md and tool_writeout.c are in sync
Daniel Stenberg [Tue, 11 Jun 2024 07:30:29 +0000 (09:30 +0200)] 
test1486: verify that write-out.md and tool_writeout.c are in sync

- also verify alphabetialal order in the source
- add two missing variables to write-out.md

Closes #13920

17 months agoGHA: add cmake MSYS2 native job
Viktor Szakats [Mon, 10 Jun 2024 21:10:47 +0000 (23:10 +0200)] 
GHA: add cmake MSYS2 native job

curl, libcurl, examples, build-only.

To compare build behaviour with autotools.

Closes #13917

17 months agoopenssl: shortcut store_expired for negative timeouts
Daniel Stenberg [Tue, 11 Jun 2024 06:03:28 +0000 (08:03 +0200)] 
openssl: shortcut store_expired for negative timeouts

Avoid some unnecessary computation if the timeout is negative.

Spotted by CodeSonar
Closes #13919

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 11 Jun 2024 07:37:32 +0000 (09:37 +0200)] 
RELEASE-NOTES: synced

17 months agocurl: support -w '%{num_retries}
Daniel Stenberg [Fri, 7 Jun 2024 22:18:44 +0000 (00:18 +0200)] 
curl: support -w '%{num_retries}

Suggested-by: Jay Guerette
Ref: https://github.com/curl/curl/discussions/13901
Closes #13910

17 months agopytest: include testenv/vsftpd.py in dist tarball
Guilherme Puida [Mon, 10 Jun 2024 23:01:38 +0000 (20:01 -0300)] 
pytest: include testenv/vsftpd.py in dist tarball

Closes #13918

17 months agoDISTROS: add MSYS2 (native) links
Viktor Szakats [Mon, 10 Jun 2024 20:36:16 +0000 (22:36 +0200)] 
DISTROS: add MSYS2 (native) links

Also rename existing 'MSYS2' to 'MSYS2 (mingw-w64)'.

Closes #13915

17 months agotool_writeout: get certinfo only when needing it
Daniel Stenberg [Mon, 10 Jun 2024 11:57:01 +0000 (13:57 +0200)] 
tool_writeout: get certinfo only when needing it

Removes a fairly expensive libcurl call when not necessary

Closes #13914

17 months agotool_writeout: bsearch the variable name
Daniel Stenberg [Mon, 10 Jun 2024 11:33:16 +0000 (13:33 +0200)] 
tool_writeout: bsearch the variable name

As the list of variable names grows, doing a simple loop to find the
name get increasingly worse. This switches to a bsearch.

Also: do a case sensitive check for the variable name. The names have
not been documented to be case insensitive and there is no point in
having them so.

Closes #13914

17 months agomulti: prepare multi_wait() for future shutdown usage
Stefan Eissing [Thu, 6 Jun 2024 10:40:38 +0000 (12:40 +0200)] 
multi: prepare multi_wait() for future shutdown usage

- new struct curl_pollfds and struct curl_waitfds
- add structs and methods to init/add/cleanup an array of pollfd and
  struct curl_waitfd. Use in multi_wait() and multi_waitfds() to
  populate the sets for polling.
- place USE_WINSOCK WSAEventSelect() setting into a separate loop over
  all collected pfds

Closes #13900

17 months agoconnection: shutdown TLS (for FTP) better
Stefan Eissing [Fri, 7 Jun 2024 08:12:39 +0000 (10:12 +0200)] 
connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.

    - add cfilter `Curl_cft_shutdown` callback
    - keep a shutdown start timestamp and timeout at connectdata
    - provide shutdown timeout default and member in
      `data->set.shutdowntimeout`.
    - provide methods for starting, interrogating and clearing
      shutdown timers
    - provide `Curl_conn_shutdown_blocking()` to shutdown the
      `sockindex` filter chain in a blocking way. Use that in FTP.
    - add `Curl_conn_cf_poll()` to wait for socket events during
      shutdown of a connection filter chain.
      This gets the monitoring sockets and events via the filters
      "adjust_pollset()" methods. This gives correct behaviour when
      shutting down a TLS connection through a HTTP/2 proxy.
    - Implement shutdown for all socket filters
      - for HTTP/2 and h2 proxying to send GOAWAY
      - for TLS backends to the best of their capabilities
      - for tcp socket filter to make a final, nonblocking
        receive to avoid unwanted RST states
    - add shutdown forwarding to happy eyeballers and
      https connect ballers when applicable.

Closes #13904

17 months agoCURLOPT_CONNECTTIMEOUT*: clarify, document the milliseond version
Daniel Stenberg [Fri, 7 Jun 2024 08:44:28 +0000 (10:44 +0200)] 
CURLOPT_CONNECTTIMEOUT*: clarify, document the milliseond version

Provide an explanation in the CURLOPT_CONNECTTIMEOUT_MS page instead of
just referring to the non-MS version.

Closes #13905

17 months agocmdline-opts: tidy up --ip-tos and --mptcp
Daniel Stenberg [Fri, 7 Jun 2024 09:02:46 +0000 (11:02 +0200)] 
cmdline-opts: tidy up --ip-tos and --mptcp

To make them render nicer in the manpage and minor polish.

Closes #13906

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 7 Jun 2024 09:08:46 +0000 (11:08 +0200)] 
RELEASE-NOTES: synced

17 months agocurl: (on linux) add MPTCP support
Dorian Craps [Thu, 4 Apr 2024 09:42:50 +0000 (11:42 +0200)] 
curl: (on linux) add MPTCP support

Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension
that enables a TCP connection to use different paths.

Multipath TCP has been used for several use cases. On smartphones, MPTCP
enables seamless handovers between cellular and Wi-Fi networks while
preserving established connections. This use-case is what pushed Apple
to use MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to automatically use the
best performing path, either IPv4 or IPv6. If one path fails, MPTCP
automatically uses the other path.

To benefit from MPTCP, both the client and the server have to support
it. Multipath TCP is a backward-compatible TCP extension that is enabled
by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...).
Multipath TCP is included in the Linux kernel since version 5.6 [3]. To
use it on Linux, an application must explicitly enable it when creating
the socket. No need to change anything else in the application.

This attached patch adds an --mptcp option which allows the creation of
an MPTCP socket instead of TCP on Linux. If Multipath TCP is not
supported on the system, an error will be reported. It is important to
note that if the end server doesn't support MPTCP, the connection will
continue after a seamless fallback to TCP.

Link: https://www.rfc-editor.org/rfc/rfc8684.html
Link: https://www.tessares.net/apples-mptcp-story-so-far/
Link: https://www.mptcp.dev
Co-developed-by: Dorian Craps (@CrapsDorian) <doriancraps@gmail.com>
Co-developed-by: Olivier Bonaventure (@obonaventure) <Olivier.Bonaventure@uclouvain.be>
Co-developed-by: Matthieu Baerts (@matttbe) <matttbe@kernel.org>
Signed-off-by: Dorian Craps <dorian.craps@student.vinci.be>
Closes #13278

17 months agocurl: support IP Type of Service / Traffic Class: --ip-tos
Orgad Shaneh [Sun, 12 May 2024 05:31:15 +0000 (08:31 +0300)] 
curl: support IP Type of Service / Traffic Class: --ip-tos

Add --ip-tos option to the command line tool for setting TOS for IPv4 or
Traffic Class for IPv6.

Closes #13606

17 months agosocketpair: provide `Curl_socketpair` only when `!CURL_DISABLE_SOCKETPAIR`
Andy Pan [Wed, 5 Jun 2024 08:08:15 +0000 (16:08 +0800)] 
socketpair: provide `Curl_socketpair` only when `!CURL_DISABLE_SOCKETPAIR`

Ref: https://curl.se/dev/log.cgi?id=20240605035856-3529577

Reported-by: Marcel Raad
Closes #13888

17 months agonoproxy: test bad ipv6 net size first
Daniel Stenberg [Thu, 6 Jun 2024 20:58:45 +0000 (22:58 +0200)] 
noproxy: test bad ipv6 net size first

No need to parse anything if the size is out of range.

Added some tests to this effect to test 1614.

Closes #13902

17 months agomanagen: warn on excessively long help texts
Daniel Stenberg [Wed, 5 Jun 2024 12:08:02 +0000 (14:08 +0200)] 
managen: warn on excessively long help texts

Help texts at 49 characters or longer get a warning displayed because
they make --help output uglier and we should make an effort to keep the
help texts short and succinct.

The warning is only for display, it does not break the build. That is
left for the future if necessary.

I picked 49 because the longest current text is 48.

Closes #13895

17 months agolib: tidy up types and casts
Viktor Szakats [Sun, 2 Jun 2024 20:30:52 +0000 (22:30 +0200)] 
lib: tidy up types and casts

Cherry-picked from #13489
Closes #13862

17 months agocmdline-opts/ech.md: shorten the help text
Daniel Stenberg [Wed, 5 Jun 2024 11:44:02 +0000 (13:44 +0200)] 
cmdline-opts/ech.md: shorten the help text

To make --help look sensible again

Closes #13894

17 months agocmdline-opts/_PROTOCOLS.md: mention WS(S)
Daniel Stenberg [Wed, 5 Jun 2024 09:08:33 +0000 (11:08 +0200)] 
cmdline-opts/_PROTOCOLS.md: mention WS(S)

Closes #13891

17 months agoGHA: disable TFTP and WebSockets tests in old-mingw-w64
Viktor Szakats [Wed, 5 Jun 2024 10:35:59 +0000 (12:35 +0200)] 
GHA: disable TFTP and WebSockets tests in old-mingw-w64

Follow-up to 03bd16e5339b069aa9409b75fcab2b21fd3a4b16 #13860
Follow-up to def7d05382743ea7aa1d356d1e41dcb22ecdd4d7

17 months agocmdline-opts/fail.md: expand and clarify
Daniel Stenberg [Wed, 5 Jun 2024 08:52:29 +0000 (10:52 +0200)] 
cmdline-opts/fail.md: expand and clarify

Closes #13890

17 months agodoh-insecure.md: expand
Daniel Stenberg [Wed, 5 Jun 2024 08:18:21 +0000 (10:18 +0200)] 
doh-insecure.md: expand

Closes #13889

17 months agocmdline: expand proxy option explanations
Daniel Stenberg [Wed, 5 Jun 2024 08:10:47 +0000 (10:10 +0200)] 
cmdline: expand proxy option explanations

- do less references to other options
- provide more specific text about proxies
- added more see-also references

Closes #13887

17 months agocmdline-opts: expand the parallel explanations
Daniel Stenberg [Wed, 5 Jun 2024 07:34:32 +0000 (09:34 +0200)] 
cmdline-opts: expand the parallel explanations

Closes #13886

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Wed, 5 Jun 2024 07:09:17 +0000 (09:09 +0200)] 
RELEASE-NOTES: synced

17 months agovtls: new io_need flags for poll handling
Stefan Eissing [Tue, 4 Jun 2024 12:00:12 +0000 (14:00 +0200)] 
vtls: new io_need flags for poll handling

- decouple need to recv/send from negotiation state, we need
  this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends

Closes #13879

17 months agocfilters: make Curl_conn_connect always assign 'done'
Daniel Stenberg [Wed, 5 Jun 2024 06:31:11 +0000 (08:31 +0200)] 
cfilters: make Curl_conn_connect always assign 'done'

It could return error without assigning it, and we have a caller in
multi.c that assumes it gets set.

Spotted by CodeSonar
Closes #13884

17 months agoCURLOPT_INTERFACE.md: quote the less-than and larger-than
Daniel Stenberg [Wed, 5 Jun 2024 06:29:50 +0000 (08:29 +0200)] 
CURLOPT_INTERFACE.md: quote the less-than and larger-than

Fixes the warnings shown on stderr.

Follow-up from 3060557af702dd5

Closes #13883

17 months agocmdline-opts/interface.md: expand the documentation
Daniel Stenberg [Tue, 4 Jun 2024 22:06:28 +0000 (00:06 +0200)] 
cmdline-opts/interface.md: expand the documentation

Explain the syntax it supports.

Closes #13882

17 months agourl: allow DoH transfers to override max connection limit
Daniel Stenberg [Tue, 4 Jun 2024 15:00:05 +0000 (17:00 +0200)] 
url: allow DoH transfers to override max connection limit

When reaching the set maximum limit of allowed connections, allow a new
connection anyway if the transfer is created for the (internal) purpose
of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can
starve out new DoH requests making it impossible to name resolve for new
transfers.

Bug: https://curl.se/mail/lib-2024-06/0001.html
Reported-by: kartatz
Closes #13880

17 months agowindows: fix UWP builds, add GHA job
Viktor Szakats [Mon, 3 Jun 2024 21:06:56 +0000 (23:06 +0200)] 
windows: fix UWP builds, add GHA job

Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).

Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
  and make `schannel_checksum()` a stub.
  Ref: e178fbd40a896f2098278ae61e1166c88e7b31d0 #1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.

See individual commits for build error details.

Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.

Closes #13870

17 months agosocket: support binding to interface *AND* IP
Orgad Shaneh [Fri, 17 May 2024 11:44:44 +0000 (14:44 +0300)] 
socket: support binding to interface *AND* IP

Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>

Binding to an interface doesn't set the address, and an interface can
have multiple addresses.

When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.

Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.

Factor out the parsing part of interface option, and add unit tests:
1663.

Closes #13719

17 months agosocketpair: add `eventfd` and use `SOCK_NONBLOCK` for `socketpair()`
Andy Pan [Sat, 1 Jun 2024 18:26:11 +0000 (02:26 +0800)] 
socketpair: add `eventfd` and use `SOCK_NONBLOCK` for `socketpair()`

Currently, we use `pipe` for `wakeup_create`, which requires ***two***
file descriptors. Furthermore, given its complexity inside, `pipe` is a
bit heavyweight for just a simple event wait/notify mechanism.

`eventfd` would be a more suitable solution for this kind of scenario,
kernel also advocates for developers to use `eventfd` instead of `pipe`
in some simple use cases:

    Applications can use an eventfd file descriptor instead of a pipe
    (see pipe(2) in all cases where a pipe is used simply to signal
    events. The kernel overhead of an eventfd file descriptor is much
    lower than that of a pipe, and only one file descriptor is required
    (versus the two required for a pipe).

This change adds the new backend of `eventfd` for `wakeup_create` and
uses it where available, eliminating the overhead of `pipe`. Also, it
optimizes the `wakeup_create` to eliminate the system calls that make
file descriptors non-blocking by moving the logic of setting
non-blocking flags on file descriptors to `socketpair.c` and using
`SOCK_NONBLOCK` for `socketpair(2)`, `EFD_NONBLOCK` for `eventfd(2)`.

Ref:
https://man7.org/linux/man-pages/man7/pipe.7.html
https://man7.org/linux/man-pages/man2/eventfd.2.html
https://man7.org/linux/man-pages/man2/socketpair.2.html
https://www.gnu.org/software/gnulib/manual/html_node/eventfd.html

Closes #13874

17 months agoci: update github/codeql-action digest to 2e230e8
renovate[bot] [Tue, 4 Jun 2024 15:52:27 +0000 (15:52 +0000)] 
ci: update github/codeql-action digest to 2e230e8

Closes #13881

17 months agoexamples/threaded-ssl: remove locking callback code
Jay Satiro [Sat, 1 Jun 2024 18:41:41 +0000 (14:41 -0400)] 
examples/threaded-ssl: remove locking callback code

- Remove the locking callback code that demonstrates how to meet
  requirements of threading libraries (mainly OpenSSL).

Locking callback code has not been needed for many years. According to
the documentation for OpenSSL and GnuTLS they are thread-safe by design,
assuming support for the underlying OS threading API is built-in.

Ref: https://github.com/curl/curl/pull/13850#issuecomment-2143538458

Closes https://github.com/curl/curl/pull/13851

17 months agotests: delete redundant `!MSDOS` guard
Viktor Szakats [Tue, 4 Jun 2024 11:40:13 +0000 (13:40 +0200)] 
tests: delete redundant `!MSDOS` guard

This fix was supposed to be committed earlier, but ended up missing from
the final commit.

Follow-up to e9a7d4a1c8377dbcf9a2d94365f60e3e5dff48f8 #12376
Closes #13878

17 months agolib: fix thread entry point to return `DWORD` on WinCE
Viktor Szakats [Tue, 4 Jun 2024 00:41:26 +0000 (02:41 +0200)] 
lib: fix thread entry point to return `DWORD` on WinCE

We already do this in `tests/server/util.c`:
https://github.com/curl/curl/blob/97e5e37cc8269660bc5d4a1936f10f2390b97c5a/tests/server/util.c#L604-L606
and in `sockfilt.c`, `lib3026.c`.

Before this patch it returned `unsigned int`.

Closes #13877

17 months agosocket: use SOCK_NONBLOCK to eliminate extra system call
Andy Pan [Sun, 2 Jun 2024 06:40:42 +0000 (14:40 +0800)] 
socket: use SOCK_NONBLOCK to eliminate extra system call

Every time function `cf_socket_open()` is called to create a socket,
`curlx_nonblock()` is called to make that socket non-blocking. And
`curlx_nonblock()` will cost us 1 or 2 system calls (2 for `fcntl()`, 1
for `ioctl()`, etc.), meanwhile, tucking `SOCK_NONBLOCK` and
`SOCK_CLOEXEC` into the `type` argument for `socket()` is widely
supported across UNIX-like OS: Linux, *BSD, Solaris, etc. With that
ability, we can save 1 or 2 system calls on each socket.

Another change in this PR is to eliminate the redundant
`curlx_nonblock()` call on the socket in `cf_udp_setup_quic()` as that
socket created by `cf_socket_open()` is already non-blocking.

Ref:
https://man7.org/linux/man-pages/man2/socket.2.html
https://man.freebsd.org/cgi/man.cgi?socket(2)
https://man.dragonflybsd.org/?command=socket&section=2
https://man.netbsd.org/socket.2
https://man.openbsd.org/socket
https://docs.oracle.com/cd/E88353_01/html/E37843/socket-3c.html
https://illumos.org/man/3SOCKET/socket
...

Closes #13855

17 months agoGHA: show cmake error log in Windows and non-native workflows
Viktor Szakats [Tue, 4 Jun 2024 00:46:05 +0000 (02:46 +0200)] 
GHA: show cmake error log in Windows and non-native workflows

CMake configure doesn't fail often, but when it does, it helps to see
its `CMakeFiles/CMakeConfigureLog.yaml` output. This file is present
since CMake v3.26:
  https://cmake.org/cmake/help/v3.26/manual/cmake-configure-log.7.html

(Older CMake versions save similar contend to
`CMakeFiles\CMakeOutput.log` and
`CMakeFiles\CMakeError.log`. This patch doesn't deal with that because
the workflows touched are all running a newer CMake.)

After this patch, we dump the content if cmake fails. Syncing this with
autotools, where we already did that.

Closes #13872

17 months agoGHA: switch a Windows job to UCRT (gcc)
Viktor Szakats [Tue, 4 Jun 2024 09:20:20 +0000 (11:20 +0200)] 
GHA: switch a Windows job to UCRT (gcc)

Cherry-picked from #13870

17 months agocurl-config: revert to backticks to support old target envs
Viktor Szakats [Tue, 4 Jun 2024 02:19:32 +0000 (04:19 +0200)] 
curl-config: revert to backticks to support old target envs

Make an exception for `curl-config` because this script that may be
running on any target system, including old ones, e.g. SunOS 5.10.

Reported-by: Alejandro R. Sedeño
Ref: https://github.com/curl/curl/pull/13307#issuecomment-2146427358
Follow-up to fa69b41c7790fab86fd363242c81d8ef2e89e183 #13307
Closes #13871

17 months agombedtls: v3.6.0 workarounds
Stefan Eissing [Fri, 31 May 2024 11:01:17 +0000 (13:01 +0200)] 
mbedtls: v3.6.0 workarounds

- add special sauce to disable unwanted peer verification by mbedtls
  when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
  return code on *writing* TLS data. We assume the data had not been
  written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
  memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0

Fixes #13653
Closes #13838

17 months agognutls: support CA caching
Stefan Eissing [Mon, 27 May 2024 14:50:15 +0000 (16:50 +0200)] 
gnutls: support CA caching

- similar to openssl, use a shared 'credentials' instance
  among TLS connections with a plain configuration.
- different to openssl, a connection with a client certificate
  is not eligible to sharing.
- document CURLOPT_CA_CACHE_TIMEOUT in man page

Closes #13795

17 months agotests: don't log buffer length in throwing away message
Dan Fandrich [Tue, 4 Jun 2024 05:19:14 +0000 (22:19 -0700)] 
tests: don't log buffer length in throwing away message

It's not available at that point, and it will be written in the
non-error case right afterward.

17 months agotests: log "Throwing away" messages before throwing away
Dan Fandrich [Mon, 3 Jun 2024 20:25:29 +0000 (13:25 -0700)] 
tests: log "Throwing away" messages before throwing away

In case the read that follows hangs we'll get a clue as to what it was
doing.

17 months agoCI: reduce memory request for FreeBSD builds
Dan Fandrich [Fri, 31 May 2024 01:39:12 +0000 (18:39 -0700)] 
CI: reduce memory request for FreeBSD builds

Also, add a comment with link to the Cirrus credit page since it's not
easy to find otherwise.

17 months agotcpkeepalive: support setting TCP keep-alive parameters on Solaris <11.4
Andy Pan [Mon, 3 Jun 2024 07:15:55 +0000 (15:15 +0800)] 
tcpkeepalive: support setting TCP keep-alive parameters on Solaris <11.4

Solaris didn't support TCP_KEEPIDLE and TCP_KEEPINTVL until 11.4,
before that it use TCP_KEEPALIVE_THRESHOLD and TCP_KEEPALIVE_ABORT_THRESHOLD
as the substitute. Therefore, for Solaris <11.4 we need to use this substitute
for setting TCP keep-alive parameters.

Ref:
https://docs.oracle.com/cd/E86824_01/html/E54777/tcp-7p.html
https://docs.oracle.com/cd/E88353_01/html/E37851/tcp-4p.html

Closes #13864

17 months agoKNOWN_BUGS: quiche: QUIC connection is draining
Daniel Stenberg [Mon, 3 Jun 2024 08:10:47 +0000 (10:10 +0200)] 
KNOWN_BUGS: quiche: QUIC connection is draining

Closes #12037
Closes #13867

17 months agoKNOWN_BUGS: aws-sigv4 has problems with particular URLs
Daniel Stenberg [Mon, 3 Jun 2024 08:04:50 +0000 (10:04 +0200)] 
KNOWN_BUGS: aws-sigv4 has problems with particular URLs

Closes #13058
Closes #13866

17 months agoKNOWN_BUGS: aws-sigv4 does not handle multipart/form-data correctly
Daniel Stenberg [Mon, 3 Jun 2024 08:02:33 +0000 (10:02 +0200)] 
KNOWN_BUGS: aws-sigv4 does not handle multipart/form-data correctly

Closes #13351
Closes #13866

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 3 Jun 2024 08:43:06 +0000 (10:43 +0200)] 
RELEASE-NOTES: synced

17 months agoGHA: fix old mingw-w64 32-bit job
Viktor Szakats [Sun, 2 Jun 2024 23:11:47 +0000 (01:11 +0200)] 
GHA: fix old mingw-w64 32-bit job

This toolchain resides in the `mingw32` directory. Make sure to
configure `PATH` accordingly.

Before this patch, it pointed to a non-existing `mingw64` directory,
making the job use the wrong compiler (gcc 12, 64-bit).

Follow-up to e838b341a08b44d4a8486fb0d3f15d12fc794c62 #12927
Closes #13863

17 months agotool_cb_hdr: return error for failed header writes
Daniel Stenberg [Sun, 2 Jun 2024 15:16:58 +0000 (17:16 +0200)] 
tool_cb_hdr: return error for failed header writes

By checking that fflush() works.

Reported-by: Sebastian Andersson
Fixes #13836
Closes #13859

17 months agoGHA: bump all build jobs to nproc+1
Viktor Szakats [Tue, 28 May 2024 07:01:49 +0000 (09:01 +0200)] 
GHA: bump all build jobs to nproc+1

- bump rest of the workflows (windows, macos, distrocheck).

- non-native virtualized envs have 2 CPUs, bump down accordingly.
  (for `vmactions/omnios-vm` it's just a guess.)

- bump all to nproc + 1.

Follow-up to e838b341a08b44d4a8486fb0d3f15d12fc794c62 #12927
Closes #13807

17 months agoGHA: disable MQTT and WebSocket tests in Windows jobs
Viktor Szakats [Sun, 2 Jun 2024 17:30:19 +0000 (19:30 +0200)] 
GHA: disable MQTT and WebSocket tests in Windows jobs

Trying to figure out which category is causing the remaining hangs.

Follow-up to def7d05382743ea7aa1d356d1e41dcb22ecdd4d7
Closes #13860

17 months agolib/v*: tidy up types and casts
Viktor Szakats [Mon, 13 May 2024 15:56:19 +0000 (17:56 +0200)] 
lib/v*: tidy up types and casts

Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622

17 months agoGHA: fix caching old mingw-w64 toolchains in the Windows workflow
Viktor Szakats [Sun, 2 Jun 2024 16:22:48 +0000 (18:22 +0200)] 
GHA: fix caching old mingw-w64 toolchains in the Windows workflow

- stop altering the `PATH` via `GITHUB_ENV`. This confused the
  `actions/cache` post-job, which needs to run in the exact same
  environment as its pre-job, to have a consistent cache entry "version"
  hash. Altering the `PATH` via `GITHUB_ENV` spills into the the
  post-job and breaks this hash. GHA doesn't reset the env automatically
  and I have not found a way to do it manually.

- add double-quotes where missing.

- move cache directory under `USERPROFILE` to not rely on absolute
  paths.

- make cache directory flatter and versionless.

Follow-up to 0914d8aadddac0d1459673d5b7f77e8f3378b22b #13759
Closes #13856

17 months agoci: pin actions/github-script action to 60a0d83
renovate[bot] [Sat, 1 Jun 2024 08:58:26 +0000 (08:58 +0000)] 
ci: pin actions/github-script action to 60a0d83

Closes #13846

17 months agox509asn1: add some common ECDSA OIDs
Bo Anderson [Sun, 2 Jun 2024 13:17:10 +0000 (14:17 +0100)] 
x509asn1: add some common ECDSA OIDs

Closes #13857

17 months agoci: update rojopolis/spellcheck-github-actions digest to e36f662
renovate[bot] [Sat, 1 Jun 2024 18:44:11 +0000 (18:44 +0000)] 
ci: update rojopolis/spellcheck-github-actions digest to e36f662

Closes #13852

17 months agox509asn1: fallback to dotted OID representation
Bo Anderson [Sun, 2 Jun 2024 13:18:20 +0000 (14:18 +0100)] 
x509asn1: fallback to dotted OID representation

Reported-by: Luke Hamburg
Fixes #13845
Closes #13858

17 months agorequest.md: language fix
Lee Li [Sun, 2 Jun 2024 01:42:52 +0000 (09:42 +0800)] 
request.md: language fix

improved for better readability and correctness

Closes #13854

17 months agovtls: deprioritize Secure Transport
MonkeybreadSoftware [Tue, 7 May 2024 07:45:15 +0000 (09:45 +0200)] 
vtls: deprioritize Secure Transport

Moved Secure Transport behind OpenSSL, so we can build CURL with both
and prefer using OpenSSL over Secure Transport by default.

Closes #13547

17 months agourlapi: add CURLU_NO_GUESS_SCHEME
Daniel Stenberg [Sat, 1 Jun 2024 10:03:34 +0000 (12:03 +0200)] 
urlapi: add CURLU_NO_GUESS_SCHEME

Used for extracting:

- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
  stored information was a guess

- when used asking for a URL, the URL is returned without a scheme, like
  when previously given to the URL parser when it was asked to guess

- as soon as the scheme is set explicitly, it is no longer internally
  marked as guessed

The idea being:

1. allow a user to figure out if a URL's scheme was set as a result of
  guessing

2. extract the URL without a guessed scheme

3. this makes it work similar to how we already deal with port numbers

Extend test 1560 to verify.

Closes #13616

17 months agowolfssl: support CA caching
Daniel Stenberg [Sat, 1 Jun 2024 10:24:42 +0000 (12:24 +0200)] 
wolfssl: support CA caching

As a bonus, add SSLSUPP_CA_CACHE to let TLS backends signal its support
for this so that *setopt() return error if there is no support.

Closes #13786

17 months agosocket: change TCP keepalive from ms to seconds on DragonFly BSD
Andy Pan [Sat, 1 Jun 2024 10:36:04 +0000 (18:36 +0800)] 
socket: change TCP keepalive from ms to seconds on DragonFly BSD

DragonFly BSD changed the time unit for TCP keep-alive from milliseconds
to seconds since v5.8, thus setting the keepalive options with
milliseconds with curl/libcurl will result in unexpected behaviors on
DragonFlyBSD 5.8+

Distinguish the DragonFly BSD versions and use the proper time units
accordingly.

Ref:
  https://lists.dragonflybsd.org/pipermail/commits/2019-July/719125.html
  https://github.com/DragonFlyBSD/DragonFlyBSD/blob/965b380e960908836b97aa034fa2753091e0172e/sys/sys/param.h#L207

Fixes #13847
Closes #13848

17 months agocurlver.h: aiming for 8.9.0
Daniel Stenberg [Sat, 1 Jun 2024 10:26:20 +0000 (12:26 +0200)] 
curlver.h: aiming for 8.9.0

17 months agonoproxy: patterns need to be comma separated
Daniel Stenberg [Sat, 1 Jun 2024 10:25:12 +0000 (12:25 +0200)] 
noproxy: patterns need to be comma separated

or they will not parse correctly.

Mentioned in DEPRECATED since Janurary 2023 (in 7ad8a7ba9ebdedc).

Closes #13789

17 months agosectransp: remove large cipher table
Jan Venekamp [Wed, 29 May 2024 11:58:53 +0000 (13:58 +0200)] 
sectransp: remove large cipher table

Previously a large table of ciphers was used to determine the default
ciphers and to lookup manually selected ciphers names.

With the lookup of the manually selected cipher names moved to
Curl_cipher_suite_walk_str() the large table is no longer needed for
that purpose.

The list of manually selected cipher can now be intersected with the
ciphers supported by Secure Transport (SSLGetSupportedCiphers()),
instead of using the fixed table for that.

The other use of the table was to filter the list of all supported
ciphers offered by Secure Transport to create a list of ciphers to
use by default, excluding ciphers in the table marked as weak.

Instead of using a complement based approach (exclude weak), switch
to using an intersection with a smaller list of ciphers deemed
appropriate.

Closes #13823

17 months agoGHA: unify http3 workflows into one
Tatsuhiro Tsujikawa [Tue, 21 May 2024 09:50:57 +0000 (18:50 +0900)] 
GHA: unify http3 workflows into one

This commit unifies the following http3 workflows into http3-linux.yml:

- ngtcp2-linux.yml
- osslq-linux.yml
- quiche-linux.yml

The idea is better use of the build cache. Previously, they
independently create caches with the same key. Some of the caches
include source code and intermediate object files, which makes cache
quite large. In this commit, only built artifacts are cached, which
drastically reduces the cache size. OpenSSL v3, mod_h2 and quiche caches
still include all stuff, but they are left for the later improvement.
Because the contents of the cache have been changed, the cache keys are
also changed to include the word "http3".

Closes #13841

17 months agoopenSSL: fix hostname handling when using ECH
Stephen Farrell [Wed, 29 May 2024 11:32:32 +0000 (12:32 +0100)] 
openSSL: fix hostname handling when using ECH

Reported-by: vvb2060
Fixes #13818
Closes #13822

17 months agoci: update github/codeql-action digest to f079b84
renovate[bot] [Fri, 31 May 2024 09:55:28 +0000 (09:55 +0000)] 
ci: update github/codeql-action digest to f079b84

Closes #13837

17 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 1 Jun 2024 08:40:20 +0000 (10:40 +0200)] 
RELEASE-NOTES: synced

17 months agocurl_multi_poll.md: expand the example with an custom file descriptor
Daniel Stenberg [Fri, 31 May 2024 12:42:45 +0000 (14:42 +0200)] 
curl_multi_poll.md: expand the example with an custom file descriptor

Closes #13842

17 months agoDISTROS: add a link to the list archive
Christian Heusel [Fri, 31 May 2024 12:44:02 +0000 (14:44 +0200)] 
DISTROS: add a link to the list archive

Related to https://github.com/curl/curl/discussions/13833

Signed-off-by: Christian Heusel <christian@heusel.eu>
Closes #13843