]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
21 months agodocs: introduce "curldown" for libcurl man page format
Daniel Stenberg [Wed, 17 Jan 2024 10:32:44 +0000 (11:32 +0100)] 
docs: introduce "curldown" for libcurl man page format

curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:

- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
  their man page section is specified)

tools:

- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown

This setup generates .3 versions of all the curldown versions at build time.

CI:

Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...

Closes #12730

21 months agolibssh2: use `libssh2_session_callback_set2()` with v1.11.1
Viktor Szakats [Mon, 22 Jan 2024 11:21:10 +0000 (11:21 +0000)] 
libssh2: use `libssh2_session_callback_set2()` with v1.11.1

To avoid a local hack to pass function pointers and to avoid
deprecation warnings when building with libssh2 v1.11.1 or newer:
```
lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/20720821100#step:3:4982

Ref: https://github.com/libssh2/libssh2/pull/1285
Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db3cf1d4b0b4
Reviewed-by: Daniel Stenberg
Closes #12754

21 months agotransfer: make the select_bits_paused condition check both directions
Daniel Stenberg [Mon, 22 Jan 2024 15:22:19 +0000 (16:22 +0100)] 
transfer: make the select_bits_paused condition check both directions

If there is activity in a direction that is not paused, return false.

Reported-by: Sergey Bronnikov
Bug: https://curl.se/mail/lib-2024-01/0049.html
Closes #12740

21 months agohttp3: initial support for OpenSSL 3.2 QUIC stack
Stefan Eissing [Thu, 18 Jan 2024 12:07:07 +0000 (13:07 +0100)] 
http3: initial support for OpenSSL 3.2 QUIC stack

- HTTP/3 for curl using OpenSSL's own QUIC stack together
  with nghttp3
- configure with `--with-openssl-quic` to enable curl to
  build this. This requires the nghttp3 library
- implementation with the following restrictions:
  * macOS has to use an unconnected UDP socket due to an
    issue in OpenSSL's datagram implementation
    See https://github.com/openssl/openssl/issues/23251
    This makes connections to non-reponsive servers hang.
  * GET requests will send the indicator that they have
    no body in a separate QUIC packet. This may result
    in processing delays or Transfer-Encodings on proxied
    requests
  * uploads that encounter blocks will use 100% cpu as
    detection of these flow control issue is not working
    (we have not figured out to pry that from OpenSSL).

Closes #12734

21 months agocmake: fix `ENABLE_MANUAL` option
Viktor Szakats [Sun, 21 Jan 2024 15:38:09 +0000 (15:38 +0000)] 
cmake: fix `ENABLE_MANUAL` option

Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`.

Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the
option designed to enable building and using the built-in curl manual.
(`USE_MANUAL=ON` option worked for this instead, by accident).

Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409
Closes #12749

21 months agoTODO: update broken link to ratelimit-headers draft
Mohammadreza Hendiani [Fri, 19 Jan 2024 13:59:03 +0000 (17:29 +0330)] 
TODO: update broken link to ratelimit-headers draft

Closes #12741

21 months agocmake: when USE_MANUAL=YES, build the curl.1 man page
Daniel Stenberg [Fri, 19 Jan 2024 14:03:50 +0000 (15:03 +0100)] 
cmake: when USE_MANUAL=YES, build the curl.1 man page

Fixes KNOWN_BUG 15.4

Closes #12742

21 months agocmdline-opts/write-out.d: remove spurious double quotes
Daniel Stenberg [Fri, 19 Jan 2024 15:37:14 +0000 (16:37 +0100)] 
cmdline-opts/write-out.d: remove spurious double quotes

21 months agortsp: Convert assertion into debug log
Stefan Eissing [Fri, 19 Jan 2024 09:15:03 +0000 (10:15 +0100)] 
rtsp: Convert assertion into debug log

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934

- write excess bytes to the client where the standard excess bytes
  checks will report any wrongness and fail the transfer

Fixes #12738
Closes #12739

21 months agoheaders: remove assert from Curl_headers_push
Daniel Stenberg [Tue, 16 Jan 2024 22:50:02 +0000 (23:50 +0100)] 
headers: remove assert from Curl_headers_push

The fuzzer managed to reach the function without a terminating CR or LF
so let's handle it normally. While there, remove the goto.

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839

Closes #12721

21 months agocurl_easy_getinfo.3: remove the wrong time value count
Daniel Stenberg [Wed, 17 Jan 2024 08:42:54 +0000 (09:42 +0100)] 
curl_easy_getinfo.3: remove the wrong time value count

It said "six" time values but they are eight by now. Remove the mention
of the amount.

Closes #12727

21 months agombedtls: fix `-Wnull-dereference` and `-Wredundant-decls`
Viktor Szakats [Tue, 16 Jan 2024 16:30:07 +0000 (16:30 +0000)] 
mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls`

- Silence warning in mbedTLS v3.5.1 public headers:
  ```
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: redundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls]
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: previous declaration of 'psa_set_key_domain_parameters' was here
  ```
  Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae785a513e157c
  Ref: https://github.com/libssh2/libssh2/pull/1226

- Fix compiler warnings seen with gcc 9.2.0 + cmake unity:
  ```
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read':
  ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull-dereference]
    189 |   nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &result);
        |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write':
  ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull-dereference]
    168 |   nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, &result);
        |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ```

- delete stray `#else`.

Closes #12720

21 months agodocs: cleanup nroff format use
Daniel Stenberg [Wed, 17 Jan 2024 13:27:16 +0000 (14:27 +0100)] 
docs: cleanup nroff format use

- remove use of .BI for code snippet
- stop using .br, just do a blank line
- remove use of .PP
- remove use for .sp
- remove backslash in .IP
- use .IP instead of .TP

Closes #12731

21 months agotest2307: fix expected failure code after ws refactoring
Stefan Eissing [Wed, 17 Jan 2024 08:53:38 +0000 (09:53 +0100)] 
test2307: fix expected failure code after ws refactoring

Fixes #12722
Closes #12728

21 months agocf-socket: show errno in tcpkeepalive error messages
Jay Satiro [Wed, 17 Jan 2024 05:10:40 +0000 (00:10 -0500)] 
cf-socket: show errno in tcpkeepalive error messages

- If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set
  then show the errno in the verbose error messages.

Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652

Closes https://github.com/curl/curl/pull/12726

21 months agotool_getparam: stop supporting `@filename` style for --cookie
Jay Satiro [Sun, 7 Jan 2024 05:07:55 +0000 (00:07 -0500)] 
tool_getparam: stop supporting `@filename` style for --cookie

The `@filename` style was never documented for --cookie <data|filename>
but prior to this change curl would accept it anyway and always treat a
@ prefixed string as a filename.

That's a problem if the string also contains a = sign because then it is
documented to be interpreted as a cookie string and not a filename.

Example:

`--cookie @foo=bar`

Before: Interpreted as load cookies from filename foo=bar.

After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`).

Other curl options with a data/filename option-value use the `@filename`
to distinguish filenames which is probably how this happened. The
--cookie option has never been documented that way.

Ref: https://curl.se/docs/manpage.html#-b

Closes https://github.com/curl/curl/pull/12645

21 months agowebsockets: refactor decode chain
Stefan Eissing [Tue, 16 Jan 2024 11:06:57 +0000 (12:06 +0100)] 
websockets: refactor decode chain

- use client writer stack for decoding frames
- move websocket protocol handler to ws.c

Closes #12713

21 months agowebsockets: check for negative payload lengths
Stefan Eissing [Mon, 15 Jan 2024 12:02:34 +0000 (13:02 +0100)] 
websockets: check for negative payload lengths

- in en- and decoding, check the websocket frame payload lengths for
  negative values (from curl_off_t) and error the operation in that case
- add test 2307 to verify

Closes #12707

21 months agodocs: mention env vars not used by schannel
Daniel Stenberg [Mon, 15 Jan 2024 22:39:55 +0000 (23:39 +0100)] 
docs: mention env vars not used by schannel

Ref: #12704

Co-authored-by: Jay Satiro <raysatiro@yahoo.com>
Closes #12711

21 months agotool_operate: make --remove-on-error only remove "real" files
Daniel Stenberg [Mon, 15 Jan 2024 15:49:20 +0000 (16:49 +0100)] 
tool_operate: make --remove-on-error only remove "real" files

Reported-by: Harry Sintonen
Assisted-by: Dan Fandrich
Closes #12710

21 months agourl: don't set default CA paths for Secure Transport backend
Jay Wu [Mon, 15 Jan 2024 04:24:45 +0000 (12:24 +0800)] 
url: don't set default CA paths for Secure Transport backend

As the default for this backend is the native CA store.

Closes #12704

21 months agoasyn-ares: with modern c-ares, use its default timeout
Lin Sun [Mon, 15 Jan 2024 06:58:09 +0000 (06:58 +0000)] 
asyn-ares: with modern c-ares, use its default timeout

Closes #12703

21 months agotool_operate: stop setting the file comment on Amiga
Daniel Stenberg [Mon, 15 Jan 2024 15:28:04 +0000 (16:28 +0100)] 
tool_operate: stop setting the file comment on Amiga

- the URL is capped at 80 cols, which ruins it if longer
- it does not strip off URL credentials
- it is done unconditonally, not on --xattr
- we don't have Amiga in the CI which makes fixing it blindly fragile

Someone who builds and tests on Amiga can add it back correctly in a
future if there is a desire.

Reported-by: Harry Sintonen
Closes #12709

21 months agortsp: deal with borked server responses
Stefan Eissing [Mon, 15 Jan 2024 10:33:13 +0000 (11:33 +0100)] 
rtsp: deal with borked server responses

- enforce a response body length of 0, if the
  response has no Content-lenght. This is according
  to the RTSP spec.
- excess bytes in a response body are forwarded to
  the client writers which will report and fail the
  transfer

Follow-up to d7b6ce6
Fixes #12701
Closes #12706

21 months agoversion: show only the libpsl version, not its dependencies
Daniel Stenberg [Sun, 14 Jan 2024 16:54:51 +0000 (17:54 +0100)] 
version: show only the libpsl version, not its dependencies

The libpsl version output otherwise also includes version number for its
dependencies, like IDN lib, but since libcurl does not use libpsl's IDN
functionality those components are not important.

Ref: https://github.com/curl/curl-for-win/issues/63
Closes #12700

21 months agocurl.h: CURLOPT_DNS_SERVERS is only available with c-ares
bch [Fri, 12 Jan 2024 21:54:40 +0000 (13:54 -0800)] 
curl.h: CURLOPT_DNS_SERVERS is only available with c-ares

Closes #12695

21 months agocmdline-opts/gen.pl: error on initital blank line
Daniel Stenberg [Sun, 14 Jan 2024 13:50:52 +0000 (14:50 +0100)] 
cmdline-opts/gen.pl: error on initital blank line

After the "---" separator, there should be no blank line and this script
now errors out if one is detected.

Ref: #12696
Closes #12698

21 months agocf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper
Daniel Stenberg [Sun, 14 Jan 2024 13:42:08 +0000 (14:42 +0100)] 
cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper

Follow-up to 693cd1679361828a which was incomplete

Ref #12680
Closes #12697

21 months agocurl_multi_fdset.3: remove mention of null pointer support
Daniel Stenberg [Sat, 13 Jan 2024 22:18:20 +0000 (23:18 +0100)] 
curl_multi_fdset.3: remove mention of null pointer support

... since this funtion has not supported null pointer fd_set arguments since
at least 2006. (That's when I stopped my git blame journey)

Fixes #12691
Reported-by: sfan5 on github
Closes #12692

21 months agodocs/cmdline: remove unnecessary line breaks
Mark Huang [Sun, 14 Jan 2024 08:56:00 +0000 (16:56 +0800)] 
docs/cmdline: remove unnecessary line breaks

Closes #12696

21 months agotransfer: remove warning: Value stored to 'blen' is never read
Daniel Stenberg [Sat, 13 Jan 2024 22:42:19 +0000 (23:42 +0100)] 
transfer: remove warning: Value stored to 'blen' is never read

Detected by scan-build

Follow-up from 1cd2f0072f

Closes #12693

21 months agolib: replace readwrite with write_resp
Stefan Eissing [Fri, 1 Dec 2023 12:50:32 +0000 (13:50 +0100)] 
lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.

The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.

Curl's protocol handler `readwrite()` method been changed:

```diff
-  CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
-                        const char *buf, size_t blen,
-                        size_t *pconsumed, bool *readmore);
+  CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+                         bool is_eos, bool *done);
```

The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:

* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
  response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
  finished

This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.

The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.

The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:

```
static CURLcode readwrite_data(data, ...)
{
  do {
    Curl_xfer_recv_resp(data, buf)
    ...
    Curl_xfer_write_resp(data, buf)
    ...
  } while(interested);
  ...
}
```

All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.

Closes #12480

21 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 13 Jan 2024 16:02:45 +0000 (17:02 +0100)] 
RELEASE-NOTES: synced

21 months agoTODO: TFTP doesn't convert LF to CRLF for mode=netascii
Daniel Stenberg [Sat, 13 Jan 2024 09:56:47 +0000 (10:56 +0100)] 
TODO: TFTP doesn't convert LF to CRLF for mode=netascii

Closes #12655
Closes #12690

21 months agogen: do italics/bold for a range of letters, not just single word
Daniel Stenberg [Sat, 13 Jan 2024 09:49:20 +0000 (10:49 +0100)] 
gen: do italics/bold for a range of letters, not just single word

Previously it would match only on a sequence of non-space, which made it
miss to highlight for example "public suffix list".

Updated the recent cookie.d edit from 5da57193b732 to use bold instead
of italics.

Closes #12689

21 months agodocs: describe and highlight super cookies
Daniel Stenberg [Fri, 12 Jan 2024 15:50:44 +0000 (16:50 +0100)] 
docs: describe and highlight super cookies

Reported-by: Yadhu Krishna M
Closes #12687

21 months agoconfigure: when enabling QUIC, check that TLS supports QUIC
Daniel Stenberg [Thu, 11 Jan 2024 13:11:19 +0000 (14:11 +0100)] 
configure: when enabling QUIC, check that TLS supports QUIC

Most importantly perhaps is when using OpenSSL that the used
build/flavor has the QUIC API: the vanilla OpenSSL does not, only
BoringSSL, libressl, AWS-LC and quictls do.

Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7fe57e#r136780413

Closes #12683

21 months agovquic: extract TLS setup into own source
Stefan Eissing [Tue, 9 Jan 2024 12:21:02 +0000 (13:21 +0100)] 
vquic: extract TLS setup into own source

- separate ngtcp2 specific parts out
- provide callback during init to allow ngtcp2 to apply its defaults

Closes #12678

21 months agomulti: remove total timer reset in file_do() while fetching file://
Sergey Markelov [Thu, 11 Jan 2024 00:23:00 +0000 (17:23 -0700)] 
multi: remove total timer reset in file_do() while fetching file://

The total timer is properly reset in MSTATE_INIT.  MSTATE_CONNECT starts
with resetting the timer that is a start point for further multi states.
If file://, MSTATE_DO calls file_do() that should not reset the total
timer.  Otherwise, the total time is always less than the pre-transfer
and the start transfer times.

Closes #12682

21 months agohttp_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT
Daniel Stenberg [Wed, 10 Jan 2024 22:20:09 +0000 (23:20 +0100)] 
http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT

Extended test 80 to verify this.

Reported-by: Stefan Eissing
Fixes #12680
Closes #12681

21 months agosectransp: do verify_cert without memdup for blobs
Daniel Stenberg [Wed, 10 Jan 2024 14:13:13 +0000 (15:13 +0100)] 
sectransp: do verify_cert without memdup for blobs

Since the information is then already stored in memory, this can avoid
an extra set of malloc + free calls.

Closes #12679

21 months agohsts: remove assert for zero length domain
Daniel Stenberg [Wed, 10 Jan 2024 08:46:19 +0000 (09:46 +0100)] 
hsts: remove assert for zero length domain

A zero length domain can happen if the HSTS parser is given invalid
input data which is not unheard of and is done by the fuzzer.

Follow-up from cfe7902111ae547873

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661

Closes #12676

21 months agoheaders: make sure the trailing newline is not stored
Daniel Stenberg [Wed, 10 Jan 2024 08:22:59 +0000 (09:22 +0100)] 
headers: make sure the trailing newline is not stored

extended test1940 to verify blank header fields too

Bug: https://curl.se/mail/lib-2024-01/0019.html
Reported-by: Dmitry Karpov
Closes #12675

21 months agocurl_easy_header.3: tiny language fix
Daniel Stenberg [Tue, 9 Jan 2024 23:42:17 +0000 (00:42 +0100)] 
curl_easy_header.3: tiny language fix

Closes #12672

21 months agoexamples/range.c: add
Daniel Stenberg [Tue, 9 Jan 2024 22:47:01 +0000 (23:47 +0100)] 
examples/range.c: add

Closes #12671

21 months agoexamples/netrc.c: add
Daniel Stenberg [Tue, 9 Jan 2024 22:44:24 +0000 (23:44 +0100)] 
examples/netrc.c: add

Closes #12671

21 months agoexamples/ipv6.c: new example showing IPv6-only internet transfer
Daniel Stenberg [Tue, 9 Jan 2024 22:42:17 +0000 (23:42 +0100)] 
examples/ipv6.c: new example showing IPv6-only internet transfer

Closes #12671

21 months agoexamples/address-scope.c: renamed from ipv6.c
Daniel Stenberg [Tue, 9 Jan 2024 22:42:01 +0000 (23:42 +0100)] 
examples/address-scope.c: renamed from ipv6.c

It shows address scope use really

Closes #12671

22 months agomulti: pollset adjust, init with FIRSTSOCKET during connect
Stefan Eissing [Tue, 9 Jan 2024 08:29:34 +0000 (09:29 +0100)] 
multi: pollset adjust, init with FIRSTSOCKET during connect

- `conn->sockfd` is set by `Curl_setup_transfer()`, but that
  is called *after* the connection has been established
- use `conn->sock[FIRSTSOCKET]` instead

Follow-up to a0f94800d507de
Closes #12664

22 months agoWEBSOCKET.md: remove dead link
Daniel Stenberg [Tue, 9 Jan 2024 10:12:02 +0000 (11:12 +0100)] 
WEBSOCKET.md: remove dead link

22 months agoCI: spellcheck/appveyor: invoke configure --without-libpsl
Daniel Stenberg [Tue, 9 Jan 2024 10:09:30 +0000 (11:09 +0100)] 
CI: spellcheck/appveyor: invoke configure --without-libpsl

Follow-up to 2998874bb61ac6

22 months agocmdline/docs/*.d: switch to using ## instead of .IP
Daniel Stenberg [Tue, 9 Jan 2024 09:36:14 +0000 (10:36 +0100)] 
cmdline/docs/*.d: switch to using ## instead of .IP

To make the editing easier. To write and to read.

Closes #12667

22 months agogen.pl: support ## for doing .IP in table-like lists
Daniel Stenberg [Tue, 9 Jan 2024 09:20:48 +0000 (10:20 +0100)] 
gen.pl: support ## for doing .IP in table-like lists

Warn on use of .RS/.IP/.RE

Closes #12667

22 months agocookie.d: Document use of empty string to enable cookie engine
Jay Satiro [Sun, 7 Jan 2024 05:25:13 +0000 (00:25 -0500)] 
cookie.d: Document use of empty string to enable cookie engine

- Explain that --cookie "" can be used to enable the cookie engine
  without reading any initial cookies.

As is documented in CURLOPT_COOKIEFILE.

Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html

Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420
Reported-by: janko-js@users.noreply.github.com
Closes https://github.com/curl/curl/pull/12646

22 months agosetopt: use memdup0 when cloning COPYPOSTFIELDS
Daniel Stenberg [Sun, 7 Jan 2024 15:11:01 +0000 (16:11 +0100)] 
setopt: use memdup0 when cloning COPYPOSTFIELDS

Closes #12651

22 months agotelnet: use dynbuf instad of malloc for escape buffer
Daniel Stenberg [Sun, 7 Jan 2024 15:28:15 +0000 (16:28 +0100)] 
telnet: use dynbuf instad of malloc for escape buffer

Previously, send_telnet_data() would malloc + free a buffer every time
for escaping IAC codes. Now, it reuses a dynbuf for this purpose.

Closes #12652

22 months agoCI: install libpsl or configure --without-libpsl in builds
Daniel Stenberg [Mon, 8 Jan 2024 15:49:21 +0000 (16:49 +0100)] 
CI: install libpsl or configure --without-libpsl in builds

As a follow-up to the stricted libpsl check in configure

22 months agoconfigure: make libpsl detection failure cause error
Daniel Stenberg [Mon, 8 Jan 2024 15:28:50 +0000 (16:28 +0100)] 
configure: make libpsl detection failure cause error

To force users to explictily disable it if they really don't want it
used and make it harder to accidentally miss it.

--without-libpsl is the option to use if PSL is not wanted.

Closes #12661

22 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 8 Jan 2024 22:00:21 +0000 (23:00 +0100)] 
RELEASE-NOTES: synced

22 months agopop3: replace calloc + memcpy with memdup0
Daniel Stenberg [Sun, 7 Jan 2024 15:06:32 +0000 (16:06 +0100)] 
pop3: replace calloc + memcpy with memdup0

... and make sure to return error on out of memory.

Closes #12650

22 months agolib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT
Daniel Stenberg [Mon, 8 Jan 2024 09:34:06 +0000 (10:34 +0100)] 
lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT

Closes #12658

22 months agomime: use memdup0 instead of malloc + memcpy
Daniel Stenberg [Sun, 7 Jan 2024 15:01:00 +0000 (16:01 +0100)] 
mime: use memdup0 instead of malloc + memcpy

Closes #12649

22 months agotool_getparam: move the --rate logic into set_rate()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --rate logic into set_rate()

22 months agotool_getparam: switch to an enum for every option
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: switch to an enum for every option

To make the big switch much easier to read/understand and to make it
easier to add new options.

22 months agotool_getparam: build post data using dynbuf (more)
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: build post data using dynbuf (more)

22 months agotool_getparam: replace malloc + copy by dynbuf for --data
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: replace malloc + copy by dynbuf for --data

22 months agotool_getparam: make data_urlencode avoid direct malloc
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: make data_urlencode avoid direct malloc

use aprintf() instead

22 months agotool_getparam: move the --url-query logic into url_query()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --url-query logic into url_query()

This function is not doing post at all so it was always weirdly placed.

22 months agotool_getparam: move the --data logic into set_data()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --data logic into set_data()

22 months agotool_getparam: unify the cmdline switch() into a single one
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: unify the cmdline switch() into a single one

- easier to follow, easier to modify, easier to extend, possibly slightly
  faster

- each case now has the long option as a comment

22 months agotool_getparam: bsearch cmdline options
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: bsearch cmdline options

- the option names are now alpha sorted and lookup is a lot faster

- use case sensitive matching. It was previously case insensitive, but that
  was not documented nor tested.

- remove "partial match" feature. It was not documented, not tested and
  was always fragile as existing use could break when we add a new
  option

- lookup short options via a table

Closes #12631

22 months agoCOPYING: update copyright year
Gabe [Mon, 8 Jan 2024 03:27:05 +0000 (21:27 -0600)] 
COPYING: update copyright year

Closes #12654

22 months agourl: init conn->sockfd and writesockfd to CURL_SOCKET_BAD
Stefan Eissing [Mon, 8 Jan 2024 14:02:12 +0000 (15:02 +0100)] 
url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD

Also add more tracing to test 19

Follow-up to a0f9480

Fixes #12657
Closes #12659

22 months agoconnect: remove margin from eyeballer alloc
Daniel Stenberg [Sun, 7 Jan 2024 11:06:12 +0000 (12:06 +0100)] 
connect: remove margin from eyeballer alloc

Presumably leftovers from debugging

Closes #12647

22 months agoftp: only consider entry path if it has a length
Daniel Stenberg [Sun, 7 Jan 2024 13:25:20 +0000 (14:25 +0100)] 
ftp: only consider entry path if it has a length

Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631

Avoids a NULL pointer deref.

Closes #12648

22 months agotransfer: adjust_pollset improvements
Stefan Eissing [Fri, 5 Jan 2024 11:28:09 +0000 (12:28 +0100)] 
transfer: adjust_pollset improvements

- let `multi_getsock()` initialize the pollset in what the
  transfer state requires in regards to SEND/RECV
- change connection filters `adjust_pollset()` implementation
  to react on the presence of POLLIN/-OUT in the pollset and
  no longer check CURL_WANT_SEND/CURL_WANT_RECV
- cf-socket will no longer add POLLIN on its own
- http2 and http/3 filters will only do adjustments if the
  passed pollset wants to POLLIN/OUT for the transfer on
  the socket. This is similar to the HTTP/2 proxy filter
  and works in stacked filters.

Closes #12640

22 months agoftp: use memdup0 to store the OS from a SYST 215 response
Daniel Stenberg [Fri, 5 Jan 2024 10:58:48 +0000 (11:58 +0100)] 
ftp: use memdup0 to store the OS from a SYST 215 response

avoid malloc + direct buffer fiddle

Closes #12639

22 months agoftp: use dynbuf to store entrypath
Daniel Stenberg [Fri, 5 Jan 2024 10:52:08 +0000 (11:52 +0100)] 
ftp: use dynbuf to store entrypath

avoid direct malloc

Closes #12638

22 months agowolfssl: load certificate *chain* for PEM client certs
Lealem Amedie [Fri, 5 Jan 2024 20:12:53 +0000 (13:12 -0700)] 
wolfssl: load certificate *chain* for PEM client certs

Closes #12634

22 months agohttp: adjust_pollset fix
Stefan Eissing [Thu, 4 Jan 2024 09:06:17 +0000 (10:06 +0100)] 
http: adjust_pollset fix

do not add a socket for POLLIN when the transfer does not want to send
(for example is paused).

Follow-up to 47f5b1a

Reported-by: bubbleguuum on github
Fixes #12632
Closes #12633

22 months agotool: make parser reject blank arguments if not supported
Daniel Stenberg [Tue, 2 Jan 2024 13:17:10 +0000 (14:17 +0100)] 
tool: make parser reject blank arguments if not supported

Already in the getstr() function that clones the input argument.

Closes #12620

22 months agobuild(deps): bump github/codeql-action from 2 to 3
dependabot[bot] [Tue, 2 Jan 2024 22:32:35 +0000 (22:32 +0000)] 
build(deps): bump github/codeql-action from 2 to 3

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12625

22 months agobuild(deps): bump actions/checkout from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:31 +0000 (22:32 +0000)] 
build(deps): bump actions/checkout from 3 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12624

22 months agobuild(deps): bump actions/upload-artifact from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:42 +0000 (22:32 +0000)] 
build(deps): bump actions/upload-artifact from 3 to 4

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12627

22 months agobuild(deps): bump actions/download-artifact from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:38 +0000 (22:32 +0000)] 
build(deps): bump actions/download-artifact from 3 to 4

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12626

22 months agohttp3/quiche: fix result code on a stream reset
Stefan Eissing [Wed, 3 Jan 2024 08:37:55 +0000 (09:37 +0100)] 
http3/quiche: fix result code on a stream reset

- fixes pytest failures in test 07_22
- aligns CURLcode values on stream reset with ngtcp2

Closes #12629

22 months agosetopt: clear mimepost when formp is freed
Daniel Stenberg [Tue, 2 Jan 2024 15:35:50 +0000 (16:35 +0100)] 
setopt: clear mimepost when formp is freed

A precaution to avoid a possibly dangling pointer left behind.

Reported-by: Thomas Ferguson
Fixes #12608
Closes #12621

22 months agoCI: Add dependabot.yml
andy5995 [Tue, 2 Jan 2024 18:41:58 +0000 (12:41 -0600)] 
CI: Add dependabot.yml

This will cause dependabot to open a PR when various actions are
updated, provided that the action maintainer has issued a release.

Closes #12623

22 months agocontent_encoding: change return code to typedef'ed enum
Gisle Vanem [Tue, 2 Jan 2024 16:02:13 +0000 (17:02 +0100)] 
content_encoding: change return code to typedef'ed enum

... to work around a clang ubsan warning.

Fixes #12618
Closes #12622

22 months agotool: prepend output_dir in header callback
Daniel Stenberg [Tue, 2 Jan 2024 12:32:18 +0000 (13:32 +0100)] 
tool: prepend output_dir in header callback

When Content-Disposition parsing is used and an output dir is prepended,
make sure to store that new file name correctly so that it can be used
for setting the file timestamp when --remote-time is used.

Extended test 3012 to verify.

Co-Authored-by: Jay Satiro
Reported-by: hgdagon on github
Fixes #12614
Closes #12617

22 months agotest1254: fix typo in name plus shorten it
Daniel Stenberg [Tue, 2 Jan 2024 13:16:43 +0000 (14:16 +0100)] 
test1254: fix typo in name plus shorten it

22 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 2 Jan 2024 08:05:19 +0000 (09:05 +0100)] 
RELEASE-NOTES: synced

22 months agoschannel: fix `-Warith-conversion` gcc 13 warning
Viktor Szakats [Mon, 1 Jan 2024 23:31:47 +0000 (23:31 +0000)] 
schannel: fix `-Warith-conversion` gcc 13 warning

```
lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' may change the sign of the result [-Warith-conversion]
 1201 |     *extension_len = *list_len +
      |                      ^
```

Closes #12616

22 months agoasyn-thread: silence `-Wcast-align` warning for Windows
Viktor Szakats [Mon, 1 Jan 2024 15:29:33 +0000 (15:29 +0000)] 
asyn-thread: silence `-Wcast-align` warning for Windows

Seen with llvm/clang 17:
```
lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align]
  310 |     CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlapped);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded from macro 'CONTAINING_RECORD'
  717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(address) - (ULONG_PTR)(&((type *)0)->field)))
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482

Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261
Closes #12615

22 months agotool_listhelp: regenerate after recent .d updates
Daniel Stenberg [Sun, 31 Dec 2023 15:29:49 +0000 (16:29 +0100)] 
tool_listhelp: regenerate after recent .d updates

Makes it survive test 1478

Closes #12612

22 months agotest1478: verify src/tool_listhelp.c
Daniel Stenberg [Sun, 31 Dec 2023 15:28:48 +0000 (16:28 +0100)] 
test1478: verify src/tool_listhelp.c

Verify that the source file on disk is identical to the output of gen.pl
listhelp, as otherwise they are out of sync and need attention.

Closes #12612

22 months agotestutil: make runtests support %include
Daniel Stenberg [Sun, 31 Dec 2023 15:27:46 +0000 (16:27 +0100)] 
testutil: make runtests support %include

Using this instruction, a test case can include the contents of a file
into the test during the preprocessing.

Closes #12612

22 months agoruntests: for mode="text" on <stdout>, fix newlines on both parts
Daniel Stenberg [Mon, 1 Jan 2024 21:48:09 +0000 (22:48 +0100)] 
runtests: for mode="text" on <stdout>, fix newlines on both parts

Closes #12612

22 months agoquiche: return CURLE_HTTP3 on send to invalid stream
Jay Satiro [Tue, 26 Dec 2023 06:55:54 +0000 (01:55 -0500)] 
quiche: return CURLE_HTTP3 on send to invalid stream

Prior to this change if a send failed on a stream in an invalid state
(according to quiche) and not marked as closed (according to libcurl)
then the send function would return CURLE_SEND_ERROR.

We already have similar code for ngtcp2 to return CURLE_HTTP3 in this
case.

Caught by test test_07_upload.py: test_07_22_upload_parallel_fail.

Fixes https://github.com/curl/curl/issues/12590
Closes https://github.com/curl/curl/pull/12597

22 months agocmdline-opts: update availability for the *-ca-native options
Daniel Stenberg [Sun, 31 Dec 2023 15:49:54 +0000 (16:49 +0100)] 
cmdline-opts: update availability for the *-ca-native options

Closes #12613