]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
21 months agocmake: speed up curldown processing, enable by default
Viktor Szakats [Tue, 23 Jan 2024 06:40:11 +0000 (06:40 +0000)] 
cmake: speed up curldown processing, enable by default

- cmake: enable `BUILD_DOCS` by default (this controls converting and
  installing `.3` files from `.md` sources)

- cmake: speed up generating `.3` files by using a single command per
  directory, instead of a single command per file. This reduces external
  commands by about a thousand. (There remains some CMake logic kicking
  in resulting in 500 -one per file- external `-E touch_nocreate` calls.)

- cd2nroff: add ability to process multiple input files.

- cd2nroff: add `-k` option to use the source filename to form the
  output filename. (instead of the default in-file `Title:` line.)

Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c
Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753
Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730

Closes #12762

21 months agodocs: install curl.1 with cmake as well
Richard Levitte [Mon, 22 Jan 2024 21:49:17 +0000 (22:49 +0100)] 
docs: install curl.1 with cmake as well

Closes #12759

21 months agoosslq: remove the TLS library from the version output
Daniel Stenberg [Tue, 23 Jan 2024 16:45:22 +0000 (17:45 +0100)] 
osslq: remove the TLS library from the version output

Since we only support using a single TLS library at any one time, we
know that the TLS library for QUIC is the same that is also shown for
regular TLS.

Fixes #12763
Reported-by: Viktor Szakats
Closes #12767

21 months agoCI: remove unnecessary OpenSSL 3 option `enable-tls1_3`
Stefan Eissing [Mon, 22 Jan 2024 16:31:25 +0000 (17:31 +0100)] 
CI: remove unnecessary OpenSSL 3 option `enable-tls1_3`

.. and switch OpenSSL 3 libdir from lib64 to lib for consistency.

Closes https://github.com/curl/curl/pull/12758

21 months agoGHA: bump nghttp2 version to v1.59.0
Stefan Eissing [Tue, 23 Jan 2024 15:35:08 +0000 (16:35 +0100)] 
GHA: bump nghttp2 version to v1.59.0

- Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version.

Closes https://github.com/curl/curl/pull/12766

21 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 23 Jan 2024 13:34:36 +0000 (14:34 +0100)] 
RELEASE-NOTES: synced

21 months agodocs/cmdline: change to .md for cmdline docs
Daniel Stenberg [Sat, 20 Jan 2024 22:18:43 +0000 (23:18 +0100)] 
docs/cmdline: change to .md for cmdline docs

 - switch all invidual files documenting command line options into .md,
   as the documentation is now markdown-looking.

 - made the parser treat 4-space indents as quotes

 - switch to building the curl.1 manpage using the "mainpage.idx" file,
   which lists the files to include to generate it, instead of using the
   previous page-footer/headers. Also, those files are now also .md
   ones, using the same format. I gave them underscore prefixes to make
   them sort separately:
   _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md,
   _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md,
   _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md,
   _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md

 - updated test cases accordingly

Closes #12751

21 months agoCI: bump actions/cache from 3 to 4
dependabot[bot] [Mon, 22 Jan 2024 14:36:50 +0000 (14:36 +0000)] 
CI: bump actions/cache from 3 to 4

Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12756

21 months agoopenssl: when verifystatus fails, remove session id from cache
Daniel Stenberg [Mon, 22 Jan 2024 22:54:08 +0000 (23:54 +0100)] 
openssl: when verifystatus fails, remove session id from cache

To prevent that it gets used in a subsequent transfer that skips the
verifystatus check since that check can't be done when the session id is
reused.

Reported-by: Hiroki Kurosawa
Closes #12760

21 months agocmake: add option to disable building docs
Viktor Szakats [Mon, 22 Jan 2024 22:16:54 +0000 (23:16 +0100)] 
cmake: add option to disable building docs

21 months agocmake: use curldown to build man pages
Richard Levitte [Mon, 22 Jan 2024 06:49:08 +0000 (07:49 +0100)] 
cmake: use curldown to build man pages

This throws away the previous HTML and PDF producers, to mimic what
Makefile.am does as faithfully as possible.

Closes #12753

21 months agomksymbolsmanpage.pl: provide references to where the symbol is used
Daniel Stenberg [Fri, 19 Jan 2024 15:18:40 +0000 (16:18 +0100)] 
mksymbolsmanpage.pl: provide references to where the symbol is used

21 months agodocs: introduce "curldown" for libcurl man page format
Daniel Stenberg [Wed, 17 Jan 2024 10:32:44 +0000 (11:32 +0100)] 
docs: introduce "curldown" for libcurl man page format

curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:

- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
  their man page section is specified)

tools:

- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown

This setup generates .3 versions of all the curldown versions at build time.

CI:

Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...

Closes #12730

21 months agolibssh2: use `libssh2_session_callback_set2()` with v1.11.1
Viktor Szakats [Mon, 22 Jan 2024 11:21:10 +0000 (11:21 +0000)] 
libssh2: use `libssh2_session_callback_set2()` with v1.11.1

To avoid a local hack to pass function pointers and to avoid
deprecation warnings when building with libssh2 v1.11.1 or newer:
```
lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/20720821100#step:3:4982

Ref: https://github.com/libssh2/libssh2/pull/1285
Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db3cf1d4b0b4
Reviewed-by: Daniel Stenberg
Closes #12754

21 months agotransfer: make the select_bits_paused condition check both directions
Daniel Stenberg [Mon, 22 Jan 2024 15:22:19 +0000 (16:22 +0100)] 
transfer: make the select_bits_paused condition check both directions

If there is activity in a direction that is not paused, return false.

Reported-by: Sergey Bronnikov
Bug: https://curl.se/mail/lib-2024-01/0049.html
Closes #12740

21 months agohttp3: initial support for OpenSSL 3.2 QUIC stack
Stefan Eissing [Thu, 18 Jan 2024 12:07:07 +0000 (13:07 +0100)] 
http3: initial support for OpenSSL 3.2 QUIC stack

- HTTP/3 for curl using OpenSSL's own QUIC stack together
  with nghttp3
- configure with `--with-openssl-quic` to enable curl to
  build this. This requires the nghttp3 library
- implementation with the following restrictions:
  * macOS has to use an unconnected UDP socket due to an
    issue in OpenSSL's datagram implementation
    See https://github.com/openssl/openssl/issues/23251
    This makes connections to non-reponsive servers hang.
  * GET requests will send the indicator that they have
    no body in a separate QUIC packet. This may result
    in processing delays or Transfer-Encodings on proxied
    requests
  * uploads that encounter blocks will use 100% cpu as
    detection of these flow control issue is not working
    (we have not figured out to pry that from OpenSSL).

Closes #12734

21 months agocmake: fix `ENABLE_MANUAL` option
Viktor Szakats [Sun, 21 Jan 2024 15:38:09 +0000 (15:38 +0000)] 
cmake: fix `ENABLE_MANUAL` option

Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`.

Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the
option designed to enable building and using the built-in curl manual.
(`USE_MANUAL=ON` option worked for this instead, by accident).

Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409
Closes #12749

21 months agoTODO: update broken link to ratelimit-headers draft
Mohammadreza Hendiani [Fri, 19 Jan 2024 13:59:03 +0000 (17:29 +0330)] 
TODO: update broken link to ratelimit-headers draft

Closes #12741

21 months agocmake: when USE_MANUAL=YES, build the curl.1 man page
Daniel Stenberg [Fri, 19 Jan 2024 14:03:50 +0000 (15:03 +0100)] 
cmake: when USE_MANUAL=YES, build the curl.1 man page

Fixes KNOWN_BUG 15.4

Closes #12742

21 months agocmdline-opts/write-out.d: remove spurious double quotes
Daniel Stenberg [Fri, 19 Jan 2024 15:37:14 +0000 (16:37 +0100)] 
cmdline-opts/write-out.d: remove spurious double quotes

21 months agortsp: Convert assertion into debug log
Stefan Eissing [Fri, 19 Jan 2024 09:15:03 +0000 (10:15 +0100)] 
rtsp: Convert assertion into debug log

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934

- write excess bytes to the client where the standard excess bytes
  checks will report any wrongness and fail the transfer

Fixes #12738
Closes #12739

21 months agoheaders: remove assert from Curl_headers_push
Daniel Stenberg [Tue, 16 Jan 2024 22:50:02 +0000 (23:50 +0100)] 
headers: remove assert from Curl_headers_push

The fuzzer managed to reach the function without a terminating CR or LF
so let's handle it normally. While there, remove the goto.

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839

Closes #12721

21 months agocurl_easy_getinfo.3: remove the wrong time value count
Daniel Stenberg [Wed, 17 Jan 2024 08:42:54 +0000 (09:42 +0100)] 
curl_easy_getinfo.3: remove the wrong time value count

It said "six" time values but they are eight by now. Remove the mention
of the amount.

Closes #12727

21 months agombedtls: fix `-Wnull-dereference` and `-Wredundant-decls`
Viktor Szakats [Tue, 16 Jan 2024 16:30:07 +0000 (16:30 +0000)] 
mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls`

- Silence warning in mbedTLS v3.5.1 public headers:
  ```
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: redundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls]
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: previous declaration of 'psa_set_key_domain_parameters' was here
  ```
  Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae785a513e157c
  Ref: https://github.com/libssh2/libssh2/pull/1226

- Fix compiler warnings seen with gcc 9.2.0 + cmake unity:
  ```
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read':
  ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull-dereference]
    189 |   nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &result);
        |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write':
  ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull-dereference]
    168 |   nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, &result);
        |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ```

- delete stray `#else`.

Closes #12720

21 months agodocs: cleanup nroff format use
Daniel Stenberg [Wed, 17 Jan 2024 13:27:16 +0000 (14:27 +0100)] 
docs: cleanup nroff format use

- remove use of .BI for code snippet
- stop using .br, just do a blank line
- remove use of .PP
- remove use for .sp
- remove backslash in .IP
- use .IP instead of .TP

Closes #12731

21 months agotest2307: fix expected failure code after ws refactoring
Stefan Eissing [Wed, 17 Jan 2024 08:53:38 +0000 (09:53 +0100)] 
test2307: fix expected failure code after ws refactoring

Fixes #12722
Closes #12728

21 months agocf-socket: show errno in tcpkeepalive error messages
Jay Satiro [Wed, 17 Jan 2024 05:10:40 +0000 (00:10 -0500)] 
cf-socket: show errno in tcpkeepalive error messages

- If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set
  then show the errno in the verbose error messages.

Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652

Closes https://github.com/curl/curl/pull/12726

21 months agotool_getparam: stop supporting `@filename` style for --cookie
Jay Satiro [Sun, 7 Jan 2024 05:07:55 +0000 (00:07 -0500)] 
tool_getparam: stop supporting `@filename` style for --cookie

The `@filename` style was never documented for --cookie <data|filename>
but prior to this change curl would accept it anyway and always treat a
@ prefixed string as a filename.

That's a problem if the string also contains a = sign because then it is
documented to be interpreted as a cookie string and not a filename.

Example:

`--cookie @foo=bar`

Before: Interpreted as load cookies from filename foo=bar.

After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`).

Other curl options with a data/filename option-value use the `@filename`
to distinguish filenames which is probably how this happened. The
--cookie option has never been documented that way.

Ref: https://curl.se/docs/manpage.html#-b

Closes https://github.com/curl/curl/pull/12645

21 months agowebsockets: refactor decode chain
Stefan Eissing [Tue, 16 Jan 2024 11:06:57 +0000 (12:06 +0100)] 
websockets: refactor decode chain

- use client writer stack for decoding frames
- move websocket protocol handler to ws.c

Closes #12713

21 months agowebsockets: check for negative payload lengths
Stefan Eissing [Mon, 15 Jan 2024 12:02:34 +0000 (13:02 +0100)] 
websockets: check for negative payload lengths

- in en- and decoding, check the websocket frame payload lengths for
  negative values (from curl_off_t) and error the operation in that case
- add test 2307 to verify

Closes #12707

21 months agodocs: mention env vars not used by schannel
Daniel Stenberg [Mon, 15 Jan 2024 22:39:55 +0000 (23:39 +0100)] 
docs: mention env vars not used by schannel

Ref: #12704

Co-authored-by: Jay Satiro <raysatiro@yahoo.com>
Closes #12711

21 months agotool_operate: make --remove-on-error only remove "real" files
Daniel Stenberg [Mon, 15 Jan 2024 15:49:20 +0000 (16:49 +0100)] 
tool_operate: make --remove-on-error only remove "real" files

Reported-by: Harry Sintonen
Assisted-by: Dan Fandrich
Closes #12710

21 months agourl: don't set default CA paths for Secure Transport backend
Jay Wu [Mon, 15 Jan 2024 04:24:45 +0000 (12:24 +0800)] 
url: don't set default CA paths for Secure Transport backend

As the default for this backend is the native CA store.

Closes #12704

21 months agoasyn-ares: with modern c-ares, use its default timeout
Lin Sun [Mon, 15 Jan 2024 06:58:09 +0000 (06:58 +0000)] 
asyn-ares: with modern c-ares, use its default timeout

Closes #12703

21 months agotool_operate: stop setting the file comment on Amiga
Daniel Stenberg [Mon, 15 Jan 2024 15:28:04 +0000 (16:28 +0100)] 
tool_operate: stop setting the file comment on Amiga

- the URL is capped at 80 cols, which ruins it if longer
- it does not strip off URL credentials
- it is done unconditonally, not on --xattr
- we don't have Amiga in the CI which makes fixing it blindly fragile

Someone who builds and tests on Amiga can add it back correctly in a
future if there is a desire.

Reported-by: Harry Sintonen
Closes #12709

21 months agortsp: deal with borked server responses
Stefan Eissing [Mon, 15 Jan 2024 10:33:13 +0000 (11:33 +0100)] 
rtsp: deal with borked server responses

- enforce a response body length of 0, if the
  response has no Content-lenght. This is according
  to the RTSP spec.
- excess bytes in a response body are forwarded to
  the client writers which will report and fail the
  transfer

Follow-up to d7b6ce6
Fixes #12701
Closes #12706

21 months agoversion: show only the libpsl version, not its dependencies
Daniel Stenberg [Sun, 14 Jan 2024 16:54:51 +0000 (17:54 +0100)] 
version: show only the libpsl version, not its dependencies

The libpsl version output otherwise also includes version number for its
dependencies, like IDN lib, but since libcurl does not use libpsl's IDN
functionality those components are not important.

Ref: https://github.com/curl/curl-for-win/issues/63
Closes #12700

21 months agocurl.h: CURLOPT_DNS_SERVERS is only available with c-ares
bch [Fri, 12 Jan 2024 21:54:40 +0000 (13:54 -0800)] 
curl.h: CURLOPT_DNS_SERVERS is only available with c-ares

Closes #12695

21 months agocmdline-opts/gen.pl: error on initital blank line
Daniel Stenberg [Sun, 14 Jan 2024 13:50:52 +0000 (14:50 +0100)] 
cmdline-opts/gen.pl: error on initital blank line

After the "---" separator, there should be no blank line and this script
now errors out if one is detected.

Ref: #12696
Closes #12698

21 months agocf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper
Daniel Stenberg [Sun, 14 Jan 2024 13:42:08 +0000 (14:42 +0100)] 
cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper

Follow-up to 693cd1679361828a which was incomplete

Ref #12680
Closes #12697

21 months agocurl_multi_fdset.3: remove mention of null pointer support
Daniel Stenberg [Sat, 13 Jan 2024 22:18:20 +0000 (23:18 +0100)] 
curl_multi_fdset.3: remove mention of null pointer support

... since this funtion has not supported null pointer fd_set arguments since
at least 2006. (That's when I stopped my git blame journey)

Fixes #12691
Reported-by: sfan5 on github
Closes #12692

21 months agodocs/cmdline: remove unnecessary line breaks
Mark Huang [Sun, 14 Jan 2024 08:56:00 +0000 (16:56 +0800)] 
docs/cmdline: remove unnecessary line breaks

Closes #12696

21 months agotransfer: remove warning: Value stored to 'blen' is never read
Daniel Stenberg [Sat, 13 Jan 2024 22:42:19 +0000 (23:42 +0100)] 
transfer: remove warning: Value stored to 'blen' is never read

Detected by scan-build

Follow-up from 1cd2f0072f

Closes #12693

21 months agolib: replace readwrite with write_resp
Stefan Eissing [Fri, 1 Dec 2023 12:50:32 +0000 (13:50 +0100)] 
lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.

The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.

Curl's protocol handler `readwrite()` method been changed:

```diff
-  CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
-                        const char *buf, size_t blen,
-                        size_t *pconsumed, bool *readmore);
+  CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+                         bool is_eos, bool *done);
```

The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:

* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
  response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
  finished

This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.

The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.

The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:

```
static CURLcode readwrite_data(data, ...)
{
  do {
    Curl_xfer_recv_resp(data, buf)
    ...
    Curl_xfer_write_resp(data, buf)
    ...
  } while(interested);
  ...
}
```

All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.

Closes #12480

21 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 13 Jan 2024 16:02:45 +0000 (17:02 +0100)] 
RELEASE-NOTES: synced

21 months agoTODO: TFTP doesn't convert LF to CRLF for mode=netascii
Daniel Stenberg [Sat, 13 Jan 2024 09:56:47 +0000 (10:56 +0100)] 
TODO: TFTP doesn't convert LF to CRLF for mode=netascii

Closes #12655
Closes #12690

21 months agogen: do italics/bold for a range of letters, not just single word
Daniel Stenberg [Sat, 13 Jan 2024 09:49:20 +0000 (10:49 +0100)] 
gen: do italics/bold for a range of letters, not just single word

Previously it would match only on a sequence of non-space, which made it
miss to highlight for example "public suffix list".

Updated the recent cookie.d edit from 5da57193b732 to use bold instead
of italics.

Closes #12689

21 months agodocs: describe and highlight super cookies
Daniel Stenberg [Fri, 12 Jan 2024 15:50:44 +0000 (16:50 +0100)] 
docs: describe and highlight super cookies

Reported-by: Yadhu Krishna M
Closes #12687

21 months agoconfigure: when enabling QUIC, check that TLS supports QUIC
Daniel Stenberg [Thu, 11 Jan 2024 13:11:19 +0000 (14:11 +0100)] 
configure: when enabling QUIC, check that TLS supports QUIC

Most importantly perhaps is when using OpenSSL that the used
build/flavor has the QUIC API: the vanilla OpenSSL does not, only
BoringSSL, libressl, AWS-LC and quictls do.

Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7fe57e#r136780413

Closes #12683

22 months agovquic: extract TLS setup into own source
Stefan Eissing [Tue, 9 Jan 2024 12:21:02 +0000 (13:21 +0100)] 
vquic: extract TLS setup into own source

- separate ngtcp2 specific parts out
- provide callback during init to allow ngtcp2 to apply its defaults

Closes #12678

22 months agomulti: remove total timer reset in file_do() while fetching file://
Sergey Markelov [Thu, 11 Jan 2024 00:23:00 +0000 (17:23 -0700)] 
multi: remove total timer reset in file_do() while fetching file://

The total timer is properly reset in MSTATE_INIT.  MSTATE_CONNECT starts
with resetting the timer that is a start point for further multi states.
If file://, MSTATE_DO calls file_do() that should not reset the total
timer.  Otherwise, the total time is always less than the pre-transfer
and the start transfer times.

Closes #12682

22 months agohttp_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT
Daniel Stenberg [Wed, 10 Jan 2024 22:20:09 +0000 (23:20 +0100)] 
http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT

Extended test 80 to verify this.

Reported-by: Stefan Eissing
Fixes #12680
Closes #12681

22 months agosectransp: do verify_cert without memdup for blobs
Daniel Stenberg [Wed, 10 Jan 2024 14:13:13 +0000 (15:13 +0100)] 
sectransp: do verify_cert without memdup for blobs

Since the information is then already stored in memory, this can avoid
an extra set of malloc + free calls.

Closes #12679

22 months agohsts: remove assert for zero length domain
Daniel Stenberg [Wed, 10 Jan 2024 08:46:19 +0000 (09:46 +0100)] 
hsts: remove assert for zero length domain

A zero length domain can happen if the HSTS parser is given invalid
input data which is not unheard of and is done by the fuzzer.

Follow-up from cfe7902111ae547873

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661

Closes #12676

22 months agoheaders: make sure the trailing newline is not stored
Daniel Stenberg [Wed, 10 Jan 2024 08:22:59 +0000 (09:22 +0100)] 
headers: make sure the trailing newline is not stored

extended test1940 to verify blank header fields too

Bug: https://curl.se/mail/lib-2024-01/0019.html
Reported-by: Dmitry Karpov
Closes #12675

22 months agocurl_easy_header.3: tiny language fix
Daniel Stenberg [Tue, 9 Jan 2024 23:42:17 +0000 (00:42 +0100)] 
curl_easy_header.3: tiny language fix

Closes #12672

22 months agoexamples/range.c: add
Daniel Stenberg [Tue, 9 Jan 2024 22:47:01 +0000 (23:47 +0100)] 
examples/range.c: add

Closes #12671

22 months agoexamples/netrc.c: add
Daniel Stenberg [Tue, 9 Jan 2024 22:44:24 +0000 (23:44 +0100)] 
examples/netrc.c: add

Closes #12671

22 months agoexamples/ipv6.c: new example showing IPv6-only internet transfer
Daniel Stenberg [Tue, 9 Jan 2024 22:42:17 +0000 (23:42 +0100)] 
examples/ipv6.c: new example showing IPv6-only internet transfer

Closes #12671

22 months agoexamples/address-scope.c: renamed from ipv6.c
Daniel Stenberg [Tue, 9 Jan 2024 22:42:01 +0000 (23:42 +0100)] 
examples/address-scope.c: renamed from ipv6.c

It shows address scope use really

Closes #12671

22 months agomulti: pollset adjust, init with FIRSTSOCKET during connect
Stefan Eissing [Tue, 9 Jan 2024 08:29:34 +0000 (09:29 +0100)] 
multi: pollset adjust, init with FIRSTSOCKET during connect

- `conn->sockfd` is set by `Curl_setup_transfer()`, but that
  is called *after* the connection has been established
- use `conn->sock[FIRSTSOCKET]` instead

Follow-up to a0f94800d507de
Closes #12664

22 months agoWEBSOCKET.md: remove dead link
Daniel Stenberg [Tue, 9 Jan 2024 10:12:02 +0000 (11:12 +0100)] 
WEBSOCKET.md: remove dead link

22 months agoCI: spellcheck/appveyor: invoke configure --without-libpsl
Daniel Stenberg [Tue, 9 Jan 2024 10:09:30 +0000 (11:09 +0100)] 
CI: spellcheck/appveyor: invoke configure --without-libpsl

Follow-up to 2998874bb61ac6

22 months agocmdline/docs/*.d: switch to using ## instead of .IP
Daniel Stenberg [Tue, 9 Jan 2024 09:36:14 +0000 (10:36 +0100)] 
cmdline/docs/*.d: switch to using ## instead of .IP

To make the editing easier. To write and to read.

Closes #12667

22 months agogen.pl: support ## for doing .IP in table-like lists
Daniel Stenberg [Tue, 9 Jan 2024 09:20:48 +0000 (10:20 +0100)] 
gen.pl: support ## for doing .IP in table-like lists

Warn on use of .RS/.IP/.RE

Closes #12667

22 months agocookie.d: Document use of empty string to enable cookie engine
Jay Satiro [Sun, 7 Jan 2024 05:25:13 +0000 (00:25 -0500)] 
cookie.d: Document use of empty string to enable cookie engine

- Explain that --cookie "" can be used to enable the cookie engine
  without reading any initial cookies.

As is documented in CURLOPT_COOKIEFILE.

Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html

Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420
Reported-by: janko-js@users.noreply.github.com
Closes https://github.com/curl/curl/pull/12646

22 months agosetopt: use memdup0 when cloning COPYPOSTFIELDS
Daniel Stenberg [Sun, 7 Jan 2024 15:11:01 +0000 (16:11 +0100)] 
setopt: use memdup0 when cloning COPYPOSTFIELDS

Closes #12651

22 months agotelnet: use dynbuf instad of malloc for escape buffer
Daniel Stenberg [Sun, 7 Jan 2024 15:28:15 +0000 (16:28 +0100)] 
telnet: use dynbuf instad of malloc for escape buffer

Previously, send_telnet_data() would malloc + free a buffer every time
for escaping IAC codes. Now, it reuses a dynbuf for this purpose.

Closes #12652

22 months agoCI: install libpsl or configure --without-libpsl in builds
Daniel Stenberg [Mon, 8 Jan 2024 15:49:21 +0000 (16:49 +0100)] 
CI: install libpsl or configure --without-libpsl in builds

As a follow-up to the stricted libpsl check in configure

22 months agoconfigure: make libpsl detection failure cause error
Daniel Stenberg [Mon, 8 Jan 2024 15:28:50 +0000 (16:28 +0100)] 
configure: make libpsl detection failure cause error

To force users to explictily disable it if they really don't want it
used and make it harder to accidentally miss it.

--without-libpsl is the option to use if PSL is not wanted.

Closes #12661

22 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 8 Jan 2024 22:00:21 +0000 (23:00 +0100)] 
RELEASE-NOTES: synced

22 months agopop3: replace calloc + memcpy with memdup0
Daniel Stenberg [Sun, 7 Jan 2024 15:06:32 +0000 (16:06 +0100)] 
pop3: replace calloc + memcpy with memdup0

... and make sure to return error on out of memory.

Closes #12650

22 months agolib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT
Daniel Stenberg [Mon, 8 Jan 2024 09:34:06 +0000 (10:34 +0100)] 
lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT

Closes #12658

22 months agomime: use memdup0 instead of malloc + memcpy
Daniel Stenberg [Sun, 7 Jan 2024 15:01:00 +0000 (16:01 +0100)] 
mime: use memdup0 instead of malloc + memcpy

Closes #12649

22 months agotool_getparam: move the --rate logic into set_rate()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --rate logic into set_rate()

22 months agotool_getparam: switch to an enum for every option
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: switch to an enum for every option

To make the big switch much easier to read/understand and to make it
easier to add new options.

22 months agotool_getparam: build post data using dynbuf (more)
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: build post data using dynbuf (more)

22 months agotool_getparam: replace malloc + copy by dynbuf for --data
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: replace malloc + copy by dynbuf for --data

22 months agotool_getparam: make data_urlencode avoid direct malloc
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: make data_urlencode avoid direct malloc

use aprintf() instead

22 months agotool_getparam: move the --url-query logic into url_query()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --url-query logic into url_query()

This function is not doing post at all so it was always weirdly placed.

22 months agotool_getparam: move the --data logic into set_data()
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: move the --data logic into set_data()

22 months agotool_getparam: unify the cmdline switch() into a single one
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: unify the cmdline switch() into a single one

- easier to follow, easier to modify, easier to extend, possibly slightly
  faster

- each case now has the long option as a comment

22 months agotool_getparam: bsearch cmdline options
Daniel Stenberg [Mon, 8 Jan 2024 16:00:05 +0000 (17:00 +0100)] 
tool_getparam: bsearch cmdline options

- the option names are now alpha sorted and lookup is a lot faster

- use case sensitive matching. It was previously case insensitive, but that
  was not documented nor tested.

- remove "partial match" feature. It was not documented, not tested and
  was always fragile as existing use could break when we add a new
  option

- lookup short options via a table

Closes #12631

22 months agoCOPYING: update copyright year
Gabe [Mon, 8 Jan 2024 03:27:05 +0000 (21:27 -0600)] 
COPYING: update copyright year

Closes #12654

22 months agourl: init conn->sockfd and writesockfd to CURL_SOCKET_BAD
Stefan Eissing [Mon, 8 Jan 2024 14:02:12 +0000 (15:02 +0100)] 
url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD

Also add more tracing to test 19

Follow-up to a0f9480

Fixes #12657
Closes #12659

22 months agoconnect: remove margin from eyeballer alloc
Daniel Stenberg [Sun, 7 Jan 2024 11:06:12 +0000 (12:06 +0100)] 
connect: remove margin from eyeballer alloc

Presumably leftovers from debugging

Closes #12647

22 months agoftp: only consider entry path if it has a length
Daniel Stenberg [Sun, 7 Jan 2024 13:25:20 +0000 (14:25 +0100)] 
ftp: only consider entry path if it has a length

Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631

Avoids a NULL pointer deref.

Closes #12648

22 months agotransfer: adjust_pollset improvements
Stefan Eissing [Fri, 5 Jan 2024 11:28:09 +0000 (12:28 +0100)] 
transfer: adjust_pollset improvements

- let `multi_getsock()` initialize the pollset in what the
  transfer state requires in regards to SEND/RECV
- change connection filters `adjust_pollset()` implementation
  to react on the presence of POLLIN/-OUT in the pollset and
  no longer check CURL_WANT_SEND/CURL_WANT_RECV
- cf-socket will no longer add POLLIN on its own
- http2 and http/3 filters will only do adjustments if the
  passed pollset wants to POLLIN/OUT for the transfer on
  the socket. This is similar to the HTTP/2 proxy filter
  and works in stacked filters.

Closes #12640

22 months agoftp: use memdup0 to store the OS from a SYST 215 response
Daniel Stenberg [Fri, 5 Jan 2024 10:58:48 +0000 (11:58 +0100)] 
ftp: use memdup0 to store the OS from a SYST 215 response

avoid malloc + direct buffer fiddle

Closes #12639

22 months agoftp: use dynbuf to store entrypath
Daniel Stenberg [Fri, 5 Jan 2024 10:52:08 +0000 (11:52 +0100)] 
ftp: use dynbuf to store entrypath

avoid direct malloc

Closes #12638

22 months agowolfssl: load certificate *chain* for PEM client certs
Lealem Amedie [Fri, 5 Jan 2024 20:12:53 +0000 (13:12 -0700)] 
wolfssl: load certificate *chain* for PEM client certs

Closes #12634

22 months agohttp: adjust_pollset fix
Stefan Eissing [Thu, 4 Jan 2024 09:06:17 +0000 (10:06 +0100)] 
http: adjust_pollset fix

do not add a socket for POLLIN when the transfer does not want to send
(for example is paused).

Follow-up to 47f5b1a

Reported-by: bubbleguuum on github
Fixes #12632
Closes #12633

22 months agotool: make parser reject blank arguments if not supported
Daniel Stenberg [Tue, 2 Jan 2024 13:17:10 +0000 (14:17 +0100)] 
tool: make parser reject blank arguments if not supported

Already in the getstr() function that clones the input argument.

Closes #12620

22 months agobuild(deps): bump github/codeql-action from 2 to 3
dependabot[bot] [Tue, 2 Jan 2024 22:32:35 +0000 (22:32 +0000)] 
build(deps): bump github/codeql-action from 2 to 3

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12625

22 months agobuild(deps): bump actions/checkout from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:31 +0000 (22:32 +0000)] 
build(deps): bump actions/checkout from 3 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12624

22 months agobuild(deps): bump actions/upload-artifact from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:42 +0000 (22:32 +0000)] 
build(deps): bump actions/upload-artifact from 3 to 4

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12627

22 months agobuild(deps): bump actions/download-artifact from 3 to 4
dependabot[bot] [Tue, 2 Jan 2024 22:32:38 +0000 (22:32 +0000)] 
build(deps): bump actions/download-artifact from 3 to 4

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12626

22 months agohttp3/quiche: fix result code on a stream reset
Stefan Eissing [Wed, 3 Jan 2024 08:37:55 +0000 (09:37 +0100)] 
http3/quiche: fix result code on a stream reset

- fixes pytest failures in test 07_22
- aligns CURLcode values on stream reset with ngtcp2

Closes #12629

22 months agosetopt: clear mimepost when formp is freed
Daniel Stenberg [Tue, 2 Jan 2024 15:35:50 +0000 (16:35 +0100)] 
setopt: clear mimepost when formp is freed

A precaution to avoid a possibly dangling pointer left behind.

Reported-by: Thomas Ferguson
Fixes #12608
Closes #12621

22 months agoCI: Add dependabot.yml
andy5995 [Tue, 2 Jan 2024 18:41:58 +0000 (12:41 -0600)] 
CI: Add dependabot.yml

This will cause dependabot to open a PR when various actions are
updated, provided that the action maintainer has issued a release.

Closes #12623