]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
20 months agodocs/libcurl: add TLS backend info for all TLS options
Daniel Stenberg [Thu, 21 Mar 2024 14:46:32 +0000 (15:46 +0100)] 
docs/libcurl: add TLS backend info for all TLS options

All man pages that are listed to be for TLS now must also specify
exactly what TLS backends the option works for, or use All if they all
work.

cd2nroff makes sure this is done and that the listed backends exist.

Closes #13168

20 months agodocs/libcurl: cleanups
Daniel Stenberg [Thu, 21 Mar 2024 14:32:26 +0000 (15:32 +0100)] 
docs/libcurl: cleanups

- CURLINFO_TLS_SESSION.md: remove mention of NSS
- CURLINFO_TLS_SSL_PTR.md: remove NSS leftover
- CURLOPT_CAINFO.md: drop mention of backends not supporting this
- CURLOPT_CAPATH.md: wolfSSL also supports this

Closes #13166

20 months agodocs: make each libcurl man specify protocol(s)
Daniel Stenberg [Thu, 21 Mar 2024 10:50:20 +0000 (11:50 +0100)] 
docs: make each libcurl man specify protocol(s)

The mandatory header now has a mandatory list of protocols for which the
manpage is relevant.

Most man pages already has a "PROTOCOLS" section, but this introduces a
stricter way to specify the relevant protocols.

cd2nroff verifies that at least one protocol is mentioned (which can be
`*`).

This information is not used just yet, but A) the PROTOCOLS section can
now instead get generated and get a unified wording across all manpages
and B) this allows us to more reliably filter/search for protocol
specific manpages/options.

Closes #13166

20 months agohttp2, http3: only return CURLE_PARTIAL_FILE when bytes were received
Stefan Eissing [Tue, 19 Mar 2024 15:06:40 +0000 (16:06 +0100)] 
http2, http3: only return CURLE_PARTIAL_FILE when bytes were received

- should resolve spurious pytest failures when stream were reset
  right after response header were received

Clsoes #13151

20 months agohttp: separate response parsing from response action
Stefan Eissing [Mon, 11 Mar 2024 16:23:15 +0000 (17:23 +0100)] 
http: separate response parsing from response action

- move code that triggers on end-of-response into separate function from
  parsing
- simplify some headp/headerlen usage
- add `httpversion` to SingleRequest to indicate the version of the
  current response

Closes #13134

20 months agohttp2: remove the third (unused) argument from http2_data_done()
Daniel Stenberg [Wed, 20 Mar 2024 11:06:13 +0000 (12:06 +0100)] 
http2: remove the third (unused) argument from http2_data_done()

Closes #13154

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 21 Mar 2024 07:28:56 +0000 (08:28 +0100)] 
RELEASE-NOTES: synced

20 months agoRELEASE-NOTES: corrected
Karlson2k [Wed, 20 Mar 2024 22:03:24 +0000 (23:03 +0100)] 
RELEASE-NOTES: corrected

Corrected link for item 118

Closes #13157

20 months agoCURLOPT_INTERFACE.md: remove spurious amp, add see-also
Daniel Stenberg [Tue, 19 Mar 2024 10:27:42 +0000 (11:27 +0100)] 
CURLOPT_INTERFACE.md: remove spurious amp, add see-also

Closes #13149

20 months agohttp: improve response header handling, save cpu cycles
Stefan Eissing [Mon, 18 Mar 2024 11:46:43 +0000 (12:46 +0100)] 
http: improve response header handling, save cpu cycles

Saving some cpu cycles in http response header processing:
- pass the length of the header line along
- use string constant sizeof() instead of strlen()
- check line length if prefix is possible
- switch on first header char to limit checks

Closes #13143

20 months agotool_getparam: accept a blank -w ""
Daniel Stenberg [Mon, 18 Mar 2024 22:01:22 +0000 (23:01 +0100)] 
tool_getparam: accept a blank -w ""

Added test 468 to verify.

Regression from 07bcae89d5d00 (shipped in 8.6.0)
Reported-by: Thomas Pyle
Fixes #13144
Closes #13145

20 months agocurl_sha512_256: work around a NetBSD bug
Evgeny Grin [Fri, 15 Mar 2024 12:11:53 +0000 (13:11 +0100)] 
curl_sha512_256: work around a NetBSD bug

Based on Michael Kaufmann analysis and suggestion

Closes #13133

20 months agohttp: expect 100 rework
Stefan Eissing [Mon, 11 Mar 2024 16:23:15 +0000 (17:23 +0100)] 
http: expect 100 rework

Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general events like a timer.

HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect:
100-continue`. That reader works as follows:

- on first invocation, records time, starts the `EXPIRE_100_TIMEOUT`
  timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0,
  eos=FALSE like a paused upload.

- on subsequent invocation it checks if the timer has expired. If so, it
  enables `KEEP_SEND` and switches to passing through reads to the
  underlying readers.

Transfer handling's `readwrite()` will be invoked when a timer expires
(like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing
`KEEP_SEND_TIMER`, it will try to upload more data, which triggers
reading from the client readers again. Which then may lead to a new
pausing or cause the upload to start.

Flags and timestamps connected to this have been moved from
`SingleRequest` into the reader's context.

Closes #13110

20 months agombedtls: fix pytest for newer versions
Stefan Eissing [Fri, 15 Mar 2024 09:10:13 +0000 (10:10 +0100)] 
mbedtls: fix pytest for newer versions

Fix the expectations in pytest for newer versions of mbedtls

Closes #13132

20 months agoipv6.md: mention IPv4 mapped addresses
Daniel Stenberg [Fri, 15 Mar 2024 08:38:21 +0000 (09:38 +0100)] 
ipv6.md: mention IPv4 mapped addresses

Reported-by: Josh Soref
Assisted-by: Jay Satiro
Fixes #13112
Closes #13131

20 months agohttp: revisit http_perhapsrewind()
Stefan Eissing [Wed, 13 Mar 2024 10:42:17 +0000 (11:42 +0100)] 
http: revisit http_perhapsrewind()

- use facilities provided by client readers better
- work also for non-uploading requests like GET/HEAD
- update documentation

Closes #13117

20 months agotest 1541: verify getinfo values on first header callback
Stefan Eissing [Thu, 14 Mar 2024 11:28:46 +0000 (12:28 +0100)] 
test 1541: verify getinfo values on first header callback

Reported-by: chensong1211 on github
Ref: #13125
Closes #13128

20 months agoTLS: start shutdown only when peer did not already close
Stefan Eissing [Fri, 8 Mar 2024 08:37:27 +0000 (09:37 +0100)] 
TLS: start shutdown only when peer did not already close

- When curl sees a TCP close from the peer, do not start a TLS shutdown.
  TLS shutdown is a handshake and if the peer already closed the
  connection, it is not interested in participating.

Reported-by: dfdity on github
Assisted-by: Jiří Bok
Assisted-by: Pēteris Caune
Fixes #10290
Closes #13087

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 14 Mar 2024 22:11:53 +0000 (23:11 +0100)] 
RELEASE-NOTES: synced

20 months agocurl: make --libcurl output better CURLOPT_*SSLVERSION
Daniel Stenberg [Thu, 14 Mar 2024 12:58:45 +0000 (13:58 +0100)] 
curl: make --libcurl output better CURLOPT_*SSLVERSION

The option is really two enums ORed together, so it needs special
attention to make the code output nice.

Added test 1481 to verify. Both the server and the proxy versions.

Reported-by: Boris Verkhovskiy
Fixes #13127
Closes #13129

20 months agoGHA/linux: add sysctl trick to work-around GitHub runner issue
Daniel Stenberg [Thu, 14 Mar 2024 11:56:39 +0000 (12:56 +0100)] 
GHA/linux: add sysctl trick to work-around GitHub runner issue

The GitHub image runner update from 20240304.1.0 to 20240310.1
introduces a problem for clang-14. The issue is caused by
incompatibility between llvm 14 provided in ubuntu-22.04 image and the
much newer kernel configured with high-entropy ASLR.

As a work-around, we issue a sysctl command to lower the entropy and get
clang-14 to work again.

URL: https://github.com/actions/runner-images/issues/9491

Closes #13124

20 months agoSPONSORS: describe the basics
Daniel Stenberg [Wed, 13 Mar 2024 12:26:07 +0000 (13:26 +0100)] 
SPONSORS: describe the basics

Closes #13119

20 months agoGOVERNANCE: document the core team
Daniel Stenberg [Wed, 13 Mar 2024 12:23:38 +0000 (13:23 +0100)] 
GOVERNANCE: document the core team

Closes #13118

20 months agovquic-tls: fix the error code returned for bad CA file
Jay Satiro [Wed, 13 Mar 2024 06:03:18 +0000 (02:03 -0400)] 
vquic-tls: fix the error code returned for bad CA file

- Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem
  reading the cert file or path.

This is a follow-up to the parent commit aedbbdf1.

Reported-by: Karthikdasari0423@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/13115

20 months agovquic-tls: return appropirate errors on wolfSSL errors
Daniel Stenberg [Mon, 11 Mar 2024 09:53:08 +0000 (10:53 +0100)] 
vquic-tls: return appropirate errors on wolfSSL errors

Reported-by: Dexter Gerig
Closes #13107

20 months agotidy-up: one comment and EOF newlines
Viktor Szakats [Tue, 12 Mar 2024 12:04:27 +0000 (12:04 +0000)] 
tidy-up: one comment and EOF newlines

Reviewed-by: Daniel Stenberg
Closes #13108

20 months agocmdline-opts: language cleanups
Daniel Stenberg [Tue, 12 Mar 2024 09:34:58 +0000 (10:34 +0100)] 
cmdline-opts: language cleanups

Use imperative mood consistently for the first sentence describing an
option.

"Set this" instead "tell curl to set" or "this sets..."

Plus some extra cleanups and rephrasing.

Closes #13106

20 months agomanagen: remove space before protocols
Daniel Stenberg [Tue, 12 Mar 2024 07:45:22 +0000 (08:45 +0100)] 
managen: remove space before protocols

For options that are listed for specific protocols, the protocols (shown
first within parentheses) are now output without the leading space in the
manpage output.

Closes #13105

20 months agombedtls: properly cleanup the thread-shared entropy
Jay Satiro [Thu, 7 Mar 2024 00:18:46 +0000 (19:18 -0500)] 
mbedtls: properly cleanup the thread-shared entropy

- Store the state of the thread-shared entropy for global init/cleanup.

- Use curl's thread support of mbedtls for all Windows builds instead of
  just when the threaded resolver is used via USE_THREADS_WIN32.

Prior to this change on global cleanup curl builds that have curl thread
support for mbedtls freed the entropy (8b1d2298) but failed to mark that
it had been freed, which caused problems on subsequent init + transfer.

Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105
Reported-by: awesomekosm@users.noreply.github.com
Closes https://github.com/curl/curl/pull/13071

20 months agotool_getparam: handle non-existing (out of range) short-options
Daniel Stenberg [Mon, 11 Mar 2024 21:39:22 +0000 (22:39 +0100)] 
tool_getparam: handle non-existing (out of range) short-options

... correctly, even when they follow an existing one without a space in
between.

Verify with test 467

Follow-up to 07dd60c05b
Reported-by: Geeknik Labs
Fixes #13101
Closes #13102

20 months agolib: move 'done' parameter to SingleRequests
Stefan Eissing [Mon, 11 Mar 2024 11:35:07 +0000 (12:35 +0100)] 
lib: move 'done' parameter to SingleRequests

A transfer may do several `SingleRequest`s for its success. This happens
regularly for authentication, follows and retries on failed connections.
The "readwrite()" calls and functions connected to those carried a `bool
*done` parameter to indicate that the current `SingleRequest` is over.
This may happen before `upload_done` or `download_done` bits of
`SingleRequest` are set.

The problem with that is now `write_resp()` protocol handlers are
invoked in places where the `bool *done` cannot be passed up to the
caller. Instead of being a bool in the call chain, it needs to become a
member of `SingleRequest`, reflecting its state.

This removes the `bool *done` parameter and adds the `done` bit to
`SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a
`SingleRequest` in a follow up, clearing `done` and other
flags/counters.

Closes #13096

20 months agorequest: clarify message when request has been sent off
Stefan Eissing [Mon, 11 Mar 2024 08:50:55 +0000 (09:50 +0100)] 
request: clarify message when request has been sent off

Change the "uploaded and fine" message for requests without a body

Reported-by: Karthikdasari0423 on github
Fixes #13093
Closes #13095

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 11 Mar 2024 06:30:01 +0000 (07:30 +0100)] 
RELEASE-NOTES: synced

20 months agolib: keep conn IP information together
Stefan Eissing [Fri, 8 Mar 2024 09:45:14 +0000 (10:45 +0100)] 
lib: keep conn IP information together

new struct ip_quadruple for holding local/remote addr+port

- used in data->info and conn and cf-socket.c
- copy back and forth complete struct
- add 'secondary' to conn
- use secondary in reporting success for ftp 2nd connection

Reported-by: DasKutti on github
Fixes #13084
Closes #13090

20 months agoscripts/managen: the new name and home for the manpage generator
Daniel Stenberg [Fri, 8 Mar 2024 10:22:12 +0000 (11:22 +0100)] 
scripts/managen: the new name and home for the manpage generator

It was previously docs/cmdline-opts/gen.pl

Closes #13089

20 months agoVULN-DISCLOSURE-POLICY.md: update detail about CVE requests
Daniel Stenberg [Fri, 8 Mar 2024 10:09:48 +0000 (11:09 +0100)] 
VULN-DISCLOSURE-POLICY.md: update detail about CVE requests

curl is a CNA now

Closes #13088

20 months agolib: client reader polish
Stefan Eissing [Thu, 7 Mar 2024 10:05:53 +0000 (11:05 +0100)] 
lib: client reader polish

- seek_func/seek_client, use transfer values only
    - remove copies held in `struct connectdata`, use only
      ever `data->set.seek_func`
    - resolves possible issues in multiuse connections
    - new mime post reader eliminates need to ever overwriting this

- websockets, remove empty Curl_ws_done() function

Closes #13079

20 months agolib1598: fix `CURLOPT_POSTFIELDSIZE` usage
Marcel Raad [Fri, 8 Mar 2024 07:45:10 +0000 (08:45 +0100)] 
lib1598: fix `CURLOPT_POSTFIELDSIZE` usage

It requires a `long` argument.

Closes https://github.com/curl/curl/pull/13085

20 months agodocs/cmdline-opts: drop the curl.1 from the dist tarball
Daniel Stenberg [Thu, 7 Mar 2024 22:25:07 +0000 (23:25 +0100)] 
docs/cmdline-opts: drop the curl.1 from the dist tarball

Since it is no longer needed for building tool_hugehelp.c and all the
docs is available in readable markdown format in the tarball, the peeps
that don't want to build the manpage still do good.

Removing it also fixes the complexity of out-of-tree builds when the
curl.1 exists in the source tree.

20 months agotest1140/1173: extend wildcards to find curl.1
Daniel Stenberg [Thu, 7 Mar 2024 21:30:50 +0000 (22:30 +0100)] 
test1140/1173: extend wildcards to find curl.1

... in its new build path.

Also update the test scripts to be more precise in error messages to
help us understand CI errors better.

Follow-up to f03c85635f35269f1
Ref: #13029
Closes #13083

20 months agohttp2: minor tweaks to optimize two struct sizes
Daniel Stenberg [Thu, 7 Mar 2024 15:41:06 +0000 (16:41 +0100)] 
http2: minor tweaks to optimize two struct sizes

- use BIT() instead of bool
- place the struct fields in (roughly) size order

Closes #13082

20 months agobuildconf.bat: remove outdated groff/nroff use
Daniel Stenberg [Thu, 7 Mar 2024 09:33:49 +0000 (10:33 +0100)] 
buildconf.bat: remove outdated groff/nroff use

- don't try to generate the real hugehelp file, because it requires
  curl.txt which needs a build
- don't attempt to do anything in a c-ares subdirectory

Follow-up to f03c85635f35269
Closes #13078

20 months agohttp2: memory errors in the push callbacks are fatal
Daniel Stenberg [Thu, 7 Mar 2024 15:02:13 +0000 (16:02 +0100)] 
http2: memory errors in the push callbacks are fatal

Use the correct nghttp2 error code accordingly.

Closes #13081

20 months agomkhelp: rename variable to fix compiler warnings
Viktor Szakats [Thu, 7 Mar 2024 09:25:39 +0000 (09:25 +0000)] 
mkhelp: rename variable to fix compiler warnings

```
src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
  _bld\src\tool_hugehelp.c(8,27):
  see declaration of 'm'
src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75cd2n0wj6lc#L614

Reviewed-by: Daniel Stenberg
Closes #13077

20 months agoKNOWN_BUGS: POP3 issue when reading small chunks
Daniel Stenberg [Thu, 7 Mar 2024 16:40:29 +0000 (17:40 +0100)] 
KNOWN_BUGS: POP3 issue when reading small chunks

Closes #12063

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 7 Mar 2024 15:50:59 +0000 (16:50 +0100)] 
RELEASE-NOTES: synced

20 months agoasyn-ares: fix data race warning
Robert Moreton [Mon, 26 Feb 2024 23:16:21 +0000 (18:16 -0500)] 
asyn-ares: fix data race warning

- Store the c-ares version during global init.

Prior to this change several threads could write the same data to a
static int variable at the same time. Though in practice it's not a
problem ThreadSanitizer may warn.

Reported-by: Nikita Taranov
Assisted-by: Jay Satiro
Fixes #13065
Closes #13000

20 months agohyper: implement unpausing via client reader
Stefan Eissing [Thu, 7 Mar 2024 09:08:35 +0000 (10:08 +0100)] 
hyper: implement unpausing via client reader

Just a tidy up to contain 'ifdef' pollution of common
code parts with implementation specifics.

- remove the ifdef hyper unpausing in easy.c
- add hyper client reader for CURL_CR_PROTOCOL phase
  that implements the unpause method for calling
  the hyper waker if it is set

Closes #13075

20 months agongtcp2: no recvbuf for stream
Stefan Eissing [Thu, 7 Mar 2024 08:23:11 +0000 (09:23 +0100)] 
ngtcp2: no recvbuf for stream

- write response data directly to the transfer via
 `Curl_xfer_write_resp()` like we do in HTTP/2.

Closes #13073

20 months agodocs/cmdline-opts/.gitignore: ignore curl.txt
Stefan Eissing [Thu, 7 Mar 2024 09:18:33 +0000 (10:18 +0100)] 
docs/cmdline-opts/.gitignore: ignore curl.txt

Closes #13076

20 months agosha512_256: add support for GnuTLS and OpenSSL
Evgeny Grin (Karlson2k) [Wed, 6 Mar 2024 23:42:04 +0000 (00:42 +0100)] 
sha512_256: add support for GnuTLS and OpenSSL

This is a follow-up for PR #12897.

Add support for SHA-512/256 digest calculation by TLS backends.
Currently only OpenSSL and GnuTLS (actually, nettle) support
SHA-512/256.

Closes #13070

20 months agodigest: add check for hashing error
Evgeny Grin [Thu, 7 Mar 2024 00:35:31 +0000 (01:35 +0100)] 
digest: add check for hashing error

Closes #13072

20 months agocmake: enable `ENABLE_CURL_MANUAL` by default
Viktor Szakats [Wed, 6 Mar 2024 22:24:56 +0000 (22:24 +0000)] 
cmake: enable `ENABLE_CURL_MANUAL` by default

Meaning `curl.1` and `src/tool_hugehelp.c` are built by default,
and `--manual` in curl tool is also enabled by default.

This syncs behaviour with autotools.

For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set
to a consistent date, e.g. the timestamp of `CHANGES`.

A pre-built manual (e.g. the one distributed in the official source
tarball) will be ignored and rebuilt after this patch, unless
explicitly disabling this option.

Fixes #13028
Closes #13069

20 months agohttp2: push headers better cleanup
Stefan Eissing [Wed, 6 Mar 2024 08:36:08 +0000 (09:36 +0100)] 
http2: push headers better cleanup

- provide common cleanup method for push headers

Closes #13054

20 months agoGIT-INFO: convert to markdown
Daniel Stenberg [Thu, 7 Mar 2024 08:28:06 +0000 (09:28 +0100)] 
GIT-INFO: convert to markdown

Closes #13074

20 months agocmake: fix libcurl.pc and curl-config library specifications
Richard Levitte [Tue, 13 Feb 2024 05:45:57 +0000 (06:45 +0100)] 
cmake: fix libcurl.pc and curl-config library specifications

Letting CMake figure out where libraries are located gives you full
paths. When generating libcurl.pc and curl-config, getting libraries as
full paths is unusual when one expects to get a list of -l<libname>.

To meet expectations, an effort is made to convert the full paths into
-l<libname>, possibly with -L<libdir> before it.

Fixes #6169
Fixes #12748
Closes #12930

20 months agotest463: HTTP with -d @file with file containing CR, LF and null byte
Daniel Stenberg [Wed, 6 Mar 2024 21:44:41 +0000 (22:44 +0100)] 
test463: HTTP with -d @file with file containing CR, LF and null byte

20 months agoparamhlp: fix CRLF-stripping files with "-d @file"
Daniel Stenberg [Wed, 6 Mar 2024 14:39:09 +0000 (15:39 +0100)] 
paramhlp: fix CRLF-stripping files with "-d @file"

All CR and LF bytes should be stripped, as documented, and all other
bytes are inluded in the data. Starting now, it also excludes null bytes
as they would otherwise also cut the data short.

Reported-by: Simon K
Fixes #13063
Closes #13064

20 months agocmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled
Viktor Szakats [Wed, 6 Mar 2024 11:43:40 +0000 (11:43 +0000)] 
cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled

Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF`
when building without the Schannel TLS backend.

This in turn may have caused Kerberos, SPNEGO and SSPI features
disappearing even with `CURL_WINDOWS_SSPI=ON` set.

This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a
default for `CURL_WINDOWS_SSPI`, but allowing a manual override.

Also update the option text to better tell its purpose.

Thanks-to: Andreas Loew
Reviewed-by: Daniel Stenberg
Ref: #13056
Closes #13061

20 months agoKNOWN_BUGS: FTPS server compatibility on Windows with Schannel
Jay Satiro [Sun, 3 Mar 2024 07:15:24 +0000 (02:15 -0500)] 
KNOWN_BUGS: FTPS server compatibility on Windows with Schannel

- Remove "2.12 FTPS with Schannel times out file list operation"

- Remove "7.12 FTPS directory listing hangs on Windows with Schannel"

- Add "7.12 FTPS server compatibility on Windows with Schannel"

This change adds a more generic bug description that explains FTPS with
the latest curl and Schannel is not widely used and may have more bugs
than other TLS backends.

The two removed FTPS Schannel bugs can't be reproduced any longer and
were likely fixed by 24d6c288.

Ref: https://github.com/curl/curl/issues/5284
Ref: https://github.com/curl/curl/issues/9161
Ref: https://github.com/curl/curl/issues/12894

Closes https://github.com/curl/curl/pull/13032

20 months agotrace-config.md: remove the mutexed options list
Jay Satiro [Sat, 2 Mar 2024 22:44:33 +0000 (17:44 -0500)] 
trace-config.md: remove the mutexed options list

- Remove the rendered manpage message that says:
  "[--trace-config] is mutually exclusive to --trace and -v, --verbose".

Actually it can be used with either of those options, which are mutually
exclusive to each other but not to --trace-config.

Ref: https://curl.se/docs/manpage.html#--trace-config

Closes https://github.com/curl/curl/pull/13031

20 months agomkhelp: simplify the generated hugehelp program
Daniel Stenberg [Tue, 5 Mar 2024 16:32:00 +0000 (17:32 +0100)] 
mkhelp: simplify the generated hugehelp program

Use a plain array and puts() every line, also allows us to provide the
strings without ending newlines.

- merge blank lines into the next one as a prefixed newline.
- turn eight consecutive spaces into a tab (since they can only be on the
  left side of text)
- the newly generated tool_hugehelp is 3K lines shorter and 50K smaller
- modifies the top logo layout a little by reducing the indent

Closes #13047

20 months agodocs: ascii version of manpage without nroff
Daniel Stenberg [Mon, 4 Mar 2024 16:22:17 +0000 (17:22 +0100)] 
docs: ascii version of manpage without nroff

Create ASCII version of manpage without nroff

 - build src/tool_hugegelp.c from the ascii manpage
 - move the the manpage and the ascii version build to docs/cmdline-opts
 - remove all use of nroff from the build process
 - should make the build entirely reproducible (by avoiding nroff)

 - partly reverts 2620aa9 to build libcurl option man pages one by one
   in cmake because the appveyor builds got all crazy until I did

The ASCII version of the manpage

 - is built with gen.pl, just like the manpage is
 - has a right-justified column making the appearance similar to the previous
   version
 - uses a 4-space indent per level (instead of the old version's 7)
 - does not do hyphenation of words (which nroff does)

History

  We first made the curl build use nroff for building the hugehelp file in
  December 1998, for curl 5.2.

Closes #13047

20 months agolib: add `void *ctx` to reader/writer instances
Stefan Eissing [Wed, 6 Mar 2024 08:52:43 +0000 (09:52 +0100)] 
lib: add `void *ctx` to reader/writer instances

- `struct Curl_cwriter` and `struct Curl_creader` now carry a
  `void *ctx` member that points to the instance as allocated.
- using `r->ctx` and `w->ctx` as pointer to the instance specific
  struct that has been allocated

Reported-by: Rudi Heitbaum
Fixes #13035
Closes #13059

20 months agohttp: fix dead code in setting post client reader
Stefan Eissing [Wed, 6 Mar 2024 11:09:05 +0000 (12:09 +0100)] 
http: fix dead code in setting post client reader

- postsize was always 0, thus the check's else never happened
  after the mime client reader was introduced

Follow-up to 0ba47146f7ff3d
Closes #13060

20 months agohttp2: fix push discard
Stefan Eissing [Wed, 6 Mar 2024 09:15:15 +0000 (10:15 +0100)] 
http2: fix push discard

- fix logic in discarding a failed pushed stream so that
  stream context is properly cleaned up

Closes #13055

20 months agotransfer.c: break receive loop in speed limited transfers
Stefan Eissing [Tue, 5 Mar 2024 10:08:55 +0000 (11:08 +0100)] 
transfer.c: break receive loop in speed limited transfers

- the change breaks looping in transfer.c receive for transfers that are
  speed limited on having gotten *some* bytes.
- the overall speed limit timing is done in multi.c

Reported-by: Dmitry Karpov
Bug: https://curl.se/mail/lib-2024-03/0001.html
Closes #13050

20 months agomime: add client reader
Stefan Eissing [Thu, 29 Feb 2024 09:12:39 +0000 (10:12 +0100)] 
mime: add client reader

Add `mime` client reader. Encapsulates reading from mime parts, getting
their length, rewinding and unpausing.

- remove special mime handling from sendf.c and easy.c
- add general "unpause" method to client readers
- use new reader in http/imap/smtp
- make some mime functions static that are now only used internally

In addition:
- remove flag 'forbidchunk' as no longer needed

Closes #13039

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 5 Mar 2024 16:15:27 +0000 (17:15 +0100)] 
RELEASE-NOTES: synced

20 months agoTODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake"
Daniel Stenberg [Tue, 5 Mar 2024 16:09:46 +0000 (17:09 +0100)] 
TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake"

Follow-up to 8e741644a229c37

20 months agocmake: add USE_OPENSSL_QUIC support
Tal Regev [Sun, 3 Mar 2024 19:57:03 +0000 (21:57 +0200)] 
cmake: add USE_OPENSSL_QUIC support

Closes #13034

20 months agoTIMER_STARTTRANSFER: set the same for everyone
Stefan Eissing [Tue, 5 Mar 2024 10:48:16 +0000 (11:48 +0100)] 
TIMER_STARTTRANSFER: set the same for everyone

- set TIMER_STARTTRANSFER on seeing the first response bytes
  in the download client writer, not coming from a CONNECT
- initialized the timer the same way for all protocols
- remove explicit setting of TIMER_STARTTRANSFER in file.c
  and c-hyper.c

Closes #13052

20 months agohttp: better error message for HTTP/1.x response without status line
Michael Kaufmann [Mon, 4 Mar 2024 17:15:08 +0000 (18:15 +0100)] 
http: better error message for HTTP/1.x response without status line

If a response without a status line is received, and the connection is
known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status
line" instead of "Received HTTP/0.9 when not allowed".

Closes #13045

20 months agoKNOWN_BUGS: fix typo
Viktor Szakats [Tue, 5 Mar 2024 10:52:04 +0000 (10:52 +0000)] 
KNOWN_BUGS: fix typo

Reviewed-by: Daniel Stenberg
Closes #13051

20 months agosmpt: fix starttls
Sebastian Neubauer [Tue, 5 Mar 2024 01:11:46 +0000 (02:11 +0100)] 
smpt: fix starttls

In cases where the connection was fast, curl sometimes failed to open a
connection. This fixes a regression of c2d973627bab12abc5486a3f3.

The regression triggered in these steps:

1. Create an smtp connection
2. Use STARTTLS
3. Receive the response
4. We are inside the loop in `smtp_statemachine`, calling
   `smtp_state_starttls_resp`
5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and
   run `smtp_perform_upgrade_tls` at the start of the function.

   In the bad flow, we stay in the while loop, calling
   `Curl_pp_readresp`, which reads part of the TLS handshake and things
   go wrong.

The reason is that `Curl_pp_moredata` changed behavior and always
returns `true`, so we stay in the loop in `smtp_statemachine`. With a
slow connection `Curl_pp_readresp` cannot read new data and returns
`CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`.

With a fast connection, `Curl_pp_readresp` reads new data from the tcp
connection, which is part of the TLS handshake.

The fix is in `Curl_pp_moredata`, which needs to take the final line
into account and return `false` if only the final line is stored.

Closes #13048

20 months agolib: enhance client reader resume + rewind
Stefan Eissing [Thu, 29 Feb 2024 09:12:39 +0000 (10:12 +0100)] 
lib: enhance client reader resume + rewind

- update client reader documentation
- client reader, add rewind capabilities
    - tell creader to rewind on next start
    - Curl_client_reset() will keep reader for future rewind if requested
    - add Curl_client_cleanup() for freeing all resources independent of
      rewinds
    - add Curl_client_start() to trigger rewinds
    - move rewind code from multi.c to sendf.c and make part of
      "cr-in"'s implementation
- http, move the "resume_from" handling into the client readers
    - the setup of a HTTP request is reshuffled to follow:
      * determine method, target, auth negotiation
      * install the client reader(s) for the request, including crlf
        conversions and "chunked" encoding
      * apply ranges to client reader
      * concat request headers, upgrades, cookies, etc.
      * complete request by determining Content-Length of installed
        readers in combination with method
      * send
    - add methods for client readers to
      * return the overall length they will generate (or -1 when unknown)
      * return the amount of data on the CLIENT level, so that
        expect-100 can decide if it want to apply itself
      * set a "resume_from" offset or fail if unsupported
    - struct HTTP has become largely empty now
- rename `Client_reader_*` to `Curl_creader_*`

Closes #13026

20 months agoopenssl-quic: fix BIO leak and Windows warning
Viktor Szakats [Mon, 4 Mar 2024 17:07:40 +0000 (17:07 +0000)] 
openssl-quic: fix BIO leak and Windows warning

Caused by an accidentally duplicated line in
d6825df334def106f735ce7e0c1a2ea87bddffb0.

```
.../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses integer precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-64-to-32]
 1095 |   bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE);
      |         ~~~~~~~~~~~~~ ~~~~~~~^~~~~~
1 warning and 2 errors generated.
```

Reviewed-by: Stefan Eissing
Closes #13043

20 months agoopenssl-quic: fix unity build, casing, indentation
Viktor Szakats [Mon, 4 Mar 2024 17:12:42 +0000 (17:12 +0000)] 
openssl-quic: fix unity build, casing, indentation

- rename static functions to avoid duplicate symbols in unity mode.
- windows -> Windows/window in error message and comment.
- fix indentation.

Reviewed-by: Stefan Eissing
Closes #13044

20 months agogen.pl: make the "manpageification" faster
Daniel Stenberg [Mon, 4 Mar 2024 14:16:11 +0000 (15:16 +0100)] 
gen.pl: make the "manpageification" faster

The function that replaces occurances of "--longoption" with "-Z,
--longoption" etc with the proper highlight applied, no longer loops
over the options.

Closes #13041

20 months agoCONTRIBUTE: update the section on documentation format
Daniel Stenberg [Mon, 4 Mar 2024 22:04:05 +0000 (23:04 +0100)] 
CONTRIBUTE: update the section on documentation format

... since most of it is markdown now.

Closes #13046

20 months agosmtp: free a temp resource
Daniel Stenberg [Mon, 4 Mar 2024 09:07:07 +0000 (10:07 +0100)] 
smtp: free a temp resource

The returned address needs to be freed.

Follow-up to e3905de8196d67b89df1602feb84c1f993211b20
Spotted by Coverity

Closes #13038

20 months ago_VARIABLES.md: improve the description
Daniel Stenberg [Mon, 4 Mar 2024 13:26:14 +0000 (14:26 +0100)] 
_VARIABLES.md: improve the description

Closes #13040

20 months agobuild(deps): bump fsfe/reuse-action from 2 to 3
dependabot[bot] [Mon, 4 Mar 2024 14:31:36 +0000 (14:31 +0000)] 
build(deps): bump fsfe/reuse-action from 2 to 3

Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3.
- [Release notes](https://github.com/fsfe/reuse-action/releases)
- [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: fsfe/reuse-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
20 months agopytest: adapt to API change
Stefan Eissing [Mon, 4 Mar 2024 09:05:52 +0000 (10:05 +0100)] 
pytest: adapt to API change

- pytest has changed the signature of the hook pytest_report_header()
  for some obscure reason and that change landed in our CI now

- remove the changed param that we never used anyway

Closes #13037

20 months agocookie: if psl fails, reject the cookie
Daniel Stenberg [Sun, 3 Mar 2024 17:41:52 +0000 (18:41 +0100)] 
cookie: if psl fails, reject the cookie

A libpsl install without data and no built-in database is now considered
bad enough to reject all cookies since they cannot be checked. It is
somewhat of a user error, but still.

Reported-by: Dan Fandrich
Closes #13033

20 months agolib: further send/upload handling polish
Stefan Eissing [Wed, 28 Feb 2024 13:51:53 +0000 (14:51 +0100)] 
lib: further send/upload handling polish

- Move all the "upload_done" handling to request.c

  - add possibility to abort sending of a request
  - add `Curl_req_done_sending()` for checks
  - transfer.c: readwrite_upload() now clean

- removing data->state.ulbuf and data->req.upload_fromhere

  - as well as data->req.upload_present
  - set data->req.upload_done on having read all from
    the client and completely flushed the send buffer

- tftp, remove setting of data->req.upload_fromhere

  - serves no purpose as `upload_present` is not set
    and the data itself is directly `sendto()` anyway

- smtp, make upload EOB conversion a client reader
- xfer_ulbuf addition

  - add xfer_ulbuf for borrowing, similar to xfer_buf
  - use in file upload
  - use in c-hyper body sending

- h1-proxy, remove init of data->state.uilbuf that is never used
- smb, add own send_buf instead of using data->state.ulbuf

Closes #13010

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 4 Mar 2024 07:19:12 +0000 (08:19 +0100)] 
RELEASE-NOTES: synced

20 months agorustls: fix two warnings related to number types
kpcyrd [Thu, 29 Feb 2024 15:01:03 +0000 (16:01 +0100)] 
rustls: fix two warnings related to number types

Reported-by: Gisle Vanem
Follow-up to #12989
Closes #13017

20 months agobufq: writing into a softlimit queue cannot be partial
Stefan Eissing [Fri, 1 Mar 2024 08:12:50 +0000 (09:12 +0100)] 
bufq: writing into a softlimit queue cannot be partial

- when unable to obtain a new chunk on a softlimit bufq,
  this is an allocation error and needs to be reported as
  such.
- writes into a soflimit bufq never must be partial success

Reported-by: Dan Fandrich
Fixes #13020
Closes #13023

20 months agoconfigure: Don't build shell completions when disabled
Dan Fandrich [Sat, 2 Mar 2024 02:27:35 +0000 (18:27 -0800)] 
configure: Don't build shell completions when disabled

With the recent changes to completion file building, the files were
built always and only installation was selectively disabled.  Now, when
they are disabled they aren't even built, avoiding a build-time error in
environments where it's not possible to run the curl binary that was
just created (e.g. if library paths were not set up correctly).

Follow-up to 0f7aba83c

Reported-by: av223119 on github
Fixes #13027
Closes #13030

20 months agocmdline-opts/_EXITCODES: sync with libcurl-errors
Jay Satiro [Thu, 29 Feb 2024 06:36:31 +0000 (01:36 -0500)] 
cmdline-opts/_EXITCODES: sync with libcurl-errors

- Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that
  can be returned by the curl tool.

Closes https://github.com/curl/curl/pull/13015

21 months agohyper: disable test1598 due to lack of trailer support
Stefan Eissing [Thu, 29 Feb 2024 10:31:55 +0000 (11:31 +0100)] 
hyper: disable test1598 due to lack of trailer support

Follow-up to 50838095

Closes #13016

21 months agoftp: Mark a const buffer as const
Dan Fandrich [Fri, 1 Mar 2024 09:07:02 +0000 (01:07 -0800)] 
ftp: Mark a const buffer as const

21 months agoappveyor: Properly skip if only CircleCI is changed
Dan Fandrich [Fri, 1 Mar 2024 09:06:28 +0000 (01:06 -0800)] 
appveyor: Properly skip if only CircleCI is changed

21 months agodocs: Update minimal binary size in INSTALL.md
Dan Fandrich [Fri, 10 Nov 2023 09:41:13 +0000 (01:41 -0800)] 
docs: Update minimal binary size in INSTALL.md

Include more options to reduce binary size.

21 months agoconfigure: Don't make shell completions without perl
Dan Fandrich [Fri, 1 Mar 2024 07:38:22 +0000 (23:38 -0800)] 
configure: Don't make shell completions without perl

The code that attempted to skip building the shell completions didn't
work properly and tried to build them even if perl wasn't available.
This step, as well as the install step, is now properly skipped without
perl.

Follow-up to 89733e2dd

Closes #13022

21 months agomisc: Fix typos in docs and lib
RainRat [Thu, 29 Feb 2024 19:55:28 +0000 (11:55 -0800)] 
misc: Fix typos in docs and lib

This fixes miscellaneous typos and duplicated words in the docs, lib
and test comments and a few user facing errorstrings.

Author: RainRat on Github
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Dan Fandrich <dan@coneharvesters.com>
Closes: #13019
21 months agoconfigure: build & install shell completions when enabled
Dan Fandrich [Thu, 8 Feb 2024 19:34:34 +0000 (11:34 -0800)] 
configure: build & install shell completions when enabled

The --with-fish-functions-dir and --with-zsh-functions-dir options
currently have no effect on a normal build because the scripts/ directory
where they're used is not built. Add scripts/ to a normal build and
change the completion options to default to off to preserve the existing
behaviour.

Closes: #12906
21 months agogithub/labeler: improve the match patterns
Dan Fandrich [Thu, 29 Feb 2024 21:04:23 +0000 (13:04 -0800)] 
github/labeler: improve the match patterns

21 months agotests: add test1598 for POST with trailers
Stefan Eissing [Wed, 28 Feb 2024 12:20:59 +0000 (13:20 +0100)] 
tests: add test1598 for POST with trailers

- test POST fields with trailers and chunked encoding

Ref: #12938
Closes #13009