]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
2 years agotls: backends use connection filters for IO, enabling HTTPS-proxy
Stefan Eissing [Fri, 25 Nov 2022 13:06:43 +0000 (14:06 +0100)] 
tls: backends use connection filters for IO, enabling HTTPS-proxy

 - OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962

2 years agoinclude/curl/curl.h: bump the deprecated requirements to gcc 6.1
Daniel Stenberg [Mon, 28 Nov 2022 07:22:02 +0000 (08:22 +0100)] 
include/curl/curl.h: bump the deprecated requirements to gcc 6.1

Reported-by: Michael Kaufmann
Fixes #9917
Closes #9987

2 years agomime: relax easy/mime structures binding
Patrick Monnerat [Wed, 16 Nov 2022 16:40:11 +0000 (17:40 +0100)] 
mime: relax easy/mime structures binding

Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).

This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).

This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.

When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.

Closes #9927

2 years agotest416: verify growing FTP file support
fractal-access [Thu, 10 Nov 2022 15:20:28 +0000 (16:20 +0100)] 
test416: verify growing FTP file support

Added setting: RETRSIZE [size] in the <servercmd> section. When set this
will cause the test FTP server to return the size set (rather than the
actual size) in the acknowledgement from a RETR request.

Closes #9772

2 years agoftp: support growing files with CURLOPT_IGNORE_CONTENT_LENGTH
fractal-access [Wed, 19 Oct 2022 13:37:44 +0000 (14:37 +0100)] 
ftp: support growing files with CURLOPT_IGNORE_CONTENT_LENGTH

When using the option CURLOPT_IGNORE_CONTENT_LENGTH (set.ignorecl in
code) to support growing files in FTP, the code should ignore the
initial size it gets from the server as this will not be the final size
of the file. This is done in ftp_state_quote() to prevent a size request
being issued in the initial sequence. However, in a later call to
ftp_state_get_resp() the code attempts to get the size of the content
again if it doesn't already have it, by parsing the response from the
RETR request. This fix prevents this parsing of the response to get the
size when the set.ignorecl option is set. This should maintain the size
value as -1, unknown, in this situation.

Closes #9772

2 years agocfilter: re-add `conn` as parameter to cfilter setup methods
Stefan Eissing [Sat, 26 Nov 2022 11:43:56 +0000 (12:43 +0100)] 
cfilter: re-add `conn` as parameter to cfilter setup methods

- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983

2 years agoKNOWN_BUGS: remove five FTP related issues
Daniel Stenberg [Tue, 22 Nov 2022 08:28:20 +0000 (09:28 +0100)] 
KNOWN_BUGS: remove five FTP related issues

- "FTP with CONNECT and slow server"

I believe this is not a problem these days.

- "FTP with NULs in URL parts"

The FTP protocol does not support them properly anyway.

- remove "FTP and empty path parts in the URL"

I don't think this has ever been reported as a real problem but was only
a hypothetical one.

- "Premature transfer end but healthy control channel"

This is not a bug, this is an optimization that *could* be performed but is
not an actual problem.

- "FTP without or slow 220 response"

Instead add to the documentation of the connect timeout that the
connection is considered complete at TCP/TLS/QUIC layer.

Closes #9979

2 years agotests: add authorityInfoAccess to generated certs
Stefan Eissing [Fri, 25 Nov 2022 13:00:48 +0000 (14:00 +0100)] 
tests: add authorityInfoAccess to generated certs

Generate stunnel.pem as well

Closes #9980

2 years agoruntests: --no-debuginfod now disables DEBUGINFOD_URLS
Daniel Stenberg [Sat, 19 Nov 2022 14:58:35 +0000 (15:58 +0100)] 
runtests: --no-debuginfod now disables DEBUGINFOD_URLS

Prior to this change, DEBUGINFOD_URLS was always disabled by runtests
due to a report of it slowing down tests. However, some setups need it
to fetch debug symbols, and if it is disabled on those systems then curl
tests with valgrind will fail.

Reported-by: Mark Gaiser
Ref: #8805
Closes #9950

2 years agotest/aws_sigv4: test cases for content-sha256
Casey Bodley [Mon, 21 Nov 2022 19:56:13 +0000 (14:56 -0500)] 
test/aws_sigv4: test cases for content-sha256

1956 adds the sha256 value corresponding to an empty buffer
1957 adds an arbitrary value and confirms that the signature differs from 1956
1958 adds whitespace to 1957 and confirms that the signature matches 1957
1959 adds a value longer than 'char sha_hex[65]' in Curl_output_aws_sigv4()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Closes #9804

2 years agoaws_sigv4: consult x-%s-content-sha256 for payload hash
Casey Bodley [Tue, 25 Oct 2022 22:46:58 +0000 (18:46 -0400)] 
aws_sigv4: consult x-%s-content-sha256 for payload hash

`Curl_output_aws_sigv4()` doesn't always have the whole payload in
memory to generate a real payload hash. this commit allows the user to
pass in a header like `x-amz-content-sha256` to provide their desired
payload hash

some services like s3 require this header, and may support other values
like s3's `UNSIGNED-PAYLOAD` and `STREAMING-AWS4-HMAC-SHA256-PAYLOAD`
with special semantics. servers use this header's value as the payload
hash during signature validation, so it must match what the client uses
to generate the signature

CURLOPT_AWS_SIGV4.3 now describes the content-sha256 interaction

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Closes #9804

2 years agoGHA: NSS use clang instead of clang-9
Philip H [Thu, 24 Nov 2022 09:32:34 +0000 (10:32 +0100)] 
GHA: NSS use clang instead of clang-9

Closes #9978

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 25 Nov 2022 07:40:19 +0000 (08:40 +0100)] 
RELEASE-NOTES: synced

2 years agotool_operate: override the numeric locale and set "C" by force
Daniel Stenberg [Wed, 23 Nov 2022 10:24:26 +0000 (11:24 +0100)] 
tool_operate: override the numeric locale and set "C" by force

Makes curl always use dot as decimal separator for options,
independently of what the locale says. Makes scripts and command lines
portable.

Updated docs accordingly.

Reported-by: Daniel Faust
Fixes #9969
Closes #9972

2 years agotest1662: verify formpost, 301 redirect, no rewind possible
Daniel Stenberg [Tue, 22 Nov 2022 07:25:50 +0000 (08:25 +0100)] 
test1662: verify formpost, 301 redirect, no rewind possible

Reproduces #9735 and verifies the subsequent fix. The original issue
uses a pipe that cannot be rewound, but this test case instead sets a
callback without rewind ability to get roughly the same properties but
being a much more portable test.

2 years agolib: rewind BEFORE request instead of AFTER previous
Daniel Stenberg [Tue, 22 Nov 2022 07:25:50 +0000 (08:25 +0100)] 
lib: rewind BEFORE request instead of AFTER previous

This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.

Reported-by: Ali Utku Selen
Fixes #9735
Closes #9958

2 years agovtls: repair build with disabled proxy
Daniel Stenberg [Wed, 23 Nov 2022 15:11:25 +0000 (16:11 +0100)] 
vtls: repair build with disabled proxy

Closes #9974

2 years agopackaging: remove traces of deleted files
Daniel Gustafsson [Wed, 23 Nov 2022 11:12:59 +0000 (12:12 +0100)] 
packaging: remove traces of deleted files

Commit a8861b6cc removed packages/DOS but left a few traces of it
which broke the distcheck CI. Remove all traces.

Closes: #9971
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2 years agoopenssl: silence compiler warning when not using IPv6
Daniel Gustafsson [Wed, 23 Nov 2022 11:10:30 +0000 (12:10 +0100)] 
openssl: silence compiler warning when not using IPv6

In non-IPv6 builds the conn parameter is unused, and compilers which
run with "-Werror=unused-parameter" (or similar) warnings turned on
fails to build. Below is an excerpt from a CI job:

  vtls/openssl.c: In function ‘Curl_ossl_verifyhost’:
  vtls/openssl.c:2016:75: error: unused parameter ‘conn’ [-Werror=unused-parameter]
   2016 | CURLcode Curl_ossl_verifyhost(struct Curl_easy *data, struct connectdata *conn,
        |                                                       ~~~~~~~~~~~~~~~~~~~~^~~~

Closes: #9970
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2 years agonetware: remove leftover traces
Daniel Gustafsson [Wed, 23 Nov 2022 10:56:52 +0000 (11:56 +0100)] 
netware: remove leftover traces

Commit 3b16575ae938dec2a29454631a12aa52b6ab9c67 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.

Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2 years agocurl_endian: remove Curl_write64_le from header
Ryan Schmidt [Tue, 22 Nov 2022 23:59:14 +0000 (17:59 -0600)] 
curl_endian: remove Curl_write64_le from header

The actual function was already removed in 4331c6dc.

See #7280
Closes #9968

2 years agodocs: add more "SEE ALSO" links to CA related pages
Daniel Stenberg [Tue, 22 Nov 2022 09:38:16 +0000 (10:38 +0100)] 
docs: add more "SEE ALSO" links to CA related pages

Closes #9959

2 years agoexamples: update descriptions
Daniel Stenberg [Tue, 22 Nov 2022 09:57:55 +0000 (10:57 +0100)] 
examples: update descriptions

Make them not say "this is an example showing..." and instead just say
what the example shows.

Closes #9960

2 years agovtls: localization of state data in filters
Stefan Eissing [Tue, 22 Nov 2022 08:55:41 +0000 (09:55 +0100)] 
vtls: localization of state data in filters

 - almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919

2 years agoexamples/10-at-a-time: fix possible skipped final transfers
Daniel Stenberg [Tue, 22 Nov 2022 08:32:42 +0000 (03:32 -0500)] 
examples/10-at-a-time: fix possible skipped final transfers

Prior to this change if curl_multi_perform returned 0 running handles
and then all remaining transfers were added, then the perform loop would
end immediately without performing those transfers.

Reported-by: Mikhail Kuznetsov
Fixes https://github.com/curl/curl/issues/9953
Closes https://github.com/curl/curl/pull/9954

2 years agoMakefile.mk: portable Makefile.m32
Viktor Szakats [Tue, 22 Nov 2022 08:25:05 +0000 (08:25 +0000)] 
Makefile.mk: portable Makefile.m32

Update bare GNU Make `Makefile.m32` to:

- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
  name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
  defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
  options. We're also not retaining all build details from the existing
  Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
  Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
  with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
  instead of bare `ZLIB_PATH`.

Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.

Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Closes #9764

2 years agocfiler: filter types have flags indicating what they do
Stefan Eissing [Mon, 21 Nov 2022 14:40:26 +0000 (15:40 +0100)] 
cfiler: filter types have flags indicating what they do

- Adding Curl_conn_is_ip_connected() to check if network connectivity
  has been reached

- having ftp wait for network connectivity before proceeding with
  transfers.

Fixes test failures 1631 and 1632 with hyper.

Closes #9952

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Mon, 21 Nov 2022 12:39:01 +0000 (13:39 +0100)] 
RELEASE-NOTES: synced

2 years agosendf: change Curl_read_plain to wrap Curl_recv_plain (take 2)
Jay Satiro [Mon, 14 Nov 2022 08:30:30 +0000 (03:30 -0500)] 
sendf: change Curl_read_plain to wrap Curl_recv_plain (take 2)

Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.

Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.

Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain

This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).

The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.

--

This is take 2 of the original fix. It preserves the original behavior
of Curl_read_plain to write 0 to the bytes read parameter on error,
since apparently some callers expect that (SOCKS tests were hanging).
The original fix which landed in 12e1def5 and was later reverted in
18383fbf failed to work properly because it did not do that.

Also, it changes Curl_write_plain the same way to complement
Curl_read_plain, and it changes Curl_send_plain to return -1 instead of
0 on CURLE_AGAIN to complement Curl_recv_plain.

Behavior on error with these changes:

Curl_recv_plain returns -1 and *code receives error code.
Curl_send_plain returns -1 and *code receives error code.
Curl_read_plain returns error code and *n (bytes read) receives 0.
Curl_write_plain returns error code and *written receives 0.

--

Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361

Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9949

2 years agohyper: classify headers as CONNECT and 1XX
Sean McArthur [Fri, 18 Nov 2022 20:29:44 +0000 (12:29 -0800)] 
hyper: classify headers as CONNECT and 1XX

Closes #9947

2 years agoftp: fix "AUTH TLS" on primary conn and for SSL in PASV second conn
Stefan Eissing [Fri, 18 Nov 2022 20:33:37 +0000 (21:33 +0100)] 
ftp: fix "AUTH TLS" on primary conn and for SSL in PASV second conn

Follow-up to dafdb20a26d0c89

Reported-by: Anthony Hu
Closes #9948

2 years agoCURLOPT_POST.3: Explain setting to 0 changes request type
Jay Satiro [Fri, 18 Nov 2022 08:54:16 +0000 (03:54 -0500)] 
CURLOPT_POST.3: Explain setting to 0 changes request type

Bug: https://github.com/curl/curl/issues/9849
Reported-by: MonkeybreadSoftware@users.noreply.github.com
Closes https://github.com/curl/curl/pull/9942

2 years agodocs/INSTALL.md: expand on static builds
Daniel Stenberg [Fri, 18 Nov 2022 10:25:24 +0000 (11:25 +0100)] 
docs/INSTALL.md: expand on static builds

Remove from KNOWN_BUGS

Closes #9944

2 years agohttp: restore h3 to working condition after connection filter introduction
Stefan Eissing [Fri, 18 Nov 2022 10:40:16 +0000 (11:40 +0100)] 
http: restore h3 to working condition after connection filter introduction

Follow-up to dafdb20a26d0c

HTTP/3 needs a special filter chain, since it does the TLS handling
itself. This PR adds special setup handling in the HTTP protocol handler
that takes are of it.

When a handler, in its setup method, installs filters, the default
behaviour for managing the filter chain is overridden.

Reported-by: Karthikdasari0423 on github
Fixes #9931
Closes #9945

2 years agourldata: change port num storage to int and unsigned short
Daniel Stenberg [Fri, 18 Nov 2022 12:17:06 +0000 (13:17 +0100)] 
urldata: change port num storage to int and unsigned short

Instead of long.

Closes #9946

2 years agoRevert "sendf: change Curl_read_plain to wrap Curl_recv_plain"
Daniel Stenberg [Fri, 18 Nov 2022 09:30:24 +0000 (10:30 +0100)] 
Revert "sendf: change Curl_read_plain to wrap Curl_recv_plain"

This reverts commit 12e1def51a75392df62e65490416007d7e68dab9.

It introduced SOCKS proxy fails, like test 700 never ending.

Reopens #9431

2 years agoHTTP-COOKIES.md: update the 6265bis link to draft-11
Daniel Stenberg [Fri, 18 Nov 2022 07:55:05 +0000 (08:55 +0100)] 
HTTP-COOKIES.md: update the 6265bis link to draft-11

Closes #9940

2 years agodocs/WEBSOCKET.md: explain the URL use
Daniel Stenberg [Fri, 18 Nov 2022 08:39:26 +0000 (09:39 +0100)] 
docs/WEBSOCKET.md: explain the URL use

Fixes #9936
Closes #9941

2 years agosendf: change Curl_read_plain to wrap Curl_recv_plain
Jay Satiro [Mon, 14 Nov 2022 08:30:30 +0000 (03:30 -0500)] 
sendf: change Curl_read_plain to wrap Curl_recv_plain

Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.

Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.

Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain

This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).

The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.

Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361

Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9904

2 years agotest3026: reduce runtime in legacy mingw builds
Jay Satiro [Mon, 14 Nov 2022 09:07:30 +0000 (04:07 -0500)] 
test3026: reduce runtime in legacy mingw builds

- Load Windows system libraries secur32 and iphlpapi beforehand, so
  that libcurl's repeated global init/cleanup only increases/decreases
  the library's refcount rather than causing it to load/unload.

Assisted-by: Marc Hoersken
Closes https://github.com/curl/curl/pull/9412

2 years agourl: move back the IDN conversion of proxy names
Daniel Stenberg [Thu, 17 Nov 2022 22:55:26 +0000 (23:55 +0100)] 
url: move back the IDN conversion of proxy names

Regression: in commit 53bcf55 we moved the IDN conversion calls to
happen before the HSTS checks. But the HSTS checks are only done on the
server host name, not the proxy names. By moving the proxy name IDN
conversions, we accidentally broke the verbose output showing the proxy
name.

This change moves back the IDN conversions for the proxy names to the
place in the code path they were before 53bcf55.

Reported-by: Andy Stamp
Fixes #9937
Closes #9939

2 years agoCURLOPT_QUICK_EXIT: don't wait for DNS thread on exit
Alexandre Ferrieux [Tue, 12 Jul 2022 21:40:05 +0000 (23:40 +0200)] 
CURLOPT_QUICK_EXIT: don't wait for DNS thread on exit

Fixes #2975
Closes #9147

2 years agoHTTP-COOKIES.md: mention that http://localhost is a secure context
Daniel Stenberg [Thu, 17 Nov 2022 18:08:56 +0000 (19:08 +0100)] 
HTTP-COOKIES.md: mention that http://localhost is a secure context

Reported-by: Trail of Bits
Closes #9938

2 years agolib: parse numbers with fixed known base 10
Daniel Stenberg [Thu, 17 Nov 2022 07:41:44 +0000 (08:41 +0100)] 
lib: parse numbers with fixed known base 10

... instead of using 0 argument that allows decimal, hex or octal when
the number is documented and assumed to use base 10.

Closes #9933

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 17 Nov 2022 13:41:20 +0000 (14:41 +0100)] 
RELEASE-NOTES: synced

2 years agoscripts/delta: adapt to curl.h changes for the opt counter
Daniel Stenberg [Thu, 17 Nov 2022 13:41:04 +0000 (14:41 +0100)] 
scripts/delta: adapt to curl.h changes for the opt counter

2 years agocookie: expire cookies at once when max-age is negative
Daniel Stenberg [Thu, 17 Nov 2022 07:26:17 +0000 (08:26 +0100)] 
cookie: expire cookies at once when max-age is negative

Update test 329 to verify

Reported-by: godmar on github
Fixes #9930
Closes #9932

2 years agoproxy: haproxy filter is only available when PROXY and HTTP are
Stefan Eissing [Thu, 17 Nov 2022 09:29:50 +0000 (10:29 +0100)] 
proxy: haproxy filter is only available when PROXY and HTTP are

Closes #9935

2 years agoOtherTests.cmake: check for cross-compile, not for toolchain
Daniel Stenberg [Wed, 16 Nov 2022 07:16:46 +0000 (08:16 +0100)] 
OtherTests.cmake: check for cross-compile, not for toolchain

Build systems like vcpkg alway sets `CMAKE_TOOLCHAIN_FILE` so it should
not be used as a sign that this is a cross-compile.

Also indented the function correctly.

Reported-by: Philip Chan
Fixes #9921
Closes #9923

2 years agontlm: improve comment for encrypt_des
Daniel Stenberg [Wed, 16 Nov 2022 08:11:17 +0000 (09:11 +0100)] 
ntlm: improve comment for encrypt_des

Reported-by: Andrei Rybak
Fixes #9903
Closes #9925

2 years agoinclude/curl/curl.h: bump the deprecated requirements to gcc 5.3
Daniel Stenberg [Tue, 15 Nov 2022 14:02:52 +0000 (15:02 +0100)] 
include/curl/curl.h: bump the deprecated requirements to gcc 5.3

Reported-by: Stephan Guilloux
Fixes #9917
Closes #9918

2 years agoproxy: refactor haproxy protocol handling as connection filter
Stefan Eissing [Mon, 14 Nov 2022 15:44:12 +0000 (16:44 +0100)] 
proxy: refactor haproxy protocol handling as connection filter

Closes #9893

2 years agolib: feature deprecation warnings in gcc >= 4.3
Patrick Monnerat [Mon, 14 Nov 2022 14:21:34 +0000 (15:21 +0100)] 
lib: feature deprecation warnings in gcc >= 4.3

Add a deprecated attribute to functions and enum values that should not
be used anymore.
This uses a gcc 4.3 dialect, thus is only available for this version of
gcc and newer. Note that the _Pragma() keyword is introduced by C99, but
is available as part of the gcc dialect even when compiling in C89 mode.

It is still possible to disable deprecation at a calling module compile
time by defining CURL_DISABLE_DEPRECATION.

Gcc type checking macros are made aware of possible deprecations.

Some testing support Perl programs are adapted to the extended
declaration syntax.

Several test and unit test C programs intentionally use deprecated
functions/options and are annotated to not generate a warning.

New test 1222 checks the deprecation status in doc and header files.

Closes #9667

2 years agolog2changes.pl: wrap long lines at 80 columns
Daniel Stenberg [Fri, 11 Nov 2022 23:07:34 +0000 (00:07 +0100)] 
log2changes.pl: wrap long lines at 80 columns

Also, only use author names in the output.

Fixes #9896
Reported-by: John Sherrill
Closes #9897

2 years agocfilters: use %zu for outputting size_t
Daniel Stenberg [Mon, 14 Nov 2022 10:18:22 +0000 (11:18 +0100)] 
cfilters: use %zu for outputting size_t

Detected by Coverity CID 1516894

Closes #9907

2 years agoCurl_closesocket: avoid using 'conn' if NULL
Daniel Stenberg [Mon, 14 Nov 2022 10:14:45 +0000 (11:14 +0100)] 
Curl_closesocket: avoid using 'conn' if NULL

... in debug-only code.

Reported by Coverity CID 1516896

Closes #9907

2 years agourl: only acknowledge fresh_reuse for non-followed transfers
Daniel Stenberg [Mon, 14 Nov 2022 09:29:54 +0000 (10:29 +0100)] 
url: only acknowledge fresh_reuse for non-followed transfers

... to make sure NTLM auth sticks to the connection it needs, as
verified by 2032.

Follow-up to fa0b9227616e

Assisted-by: Stefan Eissing
Closes #9905

2 years agonetrc.d: provide mutext info
Daniel Stenberg [Sun, 13 Nov 2022 22:14:38 +0000 (23:14 +0100)] 
netrc.d: provide mutext info

Reported-by: xianghongai on github
Fixes #9899
Closes #9901

2 years agocmdline-opts/page-footer: remove long option nroff formatting
Daniel Stenberg [Mon, 14 Nov 2022 07:38:50 +0000 (08:38 +0100)] 
cmdline-opts/page-footer: remove long option nroff formatting

As gen.pl adds them

2 years agonroff-scan.pl: detect double highlights
Daniel Stenberg [Mon, 14 Nov 2022 07:38:34 +0000 (08:38 +0100)] 
nroff-scan.pl: detect double highlights

2 years agocmdline-opts/gen.pl: fix the linkifier
Daniel Stenberg [Sun, 13 Nov 2022 22:58:47 +0000 (23:58 +0100)] 
cmdline-opts/gen.pl: fix the linkifier

Improved logic for finding existing --options in text and replacing with
the full version with nroff syntax. This also makes the web version link
options better.

Reported-by: xianghongai on github
Fixes #9899
Closes #9902

2 years agotool: use feature names instead of bit mask, when possible
Patrick Monnerat [Tue, 8 Nov 2022 23:37:49 +0000 (00:37 +0100)] 
tool: use feature names instead of bit mask, when possible

If the run-time libcurl is too old to support feature names, the name
array is created locally from the bit masks. This is the only sequence
left that uses feature bit masks.

Closes #9583

2 years agodocs: curl_version_info is not thread-safe before libcurl initialization
Patrick Monnerat [Sun, 25 Sep 2022 15:30:23 +0000 (17:30 +0200)] 
docs: curl_version_info is not thread-safe before libcurl initialization

Closes #9583

2 years agoversion: add a feature names array to curl_version_info_data
Patrick Monnerat [Tue, 8 Nov 2022 23:27:32 +0000 (00:27 +0100)] 
version: add a feature names array to curl_version_info_data

Field feature_names contains a null-terminated sorted array of feature
names. Bitmask field features is deprecated.

Documentation is updated. Test 1177 and tests/version-scan.pl updated to
match new documentation format and extended to check feature names too.

Closes #9583

2 years agonegtelnetserver.py: have it call its close() method
Stefan Eissing [Fri, 11 Nov 2022 15:39:06 +0000 (16:39 +0100)] 
negtelnetserver.py: have it call its close() method

Closes #9894

2 years agontlm: silence ubsan warning about copying from null target_info pointer.
Nathan Moinvaziri [Sat, 12 Nov 2022 00:42:18 +0000 (16:42 -0800)] 
ntlm: silence ubsan warning about copying from null target_info pointer.

runtime error: null pointer passed as argument 2, which is declared to
never be null

Closes #9898

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 11 Nov 2022 23:54:35 +0000 (00:54 +0100)] 
RELEASE-NOTES: synced

2 years agoWebsocket: fixes for partial frames and buffer updates.
Stefan Eissing [Fri, 11 Nov 2022 14:15:37 +0000 (15:15 +0100)] 
Websocket: fixes for partial frames and buffer updates.

- buffers updated correctly when handling partial frames
- callbacks no longer invoked for incomplete payload data of 0 length
- curl_ws_recv no longer returns with 0 length partial payload

Closes #9890

2 years agotool_operate: provide better errmsg for -G with bad URL
Daniel Stenberg [Fri, 11 Nov 2022 14:46:17 +0000 (15:46 +0100)] 
tool_operate: provide better errmsg for -G with bad URL

If the URL that -G would try to add a query to could not be parsed, it would
display

 curl: (27) Out of memory

It now instead shows:

 curl: (2) Could not parse the URL, failed to set query

Reported-by: Alex Xu
Fixes #9889
Closes #9892

2 years agovtls: fix build without proxy support
Daniel Stenberg [Fri, 11 Nov 2022 16:45:08 +0000 (17:45 +0100)] 
vtls: fix build without proxy support

Follow-up to dafdb20a26d0c890

Closes #9895

2 years agotool_getparam: make --no-get work as the opposite of --get
Daniel Stenberg [Fri, 11 Nov 2022 14:37:36 +0000 (15:37 +0100)] 
tool_getparam: make --no-get work as the opposite of --get

... as documented.

Closes #9891

2 years agohttp: mark it 'this_is_a_follow' in the Location: logic
Daniel Stenberg [Fri, 11 Nov 2022 07:47:12 +0000 (08:47 +0100)] 
http: mark it 'this_is_a_follow' in the Location: logic

To make regular auth "reloads" to not count as redirects.

Verified by test 3101

Fixes #9885
Closes #9887

2 years agoconfig-win32: fix SIZEOF_OFF_T for MSVC and old MinGW
Viktor Szakats [Fri, 11 Nov 2022 20:12:15 +0000 (20:12 +0000)] 
config-win32: fix SIZEOF_OFF_T for MSVC and old MinGW

The previously set default value of 8 (64-bit) is only correct for
mingw-w64 and only when we set `_FILE_OFFSET_BITS` to 64 (the default
when building curl). For MSVC, old MinGW and other Windows compilers,
the correct value is 4 (32-bit). Adjust condition accordingly. Also
drop the manual override option.

Regression in 7.86.0 (from 68fa9bf3f5d7b4fcbb57619f70cb4aabb79a51f6)

Bug: https://github.com/curl/curl/pull/9712#issuecomment-1307330551

Reported-by: Peter Piekarski
Reviewed-by: Jay Satiro
Closes #9872

2 years agolib: remove bad set.opt_no_body assignments
Daniel Stenberg [Fri, 11 Nov 2022 09:57:04 +0000 (10:57 +0100)] 
lib: remove bad set.opt_no_body assignments

This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888

2 years agolib: connection filters (cfilter) addition to curl:
Stefan Eissing [Fri, 11 Nov 2022 10:45:34 +0000 (11:45 +0100)] 
lib: connection filters (cfilter) addition to curl:

 - general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855

2 years agocurl-rustls.m4: on macOS, rustls also needs the Security framework
Stefan Eissing [Thu, 10 Nov 2022 15:30:16 +0000 (16:30 +0100)] 
curl-rustls.m4: on macOS, rustls also needs the Security framework

Closes #9883

2 years agortsp: only store first_host once
Daniel Stenberg [Thu, 10 Nov 2022 07:38:01 +0000 (08:38 +0100)] 
rtsp: only store first_host once

Suggested-by: Erik Janssen
URL: https://github.com/curl/curl/pull/9870#issuecomment-1309499744
Closes #9882

2 years agotest3028: verify PROXY
Fata Nugraha [Wed, 9 Nov 2022 11:11:05 +0000 (18:11 +0700)] 
test3028: verify PROXY

2 years agohttp: do not send PROXY more than once
Fata Nugraha [Wed, 9 Nov 2022 09:38:00 +0000 (16:38 +0700)] 
http: do not send PROXY more than once

Unlike `CONNECT`, currently we don't keep track whether `PROXY` is
already sent or not. This causes `PROXY` header to be sent twice during
`MSTATE_TUNNELING` and `MSTATE_PROTOCONNECT`.

Closes #9878
Fixes #9442

2 years agolib: add CURL_WRITEFUNC_ERROR to signal write callback error
Jay Satiro [Tue, 8 Nov 2022 23:49:21 +0000 (18:49 -0500)] 
lib: add CURL_WRITEFUNC_ERROR to signal write callback error

Prior to this change if the user wanted to signal an error from their
write callbacks they would have to use logic to return a value different
from the number of bytes (nmemb) passed to the callback. Also, the
inclination of some users has been to just return 0 to signal error,
which is incorrect as that may be the number of bytes passed to the
callback.

To remedy this the user can now return CURL_WRITEFUNC_ERROR instead.

Ref: https://github.com/curl/curl/issues/9873

Closes https://github.com/curl/curl/pull/9874

2 years agoRevert "GHA: add scorecard.yml"
Daniel Stenberg [Wed, 9 Nov 2022 09:29:19 +0000 (10:29 +0100)] 
Revert "GHA: add scorecard.yml"

This reverts commit ca76c79b34f9d90105674a2151bf228ff7b13bef.

2 years agoGHA: add scorecard.yml
Daniel Stenberg [Wed, 9 Nov 2022 09:10:33 +0000 (10:10 +0100)] 
GHA: add scorecard.yml

add a "scorecard" scanner job

2 years agotest3100: RTSP Basic authentication
Lorenzo Miniero [Wed, 7 Sep 2022 14:02:57 +0000 (16:02 +0200)] 
test3100: RTSP Basic authentication

Closes #9449

2 years agortsp: fix RTSP auth
Daniel Stenberg [Tue, 8 Nov 2022 14:34:12 +0000 (15:34 +0100)] 
rtsp: fix RTSP auth

Verified with test 3100

Fixes #4750
Closes #9870

2 years agoKNOWN_BUGS: remove eight entries
Daniel Stenberg [Thu, 27 Oct 2022 08:21:27 +0000 (10:21 +0200)] 
KNOWN_BUGS: remove eight entries

- 1.2 Multiple methods in a single WWW-Authenticate: header

This is not considered a bug anymore but a restriction and one that we
keep because we have NEVER gotten this reported by users in the wild and
because of this I consider this a fringe edge case we don't need to
support.

- 1.6 Unnecessary close when 401 received waiting for 100

This is not a bug, but possibly an optimization that *can* be done.

- 1.7 Deflate error after all content was received

This is not a curl bug. This happens due to broken servers.

- 2.1 CURLINFO_SSL_VERIFYRESULT has limited support

This is not a bug. This is just the nature of the implementation.

- 2.2 DER in keychain

This is not a bug.

- 5.7 Visual Studio project gaps

This is not a bug.

- 15.14 cmake build is not thread-safe

Fixed in 109e9730ee5e2b

- 11.3 Disconnects do not do verbose

This is not a bug.

Closes #9871

2 years agoheaders: add endif comments
Hirotaka Tagawa [Sat, 5 Nov 2022 08:03:16 +0000 (17:03 +0900)] 
headers: add endif comments

Closes #9853

2 years agotest1221: verify --url-query
Daniel Stenberg [Fri, 4 Nov 2022 23:10:25 +0000 (00:10 +0100)] 
test1221: verify --url-query

2 years agocurl: add --url-query
Daniel Stenberg [Fri, 4 Nov 2022 23:10:24 +0000 (00:10 +0100)] 
curl: add --url-query

This option adds a piece of data, usually a name + value pair, to the
end of the URL query part. The syntax is identical to that used for
--data-urlencode with one extension:

If the argument starts with a '+' (plus), the rest of the string is
provided as-is unencoded.

This allows users to "build" query parts with options and URL encoding
even when not doing GET requests, which the already provided option -G
(--get) is limited to.

This idea was born in a Twitter thread.

Closes #9691

2 years agomaketgz: set the right version in lib/libcurl.plist
Daniel Stenberg [Tue, 8 Nov 2022 07:53:05 +0000 (08:53 +0100)] 
maketgz: set the right version in lib/libcurl.plist

Follow-up to e498a9b1fe5964a18eb2a3a99dc52

Make sure the tarball gets a version of the libcurl.plist file that is
updated with the new version string.

Reported-by: jvreelanda on github
Fixes #9866
Closes #9867

2 years agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 8 Nov 2022 09:29:17 +0000 (10:29 +0100)] 
RELEASE-NOTES: synced

Bumped version to 7.87.0

2 years agocurl.h: add CURLOPT_CA_CACHE_TIMEOUT option
Michael Drake [Wed, 12 Oct 2022 11:12:08 +0000 (12:12 +0100)] 
curl.h: add CURLOPT_CA_CACHE_TIMEOUT option

Adds a new option to control the maximum time that a cached
certificate store may be retained for.

Currently only the OpenSSL backend implements support for
caching certificate stores.

Closes #9620

2 years agoopenssl: reduce CA certificate bundle reparsing by caching
Michael Drake [Wed, 28 Sep 2022 14:51:44 +0000 (15:51 +0100)] 
openssl: reduce CA certificate bundle reparsing by caching

Closes #9620

2 years agolib: fix some type mismatches and remove unneeded typecasts
Rose [Fri, 28 Oct 2022 16:32:09 +0000 (12:32 -0400)] 
lib: fix some type mismatches and remove unneeded typecasts

Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.com/curl/curl/pull/9823

Closes https://github.com/curl/curl/pull/9835

2 years agocookie: compare cookie prefixes case insensitively
Daniel Stenberg [Mon, 7 Nov 2022 16:10:56 +0000 (17:10 +0100)] 
cookie: compare cookie prefixes case insensitively

Adapted to language in rfc6265bis draft-11.

Closes #9863

Reviewed-by: Daniel Gustafsson
2 years agotool_operate: when aborting, make sure there is a non-NULL error buffer
Daniel Stenberg [Mon, 7 Nov 2022 17:01:24 +0000 (18:01 +0100)] 
tool_operate: when aborting, make sure there is a non-NULL error buffer

To store custom errors in. Or SIGSEGVs will follow.

Reported-by: Trail of Bits
Closes #9865

2 years agoWEBSOCKET.md: fix broken link
Daniel Stenberg [Mon, 7 Nov 2022 16:23:31 +0000 (17:23 +0100)] 
WEBSOCKET.md: fix broken link

Reported-by: Felipe Gasper
Bug: https://curl.se/mail/lib-2022-10/0097.html
Closes #9864

2 years agoCURLOPT_DEBUGFUNCTION.3: do not assume nul-termination in example
Daniel Stenberg [Mon, 7 Nov 2022 11:15:24 +0000 (12:15 +0100)] 
CURLOPT_DEBUGFUNCTION.3: do not assume nul-termination in example

Reported-by: Oskar Sigvardsson
Bug: https://curl.se/mail/lib-2022-11/0016.html

Closes #9862

2 years agowebsockets: fix handling of partial frames
Stefan Eissing [Mon, 7 Nov 2022 09:11:17 +0000 (10:11 +0100)] 
websockets: fix handling of partial frames

buffer used and send length calculations are fixed when a partial
websocket frame has been received.

Closes #9861

2 years agomailmap: unify Stefan Eissing
Daniel Stenberg [Mon, 7 Nov 2022 11:24:07 +0000 (12:24 +0100)] 
mailmap: unify Stefan Eissing

2 years agohyper: fix handling of hyper_task's when reusing the same address
Stefan Eissing [Mon, 7 Nov 2022 08:47:10 +0000 (09:47 +0100)] 
hyper: fix handling of hyper_task's when reusing the same address

Fixes #9840
Closes #9860