]> git.ipfire.org Git - thirdparty/curl.git/log
thirdparty/curl.git
19 months agogithub/labeler: switch from the beta to labeler v5 12458/head
Dan Fandrich [Tue, 5 Dec 2023 21:17:09 +0000 (13:17 -0800)] 
github/labeler: switch from the beta to labeler v5

Some keys were renamed and the dot option was made default.

Closes #12458

19 months agoDEPRECATE: remove NTLM_WB in June 2024
Daniel Stenberg [Tue, 5 Dec 2023 07:02:57 +0000 (08:02 +0100)] 
DEPRECATE: remove NTLM_WB in June 2024

Ref: https://curl.se/mail/lib-2023-12/0010.html

Closes #12451

19 months agorustls: implement connect_blocking
Jacob Hoffman-Andrews [Fri, 23 Jun 2023 23:58:54 +0000 (16:58 -0700)] 
rustls: implement connect_blocking

Closes #11647

19 months agoexamples/rtsp-options.c: add
Daniel Stenberg [Mon, 4 Dec 2023 16:14:56 +0000 (17:14 +0100)] 
examples/rtsp-options.c: add

Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and
CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS.

Closes #12452

20 months agongtcp2: ignore errors on unknown streams
Stefan Eissing [Mon, 4 Dec 2023 11:13:25 +0000 (12:13 +0100)] 
ngtcp2: ignore errors on unknown streams

- expecially in is_alive checks on connections, we might
  see incoming packets on streams already forgotten and closed,
  leading to errors reported by nghttp3. Ignore those.

Closes #12449

20 months agodocs: make all examples in all libcurl man pages compile
Daniel Stenberg [Mon, 4 Dec 2023 09:50:42 +0000 (10:50 +0100)] 
docs: make all examples in all libcurl man pages compile

Closes #12448

20 months agochecksrc.pl: support #line instructions
Daniel Stenberg [Mon, 4 Dec 2023 09:50:42 +0000 (10:50 +0100)] 
checksrc.pl: support #line instructions

makes it identify the correct source file and line

20 months agoGHA/man-examples: verify libcurl man page examples
Daniel Stenberg [Mon, 4 Dec 2023 09:50:42 +0000 (10:50 +0100)] 
GHA/man-examples: verify libcurl man page examples

20 months agoverify-examples.pl: verify that all man page examples compile clean
Daniel Stenberg [Mon, 4 Dec 2023 09:50:42 +0000 (10:50 +0100)] 
verify-examples.pl: verify that all man page examples compile clean

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Sat, 2 Dec 2023 22:21:21 +0000 (23:21 +0100)] 
RELEASE-NOTES: synced

20 months agohttp3: bump ngtcp2 and nghttp3 versions
Graham Campbell [Sat, 2 Dec 2023 18:17:35 +0000 (18:17 +0000)] 
http3: bump ngtcp2 and nghttp3 versions

nghttp3 v1.1.0
ngtcp2 v1.1.0

In docs and CI

Closes #12446

20 months agoCI/quiche: use `3.1.4+quic` consistently in CI workflows
Graham Campbell [Sat, 2 Dec 2023 18:24:19 +0000 (18:24 +0000)] 
CI/quiche: use `3.1.4+quic` consistently in CI workflows

Closes #12447

20 months agotest1545: disable deprecation warnings
Viktor Szakats [Sat, 2 Dec 2023 15:59:21 +0000 (15:59 +0000)] 
test1545: disable deprecation warnings

Fixes:
https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yrp6pk#L1205

Same with details:
https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmilb6wt#L1263
```
tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56.0. Use curl_mime_init() [-Werror=deprecated-declarations]
   38 |   curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file",
      |   ^~~~~~~~~~~~
[...]
```

Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421

Fixes #12445
Closes #12444

20 months agoINSTALL: update list of ports and CPU archs
Daniel Stenberg [Sat, 2 Dec 2023 16:40:48 +0000 (17:40 +0100)] 
INSTALL: update list of ports and CPU archs

20 months agosymbols-in-versions: the CLOSEPOLICY options are deprecated
Daniel Stenberg [Sat, 2 Dec 2023 16:07:34 +0000 (17:07 +0100)] 
symbols-in-versions: the CLOSEPOLICY options are deprecated

The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked.

20 months agobuild: fix builds that disable protocols but not digest auth
z2_ [Sat, 2 Dec 2023 00:09:29 +0000 (01:09 +0100)] 
build: fix builds that disable protocols but not digest auth

- Build base64 functions if digest auth is not disabled.

Prior to this change if some protocols were disabled but not digest auth
then a build error would occur due to missing base64 functions.

Fixes https://github.com/curl/curl/issues/12440
Closes https://github.com/curl/curl/pull/12442

20 months agoconnect: reduce number of transportation providers
MAntoniak [Fri, 1 Dec 2023 17:48:09 +0000 (18:48 +0100)] 
connect: reduce number of transportation providers

Use only the ones necessary - the ones that are built-in. Saves a few
bytes in the resulting code.

Closes #12438

20 months agovtls: consistently use typedef names for OpenSSL structs
David Benjamin [Fri, 1 Dec 2023 19:40:32 +0000 (14:40 -0500)] 
vtls: consistently use typedef names for OpenSSL structs

The foo_st names don't appear in OpenSSL public API documentation. The
FOO typedefs are more common. This header was already referencing
SSL_CTX via <openssl/ssl.h>. There is a comment about avoiding
<openssl/x509v3.h>, but OpenSSL actually declares all the typedefs in
<openssl/ossl_typ.h>, which is already included by <openssl/ssl.h> (and
every other OpenSSL header), so just use that. Though I've included it
just to be explicit.

(I'm also fairly sure including <openssl/ssl.h> already triggers the
Schannel conflicts anyway. The comment was probably just out of date.)

Closes #12439

20 months agolibcurl-security.3: fix typo
Lau [Fri, 1 Dec 2023 14:44:03 +0000 (15:44 +0100)] 
libcurl-security.3: fix typo

Fixed minimal typo.

Closes #12437

20 months agongtcp2: fix races in stream handling
Stefan Eissing [Fri, 1 Dec 2023 13:00:15 +0000 (14:00 +0100)] 
ngtcp2: fix races in stream handling

- fix cases where ngtcp2 invokes callbacks on streams that
  nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND
  in these cases as it is normal behaviour.

Closes #12435

20 months agotool_writeout_json: fix JSON encoding of non-ascii bytes
Emanuele Torre [Fri, 1 Dec 2023 00:51:47 +0000 (01:51 +0100)] 
tool_writeout_json: fix JSON encoding of non-ascii bytes

char variables if unspecified can be either signed or unsigned depending
on the platform according to the C standard; in most platforms, they are
signed.

This meant that the  *i<32  waas always true for bytes with the top bit
set. So they were always getting encoded as \uXXXX, and then since they
were also signed negative, they were getting extended with 1s causing
'\xe2' to be expanded to \uffffffe2, for example:

  $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/null
  \uffffffe2\uffffff80\uffffff9c

I fixed this bug by making the code use explicitly unsigned char*
variables instead of char* variables.

Test 268 verifies

Reported-by: iconoclasthero
Closes #12434

20 months agocf-socket: TCP trace output local address used in connect
Stefan Eissing [Wed, 29 Nov 2023 14:17:30 +0000 (15:17 +0100)] 
cf-socket: TCP trace output local address used in connect

Closes #12427

20 months agoCURLINFO_PRETRANSFER_TIME_T.3: fix time explanation
Jay Satiro [Thu, 30 Nov 2023 20:18:57 +0000 (15:18 -0500)] 
CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation

- Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it
  includes protocol-specific instructions that trigger a transfer.

Prior to this change it explicitly said that it did not include those
instructions in the time, but that is incorrect.

The change is a copy of the fixed explanation already in
CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b.

Reported-by: eeverettrbx@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/12431
Closes https://github.com/curl/curl/pull/12432

20 months agomulti: during ratelimit multi_getsock should return no sockets
Daniel Stenberg [Thu, 30 Nov 2023 09:32:16 +0000 (10:32 +0100)] 
multi: during ratelimit multi_getsock should return no sockets

... as there is nothing to wait for then, it just waits. Otherwise, this
causes much more CPU work and updates than necessary during ratelimit
periods.

Ref: https://curl.se/mail/lib-2023-11/0056.html
Closes #12430

20 months agotransfer: abort pause send when connection is marked for closing
Dmitry Karpov [Wed, 29 Nov 2023 20:49:06 +0000 (12:49 -0800)] 
transfer: abort pause send when connection is marked for closing

This handles cases of some bi-directional "upgrade" scenarios
(i.e. WebSockets) where sending is paused until some "upgrade" handshake
is completed, but server rejects the handshake and closes the
connection.

Closes #12428

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 28 Nov 2023 22:04:09 +0000 (23:04 +0100)] 
RELEASE-NOTES: synced

20 months agoopenssl: when a session-ID is reused, skip OCSP stapling
Daniel Stenberg [Mon, 27 Nov 2023 22:50:45 +0000 (23:50 +0100)] 
openssl: when a session-ID is reused, skip OCSP stapling

Fixes #12399
Reported-by: Alexey Larikov
Closes #12418

20 months agotest1545: test doing curl_formadd twice with missing file
Daniel Stenberg [Tue, 28 Nov 2023 10:01:54 +0000 (11:01 +0100)] 
test1545: test doing curl_formadd twice with missing file

Reproduces #12410
Verifies the fix
Closes #12421

20 months agoCurl_http_body: cleanup properly when Curl_getformdata errors
Daniel Stenberg [Tue, 28 Nov 2023 07:49:49 +0000 (15:49 +0800)] 
Curl_http_body: cleanup properly when Curl_getformdata errors

Reported-by: yushicheng7788 on github
Based-on-work-by: yushicheng7788 on github
Fixes #12410
Closes #12421

20 months agotest1477: verify that libcurl-errors.3 and public headers are synced
Daniel Stenberg [Tue, 28 Nov 2023 16:07:44 +0000 (17:07 +0100)] 
test1477: verify that libcurl-errors.3 and public headers are synced

The script errorcodes.pl extracts all error codes from all headers and
checks that they are all documented, then checks that all documented
error codes are also specified in a header file.

Closes #12424

20 months agolibcurl-errors.3: sync with current public headers
Daniel Stenberg [Tue, 28 Nov 2023 16:07:27 +0000 (17:07 +0100)] 
libcurl-errors.3: sync with current public headers

Closes #12424

20 months agotest459: fix for parallel runs
Stefan Eissing [Tue, 28 Nov 2023 11:53:46 +0000 (12:53 +0100)] 
test459: fix for parallel runs

- change warniing message to work better with varying filename
  length.
- adapt test output check to new formatting

Follow-up to 97ccc4479f77ba3191c6
Closes #12423

20 months agotool_cb_prg: make the carriage return fit for wide progress bars
Daniel Stenberg [Mon, 27 Nov 2023 14:31:00 +0000 (15:31 +0100)] 
tool_cb_prg: make the carriage return fit for wide progress bars

When the progress bar was made max width (256 columns), the fly()
function attempted to generate its output buffer too long so that the
trailing carriage return would not fit and then the output would show
wrongly. The fly function is called when the expected total transfer is
unknown, which could be one or more progress calls before the actual
progress meter get shown when the expected transfer size is provided.

This new take also replaces the msnprintf() call with a much simpler
memset() for speed.

Reported-by: Tim Hill
Fixes #12407
Closes #12415

20 months agotool_parsecfg: make warning output propose double-quoting
Daniel Stenberg [Mon, 27 Nov 2023 10:36:43 +0000 (11:36 +0100)] 
tool_parsecfg: make warning output propose double-quoting

When the config file parser detects a word that *probably* should be
quoted, mention double-quotes as a possible remedy.

Test 459 verifies.

Proposed-by: Jiehong on github
Fixes #12409
Closes #12412

20 months agocurl.rc: switch out the copyright symbol for plain ASCII
Jay Satiro [Sat, 25 Nov 2023 19:21:20 +0000 (14:21 -0500)] 
curl.rc: switch out the copyright symbol for plain ASCII

.. like we already do for libcurl.rc.

libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint"
warning so it was switched to ascii.

Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972

Suggested-by: Robert Southee
Closes https://github.com/curl/curl/pull/12403

20 months agoconncache: use the closure handle when disconnecting surplus connections
Daniel Stenberg [Sat, 25 Nov 2023 23:10:17 +0000 (00:10 +0100)] 
conncache: use the closure handle when disconnecting surplus connections

Use the closure handle for disconnecting connection cache entries so
that anything that happens during the disconnect is not stored and
associated with the 'data' handle which already just finished a transfer
and it is important that details from the unrelated disconnect does not
taint meta-data in the data handle.

Like storing the response code.

This also adjust test 1506. Unfortunately it also removes a key part of
the test that verifies that a connection is closed since when this
output vanishes (because the closure handle is used), we don't know
exactly that the connection actually gets closed in this test...

Reported-by: ohyeaah on github
Fixes #12367
Closes #12405

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Fri, 24 Nov 2023 20:01:54 +0000 (21:01 +0100)] 
RELEASE-NOTES: synced

20 months agoquic: make eyeballers connect retries stop at weird replies
Stefan Eissing [Fri, 24 Nov 2023 12:45:55 +0000 (13:45 +0100)] 
quic: make eyeballers connect retries stop at weird replies

- when a connect immediately goes into DRAINING state, do
  not attempt retries in the QUIC connection filter. Instead,
  return CURLE_WEIRD_SERVER_REPLY
- When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an
  inconclusive answer. When all addresses have been attempted,
  rewind the address list once on an inconclusive answer.
- refs #11832 where connects were retried indefinitely until
  the overall timeout fired

Closes #12400

20 months agoCI: verify libcurl function SYNPOSIS sections
Daniel Stenberg [Fri, 24 Nov 2023 19:29:52 +0000 (20:29 +0100)] 
CI: verify libcurl function SYNPOSIS sections

With the .github/scripits/verify-synopsis.pl script

Closes #12402

20 months agodocs/libcurl: SYNSOPSIS cleanup
Daniel Stenberg [Fri, 24 Nov 2023 16:52:15 +0000 (17:52 +0100)] 
docs/libcurl: SYNSOPSIS cleanup

- use the correct include file
- make sure they are declared as in the header file
- fix minor nroff syntax mistakes (missing .fi)

These are verified by verify-synopsis.pl, which extracts the SYNPOSIS
code and runs it through gcc.

Closes #12402

20 months agosendf: fix comment typo
Daniel Stenberg [Fri, 24 Nov 2023 13:36:25 +0000 (14:36 +0100)] 
sendf: fix comment typo

20 months agofopen: allocate the dir after fopen
Daniel Stenberg [Fri, 24 Nov 2023 08:46:32 +0000 (09:46 +0100)] 
fopen: allocate the dir after fopen

Move the allocation of the directory name down to after the fopen() call
to allow that shortcut code path to avoid a superfluous malloc+free
cycle.

Follow-up to 73b65e94f35311

Closes #12398

20 months agotransfer: cleanup done+excess handling
Stefan Eissing [Tue, 21 Nov 2023 10:24:18 +0000 (11:24 +0100)] 
transfer: cleanup done+excess handling

- add `SingleRequest->download_done` as indicator that
  all download bytes have been received
- remove `stop_reading` bool from readwrite functions
- move excess body handling into client download writer

Closes #12371

20 months agofopen: create new file using old file's mode
Daniel Stenberg [Thu, 23 Nov 2023 14:52:57 +0000 (15:52 +0100)] 
fopen: create new file using old file's mode

Because the function renames the temp file to the target name as a last
step, if the file was previously owned by a different user, not ORing
the old mode could otherwise end up creating a file that was no longer
readable by the original owner after save.

Reported-by: Loïc Yhuel
Fixes #12299
Closes #12395

20 months agotest1476: require proxy
Daniel Stenberg [Thu, 23 Nov 2023 14:38:21 +0000 (15:38 +0100)] 
test1476: require proxy

Follow-up from 323df4261c3542

Closes #12394

20 months agofopen: create short(er) temporary file name
Daniel Stenberg [Thu, 23 Nov 2023 07:23:17 +0000 (08:23 +0100)] 
fopen: create short(er) temporary file name

Only using random letters in the name plus a ".tmp" extension. Not by
appending characters to the final file name.

Reported-by: Maksymilian Arciemowicz
Closes #12388

20 months agotests: git ignore generated second-hsts.txt file
Stefan Eissing [Thu, 23 Nov 2023 12:06:21 +0000 (13:06 +0100)] 
tests: git ignore generated second-hsts.txt file

File is generated in test lib1900

Follow-up to 7cb03229d9e9c5

Closes #12393

20 months agoopenssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6
Viktor Szakats [Thu, 23 Nov 2023 01:14:37 +0000 (01:14 +0000)] 
openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6

Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to
OpenSSL 1.1.x, and LibreSSL 3.6 or upper.

With the caveat, that "group name" and "type name" are missing from
the log output with these TLS backends.

Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030

Reviewed-by: Daniel Stenberg
Closes #12385

20 months agourldata: fix typo in comment
Daniel Stenberg [Thu, 23 Nov 2023 11:41:13 +0000 (12:41 +0100)] 
urldata: fix typo in comment

20 months agoCI: codespell
Daniel Stenberg [Thu, 23 Nov 2023 11:08:33 +0000 (12:08 +0100)] 
CI: codespell

The list of words to ignore is in the file
.github/scripts/codespell-ignore.txt

Closes #12390

20 months agolib: fix comment typos
Daniel Stenberg [Thu, 23 Nov 2023 08:51:14 +0000 (09:51 +0100)] 
lib: fix comment typos

Five separate ones, found by codespell

Closes #12390

20 months agotest1476: verify cookie PSL mixed case
Daniel Stenberg [Thu, 23 Nov 2023 07:15:47 +0000 (08:15 +0100)] 
test1476: verify cookie PSL mixed case

20 months agocookie: lowercase the domain names before PSL checks
Daniel Stenberg [Thu, 23 Nov 2023 07:15:47 +0000 (08:15 +0100)] 
cookie: lowercase the domain names before PSL checks

Reported-by: Harry Sintonen
Closes #12387

20 months agoopenssl: fix building with v3 `no-deprecated` + add CI test
Viktor Szakats [Wed, 22 Nov 2023 15:08:09 +0000 (15:08 +0000)] 
openssl: fix building with v3 `no-deprecated` + add CI test

- build quictls with `no-deprecated` in CI to have test coverage for
  this OpenSSL 3 configuration.

- don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`.
  The caller code is meant for OpenSSL 3, while these two functions were
  only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3
  if built with option `no-deprecated`, causing build errors:
  ```
  vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_algorithms'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_digests'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?fullLog=true#L7667

  Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030
  Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669
Reviewed-by: Alex Bozarth
- vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with
  `no-deprecated` quictls 3 builds.
  Do it by moving an existing solution for this from `vtls/openssl.c`
  to `vtls/openssl.h` and adjusting caller code.
  ```
  vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_get_peer_certificate'; did you mean   'SSL_get1_peer_certificate'? [-Wimplicit-function-declaration]
  ```
  Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#step:24:1178

- curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and
  `-Wunused-function` when trying to build curl with NTLM enabled but
  without the necessary TLS backend (with DES) support.

Closes #12384

20 months agocurl.h: delete Symbian OS references
Viktor Szakats [Wed, 22 Nov 2023 01:08:33 +0000 (01:08 +0000)] 
curl.h: delete Symbian OS references

curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81
via #5989. Delete references to it from public headers, because there
is no fresh release to use those headers with.

Reviewed-by: Dan Fandrich
Reviewed-by: Jay Satiro
Closes #12378

20 months agowindows: use built-in `_WIN32` macro to detect Windows
Viktor Szakats [Tue, 21 Nov 2023 16:54:49 +0000 (16:54 +0000)] 
windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Closes #12376

20 months agourl: ConnectionExists revisited
Stefan Eissing [Tue, 21 Nov 2023 14:46:41 +0000 (15:46 +0100)] 
url: ConnectionExists revisited

- have common pattern of `if not match, continue`
- revert pages long if()s to return early
- move dead connection check to later since it may
  be relatively expensive
- check multiuse also when NOT building with NGHTTP2
- for MULTIUSE bundles, verify that the inspected
  connection indeed supports multiplexing when in use
  (bundles may contain a mix of connection, afaict)

Closes #12373

20 months agoCURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range
Daniel Stenberg [Wed, 22 Nov 2023 09:54:40 +0000 (10:54 +0100)] 
CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range

... or use the default value.

Also clarify the documentation language somewhat.

Closes #12382

20 months agourldata: make maxconnects a 32 bit value
Daniel Stenberg [Tue, 21 Nov 2023 17:18:27 +0000 (18:18 +0100)] 
urldata: make maxconnects a 32 bit value

"2^32 idle connections ought to be enough for anybody"

Closes #12375

20 months agoFEATURES: update the URL phrasing
Daniel Stenberg [Wed, 22 Nov 2023 10:34:13 +0000 (11:34 +0100)] 
FEATURES: update the URL phrasing

The URL is length limited since a while back so "no limit" simply is not
true anymore. Mention the URL RFC standard used instead.

Closes #12383

20 months agowolfssh: remove redundant static prototypes
Daniel Stenberg [Wed, 22 Nov 2023 07:57:19 +0000 (08:57 +0100)] 
wolfssh: remove redundant static prototypes

vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-Werror=redundant-decls]

Closes #12381

20 months agosetopt: remove superfluous use of ternary expressions
Daniel Stenberg [Tue, 21 Nov 2023 16:55:31 +0000 (17:55 +0100)] 
setopt: remove superfluous use of ternary expressions

Closes #12374

20 months agomime: store "form escape" as a single bit
Daniel Stenberg [Tue, 21 Nov 2023 16:44:05 +0000 (17:44 +0100)] 
mime: store "form escape" as a single bit

Closes #12374

20 months agosetopt: check CURLOPT_TFTP_BLKSIZE range on set
Daniel Stenberg [Tue, 21 Nov 2023 16:34:30 +0000 (17:34 +0100)] 
setopt: check CURLOPT_TFTP_BLKSIZE range on set

... instead of later when the transfer is about to happen.

Closes #12374

20 months agobuild: add more picky warnings and fix them
Viktor Szakats [Wed, 15 Nov 2023 14:43:36 +0000 (14:43 +0000)] 
build: add more picky warnings and fix them

Enable more picky compiler warnings. I've found these options in the
nghttp3 project when implementing the CMake quick picky warning
functionality for it [1].

`-Wunused-macros` was too noisy to keep around, but fixed a few issues
it revealed while testing.

- autotools: reflect the more precisely-versioned clang warnings.
  Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324
- autotools: sync between clang and gcc the way we set `no-multichar`.
- autotools: avoid setting `-Wstrict-aliasing=3` twice.
- autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2].
  It triggers in libtool-generated stub code.

- lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch.

- lib/curl_setup.h: delete duplicate declaration for `fileno`.
  Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d
  (1999-12-29). This suggests this may not be needed anymore, but if
  it does, we may restore this for those specific (non-Windows) systems.
- lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since
  c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394.
- lib: delete unused macro `isxdigit_ascii` since
  f65f750742068f579f4ee6d8539ed9d5f0afcb85.
- lib/mqtt: delete unused macro `MQTT_HEADER_LEN`.
- lib/multi: delete unused macro `SH_READ`/`SH_WRITE`.
- lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN`
  macro.
- lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`.
- lib/rand: fix `-Wunreachable-code` and related fallouts [3].
- lib/setopt: fix `-Wunreachable-code-break`.
- lib/system_win32 and lib/timeval: fix double declarations for
  `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4].
- lib/warnless: fix double declarations in CMake UNITY mode [5].
  This was due to force-disabling the header guard of `warnless.h` to
  to reapply it to source code coming after `warnless.c` in UNITY
  builds. This reapplied declarations too, causing the warnings.
  Solved by adding a header guard for the lines that actually need
  to be reapplied.
- lib/vauth/digest: fix `-Wunreachable-code-break` [6].
- lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant
  block.
- lib/vtls/sectransp: fix `-Wunreachable-code-break` [7].
- lib/vtls/sectransp: suppress `-Wunreachable-code`.
  Detected in `else` branches of dynamic feature checks, with results
  known at compile-time, e.g.
  ```c
  if(SecCertificateCopySubjectSummary)  /* -> true */
  ```
  Likely fixable as a separate micro-project, but given SecureTransport
  is deprecated anyway, let's just silence these locally.
- src/tool_help: delete duplicate declaration for `helptext`.
- src/tool_xattr: fix `-Wunreachable-code`.
- tests: delete duplicate declaration for `unitfail` [8].
- tests: delete duplicate declaration for `strncasecompare`.
- tests/libtest: delete duplicate declaration for `gethostname`.
  Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7
  (2010-08-02).
  Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc
  If there are still systems around with warnings, we may restore the
  prototype, but limited for those systems.
- tests/lib2305: delete duplicate declaration for
  `libtest_debug_config`.
- tests/h2-download: fix `-Wunreachable-code-break`.

[1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056f8bf9f/cmake/PickyWarningsC.cmake
[2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjauiqla5fj45?fullLog=true#L1675
[3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=12331#step:7:72
    https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=12331#step:7:109
[4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L204
[5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L218
[6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=12331#step:7:290
[7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=12331#step:9:1193
[8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=12331#step:33:1870

Closes #12331

20 months agotransfer: avoid unreachable expression
Daniel Stenberg [Tue, 21 Nov 2023 07:13:08 +0000 (08:13 +0100)] 
transfer: avoid unreachable expression

If curl_off_t and size_t have the same size (which is common on modern
64 bit systems), a condition cannot occur which Coverity pointed
out. Avoid the warning by having the code conditionally only used if
curl_off_t actually is larger.

Follow-up to 1cd2f0072fa482e25baa2

Closes #12370

20 months agotransfer: readwrite improvements
Stefan Eissing [Mon, 6 Nov 2023 16:06:06 +0000 (17:06 +0100)] 
transfer: readwrite improvements

- changed header/chunk/handler->readwrite prototypes to accept `buf`,
  `blen` and a `pconsumed` pointer. They now get the buffer to work on
  and report back how many bytes they consumed
- eliminated `k->str` in SingleRequest
- improved excess data handling to properly calculate with any body data
  left in the headerb buffer
- eliminated `k->badheader` enum to only be a bool

Closes #12283

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Tue, 21 Nov 2023 07:01:42 +0000 (08:01 +0100)] 
RELEASE-NOTES: synced

20 months agotransfer: avoid calling the read callback again after EOF
Jiří Hruška [Mon, 20 Nov 2023 08:26:59 +0000 (09:26 +0100)] 
transfer: avoid calling the read callback again after EOF

Regression since 7f43f3dc5994d01b12 (7.84.0)

Bug: https://curl.se/mail/lib-2023-11/0017.html

Closes #12363

20 months agodoh: provide better return code for responses w/o addresses
Daniel Stenberg [Mon, 20 Nov 2023 12:33:31 +0000 (13:33 +0100)] 
doh: provide better return code for responses w/o addresses

Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the
response did not contain any addresses. Now it more accurately returns
CURLE_COULDNT_RESOLVE_HOST.

Reported-by: lRoccoon on github
Fixes #12365
Closes #12366

20 months agoHTTP/2, HTTP/3: handle detach of onoing transfers
Stefan Eissing [Mon, 20 Nov 2023 10:32:19 +0000 (11:32 +0100)] 
HTTP/2, HTTP/3: handle detach of onoing transfers

- refs #12356 where a UAF is reported when closing a connection
  with a stream whose easy handle was cleaned up already
- handle DETACH events same as DONE events in h2/h3 filters

Fixes #12356
Reported-by: Paweł Wegner
Closes #12364

20 months agoautotools: stop setting `-std=gnu89` with `--enable-warnings`
Viktor Szakats [Thu, 16 Nov 2023 21:44:45 +0000 (21:44 +0000)] 
autotools: stop setting `-std=gnu89` with `--enable-warnings`

Do not alter the C standard when building with `--enable-warnings` when
building with gcc.

On one hand this alters warning results compared to a default build.
On the other, it may produce different binaries, which is unexpected.

Also fix new warnings that appeared after removing `-std=gnu89`:

- include: fix public curl headers to use the correct printf mask for
  `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64
  and Visual Studio 2013 and newer. This fixes the printf mask warnings
  in examples and tests. E.g. [1]

- conncache: fix printf format string [2].

- http2: fix potential null pointer dereference [3].
  (seen on Slackware with gcc 11.)

- libssh: fix printf format string in SFTP code [4].
  Also make MSVC builds compatible with old CRT versions.

- libssh2: fix printf format string in SFTP code for MSVC.
  Applying the same fix as for libssh above.

- unit1395: fix `argument is null` and related issues [5]:
  - stop calling `strcmp()` with NULL to avoid undefined behaviour.
  - fix checking results if some of them were NULL.
  - do not pass NULL to printf `%s`.

- ci: keep a build job with `-std=gnu89` to continue testing for
  C89-compliance. We can apply this to other gcc jobs as needed.
  Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542

[1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=logs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b
[2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=12346#step:6:67
[3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=12346#step:30:214
[4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=12346#step:29:895
[5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=12346#step:33:1689

Closes #12346

20 months agoautotools: fix/improve gcc and Apple clang version detection
Viktor Szakats [Sun, 19 Nov 2023 23:31:40 +0000 (23:31 +0000)] 
autotools: fix/improve gcc and Apple clang version detection

- Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu
  may return `n-win32` (also with `-dumpfullversion`). Causing these
  errors and failing to enable picky warnings:
  ```
  ../configure: line 23845: test: : integer expression expected
  ```
  Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/17007893718#step:5:143

  Fix that by stripping any dash-suffix and handling a dotless (major-only)
  version number by assuming `.0` in that case.

  `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0`
  Ref: https://github.com/mamedev/mame/pull/9767

- fix Apple clang version detection for releases between
  'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the
  version was under-detected as 3.7 llvm/clang equivalent.

- fix Apple clang version detection for 'Apple clang version 11.0.0'
  and newer where the Apple clang version was detected, instead of its
  llvm/clang equivalent.

- display detected clang/gcc/icc compiler version.

Via libssh2:
- https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d89875ad
  https://github.com/libssh2/libssh2/pull/1187
- https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e350008131942b592e
  https://github.com/libssh2/libssh2/pull/1232

Closes #12362

20 months agoautotools: delete LCC compiler support bits
Viktor Szakats [Sun, 19 Nov 2023 17:45:24 +0000 (17:45 +0000)] 
autotools: delete LCC compiler support bits

Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222

Closes #12357

20 months agocmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API`
Viktor Szakats [Fri, 17 Nov 2023 21:42:54 +0000 (21:42 +0000)] 
cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API`

- tests: verify CMake `DISABLE` options.

  Make an exception for 2 CMake-only ones, and one more that's
  using a different naming scheme, also in autotools and source.

- cmake: add support for `CURL_DISABLE_HEADERS_API`.

Suggested-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641

Closes #12353

20 months agohyper: temporarily remove HTTP/2 support
Jacob Hoffman-Andrews [Tue, 24 Oct 2023 14:51:05 +0000 (07:51 -0700)] 
hyper: temporarily remove HTTP/2 support

The current design of the Hyper integration requires rebuilding the
Hyper clientconn for each request. However, building the clientconn
requires resending the HTTP/2 connection preface, which is incorrect
from a protocol perspective. That in turn causes servers to send GOAWAY
frames, effectively degrading performance to "no connection reuse" in
the best case. It may also be triggering some bugs where requests get
dropped entirely and reconnects take too long.

This doesn't rule out HTTP/2 support with Hyper, but it may take a
redesign of the Hyper integration in order to make things work.

Closes #12191

20 months agoschannel: fix unused variable warning
Jay Satiro [Sun, 19 Nov 2023 23:06:55 +0000 (18:06 -0500)] 
schannel: fix unused variable warning

Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846
Reported-by: Viktor Szakats
Closes https://github.com/curl/curl/pull/12361

20 months agourl: find scheme with a "perfect hash"
Daniel Stenberg [Fri, 17 Nov 2023 12:04:13 +0000 (13:04 +0100)] 
url: find scheme with a "perfect hash"

Instead of a loop to scan over the potentially 30+ scheme names, this
uses a "perfect hash" table. This works fine because the set of schemes
is known and cannot change in a build. The hash algorithm and table size
is made to only make a single scheme index per table entry.

The perfect hash is generated by a separate tool (scripts/schemetable.c)

Closes #12347

20 months agoscripts: add schemetable.c
Daniel Stenberg [Fri, 17 Nov 2023 12:33:17 +0000 (13:33 +0100)] 
scripts: add schemetable.c

This tool generates a scheme-matching table.

It iterates over a number of different initial and shift values in order
to find the hash algorithm that needs the smallest possible table.

The generated hash function, table and table size then needs to be used
by the url.c:Curl_getn_scheme_handler() function.

20 months agovtls/vquic, keep peer name information together
Stefan Eissing [Fri, 17 Nov 2023 14:26:08 +0000 (15:26 +0100)] 
vtls/vquic, keep peer name information together

- add `struct ssl_peer` to keep hostname, dispname and sni
  for a filter
- allocate `sni` for use in VTLS backend
- eliminate `Curl_ssl_snihost()` and its use of the download buffer
- use ssl_peer in SSL and QUIC filters

Closes #12349

20 months agobuild: always revert `#pragma GCC diagnostic` after use
Viktor Szakats [Fri, 17 Nov 2023 19:13:13 +0000 (19:13 +0000)] 
build: always revert `#pragma GCC diagnostic` after use

Before this patch some source files were overriding gcc warning options,
but without restoring them at the end of the file. In CMake UNITY builds
these options spilled over to the remainder of the source code,
effecitvely disabling them for a larger portion of the codebase than
intended.

`#pragma clang diagnostic` didn't have such issue in the codebase.

Reviewed-by: Marcel Raad
Closes #12352

20 months agotidy-up: casing typos, delete unused Windows version aliases
Viktor Szakats [Wed, 15 Nov 2023 22:31:44 +0000 (22:31 +0000)] 
tidy-up: casing typos, delete unused Windows version aliases

- cmake: fix casing of `UnixSockets` to match the rest of the codebase.

- curl-compilers.m4: fix casing in a comment.

- setup-win32: delete unused Windows version constant aliases.

Reviewed-by: Marcel Raad
Closes #12351

20 months agokeylog: disable if unused
Viktor Szakats [Fri, 17 Nov 2023 16:45:40 +0000 (16:45 +0000)] 
keylog: disable if unused

Fully disable keylog code if there is no TLS or QUIC subsystem using it.

Closes #12350

20 months agocmake: add `CURL_DISABLE_BINDLOCAL` option
Viktor Szakats [Thu, 16 Nov 2023 21:08:49 +0000 (21:08 +0000)] 
cmake: add `CURL_DISABLE_BINDLOCAL` option

To match similar autotools option.

Default is `ON`.

Reviewed-by: Daniel Stenberg
Closes #12345

20 months agourl: fix `-Wzero-length-array` with no protocols
Viktor Szakats [Thu, 16 Nov 2023 18:52:13 +0000 (18:52 +0000)] 
url: fix `-Wzero-length-array` with no protocols

Fixes:
```
./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [-Wc2x-extensions]
  178 | static const struct Curl_handler * const protocols[] = {
      |                                                        ^
./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length-array]
```

Closes #12344

20 months agourl: fix builds with `CURL_DISABLE_HTTP`
Viktor Szakats [Thu, 16 Nov 2023 16:50:27 +0000 (16:50 +0000)] 
url: fix builds with `CURL_DISABLE_HTTP`

Fixes:
```
./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState'
  456 |   Curl_mime_cleanpart(data->state.formp);
      |                       ~~~~~~~~~~~ ^
```

Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682

Closes #12343

20 months agohttp: fix `-Wunused-parameter` with no auth and no proxy
Viktor Szakats [Thu, 16 Nov 2023 12:39:01 +0000 (12:39 +0000)] 
http: fix `-Wunused-parameter` with no auth and no proxy

```
lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter]
                    bool proxy)
                         ^
```

Reviewed-by: Marcel Raad
Closes #12338

20 months agoTODO: Some TLS options are not offered for HTTPS proxies
Daniel Stenberg [Thu, 16 Nov 2023 15:48:16 +0000 (16:48 +0100)] 
TODO: Some TLS options are not offered for HTTPS proxies

Closes #12286
Closes #12342

20 months agoRELEASE-NOTES: synced
Daniel Stenberg [Thu, 16 Nov 2023 14:59:12 +0000 (15:59 +0100)] 
RELEASE-NOTES: synced

20 months agoduphandle: make dupset() not return with pointers to old alloced data
Daniel Stenberg [Thu, 16 Nov 2023 09:04:35 +0000 (10:04 +0100)] 
duphandle: make dupset() not return with pointers to old alloced data

As the blob pointers are to be duplicated, the function must not return
mid-function with lingering pointers to the old handle's allocated data,
as that would lead to double-free in OOM situations.

Make sure to clear all destination pointers first to avoid this risk.

Closes #12337

20 months agohttp: fix `-Wunused-variable` compiler warning
Viktor Szakats [Thu, 16 Nov 2023 00:12:54 +0000 (00:12 +0000)] 
http: fix `-Wunused-variable` compiler warning

Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO.

E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` +
`CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` +
`CURL_DISABLE_NTLM` on non-Windows.

```
./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variable]
  CURLcode result = CURLE_OK;
           ^
./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunused-but-set-variable]
  unsigned long *availp;
                 ^
./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunused-but-set-variable]
  struct auth *authp;
               ^
```

Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490

Fixes #12228
Closes #12335

20 months agotool: support bold headers in Windows
Jay Satiro [Tue, 14 Nov 2023 09:11:04 +0000 (04:11 -0500)] 
tool: support bold headers in Windows

- If virtual terminal processing is enabled in Windows then use ANSI
  escape codes Esc[1m and Esc[22m to turn bold on and off.

Suggested-by: Gisle Vanem
Ref: https://github.com/curl/curl/discussions/11770

Closes https://github.com/curl/curl/pull/12321

20 months agobuild: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS`
Viktor Szakats [Wed, 15 Nov 2023 15:58:11 +0000 (15:58 +0000)] 
build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS`

Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` +
`-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows,
or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while
compiling due to a missing HMAC declaration.

The reason is that HMAC is required by `lib/sha256.c` which publishes
`Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when
building for libssh2 v1.8.2 (2019-05-25) or older.

Make sure to compile the HMAC bits for a successful build.

Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting
them into separate sources isn't practical.

Fixes:
```
[...]
In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:310:
./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const struct HMAC_params'
  527 | const struct HMAC_params Curl_HMAC_SHA256[] = {
      |                                          ^
./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_params'
[...]
```

Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490

Fixes #12273
Closes #12332

20 months agoduphandle: also free 'outcurl->cookies' in error path
Daniel Stenberg [Wed, 15 Nov 2023 10:48:32 +0000 (11:48 +0100)] 
duphandle: also free 'outcurl->cookies' in error path

Fixes memory-leak when OOM mid-function

Use plain free instead of safefree, since the entire struct is
freed below.

Remove some free calls that is already freed in Curl_freeset()

Closes #12329

20 months agoconfig-win32: set `HAVE_SNPRINTF` for mingw-w64
Viktor Szakats [Wed, 15 Nov 2023 02:54:43 +0000 (02:54 +0000)] 
config-win32: set `HAVE_SNPRINTF` for mingw-w64

It's available in all mingw-w64 releases. We already pre-fill this
detection in CMake.

Closes #12325

20 months agosasl: fix `-Wunused-function` compiler warning
Viktor Szakats [Wed, 15 Nov 2023 03:10:04 +0000 (03:10 +0000)] 
sasl: fix `-Wunused-function` compiler warning

In builds with disabled auths.

```
lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunused-function]
static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data,
                ^
1 warning generated.
```
Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#step:3:3822

Reviewed-by: Daniel Stenberg
Closes #12326

20 months agobuild: picky warning updates
Viktor Szakats [Wed, 15 Nov 2023 00:32:43 +0000 (00:32 +0000)] 
build: picky warning updates

- cmake: sync some picky gcc warnings with autotools.
- cmake, autotools: add `-Wold-style-definition` for clang too.
- cmake: more precise version info for old clang options.
- cmake: use `IN LISTS` syntax in `foreach()`.

Reviewed-by: Daniel Stenberg
Reviewed-by: Marcel Raad
Closes #12324

20 months agourldata: move cookielist from UserDefined to UrlState
Daniel Stenberg [Tue, 14 Nov 2023 21:52:18 +0000 (22:52 +0100)] 
urldata: move cookielist from UserDefined to UrlState

1. Because the value is not strictly set with a setopt option.

2. Because otherwise when duping a handle when all the set.* fields are
   first copied and an error happens (think out of memory mid-function),
   the function would easily free the list *before* it was deep-copied,
   which could lead to a double-free.

Closes #12323

20 months agoautotools: avoid passing `LDFLAGS` twice to libcurl
Viktor Szakats [Mon, 13 Nov 2023 00:02:45 +0000 (00:02 +0000)] 
autotools: avoid passing `LDFLAGS` twice to libcurl

autotools passes `LDFLAGS` automatically linker commands. curl's
`lib/Makefile.am` customizes libcurl linker flags. In that
customization, it added `LDFLAGS` to the custom flags. This resulted in
passing `LDFLAGS` _twice_ to the `libtool` command.

Most of the time this is benign, but some `LDFLAGS` options can break
the build when passed twice. One such example is passing `.o` files,
e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for
MUSL builds.

Passing them twice resulted in duplicate symbol errors:
```
libtool: link:  clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/aarch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...]
ld.lld-15: error: duplicate symbol: _start
>>> defined at crt1.c
>>>            /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0)
>>> defined at crt1.c
>>>            /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0)
[...]
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36
(2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217.
The patch was a works-for-me hack that ended up merged in curl:
https://sourceforge.net/p/curl/bugs/1217/#06ef
With the root cause remaining unclear.

Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring
`-L` first? This would be unusual and suggests a bug in either the
linker or in `libtool`.

The curl build does pass the list of detected libs via its own
`LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool`
command line, but it's the job of `libtool` to ensure that even
a peculiar linker gets the options in the expected order. Also because
autotools passes `LDFLAGS` last, making it hardly possible to pass
anything after it.

Perhaps in the 10 years since this issue, this already got a fix
upstream.

This patch deletes `LDFLAGS` from our customized libcurl options,
leaving a single copy of them as passed by autotools automatically.

Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36
Closes #12310

20 months agoautotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}`
Viktor Szakats [Mon, 13 Nov 2023 00:02:45 +0000 (00:02 +0000)] 
autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}`

To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and
curl tool (`CURL_LDFLAGS_BIN`).

This makes it possible to build libcurl and curl with a single
invocation with lib- and tool-specific custom linker flags.

Such flag can be enabling `.map` files, a `.def` file for libcurl DLL,
controlling static/shared, incl. requesting a static curl tool (with
`-static-libtool-libs`) while building both shared and static libcurl.

curl-for-win uses the above and some more.

These options are already supported in `Makefile.mk`. CMake has built-in
variables for this.

Closes #12312