Limited to x64, because for build testing the additional CPUs don't add
much value compared to the extra build time. They can be enabled easily
if deemed useful.
To the extent of curl-for-win configuration options, it's trivial to add
further build combinations.
Viktor Szakats [Sun, 7 Apr 2024 12:39:59 +0000 (12:39 +0000)]
OS400: fix shellcheck warnings in scripts
- use `$()` instead of backticks, and re-arrange double-quotes inside.
- add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu`.)
- add `-n` to a few `if`s.
- shorten redirections by using `{} >` (as shellcheck recommended).
- silence warnings where variables were detected as unused (SC2034).
- a couple misc updates to silence warnings.
- switch to bash shebang for `-ot` feature.
- split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, `$(dirname \`)
Stefan Eissing [Wed, 10 Apr 2024 11:30:16 +0000 (13:30 +0200)]
gnutls: lazy init the trust settings
- delay loading of trust anchors and CRLs after the ClientHello
has been sent off
- add tracing to IO operations
- on IO errors, return the CURLcode of the underlying filter
Viktor Szakats [Tue, 9 Apr 2024 19:08:18 +0000 (19:08 +0000)]
appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings
Tests were consistently flaky for a while.
Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers:
```
C:/projects/curl/lib/vtls/schannel.c(688):
warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
C:/projects/curl/lib/vtls/schannel_verify.c(642):
warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44kymgc0nif#L106
Daniel Stenberg [Mon, 8 Apr 2024 07:13:17 +0000 (09:13 +0200)]
projects: drop MSVC project files for recent versions
We encourage users to generate visual studio project files using CMake.
We keep project files in git for ancient visual studio versions that
cmake cannot generate files for, but we no longer ship the project files
in the tarballs.
appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013)
Co-Authored-by: Viktor Szakats Co-Authored-by: Jay Satiro
Closes #13311
Stefan Eissing [Tue, 9 Apr 2024 07:41:19 +0000 (09:41 +0200)]
tests: stabilitze test_02_23*
- h2-download now always opens the output file on first write callback
invocation, if it will pause the transfer or not.
- Checks on output files then does not depend on the amount of data curl
has collected for the first write.
Viktor Szakats [Tue, 9 Apr 2024 06:46:35 +0000 (06:46 +0000)]
dist: add reproducible dir entries to tarballs
In the initial implementation of reproducible tarballs, they were
missing directory entries, while .zip archives had them. It meant
that on extracting the tarball, on-disk directory entries got the
current timestamp.
This patch fixes this by including directory entries in the tarball,
with reproducible timestamps. It also moves sorting inside tar,
to ensure reproducible directory entry timestamps on extract
(without the need of `--delay-directory-restore` option, when
extracting with GNU tar. BSD tar got that right by default.)
Daniel Stenberg [Mon, 8 Apr 2024 15:14:35 +0000 (17:14 +0200)]
release-tools.sh: store the timestamp and release tag too
When maketgz invokes this script to generate the docs/RELEASE-TOOLS.md
file that gets bundled in the release, it now also passes on the exact
timestamp and version number so that those details also get mentioned in
the document. They will help users reproduce an identical tarball.
Stefan Eissing [Fri, 5 Apr 2024 13:38:11 +0000 (15:38 +0200)]
http2: emit RST when client write fails
- When the writing of response data fails, reset the stream
and do not return a callback error to nghttp2. That would
be a fatal error for the connection and harm other requests.
- add test cases for various abort scenarios
Reported-by: Konstantin Kuzov
Fixes #13292
Closes #13298
Daniel Stenberg [Thu, 4 Apr 2024 16:00:33 +0000 (18:00 +0200)]
docs/cmdline-opts: invoke managen using a relative path
... no need to use an absolute path, that makes the build unncessarily
fail if invoked using a different mount point. managen now takes options
to find the input files.
Update test1478 to provide the dir arguments to managen
Viktor Szakats [Fri, 5 Apr 2024 13:27:05 +0000 (13:27 +0000)]
dist: `set -eu`, fix shellcheck, make reproducible and smaller tarballs
- set bash `-eu` and fix fallouts.
- fix shellcheck warnings.
- set and use `SOURCE_DATE_EPOCH` for reproducibility. Authored-by: Daniel J. H.
Ref: #13280
- set `TZ=UTC` and `LC_ALL=C` for reproducibility.
- make file timestamps in tarball/zip reproducible.
- make directory timestamps in zip reproducible.
- make timestamps of tarballs/zip reproducible.
- make file order in tarball/zip reproducible.
- omit extra file metadata from zip for reproducibility.
- use maximum zip compression.
- use POSIX `ustar` tarball format to avoid supply chain vulnerability:
https://seclists.org/oss-sec/2021/q4/0
- make uid/gid in tarball reproducible.
- omit owner user/group names from tarball for reproducibility and privacy.
- omit current timestamp from .gz header for reproducibility.
- display SHA-256 hashes of produced tarballs/zip.
- fix whitespace.
`.tar.gz` also became smaller in the process: 4,462,311 -> 4,148,249 bytes (8.7.1)
Requires GNU tar, GNU date, `sha256sum`.
Reviewed-by: Daniel Stenberg
Ref: #13250
Closes #13299
Stefan Eissing [Fri, 5 Apr 2024 10:11:48 +0000 (12:11 +0200)]
vquic: use new curl_int64_t type
- add curl_int64_t signed 64-bit type for lib use
- define CURL_PRId64, CURL_PRIu64 format ids
- use curl_int64_t in vquic
curl_int64_t signed complements the existing curl_uint64_t unsigned.
Note that `curl_int64_t` and `int64_t` are assignable from each other
but not identical. Some platforms with 64 long type defint int64_t as
"long long" (staring at macOS) which messes up things like pointers and
format identifiers.
Jay Satiro [Tue, 9 Jan 2024 08:24:57 +0000 (03:24 -0500)]
lib: use multi instead of multi_easy for the active multi
- Use data->multi and not data->multi_easy to refer to the active multi.
The easy handle's active multi is always data->multi.
This is a follow up to 757dfdf which changed curl so that an easy handle
used with the easy interface and then multi interface cannot have two
different multi handles associated with it at the same time
(data->multi_easy from the easy interface and data->multi from the multi
interface).
Daniel Stenberg [Fri, 5 Apr 2024 11:01:58 +0000 (13:01 +0200)]
makefile: remove the sorting from the vc-ide action
This target generates the MSVC project files. This change removes the
extra sorting and instead makes the script use the order of the files as
listed in the variables - which are mostly sorted anyway.
This is an attempt to make the project file generation more easily
reproducible.
Viktor Szakats [Thu, 4 Apr 2024 10:45:01 +0000 (10:45 +0000)]
cmake: fix `CURL_WERROR=ON` for old CMake and use it in GHA/linux-old
- cmake: fix `-pedantic-errors` for old CMake with `CURL_WERROR=ON` set.
`-pedantic-errors` option throws a warning with GCC (all versions) and
makes `check_symbol_exists()` fail in CMake versions older than
v3.23.0 (2022-03-29), when CMake introduced a workaround:
Viktor Szakats [Thu, 4 Apr 2024 10:45:01 +0000 (10:45 +0000)]
GHA: enable unity mode for cmake jobs + tidy-ups
Unity mode is not supported by CMake v3.7.2 used in linux-old, but
enable it anyway for consistency and to kick in automatically once
migrating to a newer old Linux in the future.
Stefan Eissing [Wed, 3 Apr 2024 11:18:01 +0000 (13:18 +0200)]
request: paused upload on completed download, assess connection
A transfer with a completed download that is still uploading needs to
check the connection state when it is PAUSEd, since connection
close/errors would otherwise go unnoticed.
Viktor Szakats [Wed, 27 Mar 2024 15:54:08 +0000 (15:54 +0000)]
cmake: speed up libcurl doc building again
This time limit the number of files per command to avoid exceeding
limitations of certain OS/shell envs.
Such known env is Windows with the `cmd.exe` shell, which features an
8K command-line length limit to this day.
Allowlisting `UNIX` to have no limit and using a limit of 200 for other
envs to be safe. If there is a way to detect `cmd.exe` and/or we know
which precise envs are sensitive to this, we can tweak these conditions
further.
Even with the low limit, this patch reduces external commands by 200x,
making builds much faster.
Fabian Keil [Sun, 17 Mar 2024 14:49:58 +0000 (15:49 +0100)]
wolfssl: plug memory leak in wolfssl_connect_step2()
Fixes:
test 2034...[simple HTTPS GET with DER public key pinning]
==61829== 22,610 (3,744 direct, 18,866 indirect) bytes in 1 blocks are definitely lost in loss record 51 of 54
==61829== at 0x484BB74: malloc (vg_replace_malloc.c:446)
==61829== by 0x4B53A80: wolfSSL_Malloc (memory.c:344)
==61829== by 0x4C1C8E1: wolfSSL_X509_new (x509.c:5326)
==61829== by 0x4C3977D: d2i_X509orX509REQ (x509.c:3628)
==61829== by 0x4C1D1F4: wolfSSL_X509_d2i (x509.c:3664)
==61829== by 0x4C1C37B: wolfSSL_X509_dup (x509.c:13425)
==61829== by 0x4C197DB: wolfSSL_get_peer_certificate (ssl.c:18765)
==61829== by 0x33297C: wolfssl_connect_step2 (wolfssl.c:875)
==61829== by 0x331669: wolfssl_connect_common (wolfssl.c:1287)
==61829== by 0x3303E9: wolfssl_connect_nonblocking (wolfssl.c:1319)
==61829== by 0x32FE89: ssl_connect_nonblocking (vtls.c:510)
==61829== by 0x32DBE5: ssl_cf_connect (vtls.c:1679)
==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307)
==61829== by 0x27D9CF: cf_setup_connect (connect.c:1199)
==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307)
==61829== by 0x283CEA: cf_hc_baller_connect (cf-https-connect.c:135)
Viktor Szakats [Wed, 3 Apr 2024 03:06:16 +0000 (03:06 +0000)]
appveyor: OpenSSL 3 no longer found by CMake, revert to 1.1.1
OpenSSL moved directories, and bumped versions in AppVeyor CI.
Downgrading is not an ideal solution, but however trivial the solution
may be, I failed to come with anything that made CMake recognize either
OpenSSL 3.1 or 3.2.
Stefan Eissing [Mon, 1 Apr 2024 13:41:18 +0000 (15:41 +0200)]
http: with chunked POST forced, disable length check on read callback
- when an application forces HTTP/1.1 chunked transfer encoding
by setting the corresponding header and instructs curl to use
the CURLOPT_READFUNCTION, disregard any POST length information.
- this establishes backward compatibility with previous curl versions
Applications are encouraged to not force "chunked", but rather
set length information for a POST. By setting -1, curl will
auto-select chunked on HTTP/1.1 and work properly on other HTTP
versions.
Daniel McCarney [Sun, 31 Mar 2024 20:19:10 +0000 (16:19 -0400)]
m4: reposition USE_RUSTLS="yes" for pkg-config
It's necessary to set this var to "yes" _after_ AC_DEFINE and AC_SUBST
in order for a later `test` to pass so that `check_for_ca_bundle=1` ends
up being set. This is in turn required for the default CA certificate
bundle to be set when building w/ rustls & pkg-config.
Reported-by: Matt Jolly
Fixes #13248
Closes #13251
Daniel Stenberg [Sat, 30 Mar 2024 09:41:21 +0000 (10:41 +0100)]
curl: use curl_getenv instead of the curlx_ version
The curlx one was once introduced when we still considered dropping the
libcurl function at some point. To reduce confusion and to make it
easier to understand when curl_free() should be used, use the actual
libcurl function call directly instead.
Matt Jolly [Wed, 27 Mar 2024 12:52:26 +0000 (22:52 +1000)]
m4: fix rustls pkg-config codepath
The previous pkg-config code would successfully detect rustls but did
not set all appropriate variables and call the right macros to properly
configure cURL.
Reported-by: kpcyrd on github
Fixes #13200
Closes #13202
Daniel McCarney [Sat, 30 Mar 2024 18:18:45 +0000 (14:18 -0400)]
deps: update librustls 0.12.0 -> 0.13.0
This commit updates the optional rustls-ffi librustls dependency from
0.12.0 to 0.13.0. This version is based on the latest available rustls
release (0.23.4).
The breaking API changes from 0.12.0 to 0.13.0 are in API surface unused
by curl, so this is an in-place update without any code changes.
The `RUSTLS.md` documentation is updated to reflect the new version in
use, and to clarify that `cbindgen` isn't required to build `librustls`
- it's only used by developers to update the vendored `rustls.h` header
file maintained upstream.
Internally, libssh2 dereferences the NULL pointer if length is non-zero.
The callback function cannot return the error condition, so at least
prevent subsequent crash.