Joel Depooter [Sat, 3 Feb 2024 01:58:04 +0000 (17:58 -0800)]
docs: add missing slashes to SChannel client certificate documentation
When setting the CURLOPT_SSLCERT option to a certificate thumprint, it
is required to have a backslash between the "store location", "store
name" and "thumbprint" tokens. These slashes were present in the
previous documentation, but were missed in the transition to markdown
documentation.
Stefan Eissing [Wed, 31 Jan 2024 15:30:54 +0000 (16:30 +0100)]
HTTP/2: write response directly
- use the new `Curl_xfer_write_resp()` to write incoming responses
directly to the client
- eliminates `stream->recvbuf`
- memory consumption on parallel transfers minimized
Chris Webb [Sat, 3 Feb 2024 21:05:59 +0000 (21:05 +0000)]
configure: add --disable-docs flag
Building man pages from curldown sources now requires perl. Add a
--disable-docs flag to configure to enable building and installing
without documentation where perl is not available or man pages are not
required. This is selected automatically (with a warning) when perl is
not found by configure.
Daniel Stenberg [Thu, 1 Feb 2024 12:23:12 +0000 (13:23 +0100)]
sendf: ignore response body to HEAD
and mark the stream for close, but return OK since the response this far
was ok - if headers were received. Partly because this is what curl has
done traditionally.
Daniel Stenberg [Thu, 1 Feb 2024 10:28:22 +0000 (11:28 +0100)]
ftp: treat a 226 arriving before data as a signal to read data
For active mode transfers.
Due to some interesting timing, curl can sometimes get the 226 (transfer
complete) over the control channel first, before the data connection
signals readability. If this happens, use that as a signal to check the
data connection.
Additionally, set the socket filter in listen mode *before* the
PORT/EPRT command is issued, to reduce the risk that the little time gap
could interfere.
This issue never reproduced for me on Debian and takes several hundred
rounds for me to trigger on my mac.
Reported-by: Stefan Eissing
Fixes #12823
Closes #12841
Richard Levitte [Wed, 24 Jan 2024 12:07:00 +0000 (13:07 +0100)]
cmake: freshen up docs/INSTALL.cmake
- Turn docs/INSTALL.cmake into a proper markdown file,
docs/INSTALL-CMAKE.md
- Move things around to divide the description into configuration,
building and installing sections
- Mention the more modern cmake options to configure, build and install,
but also retain the older variants as fallbacks
Viktor Szakats [Sat, 27 Jan 2024 17:40:38 +0000 (17:40 +0000)]
build: delete/replace clang warning pragmas
- delete redundant warning suppressions for `-Wformat-nonliteral`.
This now relies on `CURL_PRINTF()` and it's theoratically possible
that this macro isn't active but the warning is. We're ignoring this
as a corner-case here.
- replace two pragmas with code changes to avoid the warnings.
Patrick Monnerat [Thu, 25 Jan 2024 12:58:19 +0000 (13:58 +0100)]
sasl: make login option string override http auth
- Use http authentication mechanisms as a default, not a preset.
Consider http authentication options which are mapped to SASL options as
a default (overriding the hardcoded default mask for the protocol) that
is ignored if a login option string is given.
Prior to this change, if some HTTP auth options were given, sasl mapped
http authentication options to sasl ones but merged them with the login
options.
That caused problems with the cli tool that sets the http login option
CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag
maps to more than one sasl mechanisms and the latter cannot be cleared
individually by the login options string.
Daniel Stenberg [Sun, 21 Jan 2024 23:04:47 +0000 (00:04 +0100)]
pingpong: stop using the download buffer
The pingpong logic now uses its own dynbuf for receiving command
response data.
When the "final" response header for a commanad has been received, that
final line is left first in the recvbuf for the protocols to parse at
will. If there is additional data behind the final response line, the
'overflow' counter is indicate how many bytes.
Viktor Szakats [Wed, 24 Jan 2024 12:25:15 +0000 (12:25 +0000)]
cmake: rework options to enable curl and libcurl docs
Rework CMake options for building/using curl tool and libcurl manuals.
- rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning:
to build man page and built-in manual for curl tool.
- rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning:
to build man pages for libcurl.
- `BUILD_LIBCURL_DOCS` now works without having to enable
`ENABLE_CURL_MANUAL` too.
- drop support for existing CMake-level `USE_MANUAL` option to avoid
confusion. (It used to work with the effect of current
`ENABLE_CURL_MANUAL`, but only by accident.)
Assisted-by: Richard Levitte
Ref: #12771
Closes #12773
Daniel Stenberg [Tue, 23 Jan 2024 14:12:09 +0000 (15:12 +0100)]
GHA: add a job scanning for "bad words" in markdown
This means words, phrases or things we have decided not to use - words that
are spelled right according to the dictionary but we want to avoid. In the
name of consistency and better documentation.
Viktor Szakats [Tue, 23 Jan 2024 06:40:11 +0000 (06:40 +0000)]
cmake: speed up curldown processing, enable by default
- cmake: enable `BUILD_DOCS` by default (this controls converting and
installing `.3` files from `.md` sources)
- cmake: speed up generating `.3` files by using a single command per
directory, instead of a single command per file. This reduces external
commands by about a thousand. (There remains some CMake logic kicking
in resulting in 500 -one per file- external `-E touch_nocreate` calls.)
- cd2nroff: add ability to process multiple input files.
- cd2nroff: add `-k` option to use the source filename to form the
output filename. (instead of the default in-file `Title:` line.)
Daniel Stenberg [Tue, 23 Jan 2024 16:45:22 +0000 (17:45 +0100)]
osslq: remove the TLS library from the version output
Since we only support using a single TLS library at any one time, we
know that the TLS library for QUIC is the same that is also shown for
regular TLS.
Fixes #12763 Reported-by: Viktor Szakats
Closes #12767
Daniel Stenberg [Sat, 20 Jan 2024 22:18:43 +0000 (23:18 +0100)]
docs/cmdline: change to .md for cmdline docs
- switch all invidual files documenting command line options into .md,
as the documentation is now markdown-looking.
- made the parser treat 4-space indents as quotes
- switch to building the curl.1 manpage using the "mainpage.idx" file,
which lists the files to include to generate it, instead of using the
previous page-footer/headers. Also, those files are now also .md
ones, using the same format. I gave them underscore prefixes to make
them sort separately:
_NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md,
_VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md,
_OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md,
_EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md
Daniel Stenberg [Wed, 17 Jan 2024 10:32:44 +0000 (11:32 +0100)]
docs: introduce "curldown" for libcurl man page format
curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:
- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
their man page section is specified)
tools:
- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown
This setup generates .3 versions of all the curldown versions at build time.
CI:
Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...
Viktor Szakats [Mon, 22 Jan 2024 11:21:10 +0000 (11:21 +0000)]
libssh2: use `libssh2_session_callback_set2()` with v1.11.1
To avoid a local hack to pass function pointers and to avoid
deprecation warnings when building with libssh2 v1.11.1 or newer:
```
lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/20720821100#step:3:4982
Ref: https://github.com/libssh2/libssh2/pull/1285
Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db3cf1d4b0b4 Reviewed-by: Daniel Stenberg
Closes #12754
Stefan Eissing [Thu, 18 Jan 2024 12:07:07 +0000 (13:07 +0100)]
http3: initial support for OpenSSL 3.2 QUIC stack
- HTTP/3 for curl using OpenSSL's own QUIC stack together
with nghttp3
- configure with `--with-openssl-quic` to enable curl to
build this. This requires the nghttp3 library
- implementation with the following restrictions:
* macOS has to use an unconnected UDP socket due to an
issue in OpenSSL's datagram implementation
See https://github.com/openssl/openssl/issues/23251
This makes connections to non-reponsive servers hang.
* GET requests will send the indicator that they have
no body in a separate QUIC packet. This may result
in processing delays or Transfer-Encodings on proxied
requests
* uploads that encounter blocks will use 100% cpu as
detection of these flow control issue is not working
(we have not figured out to pry that from OpenSSL).
Viktor Szakats [Sun, 21 Jan 2024 15:38:09 +0000 (15:38 +0000)]
cmake: fix `ENABLE_MANUAL` option
Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`.
Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the
option designed to enable building and using the built-in curl manual.
(`USE_MANUAL=ON` option worked for this instead, by accident).
Viktor Szakats [Tue, 16 Jan 2024 16:30:07 +0000 (16:30 +0000)]
mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls`
- Silence warning in mbedTLS v3.5.1 public headers:
```
./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: redundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls]
./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: previous declaration of 'psa_set_key_domain_parameters' was here
```
Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae785a513e157c
Ref: https://github.com/libssh2/libssh2/pull/1226
- Fix compiler warnings seen with gcc 9.2.0 + cmake unity:
```
./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read':
./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull-dereference]
189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &result);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write':
./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull-dereference]
168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, &result);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Daniel Stenberg [Wed, 17 Jan 2024 13:27:16 +0000 (14:27 +0100)]
docs: cleanup nroff format use
- remove use of .BI for code snippet
- stop using .br, just do a blank line
- remove use of .PP
- remove use for .sp
- remove backslash in .IP
- use .IP instead of .TP
Jay Satiro [Sun, 7 Jan 2024 05:07:55 +0000 (00:07 -0500)]
tool_getparam: stop supporting `@filename` style for --cookie
The `@filename` style was never documented for --cookie <data|filename>
but prior to this change curl would accept it anyway and always treat a
@ prefixed string as a filename.
That's a problem if the string also contains a = sign because then it is
documented to be interpreted as a cookie string and not a filename.
Example:
`--cookie @foo=bar`
Before: Interpreted as load cookies from filename foo=bar.
After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`).
Other curl options with a data/filename option-value use the `@filename`
to distinguish filenames which is probably how this happened. The
--cookie option has never been documented that way.
Stefan Eissing [Mon, 15 Jan 2024 12:02:34 +0000 (13:02 +0100)]
websockets: check for negative payload lengths
- in en- and decoding, check the websocket frame payload lengths for
negative values (from curl_off_t) and error the operation in that case
- add test 2307 to verify
projects / thirdparty / curl.git / log
