Patrick Monnerat [Tue, 23 Mar 2021 01:02:18 +0000 (02:02 +0100)]
bufref: buffer reference support
A struct bufref holds a buffer pointer, a data size and a destructor.
When freed or its contents are changed, the previous buffer is implicitly
released by the associated destructor. The data size, although not used
internally, allows binary data support.
A unit test checks its handling methods: test 1661
Marc Hoersken [Tue, 15 Dec 2020 06:22:13 +0000 (07:22 +0100)]
multi: fix slow write/upload performance on Windows
Reset FD_WRITE by sending zero bytes which is permissible
and will be treated by implementations as successful send.
Without this we won't be notified in case a socket is still
writable if we already received such a notification and did
not send any data afterwards on the socket. This would lead
to waiting forever on a writable socket being writable again.
Assisted-by: Tommy Odom Reviewed-by: Jay Satiro Reviewed-by: Marcel Raad Tested-by: tmkk on github
Bug: #6146
Closes #6245
Marc Hoersken [Sun, 6 Dec 2020 16:28:26 +0000 (17:28 +0100)]
multi: reduce Win32 API calls to improve performance
1. Consolidate pre-checks into a single Curl_poll call:
This is an attempt to restructure the code in Curl_multi_wait
in such a way that less syscalls are made by removing individual
calls to Curl_socket_check via SOCKET_READABLE/SOCKET_WRITABLE.
2. Avoid resetting the WinSock event multiple times:
We finally call WSAResetEvent anyway, so specifying it as
an optional parameter to WSAEnumNetworkEvents is redundant.
3. Wakeup directly in case no sockets are being monitoring:
Fix the WinSock based implementation to skip extra waiting by
not sleeping in case no sockets are to be waited on and just
the WinSock event is being monitored for wakeup functionality.
Assisted-by: Tommy Odom Reviewed-by: Jay Satiro Reviewed-by: Marcel Raad
Bug: #6146
Closes #6245
Add parser for CURLOPT_SSL_CIPHER_LIST option for Secure Transport (ST)
back-end. Similar to NSS and GSKit back-ends, new code parses string
value and configures ST library to use those ciphers for communication.
Create cipher spec data structure and initialize the array of specs with
cipher number, name, alias, and 'weak' flag.
Mark triple-DES ciphers as 'weak', and exclude them from the default
ciphers list.
Add cipher names to the `cipherlist` map, based on the list of ciphers
implemented by the NSS in the source code file
https://github.com/nss-dev/nss/blob/master/lib/ssl/sslenum.c
Daniel Stenberg [Tue, 20 Apr 2021 07:21:09 +0000 (09:21 +0200)]
curl_setup: provide the shutdown flags wider
By using #ifdef on the symbol names to work on anything that don't
provide them. SCO OpenServer 5.0.7, sys/socket.h does not define either
SHUT_RDWR, SHUT_RD, and SHUT_WR.
Reported-by: Kevin R. Bulgrien
Bug: https://curl.se/mail/lib-2021-04/0073.html
Closes #6925
Daniel Stenberg [Mon, 19 Apr 2021 12:49:21 +0000 (14:49 +0200)]
multi: don't close connection HTTP_1_1_REQUIRED
The ConnectionExists() function will note that the new transfer wants
less then h2 and that it can't multiplex it and therefor opt to open a
new connection instead.
Daniel Stenberg [Mon, 19 Apr 2021 11:15:05 +0000 (13:15 +0200)]
http2: move the stream error field to the per-transfer storage
Storing a stream error in the per-connection struct was an error that lead to
race conditions as subsequent stream handling could overwrite the error code
before it was used for the stream with the actual problem.
Jay Satiro [Tue, 20 Apr 2021 05:51:56 +0000 (01:51 -0400)]
tool_operate: don't discard failed parallel transfer result
- Save a parallel transfer's result code only when it fails and the
transfer is not being retried.
Prior to this change the result code was always set which meant that a
failed result could be erroneously discarded if a different transfer
later had a successful result (CURLE_OK).
UTM parameters leak referrer and various marketing/tracking information
even if these would normally be stripped by website or client policy.
This link also works fine without them. Also took the opportunity to
update the URL to the one pointed to by the previous one via permanent
redirect.
Daniel Stenberg [Mon, 19 Apr 2021 06:34:52 +0000 (08:34 +0200)]
urlapi: "normalize" numerical IPv4 host names
When the host name in a URL is given as an IPv4 numerical address, the
address can be specified with dotted numericals in four different ways:
a32, a.b24, a.b.c16 or a.b.c.d and each part can be specified in
decimal, octal (0-prefixed) or hexadecimal (0x-prefixed).
Instead of passing on the name as-is and leaving the handling to the
underlying name functions, which made them not work with c-ares but work
with getaddrinfo, this change now makes the curl URL API itself detect
and "normalize" host names specified as IPv4 numericals.
The WHATWG URL Spec says this is an okay way to specify a host name in a
URL. RFC 3896 does not allow them, but curl didn't prevent them before
and it seems other RFC 3896-using tools have not either. Host names used
like this are widely supported by other tools as well due to the
handling being done by getaddrinfo and friends.
I decided to add the functionality into the URL API itself so that all
users of these functions get the benefits, when for example wanting to
compare two URLs. Also, it makes curl built to use c-ares now support
them as well and make curl builds more consistent.
The normalization makes HTTPS and virtual hosted HTTP work fine even
when curl gets the address specified using one of the "obscure" formats.
Daniel Stenberg [Sat, 17 Apr 2021 17:00:15 +0000 (19:00 +0200)]
libssh: fix "empty expression statement has no effect" warnings
... by fixing macros to do-while constructs and moving out the calls to
"break" outside of the actual macro. It also fixes the problem where the
macro was used witin a loop and the break didn't do right.
Daniel Stenberg [Mon, 22 Mar 2021 12:39:37 +0000 (13:39 +0100)]
vtls: refuse setting any SSL version
... previously they were supported if a TLS library would (unexpectedly)
still support them, but from this change they will be refused already in
curl_easy_setopt(). SSLv2 and SSLv3 have been known to be insecure for
many years now.
Daniel Stenberg [Mon, 22 Mar 2021 10:55:27 +0000 (11:55 +0100)]
curl: ignore options asking for SSLv2 or SSLv3
Instead output a warning about it and continue with the defaults.
These SSL versions are typically not supported by the TLS libraries since a
long time back already since they are inherently insecure and broken. Asking
for them to be used will just cause an error to be returned slightly later.
In the unlikely event that a user's TLS library actually still supports these
protocol versions, this change might make the request a little less insecure.
Javier Blazquez [Wed, 14 Apr 2021 21:55:12 +0000 (14:55 -0700)]
rustls: only return CURLE_AGAIN when TLS session is fully drained
The code in cr_recv was returning prematurely as soon as the socket
reported no more data to read. However, this could be leaving some
unread plaintext data in the rustls session from a previous call,
causing causing the transfer to hang if the socket never receives
further data.
We need to ensure that the session is fully drained of plaintext data
before returning CURLE_AGAIN to the caller.
examples/hiperfifo.c: check event_initialized before delete
If event_del is called with the event struct (still) zeroed out, a
segmentation fault may occur. event_initialized checks whether the
event struct is nonzero.
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.
The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.
Tests involving NTLM are adjusted to this new behavior.
Michael Forney [Wed, 7 Apr 2021 07:32:47 +0000 (00:32 -0700)]
configure: include <time.h> unconditionally
In 2682e5f5, several instances of AC_HEADER_TIME were removed since
it is a deprecated autoconf macro. However, this was the macro that
defined TIME_WITH_SYS_TIME, which was used to indicate that <time.h>
can be included alongside <sys/time.h>. TIME_WITH_SYS_TIME is still
used in the configure test body and since it is no longer defined,
<time.h> is *not* included on systems that have <sys/time.h>.
In particular, at least on musl libc and glibc, <sys/time.h> does
not implicitly include <time.h> and does not declare clock_gettime,
gmtime_r, or localtime_r. This causes configure to fail to detect
those functions.
The AC_HEADER_TIME macro deprecation text says
> All current systems provide time.h; it need not be checked for.
> Not all systems provide sys/time.h, but those that do, all allow
> you to include it and time.h simultaneously.
So, to fix this issue, simply include <time.h> unconditionally when
testing for time-related functions and in libcurl, and don't bother
checking for it.
tool_progress: Fix progress meter in parallel mode
Make sure the total amount of DL/UL bytes are counted before the
transfer finalizes. Otherwise if a transfer finishes too quick, its
total numbers are not added, and results in a DL%/UL% that goes above
100%.
Detail:
progress_meter() is called periodically, and it may not catch a
transfer's total bytes if the value was unknown during the last call,
and the transfer is finished and deleted (i.e., lost) during the next
call.
Emil Engler [Thu, 1 Apr 2021 16:24:53 +0000 (18:24 +0200)]
libssh: get rid of PATH_MAX
This removes the last occurrence of PATH_MAX inside our libssh
implementation by calculating the path length from the string length of
the two components.
After 957bc1881e686f9714c4e6a01bf33535091f0e21, we no longer compute an
expected_size for directories. This has the upshot that when we compare
even an empty Range with the available size, we fail.
This brings back the previous behaviour, which was to succeed, but with
empty content. This also removes the "Accept-ranges: bytes" header,
which is nonsensical on directories.
Jay Satiro [Thu, 1 Apr 2021 19:15:01 +0000 (15:15 -0400)]
hostip: Fix for builds that disable all asynchronous DNS
- Define Curl_resolver_error function only when USE_CURL_ASYNC.
Prior to this change building curl without an asynchronous resolver
backend (c-ares or threaded) and without DoH (DNS-over-HTTPS, which is
also asynchronous but independent of resolver backend) would cause a
build error since Curl_resolver_error is called by and evaluates
variables only available in asynchronous builds.
Daniel Stenberg [Fri, 26 Mar 2021 13:25:45 +0000 (14:25 +0100)]
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.
Jay Satiro [Fri, 26 Mar 2021 07:14:28 +0000 (03:14 -0400)]
KNOWN_BUGS: Update 11.9 - DoH option inheritance
- Add description: Explain that some options aren't inherited because
they are not relevant for the DoH SSL connections or may result in
unexpected behavior.
- Remove the reference to #4578 (SSL verify options not inherited) since
that was fixed by #6597 (separate DoH-specific options for verify).
- Explain that DoH-specific options (those created by #6597) are
available: CURLOPT_DOH_SSL_VERIFYHOST, CURLOPT_DOH_SSL_VERIFYPEER and
CURLOPT_DOH_SSL_VERIFYSTATUS.
- Add a reference to #6605 and explain that the user's debug function is
not inherited because it would be unexpected to pass internal handles
(ie DoH handles) to the user's callback.
projects / thirdparty / curl.git / log
