Jay Satiro [Thu, 4 Mar 2021 08:02:38 +0000 (03:02 -0500)]
schannel: Evaluate CURLOPT_SSL_OPTIONS via SSL_SET_OPTION macro
- Change use of those options from CURLOPT_SSL_OPTIONS that are not
already evaluated via SSL_SET_OPTION in schannel and secure transport
to use that instead of data->set.ssl.optname.
Example:
Evaluate SSL_SET_OPTION(no_revoke) instead of data->set.ssl.no_revoke.
This change is because options set via CURLOPT_SSL_OPTIONS
(data->set.ssl.optname) are separate from those set for HTTPS proxy via
CURLOPT_PROXY_SSL_OPTIONS (data->set.proxy_ssl.optname). The
SSL_SET_OPTION macro determines whether the connection is for HTTPS
proxy and based on that which option to evaluate.
Since neither Schannel nor Secure Transport backends currently support
HTTPS proxy in libcurl, this change is for posterity and has no other
effect.
Jay Satiro [Mon, 1 Mar 2021 08:20:58 +0000 (03:20 -0500)]
projects: Update VS projects for OpenSSL 1.1.x
- Update VS project templates to use the OpenSSL lib names and include
directories for OpenSSL 1.1.x.
This change means the VS project files will now build only with OpenSSL
1.1.x when an OpenSSL configuration is chosen. Prior to this change the
project files built only with OpenSSL 1.0.x (end-of-life) when an
OpenSSL configuration was chosen.
The template changes in this commit were made by script:
And since the output directory now contains the includes it's prepended:
..\..\..\..\..\openssl\build\Win{32,64}\VC{6..15}\{DLL,LIB}
{Debug,Release}\include
- Change build-openssl.bat to copy the build's include directory to the
output directory (as seen above).
Each build has its own opensslconf.h which is different so we can't just
include the source include directory any longer.
Note the include directory in the output directory is a full copy from
the build so technically we don't need to include the OpenSSL source
include directory in the template. However, I left it last in case the
user made a custom OpenSSL build using the old method which would put
opensslconf in the OpenSSL source include directory.
- Change build-openssl.bat to use a temporary install directory that is
different from the temporary build directory.
For OpenSSL 1.1.x the temporary paths must be separate not a descendant
of the other, otherwise pdb files will be lost between builds.
Jay Satiro [Thu, 25 Feb 2021 20:49:00 +0000 (15:49 -0500)]
doh: Inherit CURLOPT_STDERR from user's easy handle
Prior to this change if the user set their easy handle's error stream
to something other than stderr it was not inherited by the doh handles,
which meant that they would still write to the default standard error
stream (stderr) for verbose output.
unescaped is coming from Curl_urldecode and not a unicode conversion
function, so reclaiming its memory should be performed with a normal
call to free rather than curlx_unicodefree. In reality, this is the
same thing as curlx_unicodefree is implemented as a call to free but
that's not guaranteed to always hold. Using the curlx macro present
issues with memory debugging as well.
Closes #6671 Reviewed-by: Jay Satiro <raysatiro@yahoo.com> Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Jay Satiro [Sat, 27 Feb 2021 23:08:53 +0000 (18:08 -0500)]
tool_help: Increase space between option and description
- Increase the minimum number of spaces between the option and the
description from 1 to 2.
Before:
~~~
-u, --user <user:password> Server user and password
-A, --user-agent <name> Send User-Agent <name> to server
-v, --verbose Make the operation more talkative
-V, --version Show version number and quit
-w, --write-out <format> Use output FORMAT after completion
--xattr Store metadata in extended file attributes
~~~
After:
~~~
-u, --user <user:password> Server user and password
-A, --user-agent <name> Send User-Agent <name> to server
-v, --verbose Make the operation more talkative
-V, --version Show version number and quit
-w, --write-out <format> Use output FORMAT after completion
--xattr Store metadata in extended file attributes
~~~
openssl: remove get_ssl_version_txt in favor of SSL_get_version
openssl: use SSL_get_version to get connection protocol
Replace our bespoke get_ssl_version_txt in favor of SSL_get_version.
We can get rid of few lines of code, since SSL_get_version achieve
the exact same thing
Closes #6665 Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Signed-off-by: Jean-Philippe Menil <jpmenil@gmail.com>
Commit e06fa7462ac258c removed support for libgcrypt leaving only
support for nettle which has been the default crypto library in
GnuTLS for a long time. There were however a few conditionals on
USE_GNUTLS_NETTLE which cause compilation errors in the metalink
code (as it used the gcrypt fallback instead as a result). See the
below autobuild for an example of the error:
Previously only a single -b cookie parameter was supported with the last
one winning. This adds support for supplying multiple -b params to have
them serialized semicolon separated. Both cookiefiles and cookies can be
entered multiple times.
Closes #6649 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg [Mon, 22 Feb 2021 12:03:02 +0000 (13:03 +0100)]
multi: do once-per-transfer inits in before_perform in DID state
... since the state machine might go to RATELIMITING and then back to
PERFORMING doing once-per-transfer inits in that function is wrong and
it caused problems with receiving chunked HTTP and it set the
PRETRANSFER time much too often...
Jay Satiro [Fri, 19 Feb 2021 23:30:18 +0000 (18:30 -0500)]
test1188: Check for --fail HTTP status
- Change the test to check for curl error on HTTP 404 Not Found.
test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr".
Prior to this change it did that by specifying a non-existent host which
would cause an error. ISPs may hijack DNS and resolve non-existent hosts
so the test would not work if that was the case.
Jay Satiro [Wed, 17 Feb 2021 22:46:16 +0000 (17:46 -0500)]
memdebug: close debug logfile explicitly on exit
- Use atexit to register a dbg cleanup function that closes the logfile.
LeakSantizier (LSAN) calls _exit() instead of exit() when a leak is
detected on exit so the logfile must be closed explicitly or data could
be lost. Though _exit() does not call atexit handlers such as this,
LSAN's call to _exit() comes after the atexit handlers are called.
Prior to this change the logfile was not explicitly closed so it was
possible that if LSAN detected a leak and called _exit (which does
not flush or close files like exit) then the logfile could be missing
data. That could then cause curl's memanalyze to report false leaks
(eg a malloc was recorded to the logfile but the corresponding free was
discarded from the buffer instead of written to the logfile, then
memanalyze reports that as a leak).
Jay Satiro [Sat, 13 Feb 2021 05:51:28 +0000 (00:51 -0500)]
curl_multibyte: always return a heap-allocated copy of string
- Change the Windows char <-> UTF-8 conversion functions to return an
allocated copy of the passed in string instead of the original.
Prior to this change the curlx_convert_ functions would, as what I
assume was an optimization, not make a copy of the passed in string if
no conversion was required. No conversion is required in non-UNICODE
Windows builds since our tchar strings are type char and remain in
whatever the passed in encoding is, which is assumed to be UTF-8 but may
be other encoding.
In contrast the UNICODE Windows builds require conversion
(wchar <-> char) and do return a copy. That inconsistency could lead to
programming errors where the developer expects a copy, and does not
realize that won't happen in all cases.
Viktor Szakats [Fri, 19 Feb 2021 13:57:19 +0000 (13:57 +0000)]
http: add support to read and store the referrer header
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
attribute with the referrer (if there was any)
Viktor Szakats [Thu, 18 Feb 2021 11:18:36 +0000 (11:18 +0000)]
ci: stop building on freebsd-12-1
An updated freebsd-12-2 image was added a few months ago, and this
older one is consistently failing to go past `pkginstall`:
```
Newer FreeBSD version for package py37-mlt:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1202000
- running kernel: 1201000
Ignore the mismatch and continue? [Y/n]: pkg: repository FreeBSD contains packages for wrong OS version: FreeBSD:12:amd64
```
FreeBSD thread suggests that 12.1 is EOL, and best to avoid.
Daniel Stenberg [Thu, 18 Feb 2021 07:30:35 +0000 (08:30 +0100)]
test1188: change error from connect to resolve error
Using the %NOLISTENPORT to trigger a connection failure is somewhat
"risky" (since it isn't guaranteed to not be listened to) and caused
occasional CI problems. This fix changes the infused error to be a more
reliable one but still verifies the --write-out functionality properly -
which is the purpose of this test.
Daniel Stenberg [Wed, 17 Feb 2021 13:19:57 +0000 (14:19 +0100)]
wolfssl: don't store a NULL sessionid
This caused a memory leak as the session id cache entry was still
erroneously stored with a NULL sessionid and that would later be treated
as not needed to get freed.
Daniel Stenberg [Wed, 17 Feb 2021 10:53:32 +0000 (11:53 +0100)]
parse_proxy: fix a memory leak in the OOM path
Reported-by: Jay Satiro Reviewed-by: Jay Satiro Reviewed-by: Emil Engler
Closes #6614
Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541
Jay Satiro [Tue, 16 Feb 2021 22:13:22 +0000 (17:13 -0500)]
url: fix possible use-after-free in default protocol
Prior to this change if the user specified a default protocol and a
separately allocated non-absolute URL was used then it was freed
prematurely, before it was then used to make the replacement URL.
Daniel Stenberg [Tue, 16 Feb 2021 07:19:24 +0000 (08:19 +0100)]
multi: rename the multi transfer states
While working on documenting the states it dawned on me that step one is
to use more descriptive names on the states. This also changes prefix on
the states to make them shorter in the source.
State names NOT ending with *ing are transitional ones.
Viktor Szakats [Tue, 16 Feb 2021 10:19:37 +0000 (10:19 +0000)]
http: do not add a referrer header with empty value
Previously an empty 'Referer:' header was added to the HTTP request when
passing `--referer ';auto'` or `--referer ''` on the command-line. This
patch makes `--referer` work like `--header 'Referer:'` and will only add
the header if it has a non-zero length value.
Daniel Stenberg [Mon, 15 Feb 2021 09:15:46 +0000 (10:15 +0100)]
lib: remove 'conn->data' completely
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.
Jay Satiro [Mon, 15 Feb 2021 05:25:59 +0000 (00:25 -0500)]
doh: Fix sharing user's resolve list with DOH handles
- Share the shared object from the user's easy handle with the DOH
handles.
Prior to this change if the user had set a shared object with shared
cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't
used by any associated DOH handles, since they used the multi's default
hostcache.
This change means all the handles now use the same hostcache, which is
either the shared hostcache from the user created shared object if it
exists or if not then the multi's default hostcache.
Daniel Stenberg [Mon, 15 Feb 2021 07:35:32 +0000 (08:35 +0100)]
http2: remove conn->data use
... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.
Daniel Stenberg [Mon, 25 Jan 2021 13:18:31 +0000 (14:18 +0100)]
openssl: remove conn->data use
We still make the trace callback function get the connectdata struct
passed to it, since the callback is anchored on the connection.
Repeatedly updating the callback pointer to set 'data' with
SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because
there might already be messages in the queue with the old pointer.
This code therefore makes sure to set the "logger" handle before using
OpenSSL calls so that the right easy handle gets used for tracing.
Jay Satiro [Thu, 11 Feb 2021 22:09:59 +0000 (17:09 -0500)]
doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their respective counterparts.
- New curl tool options --doh-insecure and --doh-cert-status do the same
as their respective counterparts.
Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.
Jay Satiro [Sat, 13 Feb 2021 07:45:33 +0000 (02:45 -0500)]
hostip: fix crash in sync resolver builds that use DOH
- Guard some Curl_async accesses with USE_CURL_ASYNC instead of
!CURLRES_SYNCH.
This is another follow-up to 8335c64 which moved the async struct from
the connectdata struct into the Curl_easy struct. A previous follow-up 6cd167a fixed building for sync resolver by guarding some async struct
accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS)
is available as an asynchronous secondary resolver the async struct may
be used even when libcurl is built for the sync resolver. That means
that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time.
Daniel Stenberg [Fri, 12 Feb 2021 09:27:42 +0000 (10:27 +0100)]
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.
The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*
Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.
projects / thirdparty / curl.git / log
