Stefan Eissing [Mon, 17 Jul 2023 10:38:28 +0000 (12:38 +0200)]
http2: treat initial SETTINGS as a WINDOW_UPDATE
- refs #11426 where spurious stalls on large POST requests
are reported
- the issue seems to involve the following
* first stream on connection adds up to 64KB of POST
data, which is the max default HTTP/2 stream window size
transfer is set to HOLD
* initial SETTINGS from server arrive, enlarging the stream
window. But no WINDOW_UPDATE is received.
* curl stalls
- the fix un-HOLDs a stream on receiving SETTINGS, not
relying on a WINDOW_UPDATE from lazy servers
cf-socket: don't bypass fclosesocket callback if cancelled before connect
After upgrading to 8.1.2 from 7.84.0, I found that sockets were being
closed without calling the fclosesocket callback if a request was
cancelled after the associated socket was created, but before the socket
was connected. This lead to an imbalance of fopensocket & fclosesocket
callbacks, causing problems with a custom event loop integration using
the multi-API.
This was caused by cf_socket_close() calling sclose() directly instead
of calling socket_close() if the socket was not active. For regular TCP
client connections, the socket is activated by cf_socket_active(), which
is only called when the socket completes the connect.
As far as I can tell, this issue has existed since 7.88.0. That is,
since the code in question was introduced by:
commit 71b7e0161032927cdfb4e75ea40f65b8898b3956
Author: Stefan Eissing <stefan@eissing.org>
Date: Fri Dec 30 09:14:55 2022 +0100
Stefan Eissing [Thu, 13 Jul 2023 09:40:09 +0000 (11:40 +0200)]
CI: brew fix for openssl in default path
If brew install/update links openssl into /usr/local, it will be found
before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of
that by unlinking the keg.
Stefan Eissing [Mon, 10 Jul 2023 07:39:19 +0000 (09:39 +0200)]
macOS: fix taget detection
- TARGET_OS_OSX is not always defined on macOS
- this leads to missing symbol Curl_macos_init()
- TargetConditionals.h seems to define these only when
dynamic targets are enabled (somewhere?)
- this PR fixes that on my macOS 13.4.1
- I have no clue why CI builds worked without it
Stan Hu [Mon, 5 Jun 2023 17:05:48 +0000 (10:05 -0700)]
hostip.c: Move macOS-specific calls into global init call
https://github.com/curl/curl/pull/7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.
However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.
In addition, this change is beneficial because it:
1. Avoids extra macOS system calls for every IP lookup.
2. Consolidates macOS-specific initialization in a separate file.
Stefan Eissing [Tue, 27 Jun 2023 10:06:21 +0000 (12:06 +0200)]
http3/ngtcp2: upload EAGAIN handling
- refs #11389 where IDLE timeouts on upload are reported
- reword ngtcp2 expiry handling to apply to both send+recv
calls into the filter
- EAGAIN uploads similar to the recent changes in HTTP/2, e.g.
report success only when send data was ACKed.
- HOLD sending of EAGAINed uploads to avoid cpu busy loops
- rename internal function for consistency with HTTP/2
implementation
Stefan Eissing [Fri, 7 Jul 2023 08:52:05 +0000 (10:52 +0200)]
http2: raise header limitations above and beyond
- not quite to infinity
- rewrote the implementation of our internal HTTP/1.x request
parsing to work with very large lines using dynbufs.
- new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which
is also the limit of curl's general HTTP request processing.
Stefan Eissing [Mon, 26 Jun 2023 07:03:47 +0000 (09:03 +0200)]
http2: fix crash in handling stream weights
- Delay the priority handling until the stream has been opened.
- Add test2404 to reproduce and verify.
Weights may change "on the run", which is why there are checks in
general egress handling. These must not trigger when the stream has not
been opened yet.
Dan Fandrich [Fri, 23 Jun 2023 19:51:41 +0000 (12:51 -0700)]
CI: enable parallel make in more builds
Most CI services provide at least two cores, so enable parallel make
jobs to take advantage of that for builds. Some dependencies aren't safe
to build in parallel so leave those as-is. Also, rename a few
workflows to eliminate duplicate names and provide a better idea what
they're about.
Daniel Stenberg [Thu, 22 Jun 2023 12:34:49 +0000 (14:34 +0200)]
configure: add check for ldap_init_fd
... as otherwise the configure script will say it is OpenLDAP in the
summary, but not set the USE_OPENLDAP define, therefor not using the
intended OpenLDAP code paths.
Regression since 4d7385446 (7.85.0)
Fixes #11372
Closes #11374 Reported-by: vlkl-sap on github
Michał Petryka [Thu, 22 Jun 2023 19:30:43 +0000 (21:30 +0200)]
cmake: stop CMake from quietly ignoring missing Brotli
The CMake project was set to `QUIET` for Brotli instead of
`REQUIRED`. This makes builds unexpectedly ignore missing Brotli even
when `CURL_BROTLI` is enabled.
Stefan Eissing [Thu, 22 Jun 2023 09:01:53 +0000 (11:01 +0200)]
http2: h2 and h2-PROXY connection alive check fixes
- fix HTTP/2 check to not declare a connection dead when
the read attempt results in EAGAIN
- add H2-PROXY alive check as for HTTP/2 that was missing
and is needed
- add attach/detach around Curl_conn_is_alive() and remove
these in filter methods
- add checks for number of connections used in some test_10
proxy tunneling tests
Stefan Eissing [Wed, 21 Jun 2023 13:59:42 +0000 (15:59 +0200)]
http2: error stream resets with code CURLE_HTTP2_STREAM
- refs #11357, where it was reported that HTTP/1.1 downgrades
no longer works
- fixed with suggested change
- added test_05_03 and a new handler in the curltest module
to reproduce that downgrades work
Fixes #11357
Closes #11362 Reported-by: Jay Satiro
Emanuele Torre [Thu, 22 Jun 2023 11:18:30 +0000 (13:18 +0200)]
quote.d: fix indentation of generated paragraphs
quote.d was missing a .IP at the end which caused the paragraphs
generated for See-also, Multi, and Example to not be indented correctly.
I also remove a redundant "This option can be used multiple times.", and
replaced .IP "item" with .TP .B "item" to make more clear which lines
are part of the list of commands and which aren't.
Dan Fandrich [Wed, 21 Jun 2023 19:43:48 +0000 (12:43 -0700)]
runtests: use more consistent failure lines
After a test failure log a consistent log message to make it easier to
parse the log file. Also, log a consistent message with "ignored" for
failures that cause the test to be not considered at all. These should
perhaps be counted in the skipped category, but this commit does not
change that behaviour.
Dan Fandrich [Wed, 21 Jun 2023 19:39:51 +0000 (12:39 -0700)]
runtests: consistently write the test check summary block
The memory check character was erroneously omitted if the memory
checking file was not available for some reason, making the block of
characters an inconsistent length.
Stefan Eissing [Mon, 19 Jun 2023 10:11:53 +0000 (12:11 +0200)]
hyper: unslow
- refs #11203 where hyper was reported as being slow
- fixes hyper_executor_poll to loop until it is out of
tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issues/3237
- added a fix in hyper io handling for detecting EAGAIN
- added some debug logs to see IO results
- pytest http/1.1 test cases pass
- pytest h2 test cases fail on connection reuse. HTTP/2
connection reuse does not seem to work. Hyper submits
a request on a reused connection, curl's IO works and
thereafter hyper declares `Hyper: [1] operation was canceled: connection closed`
on stderr without any error being logged before.
Fixes #11203 Reported-by: Gisle Vanem Advised-by: Sean McArthur
Closes #11344
Stefan Eissing [Mon, 19 Jun 2023 08:53:23 +0000 (10:53 +0200)]
HTTP/2: upload handling fixes
- fixes #11242 where 100% CPU on uploads was reported
- fixes possible stalls on last part of a request body when
that information could not be fully send on the connection
due to an EAGAIN
- applies the same EGAIN handling to HTTP/2 proxying
Daniel Stenberg [Mon, 19 Jun 2023 07:19:56 +0000 (09:19 +0200)]
example/opensslthreadlock: remove
This shows how to setup OpenSSL mutex callbacks, but this is not
necessary since OpenSSL 1.1.0 - meaning that no currently supported
OpenSSL version requires this anymore
Dan Fandrich [Fri, 16 Jun 2023 03:10:24 +0000 (20:10 -0700)]
runtests: better detect and handle pipe errors in the controller
Errors reading and writing to the pipes are now better detected and
propagated up to the main test loop so it can be cleanly shut down. Such
errors are usually due to a runner dying so it doesn't make much sense
to try to continue the test run.
Dan Fandrich [Wed, 14 Jun 2023 22:28:39 +0000 (15:28 -0700)]
runtests: cleanly abort the runner if the controller dies
If the controller dies unexpectedly, have the runner stop its servers
and exit cleanly. Otherwise, the orphaned servers will stay running in
the background.
Dan Fandrich [Wed, 14 Jun 2023 22:27:07 +0000 (15:27 -0700)]
runtests: improve error logging
Give more information about test harness error conditions to help figure
out what might be wrong. Print some internal test state when SIGUSR1 is
sent to runtests.pl.
Dan Fandrich [Wed, 14 Jun 2023 22:18:36 +0000 (15:18 -0700)]
runtests: better handle ^C during slow tests
Since the SIGINT handler now just sets a flag that must be checked in the
main controller loop, make sure that runs periodically. Rather than
blocking on a response from a test runner near the end of the test run,
add a short timeout to allow it.
Dan Fandrich [Wed, 14 Jun 2023 21:19:26 +0000 (14:19 -0700)]
tests: improve reliability of TFTP tests
Stop checking the timeout used by the client under test (for most
tests). The timeout will change if the TFTP test server is slow (such as
happens on an overprovisioned CI server) because the client will retry
and reduce its timeout, and the actual value is not important for most
tests.
test285 is changed a different way, by increasing the connect timeout.
This improves test coverage by allowing the changed timeout value to be
checked, but improves reliability with a carefully-chosen timeout that
not only allows twice the time to respond as before, but also allows
several retries before the client will change its timeout value.
Daniel Stenberg [Sat, 17 Jun 2023 22:18:12 +0000 (00:18 +0200)]
cf-socket: skip getpeername()/getsockname for TFTP
Since the socket is not connected then the call fails. When the call
fails, failf() is called to write an error message that is then
surviving and is returned when the *real* error occurs later. The
earlier, incorrect, error therefore hides the actual error message.
This could be seen in stderr for test 1007
Test 1007 has now been extended to verify the stderr message.
Jay Satiro [Fri, 16 Jun 2023 03:02:48 +0000 (23:02 -0400)]
lib: fix some format specifiers
- Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some
curl_off_t variables.
- Use %zu where %zd was erroneously used for some size_t variables.
Prior to this change some of the Windows CI tests were failing because
in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t.
When %zd was used for some curl_off_t variables then only the lower
32-bits was read and the upper 32-bits would be read for part or all of
the next specifier.
Chris Talbot [Mon, 5 Dec 2022 23:05:01 +0000 (18:05 -0500)]
imap: Provide method to disable SASL if it is advertised
- Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext
LOGIN over SASL auth.
Prior to this change there was no method to be able to fall back to
LOGIN if an IMAP server advertises SASL capabilities. However, this may
be desirable for e.g. a misconfigured server.
Marcel Raad [Tue, 13 Jun 2023 09:13:59 +0000 (11:13 +0200)]
fopen: fix conversion warning on 32-bit Android
When building for 32-bit ARM or x86 Android, `st_mode` is defined as
`unsigned int` instead of `mode_t`, resulting in a
-Wimplicit-int-conversion clang warning because `mode_t` is
`unsigned short`. Add a cast to silence the warning.
Marcel Raad [Mon, 12 Jun 2023 14:02:00 +0000 (16:02 +0200)]
http2: fix variable type
`max_recv_speed` is `curl_off_t`, so using `size_t` might result in
-Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD
ARM autobuilds.
projects / thirdparty / curl.git / log
