Viktor Szakats [Thu, 22 May 2025 07:35:25 +0000 (09:35 +0200)]
lib3026: drop DLL pre-load perf mitigation for old mingw
curl no longer supports old/legacy/classic mingw.
This mitigation was addressing slow perf seen in CI with old mingw.
The slow perf is not seen in current CI with supported compilers.
Remove the duplicate DLL load function from libtest. It's no longer
used after this patch.
Current CI run times for test3026 on GHA/windows:
```
test 3026...[curl_global_init thread-safety]
# mingw, CM clang-x86_64 gnutls libssh
-------e--- OK (1715 out of 1738, remaining: 00:02, took 0.196s, duration: 02:55)
# dl-mingw, CM 9.5.0-x86_64 schannel
-------e--- OK (1554 out of 1577, remaining: 00:02, took 0.217s, duration: 02:29)
# msvc, CM x64-windows schannel +examples
-------e--- OK (1578 out of 1601, remaining: 00:02, took 0.205s, duration: 02:50)
```
Viktor Szakats [Thu, 22 May 2025 10:16:11 +0000 (12:16 +0200)]
windows: fix builds targeting WinXP, test it in CI
- appveyor: make a job target Windows XP.
- examples/block_ip: force this specific example to target Vista to make
it compile when building curl for Windows XP. Fixing:
```
docs\examples\block_ip.c(157): warning C4013: 'inet_pton' undefined; assuming extern returning int
docs\examples\block_ip.c(272): warning C4013: 'inet_ntop' undefined; assuming extern returning int
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/52102142/job/2ajdluhc20r4gmmw#L530
Matt Jolly [Wed, 21 May 2025 10:30:11 +0000 (20:30 +1000)]
configure: suppress command not found for brew
42331cb48a1f66efaa0920ee8ccba5a74d67de27 made configure checks more
CMake-like, but now calls `brew` resulting in "command not found" in
the configure output if the package is not installed.
Redirect stderr to suppress this; it's not an issue for us if `brew`
isn't found.
Daniel Stenberg [Wed, 21 May 2025 06:24:39 +0000 (08:24 +0200)]
http_aws_sigv4: avoid risk of overflowed constant
- Simplify canon_query() a bit. Avoid unconditionally using length -1
where length risks being zero at times. Pointed out by Coverity.
- Fix indent errors
- narrow some variable scopes
- fix keywords in tests
Stefan Eissing [Mon, 19 May 2025 10:25:58 +0000 (12:25 +0200)]
dns_entry: move from conn to data->state
The `struct Curl_dns_entry *` used to established a connection
do not have the connection's lifetime, but the transfer's lifetime
(of the transfer that initiates the connect).
`Curl_dns_entry *` is reference counted with the "dns cache". That
cache might be owned by the multi or the transfer's share. In the
share, the reference count needs updating under lock.
Therefore, the dns entry can only be kept *and* released using the
same transfer it was initially looked up from. But a connection is
often discarded using another transfer.
So far, the problem of this has been avoided in clearing the connection's
dns entries in the "multi_don()" handling. So, connections had NULL
dns entries after the initial transfers and its connect had been handled.
Keeping the dns entries in data->state seems therefore a better choice.
Also: remove the `struct Curl_dns_entry *` from the connect filters
contexts. Use `data->state.dns` every time instead and fail correctly
when not present and needed.
Viktor Szakats [Sat, 17 May 2025 00:02:23 +0000 (02:02 +0200)]
checksrc: avoid extra runs in CI, enable more check locally, fix fallouts
To avoid redundant work in CI and to avoid a single checksrc issue make
all autotools jobs fail. After this patch checksrc issues make fail
the checksrc job, the `dist / verify-out-of-tree-autotools-debug`,
`dist / maketgz-and-verify-in-tree` jobs and the fuzzer job (if run).
Of these, the `dist` jobs replicate local builds, also testing the build
logic.
Also add a script to check the complete local repository, optionally
with the build tree to verify generated C files.
Also:
- automatically run checksrc in subdirectories having a `checksrc`
target. (examples, OS400, tests http/client, unit and tunit)
- tests/libtest: make sure to run `checksrc` on generated `lib1521.c`.
(requires in-tree autotools build.)
- tests: run `checksrc` on targets also for non-`DEBUGBUILD`
builds. It ensures to check `lib1521.c` in CI via job
`dist / maketgz-and-verify-in-tree`.
- src: drop redundant `$(builddir)` in autotools builds.
- scripts: add `checksrc-all.sh` script to check all C sources and
the build directory as an option.
- use the above from CI, also make it verify all generated sources.
- silence `checksrc` issues in generated C sources.
- checksrc: add `-v` option to enable verbose mode.
- checksrc: make verbose mode show checked filename and fix to only
return error on failure.
- make sure that generated C files pass `checksrc`.
Viktor Szakats [Thu, 15 May 2025 20:11:06 +0000 (22:11 +0200)]
pytest: enable in 5 GHA jobs, fix fallouts
They take about 4.5 minutes of CI time in GHA/macos.
Also:
- autotools: improve `caddy`, `vsftpd` detection.
Bringing it closer to cmake.
- autotools: fix `--with-test-caddy=no`, `--with-test-vsftps=no`,
`--with-test-nghttpx=no` options.
- cmake: sync `nghttpx` default with autotools.
- pytest: disable failing mbedTLS tests on macOS CI.
- pytest: disable failing earlydata tests on macOS CI.
- GHA/macos: keep vsftpd pytests disabled due to lengthy run times.
- pytest: fix test_05_04 for LibreSSL. Ref: #17367 Authored-by: Stefan Eissing
Remaining issues:
- some unidentified tests taking a long time with mbedTLS:
`================= 462 passed, 278 skipped in 347.93s (0:05:47) =================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374999041#step:17:1536
Workaround: not enabling pytest for mbedTLS jobs
- 17 FTP tests taking a long time (affecting all TLS backends):
without vsftpd:
`====================== 496 passed, 244 skipped in 56.15s =======================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374998230#step:17:1536
with vsftpd:
`================= 513 passed, 227 skipped in 409.37s (0:06:49) =================`
Ref: https://github.com/curl/curl/actions/runs/15073678568/job/42376039672?pr=17362#step:17:1537
Workaround: force-disable vsftpd.
- 100 tests failing with SecureTransport. Let's ignore that due to imminent deprecation.
Ref: https://github.com/curl/curl/actions/runs/15055652333/job/42320873732#step:17:15362
Daniel Stenberg [Fri, 16 May 2025 21:18:34 +0000 (23:18 +0200)]
formdata: cleanups
- use memchr() instead of for() loop
- add and use free_formlist() instead of duplicate code
- shorten some variable names
- reduce flag struct field from 'long' to 'unsigned char'
- pass in struct pointer, not individual fields, to addhttppost()
Stefan Eissing [Fri, 16 May 2025 08:38:13 +0000 (10:38 +0200)]
asnyc-thrdd: explain how this is okay with a comment
Coverity assess correctly that a variable write under mutex lock could
overwrite values from another thread - if the function were ever called
from multiple thread for the same transfer - which it is not.
Stefan Eissing [Mon, 12 May 2025 13:49:49 +0000 (15:49 +0200)]
pytest-xdist: pytest in parallel
Require now pytest-xdist from tests/http/requirements.txt and
run pytest in 'auto' parallel mode (counts cpu cores).
For CI runs, set the worker count to 4, overriding the
core count of 2 exposed in the images.
- use Filelock to generate allocated ports at start for all
workers and have subsequent workers just read the file and
take the ports for their slot
- make httpd config clearing a function fixture so every test
starts with a clean httpd config
- have fixture `configures_httpd` as parameter of test cases
that configure httpd anyway, saving one reload
- add pytest-xdist and filelock to required pyhton modules
- add installs to ruff CI
- give live checks waiting for a server to start up longer time
- add fixtures to tests that rely on a server
- do not stop servers unnecessarily. failures may not start them
properly again, leading to unexpected fails in whatever follows
- add a https: port to httpd that is *not* back by QUIC to allow
failover tests without stopping the QUIC server
Daniel Stenberg [Thu, 15 May 2025 07:49:47 +0000 (09:49 +0200)]
tool_operate: move config2setopts to separate file, split into subs
To decrease size and complexity. Complexity taken down from 190 to 80.
Bonus:
- remove leftover HTTP/0.9 warning never triggered since hyper was dropped
- remove the ftp-skip-ip option unless FTP is used
- only set HTTP options if HTTP(S) is used
- remove use of the pointless SETOPT_CHECK macro
Side-effect:
- The order of the options in --libcurl is modified
Daniel Stenberg [Mon, 12 May 2025 10:55:20 +0000 (12:55 +0200)]
CONTRIBUTE: add project guidelines for AI use
Feedback-by: Daniel Fosco Feedback-by: Jimmy Sjölund Feedback-by: Christoph Jabs Feedback-by: Manuel Strehl Feedback-by: Dan Fandrich Feedback-by: Sarah Gooding
Closes #17325
Stefan Eissing [Thu, 15 May 2025 08:48:12 +0000 (10:48 +0200)]
ngtcp2: clarify ignoring of result
In shutdown, the result of a bufq_write() is intentionally ignored, but
it was not obvious why. Add a (void) cast to declare intent and a
comment explaining why.
Dan Fandrich [Tue, 13 May 2025 23:14:57 +0000 (16:14 -0700)]
scripts: move the ruff linter command into a script
This makes it easier for users to run it with the correct options
locally. Also, update the ruff, pytype and cmakelint versions to the
latest current ones.
Viktor Szakats [Tue, 13 May 2025 09:11:55 +0000 (11:11 +0200)]
spacecheck.pl: drop more exceptions
- replace ß (scharfes S) with links.
- replace § (section sign) with links.
- replace 🙏 emoji with `:pray:`.
Supported by GitHub, Forgejo/Gitea and most likely GitLab.
- docs/libcurl/curl_mprintf.md: replace Unicode ± with `{+|-}`.
- docs/CIPHERS.md: URL encode Unicode in URLs.
- lib1560: use hex encoding in `räksmörgås.se`.
- unit1307: use hex encoding in `Lindmätarv`.
- drop LATIN SMALL LETTER A WITH ACUTE exception.
No longer appears in tests.
This leaves the single character exception: `ö`
And file exceptions holding contributor names.
Daniel Stenberg [Tue, 13 May 2025 09:25:43 +0000 (11:25 +0200)]
setopt: provide info for CURLE_BAD_FUNCTION_ARGUMENT
If CURLE_BAD_FUNCTION_ARGUMENT is returned and failf() has not provided
any details, this adds a generic error string that includes the option
number.
This helps debugging for example the curl tool which does a lot of
setopt calls and in reading post fact logs it is not always easy to tell
exactly which call that failed.
Daniel Stenberg [Mon, 12 May 2025 20:48:47 +0000 (22:48 +0200)]
tests/server: stop using libcurl string comparisons
Further untangle the test server code from curl code. While the string
comparison functions are available in the libcurl API, the tests servers
don't link with libcurl. Use native functions instead.
Dan Fandrich [Wed, 7 May 2025 06:39:22 +0000 (23:39 -0700)]
test1621: Improve stripcredentials tests
- add more unusual input cases
- add a valid non-http protocol
- fix tests so an input that should be stripped but isn't is a failure
- fix detection of when stripcredentials() would be available to test
- avoid using a NULL pointer
Viktor Szakats [Fri, 9 May 2025 13:18:51 +0000 (15:18 +0200)]
windows: fix to preserve error code in `curlx_winapi_strerror()`
Drop the interim macro `PRESERVE_WINDOWS_ERROR_CODE` and always preserve
error code for `_WIN32`. To make sure this is always done in
`curlx_winapi_strerror()`.