Viktor Szakats [Sun, 7 Jul 2024 12:30:57 +0000 (14:30 +0200)]
tests: include current directory when running test Perl commands
Necessary to find generated files in the out-of-tree build directory.
E.g. `tests/configurehelp.pm`, for tests 1119 and 1167.
Before this patch macOS autotools builds were failing these two tests
due to falling back to the default preprocessor (`cpp`) instead of
the actual one configured. Then `cpp` failing to compile Apple SDK
headers referenced by curl headers.
Viktor Szakats [Sun, 7 Jul 2024 15:23:51 +0000 (17:23 +0200)]
GHA/windows: usability improvements
- move `curl --version` into separate step.
- move configure log to separate step. Run on success, too.
- add step with `curl_config.h` dump (full and brief/sorted).
- make `autoreconf` a separate step.
- add each job configuration a short name.
- shorten job names.
Dedupe/drop redundant info, introduce abbreviations:
AM = autotools, CM = CMake, U = Unicode, R = Release, not -> `!`, etc.
Instead of mentioning `debug`, mentioned when it's not.
- simplify `PATH` forming for MSVC jobs.
It's sufficient to add the release binary directory of vcpkg, the debug one
is redundant.
Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979
Viktor Szakats [Fri, 5 Jul 2024 00:25:27 +0000 (02:25 +0200)]
GHA/macos: delete misplaced `CFLAGS`, drop redundant CMake option
With macOS there is a long-term struggle with deprecation warnings.
In curl they occur with LDAP, SecureTransport and in docs/examples.
There are three ways to fix them:
- by CFLAGS `-Wno-deprecated-declarations` as a workaround.
- by CFLAGS `-mmacosx-version-min` set to a version where the the
feature was not deprecated.
- by CMake option `-DCMAKE_OSX_DEPLOYMENT_TARGET=`.
In GHA CMake jobs, all three were used, and `-mmacosx-version-min` was
set in a bogus way. Delete that bogus option, and delete the lone,
redundant CMake option too.
In a future commit I might replace the suppression option to properly
setting the target OS.
Viktor Szakats [Thu, 4 Jul 2024 03:26:21 +0000 (05:26 +0200)]
macos: add workaround for gcc, non-c-ares, IPv6, compile error
Apple macOS SDK 13.0 and later are increasingly incompatible with gcc,
which started causing CI errors with the 20240701.9 revision of the
`macos-latest` (= `macos-14-arm64`) runner image.
This error is happening inside an Apple SDK header. We use the header
for calling a function in a resolver-related hack, in non-c-ares, IPv6
builds. You can avoid the problem by using c-ares or disabling IPv6
(or using clang, llvm, or a compatible gcc + SDK combination).
This patch fixes affected builds by declaring the ncessary framework
function manually, and not including the problematic header.
This workaround is ugly, doesn't cover all combinations, and fragile.
Other options are to disable this resolver-related hack for GCC, or to
replace it with a solution that doesn't rely on Apple SDK.
If you are aware of a stable fix or workaround, let us know.
gcc 12.4.0 + macOS SDK 14.0 (Xcode 15.0.1) error example:
```
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:54,
from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SCDynamicStoreCopySpecific.h:30,
from /Users/runner/work/curl/curl/lib/macos.c:33,
from /Users/runner/work/curl/curl/build/lib/CMakeFiles/libcurl_shared.dir/Unity/unity_0_c.c:244:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:126:1: error: attributes should be specified before the declarator in a function definition
126 | CF_INLINE CFOptionFlags CFUserNotificationCheckBoxChecked(CFIndex i) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(1UL << (8 + i)));}
| ^~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:127:1: error: attributes should be specified before the declarator in a function definition
127 | CF_INLINE CFOptionFlags CFUserNotificationSecureTextField(CFIndex i) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(1UL << (16 + i)));}
| ^~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:128:1: error: attributes should be specified before the declarator in a function definition
128 | CF_INLINE CFOptionFlags CFUserNotificationPopUpSelection(CFIndex n) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(n << 24));}
| ^~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9787982387/job/27025351601?pr=14096#step:7:18
The exact conditions are fuzzy. Oddly enough gcc 12.3.0 and the SDK
same as above are _compatible_:
https://github.com/curl/curl/actions/runs/9791701214/job/27036037162
Also notice that similar errors can also happen in SecureTransport
builds, due to the SDK headers required.
Ref: https://github.com/curl/curl/pull/14097#issuecomment-2208639046
Ref: https://github.com/curl/curl/pull/14091#issuecomment-2205870854
Cherry-picked from #14097
Closes #14119
Viktor Szakats [Sun, 7 Jul 2024 17:29:08 +0000 (19:29 +0200)]
cmake: feature casing fix and tidy-ups
- fix casing of a feature (`Unicode`) in the feature list.
- sort TLS backends case-insensitively.
- sync feature/protocol list heading with `curl -V` and autotools.
Viktor Szakats [Sun, 7 Jul 2024 13:39:31 +0000 (15:39 +0200)]
GHA: improve vcpkg cache, add BoringSSL ECH and LibreSSL MSVC jobs
- cache on a per-package basis.
Replace manual caching with a built-in solution. It shares cached
package builds between jobs, e.g. libssh2 only builds once
per platform (instead of once per job). Individual packages are built
as needed (not the whole per-job tree). It also fixes the duplicate
cache entry issues.
Ref: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-github-actions-cache
Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979
Follow-up to cb22cfca69bded45bf7f9c72c8e6764990490f11 #14077
- add BoringSSL job with ECH enabled. The first such job in the curl CI.
- add LibreSSL job.
- use vcpkg pre-installed on the runner image, instead of rolling our
own. This is quicker, simpler and more robust.
Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979
- show pre-installed vcpkg and ports version.
- drop `gsasl` dependency till it reaches the pre-installed vcpkg ports.
- re-add `find .` to see the binaries generated.
- simplify setting up `PATH`.
- exclude failing tests for any job enabling WinIDN.
- drop collecting and uploading log archives. We already dump CMake
logs, and our build doesn't use Ninja. Rest of files weren't generated
by the curl build. We don't aim to debug vcpkg package builds.
Viktor Szakats [Thu, 4 Jul 2024 03:04:35 +0000 (05:04 +0200)]
build: add Debug, TrackMemory, ECH to feature list
Also:
- remove stray `ECH` and `HTTPSRR` from cmake protocol list.
- stop excluding `Debug` and `TrackMemory` in `test1013.pl`.
- configure: delete `CURL_CHECK_CURLDEBUG` check.
Ref: 065047dc62cba3efde597fa5420d112fc2f4c500
This check was effectively doing nothing, except disabling
`--enable-curldebug` in `curl-config` for
Cygwin/MSYS/cegcc/OS2/AIX targets with c-ares enabled.
Dan Fandrich [Fri, 28 Jun 2024 21:41:29 +0000 (14:41 -0700)]
curl: list categories in --help
This eliminates the need to run an extra help subcommand to get the
possible categories, reducing the friction in getting relevant help. The
help wording was also slightly tweaked for grammatical accuracy.
Stefan Eissing [Fri, 5 Jul 2024 12:09:22 +0000 (14:09 +0200)]
multi: pollset assertion only when IP connected
Give warning for an empty pollset only when the connection has at least
IP connectivity. There are cases where the connect in QUIC makes another
attempt on a timeout and no socket will be available during that.
Daniel Stenberg [Thu, 4 Jul 2024 11:38:18 +0000 (13:38 +0200)]
cmdline-opts: category cleanup
Option cleanups:
--get is not upload
--form* are post
- added several options into ldap, smtp, imap and pop3
- shortened the category descriptions in the list
category curl fixes:
--create-dirs removed from 'curl'
--ftp-create-dirs removed from 'curl'
--netrc moved to 'auth' from 'curl'
--netrc-file moved to 'auth' from 'curl'
--netrc-optional moved to 'auth' from 'curl'
--no-buffer moved to 'output' from 'curl'
--no-clobber removed from 'curl'
--output removed from 'curl'
--output-dir removed from 'curl'
--remove-on-error removed from 'curl'
Add a "global" category:
- Made all "global" options set this category
Add a "deprecated" category:
- Moved the deprecated options to it (maybe they should not be in any
category long term)
Add a 'timeout' category
- Put a number of appropriate options in it
Add an 'ldap' category
- Put the LDAP related option in there
Remove categories "ECH" and "ipfs"
- They should not be categories. Had only one single option each.
Remove category "misc"
- It should not be a category as it is impossible to know when to browse
it.
--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth
Managen fixes:
- errors if an option is given a category name that is not already setup
for in code
- verifies that options set `scope: global` also is put in category
`global´
Stefan Eissing [Thu, 4 Jul 2024 14:39:20 +0000 (16:39 +0200)]
GHA: configure OpenSSL's libdir as 'lib' only
Also mention in HTTP3.md
OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.
Ref: #14099
See also: https://github.com/openssl/openssl/issues/23569
Daniel Stenberg [Thu, 4 Jul 2024 21:52:42 +0000 (23:52 +0200)]
tool_operate: simplify return code handling from url_proto()
The additional checks were superfluous as it would only ever return
error if one of those protocols were set. Also: a returned error
*should* mean get out of there, without having to check more conditions.
Stefan Eissing [Thu, 4 Jul 2024 09:14:05 +0000 (11:14 +0200)]
transfer: avoid polling socket every transfer loop
Improve download performance, minimal effort.
Do not poll the socket for pending data every transfer loop iteration.
This gives 10-20% performance gains on large HTTP/1.1 downloads (on my
machine).
Viktor Szakats [Tue, 2 Jul 2024 17:35:07 +0000 (19:35 +0200)]
GHA: Windows job exclusions tweaks
- disable SMTP tests in MSYS2/mingw-w64 and MSVC jobs.
On the suspicion of sometimes hanging:
https://github.com/curl/curl/actions/runs/9346162475/job/25720437944?pr=13855#step:14:2838
https://github.com/curl/curl/actions/runs/9758011305/job/26931678639?pr=14084#step:14:2834
https://github.com/curl/curl/actions/runs/9774468536/job/26982805294#step:11:4731
- run TFTP, MQTT, WebSockets tests in MSYS2/msys jobs again.
- switch hanging old-mingw-w64 7.3.0 job to Release (from Debug).
Guessing here, 9.5.0 is more solid, and one difference is
Debug/Release mode. Let's match 7.3.0 with that and see how it changes
hangs and flakiness.
The other difference is Unicode ON in 7.3.0. Flaky 6.3.0 was also
Debug, with Unicode OFF: 217878bade884202ee5fb2e80186c5fd130392e8 #13566.
(Unicode unlikely to play a role here IMO.)
If 7.3.0 keeps hanging / remains flaky I'll consider disabling its
test runs.
Viktor Szakats [Tue, 2 Jul 2024 08:41:35 +0000 (10:41 +0200)]
winbuild: MS-DOS batch tidy-ups
- prefer `.bat` extension over `.cmd` for MS-DOS batch, which also
avoids confusion with OS/400 `.cmd` files.
- cleanup `echo` quotes, drop them consistently.
- delete empty output line from one of the error branches.
- prefer lowercase commands like the rest of MS-DOS batches.
- delete a contraction.
- drop backticks from error message.
- use `nmake.exe` consistently.
- use equal/not-equal operator style consistently.
- inline a single-line `if` branch.
- delete exceptions and rules dealing with Windows `.cmd` extension.
Stefan Eissing [Mon, 1 Jul 2024 12:56:27 +0000 (14:56 +0200)]
multi: fix pollset during RESOLVING phase
- add a DEBUGASSERT for when a transfer's pollset should not be empty.
- move write unpausing from transfer loop into curl_easy_pause. This
make sure that the url_updatesocket() finds the correct state when
updating socket events.
- fix HTTP/2 proxy during connect phase to set sockets correctly
- fix test2600 to simulate a socket set
- move write unpausing from transfer loop into curl_easy_pause. This
make sure that the url_updatesocket() finds the correct state when
updating socket events.
- waiting for the resolver to deliver might not involve any sockets to
wait for. Do not generate a warning.
Daniel Stenberg [Mon, 1 Jul 2024 14:47:21 +0000 (16:47 +0200)]
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.
- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period
- fix to show `smb` and `smbs` in cmake protocol list.
- add wolfSSL CMake job to GHA (for macOS).
- fix mqtt and wolfSSL symbol clash.
```
./curl/lib/mqtt.c: In function 'mqtt_doing':
./curl/lib/mqtt.c:746:17: error: declaration of 'byte' shadows a global declaration [-Werror=shadow]
746 | unsigned char byte;
| ^~~~
/opt/homebrew/Cellar/wolfssl/5.7.0_1/include/wolfssl/wolfcrypt/types.h:85:36: note: shadowed declaration is here
85 | typedef unsigned char byte;
| ^~~~
```
- format `FindWolfSSL.cmake` closer to neighbours.
Daniel Stenberg [Mon, 1 Jul 2024 07:05:19 +0000 (09:05 +0200)]
curl_str[n]equal.md: tidy up text to make them stand-alone
Previously this was one single manpage for two functions but as they are
two separate ones since a while back, they should each clearly document
their single specific functions.
Daniel Stenberg [Sat, 29 Jun 2024 22:08:06 +0000 (00:08 +0200)]
docs/libcurl: polish the single-line descriptions
- use imperative form
- use lowercase
- no period
- unify some phrases
- fix curl_multi_socket and curl_multi_socket_all to keep their own
descriptions
Daniel Stenberg [Fri, 28 Jun 2024 07:01:08 +0000 (09:01 +0200)]
managen: insert final .fi for files ending with a quote
When an individual file ended with a quote (typically an example), the
render function would return without ending the quote correctly with a
".fi" (fill in) in the manpage output.
This made the additional text provided below to render wrongly.
Daniel Stenberg [Thu, 27 Jun 2024 14:37:25 +0000 (16:37 +0200)]
KNOWN_BUGS: three new bugs
These have lingered in the issue tracker for a long time without action.
We don't expect any fixes in the near term either. Move them to the
KNOWN_BUGS document.
Viktor Szakats [Thu, 27 Jun 2024 00:38:38 +0000 (02:38 +0200)]
CI: add whitespace checker
Fix issues detected.
Also:
- One of the `.vc` files used LF EOLs, while the other didn't.
Make that one also use LF EOLs, as this is apparently supported by
`nmake`.
- Drop `.dsw` and `.btn` types from `.gitattributes`.
The repository doesn't use them.
- Sync section order with the rest of files in
`tests/certs/EdelCurlRoot-ca.prm`.
- Indent/align `.prm` and `.pem` files.
- Delete dummy `[something]` section from `.prm` and `.pem` files.
Mental note:
MSVC `.sln` files seem to accept spaces for indentation and also support
LF line-endings. I cannot test this and I don't know what's more
convenient when updating them, so left them as-is, with specific
exclusions.