tidy-up: URLs (cont.) and mdlinkcheck

- add missing ending slashes.
  To avoid duplicates and to use canonical URLs.
- reapply lost updates.
  Follow-up to 2ae983bf4ea5ef86f0e68ea0ff219a91b1aa3428 #19879
- mdlinkcheck: include the `include` directory.
- mdlinkcheck: show unused whitelist items.
- mdlinkcheck: improve debug output.
- mdlinkcheck: delete redundant whitelist items.
- examples/simplessl: lowercase the protocol part.
- BINDINGS: replace one remaining HTTP URL with HTTPS.
  Issue: https://github.com/pycurl/pycurl/issues/892
- BINDINGS: fix a broken link.
- BINDINGS: follow a refresh content redirect.
- KNOWN_BUGS: whitespace.

Closes #19911

diff --git a/docs/BINDINGS.md b/docs/BINDINGS.md
index 065126efd72376a6d407e10571922465c7a2ff08..56f0376b6415c7e826c108a3f5896db392b12bc8 100644 (file)
--- a/docs/BINDINGS.md
+++ b/docs/BINDINGS.md
@@ -49,7 +49,7 @@ Clojure: [clj-curl](https://github.com/lsevero/clj-curl) by Lucas Severo
 
 [Fortran](https://github.com/interkosmos/fortran-curl) Written by Philipp Engel
 
-[Gambas](https://gambas.sourceforge.net/)
+[Gambas](https://gambaswiki.org/website/en/main.html)
 
 [glib/GTK+](https://web.archive.org/web/20100526203452/atterer.net/glibcurl) Written by Richard Atterer
 
@@ -102,7 +102,7 @@ Bailiff and Bálint Szilakszi,
 
 [PureBasic](https://web.archive.org/web/20250325015028/www.purebasic.com/documentation/http/index.html) uses libcurl in its "native" HTTP subsystem
 
-[Python](http://pycurl.io/) PycURL by Kjetil Jacobsen
+[Python](https://github.com/pycurl/pycurl) PycURL by Kjetil Jacobsen
 
 [Python](https://pypi.org/project/pymcurl/) mcurl by Ganesh Viswanathan
 
@@ -112,7 +112,7 @@ Bailiff and Bálint Szilakszi,
 
 [Rexx](https://rexxcurl.sourceforge.net/) Written Mark Hessling
 
-[Ring](https://ring-lang.sourceforge.io/doc1.3/libcurl.html) RingLibCurl by Mahmoud Fayed
+[Ring](https://ring-lang.github.io/doc1.24/libcurl.html) RingLibCurl by Mahmoud Fayed
 
 RPG, support for ILE/RPG on OS/400 is included in source distribution
 

diff --git a/docs/FAQ.md b/docs/FAQ.md
index f6911990d94bbb72058550ef414d87911611ae0c..eba30f1fefd520d1cba297c3b8aa8131449ec772 100644 (file)
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@@ -303,7 +303,7 @@ library comparison](https://curl.se/docs/ssl-compared.html).
 The curl tool that is shipped as an integrated component of Windows 10 and
 Windows 11 is managed by Microsoft. If you were to delete the file or replace
 it with a newer version downloaded from [the curl
-website](https://curl.se/windows), then Windows Update will cease to work on
+website](https://curl.se/windows/), then Windows Update will cease to work on
 your system.
 
 There is no way to independently force an upgrade of the curl.exe that is part
@@ -312,7 +312,7 @@ also nothing the curl project itself can do about this, since this is managed
 and controlled entirely by Microsoft as owners of the operating system.
 
 You can always download and install [the latest version of curl for
-Windows](https://curl.se/windows) into a separate location.
+Windows](https://curl.se/windows/) into a separate location.
 
 ## Does curl support SOCKS (RFC 1928) ?
 
@@ -908,12 +908,12 @@ format:
 you will find that even if `D:\blah.txt` does exist, curl returns a 'file not
 found' error.
 
-According to [RFC 1738](https://www.ietf.org/rfc/rfc1738.txt), `file://` URLs
-must contain a host component, but it is ignored by most implementations. In
-the above example, `D:` is treated as the host component, and is taken away.
-Thus, curl tries to open `/blah.txt`. If your system is installed to drive C:,
-that will resolve to `C:\blah.txt`, and if that does not exist you will get
-the not found error.
+According to [RFC 1738](https://datatracker.ietf.org/doc/html/rfc1738),
+`file://` URLs must contain a host component, but it is ignored by most
+implementations. In the above example, `D:` is treated as the host component,
+and is taken away. Thus, curl tries to open `/blah.txt`. If your system is
+installed to drive C:, that will resolve to `C:\blah.txt`, and if that does
+not exist you will get the not found error.
 
 To fix this problem, use `file://` URLs with *three* leading slashes:
 

diff --git a/docs/INFRASTRUCTURE.md b/docs/INFRASTRUCTURE.md
index 13240b31b5face729b028fe217c7f96b38543b22..7a6bafb9dbabab93d9a801d410043af264347176 100644 (file)
--- a/docs/INFRASTRUCTURE.md
+++ b/docs/INFRASTRUCTURE.md
@@ -100,11 +100,11 @@ company). The machine is physically located in Sweden.
 curl release tarballs are hosted on https://curl.se/download.html. They are
 uploaded there at release-time by the release manager.
 
-curl-for-win downloads are hosted on https://curl.se/windows and are uploaded
+curl-for-win downloads are hosted on https://curl.se/windows/ and are uploaded
 to the server by Viktor Szakats.
 
-curl-for-QNX downloads are hosted on <https://curl.se/qnx> and are uploaded to
-the server by Daniel Stenberg.
+curl-for-QNX downloads are hosted on <https://curl.se/qnx/> and are uploaded
+to the server by Daniel Stenberg.
 
 Daily release tarball-like snapshots are generated automatically and are
 provided for download at <https://curl.se/snapshots/>.

diff --git a/docs/KNOWN_BUGS.md b/docs/KNOWN_BUGS.md
index 1f5cebee27c7d21ce82a9d0fb12bcf5bf295aa3d..83a30fd9e897c4c1474bbf3ef1d5bf85d9b96a95 100644 (file)
--- a/docs/KNOWN_BUGS.md
+++ b/docs/KNOWN_BUGS.md
@@ -161,7 +161,7 @@ Passing in a Unicode filename with -o:
 
 Passing in Unicode character with -d:
 
- [curl issue 12231](https://github.com/curl/curl/issues/12231)
+[curl issue 12231](https://github.com/curl/curl/issues/12231)
 
 Windows Unicode builds use the home directory in current locale.
 
@@ -215,8 +215,10 @@ what `winhttp` does. See https://curl.se/bug/view.cgi?id=535
 
 ## NTLM does not support password with Unicode 'SECTION SIGN' character
 
- https://en.wikipedia.org/wiki/Section_sign
- [curl issue 2120](https://github.com/curl/curl/issues/2120)
+Code point: U+00A7
+
+https://en.wikipedia.org/wiki/Section_sign
+[curl issue 2120](https://github.com/curl/curl/issues/2120)
 
 ## libcurl can fail to try alternatives with `--proxy-any`
 
@@ -231,7 +233,7 @@ using NTLM.
 
 ## Do not clear digest for single realm
 
- [curl issue 3267](https://github.com/curl/curl/issues/3267)
+[curl issue 3267](https://github.com/curl/curl/issues/3267)
 
 ## SHA-256 digest not supported in Windows SSPI builds
 
@@ -243,7 +245,7 @@ with `SEC_E_QOP_NOT_SUPPORTED` which causes curl to fail with
 Microsoft does not document supported digest algorithms and that `SEC_E` error
 code is not a documented error for `InitializeSecurityContext` (digest).
 
- [curl issue 6302](https://github.com/curl/curl/issues/6302)
+[curl issue 6302](https://github.com/curl/curl/issues/6302)
 
 ## curl never completes Negotiate over HTTP
 
@@ -306,7 +308,7 @@ In the `SSH_SFTP_INIT` state for libssh, the ssh session working mode is set
 to blocking mode. If the network is suddenly disconnected during sftp
 transmission, curl is stuck, even if curl is configured with a timeout.
 
- [curl issue 8632](https://github.com/curl/curl/issues/8632)
+[curl issue 8632](https://github.com/curl/curl/issues/8632)
 
 ## Cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!"
 
@@ -494,7 +496,7 @@ cannot be built.
 
 ## HTTP/2 prior knowledge over proxy
 
- [curl issue 12641](https://github.com/curl/curl/issues/12641)
+[curl issue 12641](https://github.com/curl/curl/issues/12641)
 
 ## HTTP/2 frames while in the connection pool kill reuse
 

diff --git a/docs/TODO.md b/docs/TODO.md
index 2b77b8ce0b9e6b1c04188c35cb9ba380018cc627..0224ae87332c77c4dbb5030d04a41697dd263f8f 100644 (file)
--- a/docs/TODO.md
+++ b/docs/TODO.md
@@ -530,8 +530,8 @@ for 1) efficiency and 2) safety.
 ## Support DANE
 
 [DNS-Based Authentication of Named Entities
-(DANE)](https://www.rfc-editor.org/rfc/rfc6698.txt) is a way to provide SSL
-keys and certs over DNS using DNSSEC as an alternative to the CA model.
+(DANE)](https://datatracker.ietf.org/doc/html/rfc6698) is a way to provide
+SSL keys and certs over DNS using DNSSEC as an alternative to the CA model.
 
 A patch was posted on March 7 2013
 (https://curl.se/mail/lib-2013-03/0075.html) but it was a too simple approach.

diff --git a/docs/TheArtOfHttpScripting.md b/docs/TheArtOfHttpScripting.md
index f7cf43602d908c65e8a118d283173d27ce8ce304..ab4961c5781c004df647016a1b5f6f5c4fa7d316 100644 (file)
--- a/docs/TheArtOfHttpScripting.md
+++ b/docs/TheArtOfHttpScripting.md
@@ -91,7 +91,7 @@ SPDX-License-Identifier: curl
 
  The Uniform Resource Locator format is how you specify the address of a
  particular resource on the Internet. You know these, you have seen URLs like
- https://curl.se or https://example.com a million times. RFC 3986 is the
+ https://curl.se/ or https://example.com/ a million times. RFC 3986 is the
  canonical spec. The formal name is not URL, it is **URI**.
 
 ## Host
@@ -158,7 +158,7 @@ SPDX-License-Identifier: curl
  issues a GET request to the server and receives the document it asked for.
  If you issue the command line
 
-    curl https://curl.se
+    curl https://curl.se/
 
  you get a webpage returned in your terminal window. The entire HTML document
  this URL identifies.

diff --git a/docs/cmdline-opts/_WWW.md b/docs/cmdline-opts/_WWW.md
index 35d946697f9476d831aa1d32b76ed506c5539f26..8656e9ee8c8f07181ad75cf4f34b980ac199ca32 100644 (file)
--- a/docs/cmdline-opts/_WWW.md
+++ b/docs/cmdline-opts/_WWW.md
@@ -1,4 +1,4 @@
 <!-- Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al. -->
 <!-- SPDX-License-Identifier: curl -->
 # WWW
-https://curl.se
+https://curl.se/

diff --git a/docs/examples/10-at-a-time.c b/docs/examples/10-at-a-time.c
index 08f1f25f7f7f08f9f5f0e2b58643a85404157422..db420eebfb67263170296fd674e20f3805b8a698 100644 (file)
--- a/docs/examples/10-at-a-time.c
+++ b/docs/examples/10-at-a-time.c
@@ -31,53 +31,53 @@
 #include <curl/curl.h>
 
 static const char *urls[] = {
-  "https://www.microsoft.com",
-  "https://opensource.org",
-  "https://www.google.com",
-  "https://www.yahoo.com",
-  "https://www.ibm.com",
-  "https://www.mysql.com",
-  "https://www.oracle.com",
-  "https://www.ripe.net",
-  "https://www.iana.org",
-  "https://www.amazon.com",
-  "https://www.netcraft.com",
-  "https://www.heise.de",
-  "https://www.chip.de",
-  "https://www.ca.com",
-  "https://www.cnet.com",
-  "https://www.mozilla.org",
-  "https://www.cnn.com",
-  "https://www.wikipedia.org",
-  "https://www.dell.com",
-  "https://www.hp.com",
-  "https://www.cert.org",
-  "https://www.mit.edu",
-  "https://www.nist.gov",
-  "https://www.ebay.com",
-  "https://www.playstation.com",
-  "https://www.uefa.com",
-  "https://www.ieee.org",
-  "https://www.apple.com",
-  "https://www.symantec.com",
-  "https://www.zdnet.com",
+  "https://www.microsoft.com/",
+  "https://opensource.org/",
+  "https://www.google.com/",
+  "https://www.yahoo.com/",
+  "https://www.ibm.com/",
+  "https://www.mysql.com/",
+  "https://www.oracle.com/",
+  "https://www.ripe.net/",
+  "https://www.iana.org/",
+  "https://www.amazon.com/",
+  "https://www.netcraft.com/",
+  "https://www.heise.de/",
+  "https://www.chip.de/",
+  "https://www.ca.com/",
+  "https://www.cnet.com/",
+  "https://www.mozilla.org/",
+  "https://www.cnn.com/",
+  "https://www.wikipedia.org/",
+  "https://www.dell.com/",
+  "https://www.hp.com/",
+  "https://www.cert.org/",
+  "https://www.mit.edu/",
+  "https://www.nist.gov/",
+  "https://www.ebay.com/",
+  "https://www.playstation.com/",
+  "https://www.uefa.com/",
+  "https://www.ieee.org/",
+  "https://www.apple.com/",
+  "https://www.symantec.com/",
+  "https://www.zdnet.com/",
   "https://www.fujitsu.com/global/",
-  "https://www.supermicro.com",
-  "https://www.hotmail.com",
-  "https://www.ietf.org",
-  "https://www.bbc.co.uk",
-  "https://news.google.com",
-  "https://www.foxnews.com",
-  "https://www.msn.com",
-  "https://www.wired.com",
-  "https://www.sky.com",
-  "https://www.usatoday.com",
-  "https://www.cbs.com",
+  "https://www.supermicro.com/",
+  "https://www.hotmail.com/",
+  "https://www.ietf.org/",
+  "https://www.bbc.co.uk/",
+  "https://news.google.com/",
+  "https://www.foxnews.com/",
+  "https://www.msn.com/",
+  "https://www.wired.com/",
+  "https://www.sky.com/",
+  "https://www.usatoday.com/",
+  "https://www.cbs.com/",
   "https://www.nbc.com/",
-  "https://slashdot.org",
-  "https://www.informationweek.com",
-  "https://apache.org",
-  "https://www.un.org",
+  "https://slashdot.org/",
+  "https://www.informationweek.com/",
+  "https://apache.org/",
+  "https://www.un.org/",
 };
 
 #define MAX_PARALLEL 10  /* number of simultaneous transfers */

diff --git a/docs/examples/crawler.c b/docs/examples/crawler.c
index 09c0c7d6923f0f6b4bdf32ced0fc33e539d7289b..d153cc21c5ac52edeaa071b6d22b4156e2db8ece 100644 (file)
--- a/docs/examples/crawler.c
+++ b/docs/examples/crawler.c
@@ -47,7 +47,7 @@ static int max_total = 20000;
 static int max_requests = 500;
 static size_t max_link_per_page = 5;
 static int follow_relative_links = 0;
-static const char *start_page = "https://www.reuters.com";
+static const char *start_page = "https://www.reuters.com/";
 
 static int pending_interrupt = 0;
 static void sighandler(int dummy)

diff --git a/docs/examples/maxconnects.c b/docs/examples/maxconnects.c
index 2ee1d305a93fcce59f2c821eb8f00b7c02b525e5..95231d600268edd0af4fc87b167e6e079971294d 100644 (file)
--- a/docs/examples/maxconnects.c
+++ b/docs/examples/maxconnects.c
@@ -40,8 +40,8 @@ int main(void)
   curl = curl_easy_init();
   if(curl) {
     const char *urls[] = {
-      "https://example.com",
-      "https://curl.se",
+      "https://example.com/",
+      "https://curl.se/",
       "https://www.example/",
       NULL /* end of list */
     };

diff --git a/docs/examples/simplessl.c b/docs/examples/simplessl.c
index d08da207bfbfcc445bb02943bfeeaa77f3d8c702..c60b91057e9ae248a2f6df0951f2f1f16ed68cf8 100644 (file)
--- a/docs/examples/simplessl.c
+++ b/docs/examples/simplessl.c
@@ -84,7 +84,7 @@ int main(void)
     goto error;
 
   /* what call to write: */
-  curl_easy_setopt(curl, CURLOPT_URL, "HTTPS://secure.site.example");
+  curl_easy_setopt(curl, CURLOPT_URL, "https://secure.site.example/");
   curl_easy_setopt(curl, CURLOPT_HEADERDATA, headerfile);
 
 #ifdef USE_ENGINE

diff --git a/docs/internals/CONNECTION-FILTERS.md b/docs/internals/CONNECTION-FILTERS.md
index 09acc7ad9560471b74d4e268c24017fa60e21959..08fb367385786d257c7049590905f3a1f7bdb964 100644 (file)
--- a/docs/internals/CONNECTION-FILTERS.md
+++ b/docs/internals/CONNECTION-FILTERS.md
@@ -233,7 +233,7 @@ Users of `curl` may activate them by adding the name of the filter type to the
 of an HTTP/2 request, invoke curl with:
 
 ```
-> curl -v --trace-config ids,time,http/2  https://curl.se
+> curl -v --trace-config ids,time,http/2 https://curl.se/
 ```
 
 Which gives you trace output with time information, transfer+connection ids
@@ -260,7 +260,7 @@ into IPv4 and IPv6 and makes parallel attempts. The connection filter chain
 looks like this:
 
 ```
-* create connection for http://curl.se
+* create connection for http://curl.se/
 conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
 * start connect
 conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
@@ -276,7 +276,7 @@ The modular design of connection filters and that we can plug them into each oth
 The `HAPPY-EYEBALLS` on the other hand stays focused on its side of the problem. We can use it also to make other type of connection by just giving it another filter type to try to have happy eyeballing for QUIC:
 
 ```
-* create connection for --http3-only https://curl.se
+* create connection for --http3-only https://curl.se/
 conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
 * start connect
 conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
@@ -292,7 +292,7 @@ type that is used for `--http3` when **both** HTTP/3 and HTTP/2 or HTTP/1.1
 shall be attempted:
 
 ```
-* create connection for --http3 https://curl.se
+* create connection for --http3 https://curl.se/
 conn[curl.se] --> HTTPS-CONNECT --> NULL
 * start connect
 conn[curl.se] --> HTTPS-CONNECT --> NULL

diff --git a/docs/tests/TEST-SUITE.md b/docs/tests/TEST-SUITE.md
index 527a8599eafd95d63443356903af9556ac7fd5c0..6a2e60c81759a3f8e6fc11766b9da02525292f08 100644 (file)
--- a/docs/tests/TEST-SUITE.md
+++ b/docs/tests/TEST-SUITE.md
@@ -214,13 +214,13 @@ SPDX-License-Identifier: curl
   to drown in output. The newly introduced *connection filters* allows one to
   dynamically increase log verbosity for a particular *filter type*. Example:
 
-    CURL_DEBUG=ssl curl -v https://curl.se
+    CURL_DEBUG=ssl curl -v https://curl.se/
 
   makes the `ssl` connection filter log more details. One may do that for
   every filter type and also use a combination of names, separated by `,` or
   space.
 
-    CURL_DEBUG=ssl,http/2 curl -v https://curl.se
+    CURL_DEBUG=ssl,http/2 curl -v https://curl.se/
 
    The order of filter type names is not relevant. Names used here are
    case insensitive. Note that these names are implementation internals and

diff --git a/lib/curl_rtmp.c b/lib/curl_rtmp.c
index e0597cdcca9237f787dcc2835d976f1213fce890..45a50674f50bb6712ca95e0c7d6245399640f740 100644 (file)
--- a/lib/curl_rtmp.c
+++ b/lib/curl_rtmp.c
@@ -62,7 +62,7 @@ static Curl_recv rtmp_recv;
 static Curl_send rtmp_send;
 
 /*
- * RTMP protocol handler.h, based on https://rtmpdump.mplayerhq.hu
+ * RTMP protocol handler.h, based on https://rtmpdump.mplayerhq.hu/
  */
 
 const struct Curl_handler Curl_handler_rtmp = {

diff --git a/packages/vms/curl_gnv_build_steps.txt b/packages/vms/curl_gnv_build_steps.txt
index 0c30924998ea944d8d36901aff175d0f29b219e3..c84431f7faeee466665a610c0ba0065f1a671918 100644 (file)
--- a/packages/vms/curl_gnv_build_steps.txt
+++ b/packages/vms/curl_gnv_build_steps.txt
@@ -38,7 +38,7 @@ time that this document was written.
 [gnv.common_src]curl_*_original_src.bck is the original source of the curl kit
 as provided by the curl project.  [gnv.vms_src]curl-*_vms_src.bck, if present,
 has the OpenVMS specific files that are used for building that are not yet in
-the curl source kits for that release distributed https://curl.se
+the curl source kits for that release distributed https://curl.se/
 
 These backup savesets should be restored to different directory trees on
 an ODS-5 volume(s) which are referenced by concealed rooted logical names.

diff --git a/scripts/mdlinkcheck b/scripts/mdlinkcheck
index 446cb9f1dd6ed84518a2eab061745a71f52e0c7c..f50d77527a29666169870504f4e9b0e580740e41 100755 (executable)
--- a/scripts/mdlinkcheck
+++ b/scripts/mdlinkcheck
@@ -71,14 +71,12 @@ my %whitelist = (
     'https://curl.se/rfc/rfc2255.txt' => 1,
     'https://curl.se/sponsors.html' => 1,
     'https://curl.se/support.html' => 1,
-    'https://curl.se/windows' => 1,
     'https://curl.se/windows/' => 1,
 
     'https://testclutch.curl.se/' => 1,
 
     'https://github.com/curl/curl-fuzzer' => 1,
     'https://github.com/curl/curl-www' => 1,
-    'https://github.com/curl/curl.git' => 1,
     'https://github.com/curl/curl/wcurl' => 1,
 
     );
@@ -87,7 +85,7 @@ my %url;
 my %flink;
 
 # list all files to scan for links
-my @files=`git ls-files docs src lib scripts`;
+my @files=`git ls-files docs include lib scripts src`;
 
 sub storelink {
     my ($f, $line, $link) = @_;
@@ -103,6 +101,7 @@ sub storelink {
     if($link =~ /^(https|http):/) {
         if($whitelist{$link}) {
             #print "-- whitelisted: $link\n";
+            $whitelist{$link}++;
         }
         # example.com is just example
         elsif($link =~ /^https:\/\/(.*)example.(com|org|net)/) {
@@ -164,7 +163,7 @@ sub findlinks {
         # comma, question mark, colon, closing parenthesis, backslash,
         # closing angle bracket, whitespace, pipe, backtick, semicolon
         elsif(/(https:\/\/[a-z0-9.\/:%_+@-]+[^."'*\#,?:\)> \t|`;\\])/i) {
-            #print "RAW ";
+            #print "RAW '$_'\n";
             storelink($f, $line, $1);
         }
         $line++;
@@ -202,10 +201,16 @@ for my $f (@files) {
     }
 }
 
-#for my $u (sort keys %url) {
-#    print "$u\n";
-#}
-#exit;
+for my $u (sort keys %whitelist) {
+    if($whitelist{$u} == 1) {
+        printf "warning: unused whitelist entry: '$u'\n";
+    }
+}
+
+for my $u (sort keys %url) {
+    print "$u\n";
+}
+exit;
 
 my $error;
 my @errlist;