clen
CNA
hel
-htpt
+htpts
inout
PASE
passwor
extend-ignore-identifiers-re = [
"^(ba|fo|pn|PN|UE)$",
"^(CNA|ser)$",
- "^(ECT0|ECT1|HELO|htpt|PASE)$",
+ "^(ECT0|ECT1|HELO|htpts|PASE)$",
"^[A-Za-z0-9_-]*(EDE|GOST)[A-Z0-9_-]*$", # ciphers
"^0x[0-9a-fA-F]+FUL$", # unsigned long hex literals ending with 'F'
"^(eyeballers|HELO_smtp|Januar|optin|passin|perfec|SMTP_HELO)$",
that informs the client about this is only interpreted if you are using the
-L/--location option. As in:
- curl -L http://example.com
+ curl -L https://example.com
Not all redirects are HTTP ones, see 4.14
Set a custom Host: header that identifies the server name you want to reach
but use the target IP address in the URL:
- curl --header "Host: www.example.com" http://127.0.0.1/
+ curl --header "Host: www.example.com" https://somewhere.example/
You can also opt to add faked hostname entries to curl with the --resolve
option. That has the added benefit that things like redirects will also work
properly. The above operation would instead be done as:
- curl --resolve www.example.com:80:127.0.0.1 http://www.example.com/
+ curl --resolve www.example.com:80:127.0.0.1 https://www.example.com/
3.20 How to SFTP from my user's home directory?
be disabled or not supported.
Note that this error will also occur if you pass a wrongly spelled protocol
- part as in "htpt://example.com" or as in the less evident case if you prefix
- the protocol part with a space as in " http://example.com/".
+ part as in "htpts://example.com" or as in the less evident case if you
+ prefix the protocol part with a space as in " https://example.com/".
3.22 curl -X gives me HTTP problems
By default you use curl without explicitly saying which request method to
use when the URL identifies an HTTP transfer. If you just pass in a URL like
- "curl http://example.com" it will use GET. If you use -d or -F curl will use
+ "curl https://example.com" it will use GET. If you use -d or -F curl will use
POST, -I will cause a HEAD and -T will make it a PUT.
If for whatever reason you are not happy with these default choices that curl
An example that would invoke a remote CGI that uses &-symbols could be:
- curl 'http://www.example.com/cgi-bin/query?text=yes&q=curl'
+ curl 'https://www.example.com/cgi-bin/query?text=yes&q=curl'
In Windows, the standard DOS shell treats the percent sign specially and you
need to use TWO percent signs for each single one you want to use in the
If you get this return code and an HTML output similar to this:
<H1>Moved Permanently</H1> The document has moved <A
- HREF="http://same_url_now_with_a_trailing_slash/">here</A>.
+ HREF="https://same_url_now_with_a_trailing_slash.example/">here</A>.
it might be because you requested a directory URL but without the trailing
slash. Try the same operation again _with_ the trailing URL, or use the
Fetch two documents at once:
- curl ftp://ftp.example.com/ http://www.example.com:8000/
+ curl ftp://ftp.example.com/ https://www.example.com:8000/
Get a file off an FTPS server:
Get a webpage and store in a local file with a specific name:
- curl -o thatpage.html http://www.example.com/
+ curl -o thatpage.html https://www.example.com/
Get a webpage and store in a local file, make the local file get the name of
the remote document (if no filename part is specified in the URL, this fails):
- curl -O http://www.example.com/index.html
+ curl -O https://www.example.com/index.html
Fetch two files and store them with their remote names:
### HTTP
-curl also supports user and password in HTTP URLs, thus you can pick a file
+curl also supports user and password in HTTP(S) URLs. You can download a file
like:
- curl http://name:passwd@http.server.example/full/path/to/file
+ curl https://name:passwd@http.server.example/full/path/to/file
or specify user and password separately like in
- curl -u name:passwd http://http.server.example/full/path/to/file
+ curl -u name:passwd https://http.server.example/full/path/to/file
HTTP offers many different methods of authentication and curl supports
several: Basic, Digest, NTLM and Negotiate (SPNEGO). Without telling which
curl -x my-proxy:888 ftp://ftp.example.com/README
-Get a file from an HTTP server that requires user and password, using the
+Get a file from an HTTPS server that requires user and password, using the
same proxy as above:
- curl -u user:passwd -x my-proxy:888 http://www.example.com/
+ curl -u user:passwd -x my-proxy:888 https://www.example.com/
Some proxies require special authentication. Specify by using -U as above:
- curl -U user:passwd -x my-proxy:888 http://www.example.com/
+ curl -U user:passwd -x my-proxy:888 https://www.example.com/
A comma-separated list of hosts and domains which do not use the proxy can be
specified as:
- curl --noproxy example.com -x my-proxy:888 http://www.example.com/
+ curl --noproxy example.com -x my-proxy:888 https://www.example.com/
If the proxy is specified with `--proxy1.0` instead of `--proxy` or `-x`, then
curl uses HTTP/1.0 instead of HTTP/1.1 for any `CONNECT` attempts.
Get the first 100 bytes of a document:
- curl -r 0-99 http://www.example.com/
+ curl -r 0-99 https://www.example.com/
Get the last 500 bytes of a document:
- curl -r -500 http://www.example.com/
+ curl -r -500 https://www.example.com/
curl also supports simple ranges for FTP files as well. Then you can only
specify start and stop position.
### HTTP
-Upload all data on stdin to a specified HTTP site:
+Upload all data on stdin to a specified HTTPS site:
- curl -T - http://www.example.com/myfile
+ curl -T - https://www.example.com/myfile
Note that the HTTP server must have been configured to accept PUT before this
can be done successfully.
Post a simple `name` and `phone` guestbook.
- curl -d "name=Rafael%20Sagula&phone=3320780" http://www.example.com/guest.cgi
+ curl -d "name=Rafael%20Sagula&phone=3320780" https://www.example.com/guest.cgi
Or automatically [URL encode the data](https://everything.curl.dev/http/post/url-encode).
curl --data-urlencode "name=Rafael Sagula&phone=3320780"
- http://www.example.com/guest.cgi
+ https://www.example.com/guest.cgi
How to post a form with curl, lesson #1:
Example:
-(say if `http://example.com` had the following html)
+(say if `https://example.com` had the following html)
```html
<form action="post.cgi" method="post">
To post to this, you would enter a curl command line like:
curl -d "user=foobar&pass=12345&id=blablabla&ding=submit"
- http://example.com/post.cgi
+ https://example.com/post.cgi
While `-d` uses the application/x-www-form-urlencoded mime-type, generally
understood by CGI's and similar, curl also supports the more capable
different content types using the following syntax:
curl -F "coolfiles=@fil1.gif;type=image/gif,fil2.txt,fil3.html"
- http://www.example.com/postit.cgi
+ https://www.example.com/postit.cgi
If the content-type is not specified, curl tries to guess from the file
extension (it only knows a few), or use the previously specified type (from an
curl -F "file=@cooltext.txt" -F "yourname=Daniel"
-F "filedescription=Cool text file with cool text inside"
- http://www.example.com/postit.cgi
+ https://www.example.com/postit.cgi
To send two files in one post you can do it in two ways:
servers or CGI scripts that rely on that information being available or
contain certain data.
- curl -e www.example.org http://www.example.com/
+ curl -e www.example.org https://www.example.com/
## User Agent
Example:
- curl -A 'Mozilla/3.0 (Win95; I)' http://www.bank.example.com/
+ curl -A 'Mozilla/3.0 (Win95; I)' https://www.bank.example.com/
Other common strings:
URL by making a config file similar to:
# default url to get
- url = "http://help.with.curl.example.com/curlhelp.html"
+ url = "https://help.with.curl.example.com/curlhelp.html"
You can specify another config file to be read by using the `-K`/`--config`
flag. If you set config filename to `-` it reads the config from stdin, which
can be handy if you want to hide options from being visible in process tables
etc:
- echo "user = user:passwd" | curl -K - http://that.secret.example.com
+ echo "user = user:passwd" | curl -K - https://that.secret.example.com
## Extra Headers
Get a webpage from a server using a specified port for the interface:
- curl --interface eth0:1 http://www.example.com/
+ curl --interface eth0:1 https://www.example.com/
or
- curl --interface 192.168.1.10 http://www.example.com/
+ curl --interface 192.168.1.10 https://www.example.com/
## HTTPS
Continue downloading a document from a web server
- curl -C - -o file http://www.example.com/
+ curl -C - -o file https://www.example.com/
## Time Conditions
For example, you can easily make a download that only gets performed if the
remote file is newer than a local copy. It would be made like:
- curl -z local.html http://remote.example.com/remote.html
+ curl -z local.html https://remote.example.com/remote.html
Or you can download a file only if the local file is newer than the remote
one. Do this by prepending the date string with a `-`, as in:
- curl -z -local.html http://remote.example.com/remote.html
+ curl -z -local.html https://remote.example.com/remote.html
You can specify a plain text date as condition. Tell curl to only download the
file if it was updated since January 12, 2012:
- curl -z "Jan 12 2012" http://remote.example.com/remote.html
+ curl -z "Jan 12 2012" https://remote.example.com/remote.html
curl accepts a wide range of date formats. You always make the date check the
other way around by prepending it with a dash (`-`).
For example: get two files and use `-O` for the first and a custom file
name for the second:
- curl -O http://example.com/file.txt ftp://example.com/moo.exe -o moo.jpg
+ curl -O https://example.com/file.txt ftp://example.com/moo.exe -o moo.jpg
You can also upload multiple files in a similar fashion:
options can specify which address to use when both are available. IPv6
addresses can also be specified directly in URLs using the syntax:
- http://[2001:1890:1112:1::20]/overview.html
+ https://[2001:1890:1112:1::20]/overview.html
When this style is used, the `-g` option must be given to stop curl from
interpreting the square brackets as special globbing characters. Link local
offer even more details as they show **everything** curl sends and
receives. Use it like this:
- curl --trace-ascii debugdump.txt http://www.example.com/
+ curl --trace-ascii debugdump.txt https://www.example.com/
## See the Timing
[`--trace-time`](https://curl.se/docs/manpage.html#--trace-time) option is
what you need. It prepends the time to each trace output line:
- curl --trace-ascii d.txt --trace-time http://example.com/
+ curl --trace-ascii d.txt --trace-time https://example.com/
## See which Transfer
you need. It prepends the transfer and connection identifier to each trace
output line:
- curl --trace-ascii d.txt --trace-ids http://example.com/
+ curl --trace-ascii d.txt --trace-ids https://example.com/
## See the Response
IP address for a hostname than what would otherwise be used, by using curl's
[`--resolve`](https://curl.se/docs/manpage.html#--resolve) option:
- curl --resolve www.example.org:80:127.0.0.1 http://www.example.org/
+ curl --resolve www.example.org:80:127.0.0.1 https://www.example.org/
## Port number
number immediately following the hostname. Like when doing HTTP to port
1234:
- curl http://www.example.org:1234/
+ curl https://www.example.org:1234/
The port number you specify in the URL is the number that the server uses to
offer its services. Sometimes you may use a proxy, and then you may
need to specify that proxy's port number separately from what curl needs to
connect to the server. Like when using an HTTP proxy on port 4321:
- curl --proxy http://proxy.example.org:4321 http://remote.example.org/
+ curl --proxy http://proxy.example.org:4321 https://remote.example.org/
## Username and password
You can opt to either insert the user and password in the URL or you can
provide them separately:
- curl http://user:password@example.org/
+ curl https://user:password@example.org/
or
- curl -u user:password http://example.org/
+ curl -u user:password https://example.org/
You need to pay attention that this kind of HTTP authentication is not what
is usually done and requested by user-oriented websites these days. They tend
Example, send two GET requests:
- curl http://url1.example.com http://url2.example.com
+ curl https://url1.example.com https://url2.example.com
If you use [`--data`](https://curl.se/docs/manpage.html#-d) to POST to
the URL, using multiple URLs means that you send that same POST to all the
Example, send two POSTs:
- curl --data name=curl http://url1.example.com http://url2.example.com
+ curl --data name=curl https://url1.example.com https://url2.example.com
## Multiple HTTP methods in a single command line
Perhaps this is best shown with a few examples. To send first a HEAD and then
a GET:
- curl -I http://example.com --next http://example.com
+ curl -I https://example.com --next https://example.com
To first send a POST and then a GET:
- curl -d score=10 http://example.com/post.cgi --next http://example.com/results.html
+ curl -d score=10 https://example.com/post.cgi --next https://example.com/results.html
# HTML forms
To make curl do the GET form post for you, just enter the expected created
URL:
- curl "http://www.example.com/when/junk.cgi?birthyear=1905&press=OK"
+ curl "https://www.example.com/when/junk.cgi?birthyear=1905&press=OK"
## POST
And to use curl to post this form with the same data filled in as before, we
could do it like:
- curl --data "birthyear=1905&press=%20OK%20" http://www.example.com/when/junk.cgi
+ curl --data "birthyear=1905&press=%20OK%20" https://www.example.com/when/junk.cgi
This kind of POST uses the Content-Type `application/x-www-form-urlencoded`
and is the most widely used POST kind.
Recent curl versions can in fact url-encode POST data for you, like this:
- curl --data-urlencode "name=I am Daniel" http://www.example.com
+ curl --data-urlencode "name=I am Daniel" https://www.example.com
If you repeat `--data` several times on the command line, curl concatenates
all the given data pieces - and put a `&` symbol between each data segment.
Put a file to an HTTP server with curl:
- curl --upload-file uploadfile http://www.example.com/receive.cgi
+ curl --upload-file uploadfile https://www.example.com/receive.cgi
# HTTP Authentication
To tell curl to use a user and password for authentication:
- curl --user name:password http://www.example.com
+ curl --user name:password https://www.example.com
## Other Authentication
Use curl to set the referer field with:
- curl --referer http://www.example.come http://www.example.com
+ curl --referer https://www.example.come https://www.example.com
## User Agent
To tell curl to follow a Location:
- curl --location http://www.example.com
+ curl --location https://www.example.com
If you use curl to POST to a site that immediately redirects you to another
page, you can safely use [`--location`](https://curl.se/docs/manpage.html#-L)
The simplest way to send a few cookies to the server when getting a page with
curl is to add them on the command line like:
- curl --cookie "name=Daniel" http://www.example.com
+ curl --cookie "name=Daniel" https://www.example.com
Cookies are sent as common HTTP headers. This is practical as it allows curl
to record cookies simply by recording headers. Record cookies with curl by
using the [`--dump-header`](https://curl.se/docs/manpage.html#-D) (`-D`)
option like:
- curl --dump-header headers_and_cookies http://www.example.com
+ curl --dump-header headers_and_cookies https://www.example.com
(Take note that the
[`--cookie-jar`](https://curl.se/docs/manpage.html#-c) option described
believing you had a previous connection). To use previously stored cookies,
you run curl like:
- curl --cookie stored_cookies_in_file http://www.example.com
+ curl --cookie stored_cookies_in_file https://www.example.com
curl's "cookie engine" gets enabled when you use the
[`--cookie`](https://curl.se/docs/manpage.html#-b) option. If you only
page and follow a location (and thus possibly send back cookies it received),
you can invoke it like:
- curl --cookie nada --location http://www.example.com
+ curl --cookie nada --location https://www.example.com
curl has the ability to read and write cookie files that use the same file
format that Netscape and Mozilla once used. It is a convenient way to share
cookie file at the end of an operation:
curl --cookie cookies.txt --cookie-jar newcookies.txt \
- http://www.example.com
+ https://www.example.com
# HTTPS
You can delete a default header by providing one without content. Like you
can ruin the request by chopping off the `Host:` header:
- curl --header "Host:" http://www.example.com
+ curl --header "Host:" https://www.example.com
You can add headers the same way. Your server may want a `Destination:`
header, and you can add it:
- curl --header "Destination: http://nowhere" http://example.com
+ curl --header "Destination: nowhere" https://example.com
## More on changed methods
thinks it sends a POST. You can change the normal GET to a POST method by
simply adding `-X POST` in a command line like:
- curl -X POST http://example.org/
+ curl -X POST https://example.org/
curl however still acts as if it sent a GET so it does not send any request
body etc.
For example, this could look like:
- http://user:password@www.example.com:80/index.html?foo=bar#top
+ https://user:password@www.example.com:80/index.html?foo=bar#top
## Scheme
server or machine represented by either an IPv4 or IPv6 address (within
brackets). For example:
- http://www.example.com/
+ https://www.example.com/
- http://hostname/
+ https://hostname.example/
- http://192.168.0.1/
+ https://192.168.0.1/
- http://[2001:1890:1112:1::20]/
+ https://[2001:1890:1112:1::20]/
### "localhost"
the --proxy option. That is
NO_PROXY=direct.example.com curl -x http://proxy.example.com
- http://direct.example.com
+ https://direct.example.com
accesses the target URL directly, and
NO_PROXY=direct.example.com curl -x http://proxy.example.com
- http://somewhere.example.com
+ https://somewhere.example.com
accesses the target URL through the proxy.
Provide a list with three different names like this:
- http://site.{one,two,three}.com
+ https://fun.example/{one,two,three}.jpg
+
+ sftp://{one,two,three}.example/README
Do sequences of alphanumeric series by using [] as in:
Nested sequences are not supported, but you can use several ones next to each
other:
- http://example.com/archive[1996-1999]/vol[1-4]/part{a,b,c}.html
+ https://example.com/archive[1996-1999]/vol[1-4]/part{a,b,c}.html
You can specify a step counter for the ranges to get every Nth number or
letter:
- http://example.com/file[1-100:10].txt
+ https://example.com/file[1-100:10].txt
- http://example.com/file[a-z:2].txt
+ https://example.com/file[a-z:2].txt
When using [] or {} sequences when invoked from a command line prompt, you
probably have to put the full URL within double quotes to avoid the shell from
used to mitigate against this kind of attack.
A redirect can also specify a location available only on the machine running
-libcurl, including servers hidden behind a firewall from the attacker.
-E.g. **http://127.0.0.1/** or **http://intranet/delete-stuff.cgi?delete=all** or
-**tftp://bootp-server/pc-config-data**
+libcurl, including servers hidden behind a firewall from the attacker. E.g.
+**https://127.0.0.1/** or
+**https://intranet.example/delete-stuff.cgi?delete=all** or
+**tftp://bootp-server.example/pc-config-data**
Applications can mitigate against this by disabling
CURLOPT_FOLLOWLOCATION(3) and handling redirects itself, sanitizing URLs
A user who can control the DNS server of a domain being passed in within a URL
can change the address of the host to a local, private address which a
server-side libcurl-using application could then use. E.g. the innocuous URL
-**http://fuzzybunnies.example.com/** could actually resolve to the IP
-address of a server behind a firewall, such as 127.0.0.1 or
-10.1.2.3. Applications can mitigate against this by setting a
-CURLOPT_OPENSOCKETFUNCTION(3) or CURLOPT_PREREQFUNCTION(3) and
-checking the address before a connection.
+**https://fuzzybunnies.example.com/** could actually resolve to the IP address
+of a server behind a firewall, such as 127.0.0.1 or 10.1.2.3. Applications can
+mitigate against this by setting a CURLOPT_OPENSOCKETFUNCTION(3) or
+CURLOPT_PREREQFUNCTION(3) and checking the address before a connection.
All the malicious scenarios regarding redirected URLs apply just as well to
non-redirected URLs, if the user is allowed to specify an arbitrary URL that
If cookies are enabled and cached, then a user could craft a URL which
performs some malicious action to a site whose authentication is already
stored in a cookie. E.g.
-**http://mail.example.com/delete-stuff.cgi?delete=all** Applications can
+**https://mail.example.com/delete-stuff.cgi?delete=all** Applications can
mitigate against this by disabling cookies or clearing them between requests.
# Dangerous SCP URLs
If the user can set the URL, the user can also specify the scheme part to
other protocols that you did not intend for users to use and perhaps did not
-consider. curl supports over 20 different URL schemes. "http://" might be what
-you thought, "ftp://" or "imap://" might be what the user gives your
+consider. curl supports over 27 different URL schemes. `https://` might be
+what you expect, `ftp://` or `imap://` might be what the user gives your
application. Also, cross-protocol operations might be done by using a
particular scheme in the URL but point to a server doing a different protocol
on a non-standard port.
preferred URL to transfer with CURLOPT_URL(3) in a manner similar to:
~~~c
- curl_easy_setopt(handle, CURLOPT_URL, "http://example.com/");
+ curl_easy_setopt(handle, CURLOPT_URL, "https://example.com/");
~~~
Let's assume for a while that you want to receive data as the URL identifies a
~~~c
char *data = "name=daniel&project=curl";
curl_easy_setopt(handle, CURLOPT_POSTFIELDS, data);
- curl_easy_setopt(handle, CURLOPT_URL, "http://posthere.com/");
+ curl_easy_setopt(handle, CURLOPT_URL, "https://posthere.example/");
curl_easy_perform(handle); /* post away! */
~~~
if(curl) {
CURLcode res;
curl_easy_setopt(curl, CURLOPT_URL, "https://example.com");
- curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy.example.com");
+ curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy.example");
curl_easy_setopt(curl, CURLOPT_PROXYAUTH,
CURLAUTH_BASIC | CURLAUTH_DIGEST);
curl_easy_setopt(curl, CURLOPT_PROXYUSERNAME, "shrek");
/* accept various URLs */
curl_easy_setopt(curl, CURLOPT_URL, "https://example.com/");
/* use this proxy */
- curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy:80");
+ curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy.example:80");
/* ... but make sure this host name is not proxied */
curl_easy_setopt(curl, CURLOPT_NOPROXY, "www.example.com");
curl_easy_perform(curl);
CURL *curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "https://example.com/file.txt");
- curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy:80");
+ curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy.example:80");
curl_easy_perform(curl);
}
}
curl_easy_setopt(curl, CURLOPT_URL, "https://example.com");
curl_easy_setopt(curl, CURLOPT_HEADER, 1L);
- curl_easy_setopt(curl, CURLOPT_PROXY, "http://foo:3128");
+ curl_easy_setopt(curl, CURLOPT_PROXY, "http://proxy.example:3128");
curl_easy_setopt(curl, CURLOPT_HTTPPROXYTUNNEL, 1L);
curl_easy_setopt(curl, CURLOPT_SUPPRESS_CONNECT_HEADERS, 1L);
projects / thirdparty / curl.git / commitdiff
