From: Daniel Stenberg Date: Fri, 9 Sep 2022 13:11:13 +0000 (+0200) Subject: docs/WebSockets.md: docs X-Git-Tag: curl-7_86_0~281 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=6698e6ca24dfb02b78915d00840275bda97b83bc;p=thirdparty%2Fcurl.git docs/WebSockets.md: docs --- diff --git a/docs/FAQ b/docs/FAQ index 375715dc96..1a0b31c9f3 100644 --- a/docs/FAQ +++ b/docs/FAQ @@ -136,7 +136,8 @@ FAQ A client-side URL transfer library, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, - RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. + RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS + and WSS. libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, Kerberos, SPNEGO, HTTP form based upload, proxies, cookies, user+password diff --git a/docs/WebSockets.md b/docs/WebSockets.md new file mode 100644 index 0000000000..736f965382 --- /dev/null +++ b/docs/WebSockets.md @@ -0,0 +1,119 @@ + + +# WebSockets in curl + +## API + +The Websockets API is described in the individual man pages for the new API. + +Websockets with libcurl can be done two ways. + +1. Get the websockets frames from the server sent to a WS write callback. You + can then respond with `curl_ws_send()` from within the callback or outside + of it. + +2. Set `CURLOPT_CONNECT_ONLY` to 2L (new for websockets), which makes libcurl + do the `Upgrade:` dance in the `curl_easy_perform()` call and then you can + use `curl_ws_recv()` and `curl_ws_send()` to receive and send websocket + frames from and to the server. + +The new options to `curl_easy_setopt()`: + + `CURLOPT_WS_OPTIONS` - to control specific behavior (no bits implemented yet) + +The new function calls: + + `curl_ws_recv()` - receive a websockets frame + + `curl_ws_send()` - send a websockets frame + + `curl_ws_meta()` - return websockets metadata within a write callback + +## Max frame size + +The current implementation only supports frame sizes up to a max (64K right +now). This is because the API delivers full frames and it then cannot manage +the full 2^63 bytes size. + +If we decide we need to support (much) larger frames than 64K, we need to +adjust the API accordingly to be able to deliver partial frames in both +directions. + +## Errors + +If the given WebSocket URL (using `ws://` or `wss://`) fails to get upgraded +via a 101 response code and instead gets another response code back from the +HTTP server - the transfer will return `CURLE_HTTP_RETURNED_ERROR` for that +transfer. Note then that even 2xx response codes are then considered error +since it failed to provide a WebSocket transfer. + +## Test suite + +I looked for an existing small WebSockets server implementation with maximum +flexibility to dissect and cram into the test suite but I ended up deciding +that extending the existing test suite server sws to deal with WebSockets +might be the better way. + +- This server is already integrated and working in the test suite + +- We want maximum control and ability to generate broken protocol and negative + tests as well. A dumber and simpler TCP server could then be easier to + massage into this than a "proper" websockets server. + +## Command line tool websockets + +The plan is to make curl do websockets similar to telnet/nc. That part of the +work has not been started. + +Ideas: + + - Read stdin and send off as messages. Consider newline as end of fragment. + (default to text? offer option to set binary) + - Respond to PINGs automatically + - Issue PINGs at some default interval (option to switch off/change interval?) + - Allow `-d` to specify (initial) data to send (should the format allow for + multiple separate frames?) + - Exit after N messages received, where N can be zero. + +## Future work + +- Verify the Sec-WebSocket-Accept response. It requires a sha-1 function. +- Verify Sec-Websocket-Extensions and Sec-Websocket-Protocol in the response +- Make websockets work with hyper +- Consider a `curl_ws_poll()` +- Make sure Websockets code paths are fuzzed +- Add client-side PING interval +- Provide option to disable PING-PONG automation +- Support compression (`CURLWS_COMPRESS`) + +## Why not libwebsockets + +[libwebsockets](https://libwebsockets.org/) is said to be a solid, fast and +efficient WebSockets library with a vast amount of users. My plan was +originally to build upon it to skip having to implement the lowlevel parts of +WebSockets myself. + +Here are the reasons why I have decided to move forward with WebSockets in +curl **without using libwebsockets**: + +- doxygen generated docs only makes them very hard to navigate. No tutorial, + no clearly written explanatory pages for specific functions. + +- seems (too) tightly integrated with a specific TLS library, while we want to + support websockets with whatever TLS library libcurl was already made to + work with. + +- seems (too) tightly integrated with event libraries + +- the references to threads and thread-pools in code and APIs indicate too + much logic for our purposes + +- "bloated" - it is a *huge* library that is actually more lines of code than + libcurl itself + +- websockets is a fairly simple protocol on the network/framing layer so + making a homegrown handling of it should be fine diff --git a/docs/cmdline-opts/page-header b/docs/cmdline-opts/page-header index 84bd39d091..1737ab5bd0 100644 --- a/docs/cmdline-opts/page-header +++ b/docs/cmdline-opts/page-header @@ -33,7 +33,7 @@ curl \- transfer a URL **curl** is a tool for transferring data from or to a server. It supports these protocols: DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, -SMTPS, TELNET or TFTP. The command is designed to work without user +SMTPS, TELNET, TFTP, WS and WSS. The command is designed to work without user interaction. curl offers a busload of useful tricks like proxy support, user diff --git a/docs/curl-config.1 b/docs/curl-config.1 index 8151b26288..ce8d62274c 100644 --- a/docs/curl-config.1 +++ b/docs/curl-config.1 @@ -62,9 +62,9 @@ on. The prefix is set with "configure --prefix". .IP "--protocols" Lists what particular protocols the installed libcurl was built to support. At the time of writing, this list may include HTTP, HTTPS, FTP, FTPS, FILE, -TELNET, LDAP, DICT. Do not assume any particular order. The protocols will -be listed using uppercase and are separated by newlines. There may be none, -one, or several protocols in the list. (Added in 7.13.0) +TELNET, LDAP, DICT and many more. Do not assume any particular order. The +protocols will be listed using uppercase and are separated by newlines. There +may be none, one, or several protocols in the list. (Added in 7.13.0) .IP "--ssl-backends" Lists the SSL backends that were enabled when libcurl was built. It might be no, one or several names. If more than one name, they will appear