doc/man3/OSSL_HTTP_transfer.pod

   1 =pod
   2
   3 =head1 NAME
   4
   5 OSSL_HTTP_get,
   6 OSSL_HTTP_get_asn1,
   7 OSSL_HTTP_post_asn1,
   8 OSSL_HTTP_transfer,
   9 OSSL_HTTP_bio_cb_t,
  10 OSSL_HTTP_proxy_connect,
  11 OSSL_HTTP_parse_url
  12 - http client functions
  13
  14 =head1 SYNOPSIS
  15
  16  #include <openssl/http.h>
  17
  18  typedef BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg,
  19                                     int connect, int detail);
  20  BIO *OSSL_HTTP_get(const char *url, const char *proxy, const char *no_proxy,
  21                     BIO *bio, BIO *rbio,
  22                     OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
  23                     const STACK_OF(CONF_VALUE) *headers,
  24                     int maxline, unsigned long max_resp_len, int timeout,
  25                     const char *expected_content_type, int expect_asn1);
  26  ASN1_VALUE *OSSL_HTTP_get_asn1(const char *url,
  27                                 const char *proxy, const char *no_proxy,
  28                                 BIO *bio, BIO *rbio,
  29                                 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
  30                                 const STACK_OF(CONF_VALUE) *headers,
  31                                 int maxline, unsigned long max_resp_len,
  32                                 int timeout, const char *expected_content_type,
  33                                 const ASN1_ITEM *it);
  34  ASN1_VALUE *OSSL_HTTP_post_asn1(const char *server, const char *port,
  35                                  const char *path, int use_ssl,
  36                                  const char *proxy, const char *no_proxy,
  37                                  BIO *bio, BIO *rbio,
  38                                  OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
  39                                  const STACK_OF(CONF_VALUE) *headers,
  40                                  const char *content_type,
  41                                  const ASN1_VALUE *req, const ASN1_ITEM *req_it,
  42                                  int maxline, unsigned long max_resp_len,
  43                                  int timeout, const char *expected_ct,
  44                                  const ASN1_ITEM *rsp_it);
  45  BIO *OSSL_HTTP_transfer(const char *server, const char *port, const char *path,
  46                          int use_ssl, const char *proxy, const char *no_proxy,
  47                          BIO *bio, BIO *rbio,
  48                          OSSL_HTTP_bio_cb_t bio_update_fn, void *arg,
  49                          const STACK_OF(CONF_VALUE) *headers,
  50                          const char *content_type, BIO *req_mem,
  51                          int maxline, unsigned long max_resp_len, int timeout,
  52                          const char *expected_ct, int expect_asn1,
  53                          char **redirection_url);
  54  int OSSL_HTTP_proxy_connect(BIO *bio, const char *server, const char *port,
  55                              const char *proxyuser, const char *proxypass,
  56                              int timeout, BIO *bio_err, const char *prog);
  57  int OSSL_HTTP_parse_url(const char *url, char **phost, char **pport,
  58                          int *pport_num, char **ppath, int *pssl);
  59
  60 =head1 DESCRIPTION
  61
  62 OSSL_HTTP_get() uses HTTP GET to obtain data (of any type) from the given I<url>
  63 and returns it as a memory BIO.
  64
  65 OSSL_HTTP_get_asn1() uses HTTP GET to obtain an ASN.1-encoded value
  66 (e.g., an X.509 certificate) with the expected structure specified by I<it>
  67 (e.g., I<ASN1_ITEM_rptr(X509)>) from the given I<url>
  68 and returns it on success as a pointer to I<ASN1_VALUE>.
  69
  70 OSSL_HTTP_post_asn1() uses the HTTP POST method to send a request I<req>
  71 with the ASN.1 structure defined in I<req_it> and the given I<content_type> to
  72 the given I<server> and optional I<port> and I<path>.
  73 If I<use_ssl> is nonzero a TLS connection is requested and the I<bio_update_fn>
  74 parameter, described below, must be provided.
  75 The optional list I<headers> may contain additional custom HTTP header lines.
  76 The expected structure of the response is specified by I<rsp_it>.
  77 On success it returns the response as a pointer to B<ASN1_VALUE>.
  78
  79 OSSL_HTTP_transfer() exchanges any form of HTTP request and response.
  80 It implements the core of the functions described above.
  81 If I<path> parameter is NULL it defaults to "/".
  82 If I<use_ssl> is nonzero a TLS connection is requested
  83 and the I<bio_update_fn> parameter, described below, must be provided.
  84 If I<req_mem> is NULL it uses the HTTP GET method, else it uses HTTP POST to
  85 send a request with the contents of the memory BIO and optional I<content_type>.
  86 The optional list I<headers> may contain additional custom HTTP header lines.
  87 If I<req_mem> is NULL (i.e., the HTTP method is GET) and I<redirection_url>
  88 is not NULL the latter pointer is used to provide any new location that
  89 the server may return with HTTP code 301 (MOVED_PERMANENTLY) or 302 (FOUND).
  90 In this case the caller is responsible for deallocating this URL with
  91 L<OPENSSL_free(3)>.
  92
  93 The above functions have the following parameters in common.
  94
  95 Typically the OpenSSL build supports sockets
  96 and the I<bio> and I<rbio> parameters are both NULL.
  97 In this case the client creates a network BIO internally
  98 for connecting to the given I<server>
  99 at the specified I<port> (if any, defaulting to 80 for HTTP or 443 for HTTPS),
 100 optionally via a I<proxy> (respecting I<no_proxy>) as described below.
 101 Then the client uses this internal BIO for exchanging the request and response.
 102 If I<bio> is given and I<rbio> is NULL then the client uses this I<bio> instead.
 103 If both I<bio> and I<rbio> are given (which may be memory BIOs for instance)
 104 then no explicit connection is attempted,
 105 I<bio> is used for writing the request, and I<rbio> for reading the response.
 106 As soon as the client has flushed I<bio> the server must be ready to provide
 107 a response or indicate a waiting condition via I<rbio>.
 108
 109 The optional I<proxy> parameter can be used to set the address of the an
 110 HTTP(S) proxy to use (unless overridden by "no_proxy" settings).
 111 If TLS is not used this defaults to the environment variable C<http_proxy>
 112 if set, else C<HTTP_PROXY>.
 113 If I<use_ssl> != 0 it defaults to C<https_proxy> if set, else C<HTTPS_PROXY>.
 114 An empty proxy string specifies not to use a proxy.
 115 Else the format is C<[http[s]://]address[:port][/path]>,
 116 where any path given is ignored.
 117 The default proxy port number is 80, or 443 in case "https:" is given.
 118 The HTTP client functions connect via the given proxy unless the I<server>
 119 is found in the optional list I<no_proxy> of proxy hostnames (if not NULL;
 120 default is the environment variable C<no_proxy> if set, else C<NO_PROXY>).
 121 Proxying plain HTTP is supported directly,
 122 while using a proxy for HTTPS connections requires a suitable callback function
 123 such as OSSL_HTTP_proxy_connect(), described below.
 124
 125 The I<maxline> parameter specifies the response header maximum line length,
 126 where a value <= 0 indicates using the B<HTTP_DEFAULT_MAX_LINE_LENGTH> of 4KiB.
 127 This length is also used as the number of content bytes read at a time.
 128 The I<max_resp_len> parameter specifies the maximum response length,
 129 where 0 indicates the B<HTTP_DEFAULT_MAX_RESP_LEN>, which currently is 100 KiB.
 130
 131 An ASN.1-encoded response is expected by OSSL_HTTP_get_asn1() and
 132 OSSL_HTTP_post_asn1(), while for OSSL_HTTP_get() or OSSL_HTTP_transfer()
 133 this is only the case if the I<expect_asn1> parameter is nonzero.
 134 If the response header contains one or more "Content-Length" header lines and/or
 135 an ASN.1-encoded response is expected, which should include a total length,
 136 the length indications received are checked for consistency
 137 and for not exceeding the maximum response length.
 138
 139 If the parameter I<expected_content_type> (or I<expected_ct>, respectively)
 140 is not NULL then the HTTP client checks that the given content type string
 141 is included in the HTTP header of the response and returns an error if not.
 142
 143 If the I<timeout> parameter is > 0 this indicates the maximum number of seconds
 144 to wait until the transfer is complete.
 145 A value of 0 enables waiting indefinitely,
 146 while a value < 0 immediately leads to a timeout condition.
 147
 148 The optional parameter I<bio_update_fn> with its optional argument I<arg> may
 149 be used to modify the connection BIO used by the HTTP client (and cannot be
 150 used when both I<bio> and I<rbio> are given).
 151 I<bio_update_fn> is a BIO connect/disconnect callback function with prototype
 152
 153  BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, int connect, int detail)
 154
 155 The callback may modify the HTTP BIO provided in the I<bio> argument,
 156 whereby it may make use of a custom defined argument I<arg>,
 157 which may for instance refer to an I<SSL_CTX> structure.
 158 During connection establishment, just after calling BIO_do_connect_retry(),
 159 the function is invoked with the I<connect> argument being 1 and the I<detail>
 160 argument being 1 if HTTPS is requested, i.e., SSL/TLS should be enabled.
 161 On disconnect I<connect> is 0 and I<detail> is 1 if no error occurred, else 0.
 162 For instance, on connect the function may prepend a TLS BIO to implement HTTPS;
 163 after disconnect it may do some diagnostic output and/or specific cleanup.
 164 The function should return NULL to indicate failure.
 165 Here is a simple example that supports TLS connections (but not via a proxy):
 166
 167  BIO *http_tls_cb(BIO *hbio, void *arg, int connect, int detail)
 168  {
 169      SSL_CTX *ctx = (SSL_CTX *)arg;
 170
 171      if (connect && detail) { /* connecting with TLS */
 172          BIO *sbio = BIO_new_ssl(ctx, 1);
 173          hbio = sbio != NULL ? BIO_push(sbio, hbio) : NULL;
 174      } else if (!connect && !detail) { /* disconnecting after error */
 175          /* optionally add diagnostics here */
 176      }
 177      return hbio;
 178  }
 179
 180 After disconnect the modified BIO will be deallocated using BIO_free_all().
 181
 182 OSSL_HTTP_proxy_connect() may be used by an above BIO connect callback function
 183 to set up an SSL/TLS connection via an HTTPS proxy.
 184 It promotes the given BIO I<bio> representing a connection
 185 pre-established with a TLS proxy using the HTTP CONNECT method,
 186 optionally using proxy client credentials I<proxyuser> and I<proxypass>,
 187 to connect with TLS protection ultimately to I<server> and I<port>.
 188 If the I<port> argument is NULL or the empty string it defaults to "443".
 189 The I<timeout> parameter is used as described above.
 190 Since this function is typically called by applications such as
 191 L<openssl-s_client(1)> it uses the I<bio_err> and I<prog> parameters (unless
 192 NULL) to print additional diagnostic information in a user-oriented way.
 193
 194 OSSL_HTTP_parse_url() parses its input string I<url> as a URL
 195 of the form C<[http[s]://]address[:port][/path]> and splits it up into host,
 196 port, and path components and a flag indicating whether it begins with 'https'.
 197 The host component may be a DNS name or an IP address
 198 where IPv6 addresses should be enclosed in square brackets C<[> and C<]>.
 199 The port component is optional and defaults to "443" for HTTPS, else "80".
 200 If the I<pport_num> argument is NULL the port specification
 201 can be in mnemonic form such as "http" like with L<BIO_set_conn_port(3)>, else
 202 it must be in numerical form and its integer value is assigned to I<*pport_num>.
 203 The path component is also optional and defaults to "/".
 204 On success the function assigns via each non-NULL result pointer argument
 205 I<phost>, I<pport>, I<pport_num>, I<ppath>, and I<pssl>
 206 the respective url component.
 207 On error, I<*phost>, I<*pport>, and I<*ppath> are assigned to NULL,
 208 else they are guaranteed to contain non-NULL string pointers.
 209 It is the reponsibility of the caller to free them using L<OPENSSL_free(3)>.
 210 A string returned via I<*ppath> is guaranteed to begin with a C</> character.
 211
 212 =head1 NOTES
 213
 214 The names of the environment variables used by this implementation:
 215 C<http_proxy>, C<HTTP_PROXY>, C<https_proxy>, C<HTTPS_PROXY>, C<no_proxy>, and
 216 C<NO_PROXY>, have been chosen for maximal compatibility with
 217 other HTTP client implementations such as wget, curl, and git.
 218
 219 =head1 RETURN VALUES
 220
 221 OSSL_HTTP_get(), OSSL_HTTP_get_asn1(), OSSL_HTTP_post_asn1(), and
 222 OSSL_HTTP_transfer() return on success the data received via HTTP, else NULL.
 223 Error conditions include connection/transfer timeout, parse errors, etc.
 224
 225 OSSL_HTTP_proxy_connect() and OSSL_HTTP_parse_url()
 226 return 1 on success, 0 on error.
 227
 228 =head1 SEE ALSO
 229
 230 L<BIO_set_conn_port(3)>
 231
 232 =head1 HISTORY
 233
 234 OSSL_HTTP_get(), OSSL_HTTP_get_asn1(), OSSL_HTTP_post_asn1(),
 235 OSSL_HTTP_proxy_connect(), and OSSL_HTTP_parse_url() were added in OpenSSL 3.0.
 236
 237 =head1 COPYRIGHT
 238
 239 Copyright 2019-2021 The OpenSSL Project Authors. All Rights Reserved.
 240
 241 Licensed under the Apache License 2.0 (the "License").  You may not use
 242 this file except in compliance with the License.  You can obtain a copy
 243 in the file LICENSE in the source distribution or at
 244 L<https://www.openssl.org/source/license.html>.
 245
 246 =cut