[MAJOR] http: add support for HTTP 1xx informational responses

HTTP supports status codes 100 and 101 to report protocol indications,
which are followed by the requests's response. Till now, haproxy would
only see those responses without parsing subsequent ones. That means
that cookie additions were only performed on 1xx messages for instance,
which does not work since headers must be ignored with 1xx messages.
Also, logs were not terribly useful with the common 100 status code
in response to "Expect: 100-continue" during POST some requests.

This change adds support for such messages. Now haproxy sees them,
forwards them and skips them until it finds a correct response, which
it logs and processes. As an exception, header removal/rewriting still
work on 1xx responses in order to be able to strip out sensible
information that may have accidentely been left by another equipment
(possibly an older haproxy itself). But headers addition are disabled
however.

This change brings the ability to loop on response without data, which
is a starting point to support keepalive. The change is marked as major
as a few fixes had to be performed in the HTTP message parser.

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 4a287b42bec750792543e723faa8437a0add6361..1ad0d4ea5b56ba24839b15f4407bbcd4fd0f3b74 100644 (file)
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -259,6 +259,17 @@ messages. Let's consider this HTTP response :
      2     Content-length: 350
      3     Content-Type: text/html
 
+As a special case, HTTP supports so called "Informational responses" as status
+codes 1xx. These messages are special in that they don't convey any part of the
+response, they're just used as sort of a signaling message to ask a client to
+continue to post its request for instance. The requested information will be
+carried by the next non-1xx response message following the informational one.
+This implies that multiple responses may be sent to a single request, and that
+this only works when keep-alive is enabled (1xx messages are HTTP/1.1 only).
+HAProxy handles these messages and is able to correctly forward and skip them,
+and only process the next non-1xx response. As such, these messages are neither
+logged nor transformed, unless explicitly state otherwise.
+
 
 1.3.1. The Response line
 ------------------------
@@ -270,6 +281,7 @@ Line 1 is the "response line". It is always composed of 3 fields :
   - a reason      : OK
 
 The status code is always 3-digit. The first digit indicates a general status :
+ - 1xx = informational message to be skipped (eg: 100, 101)
  - 2xx = OK, content is following   (eg: 200, 206)
  - 3xx = OK, no content following   (eg: 302, 304)
  - 4xx = error caused by the client (eg: 401, 403, 404)
@@ -4600,6 +4612,15 @@ passed during the first request of a TCP session will be seen. All subsequent
 headers will be considered data only and not analyzed. Furthermore, HAProxy
 never touches data contents, it stops analysis at the end of headers.
 
+There is an exception though. If HAProxy encounters an "Informational Response"
+(status code 1xx), it is able to process all rsp* rules which can allow, deny,
+rewrite or delete a header, but it will refuse to add a header to any such
+messages as this is not HTTP-compliant. The reason for still processing headers
+in such responses is to stop and/or fix any possible information leak which may
+happen, for instance because another downstream equipment would inconditionally
+add a header, or if a server name appears there. When such messages are seen,
+normal processing still occurs on the next non-informational messages.
+
 This section covers common usage of the following keywords, described in detail
 in section 4.2 :
 

diff --git a/src/proto_http.c b/src/proto_http.c
index d5077e1e47011670d18365a9cbd48aef6c5ba80f..1ecdc0b9155a3429a1f02b688601200d10d9695d 100644 (file)
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -1395,15 +1395,8 @@ void http_msg_analyzer(struct buffer *buf, struct http_msg *msg, struct hdr_idx
        http_msg_rpbefore:
        case HTTP_MSG_RPBEFORE:
                if (likely(HTTP_IS_TOKEN(*ptr))) {
-                       if (likely(ptr == buf->data)) {
-                               msg->sol = ptr;
-                               msg->som = 0;
-                       } else {
-#if PARSE_PRESERVE_EMPTY_LINES
-                               /* only skip empty leading lines, don't remove them */
-                               msg->sol = ptr;
-                               msg->som = ptr - buf->data;
-#else
+#if !defined(PARSE_PRESERVE_EMPTY_LINES)
+                       if (likely(ptr != buf->data)) {
                                /* Remove empty leading lines, as recommended by
                                 * RFC2616. This takes a lot of time because we
                                 * must move all the buffer backwards, but this
@@ -1411,14 +1404,12 @@ void http_msg_analyzer(struct buffer *buf, struct http_msg *msg, struct hdr_idx
                                 * cleaner when we'll be able to start sending
                                 * the request from any place in the buffer.
                                 */
-                               buf->lr = ptr;
-                               buffer_replace2(buf, buf->data, buf->lr, NULL, 0);
-                               msg->som = 0;
-                               msg->sol = buf->data;
-                               ptr = buf->data;
+                               ptr += buffer_replace2(buf, buf->lr, ptr, NULL, 0);
                                end = buf->r;
-#endif
                        }
+#endif
+                       msg->sol = ptr;
+                       msg->som = ptr - buf->data;
                        hdr_idx_init(idx);
                        state = HTTP_MSG_RPVER;
                        goto http_msg_rpver;
@@ -2766,6 +2757,7 @@ int process_response(struct session *t)
        struct buffer *req = t->req;
        struct buffer *rep = t->rep;
 
+ next_response:
        DPRINTF(stderr,"[%u] %s: session=%p b=%p, exp(r,w)=%u,%u bf=%08x bl=%d analysers=%02x\n",
                now_ms, __FUNCTION__,
                t,
@@ -2928,7 +2920,15 @@ int process_response(struct session *t)
                                        t->flags |= SN_FINST_H;
                                return 0;
                        }
-                       buffer_write_dis(rep);
+
+                       /* We disable sending only if we have nothing to send.
+                        * Note that we should not need to do this since the
+                        * buffer is protected by the fact that at least one
+                        * analyser remains. But close events could still be
+                        * forwarded if we don't disable the BF_WRITE_ENA flag.
+                        */
+                       if (!rep->send_max)
+                               buffer_write_dis(rep);
                        return 0;
                }
 
@@ -3034,7 +3034,8 @@ int process_response(struct session *t)
 
                        /* We might have to check for "Connection:" */
                        if (((t->fe->options | t->be->options) & (PR_O_HTTP_CLOSE|PR_O_FORCE_CLO)) &&
-                           !(t->flags & SN_CONN_CLOSED)) {
+                           !(t->flags & SN_CONN_CLOSED) &&
+                           txn->status >= 200) {
                                char *cur_ptr, *cur_end, *cur_next;
                                int cur_idx, old_idx, delta, val;
                                struct hdr_idx_elem *cur_hdr;
@@ -3084,6 +3085,8 @@ int process_response(struct session *t)
 
                        /* add response headers from the rule sets in the same order */
                        for (cur_idx = 0; cur_idx < rule_set->nb_rspadd; cur_idx++) {
+                               if (txn->status < 200)
+                                       break;
                                if (unlikely(http_header_add_tail(rep, &txn->rsp, &txn->hdr_idx,
                                                                  rule_set->rsp_add[cur_idx])) < 0)
                                        goto return_bad_resp;
@@ -3098,22 +3101,23 @@ int process_response(struct session *t)
                /*
                 * 4: check for server cookie.
                 */
-               if (t->be->cookie_name || t->be->appsession_name || t->fe->capture_name
-                   || (t->be->options & PR_O_CHK_CACHE))
+               if ((t->be->cookie_name || t->be->appsession_name || t->fe->capture_name
+                    || (t->be->options & PR_O_CHK_CACHE)) && txn->status >= 200)
                        manage_server_side_cookies(t, rep);
 
 
                /*
                 * 5: check for cache-control or pragma headers if required.
                 */
-               if ((t->be->options & (PR_O_COOK_NOC | PR_O_CHK_CACHE)) != 0)
+               if ((t->be->options & (PR_O_COOK_NOC | PR_O_CHK_CACHE)) != 0 && txn->status >= 200)
                        check_response_for_cacheability(t, rep);
 
                /*
                 * 6: add server cookie in the response if needed
                 */
                if ((t->srv) && !(t->flags & SN_DIRECT) && (t->be->options & PR_O_COOK_INS) &&
-                   (!(t->be->options & PR_O_COOK_POST) || (txn->meth == HTTP_METH_POST))) {
+                   (!(t->be->options & PR_O_COOK_POST) || (txn->meth == HTTP_METH_POST)) &&
+                   txn->status >= 200) {
                        int len;
 
                        /* the server is known, it's not the one the client requested, we have to
@@ -3156,7 +3160,8 @@ int process_response(struct session *t)
                 */
                if (((txn->flags & (TX_CACHEABLE | TX_CACHE_COOK | TX_SCK_ANY)) ==
                     (TX_CACHEABLE | TX_CACHE_COOK | TX_SCK_ANY)) &&
-                   (t->be->options & PR_O_CHK_CACHE)) {
+                   (t->be->options & PR_O_CHK_CACHE) &&
+                   txn->status >= 200) {
 
                        /* we're in presence of a cacheable response containing
                         * a set-cookie header. We'll block it as requested by
@@ -3179,7 +3184,8 @@ int process_response(struct session *t)
                 * Note that we do not need to add it in case of HTTP/1.0.
                 */
                if (!(t->flags & SN_CONN_CLOSED) &&
-                   ((t->fe->options | t->be->options) & (PR_O_HTTP_CLOSE|PR_O_FORCE_CLO))) {
+                   ((t->fe->options | t->be->options) & (PR_O_HTTP_CLOSE|PR_O_FORCE_CLO)) &&
+                   txn->status >= 200) {
                        if ((unlikely(msg->sl.st.v_l != 8) ||
                             unlikely(req->data[msg->som + 7] != '0')) &&
                            unlikely(http_header_add_tail2(rep, &txn->rsp, &txn->hdr_idx,
@@ -3188,6 +3194,21 @@ int process_response(struct session *t)
                        t->flags |= SN_CONN_CLOSED;
                }
 
+               /*
+                * 9: we may be facing a 1xx response (100 continue, 101 switching protocols),
+                * in which case this is not the right response, and we're waiting for the
+                * next one. Let's allow this response to go to the client and wait for the
+                * next one.
+                */
+               if (txn->status < 200) {
+                       hdr_idx_init(&txn->hdr_idx);
+                       buffer_forward(rep, rep->lr - (rep->data + msg->som));
+                       msg->msg_state = HTTP_MSG_RPBEFORE;
+                       txn->status = 0;
+                       rep->analysers |= AN_RTR_HTTP_HDR;
+                       goto next_response;
+               }
+
                /*************************************************************
                 * OK, that's finished for the headers. We have done what we *
                 * could. Let's switch to the DATA state.                    *