BUG/MEDIUM: h2: always consume any trailing data after end of output buffers

In case a stream tries to emit more data than advertised by the chunks
or content-length headers, the extra data remains in the channel's output
buffer until the channel's timeout expires. It can easily happen when
sending malformed error files making use of a wrong content-length or
having extra CRLFs after the empty chunk. It may also be possible to
forge such a bad response using Lua.

The H1 to H2 encoder must protect itself against this by marking the data
presented to it as consumed if it decides to discard them, so that the
sending stream doesn't wait for the timeout to trigger.

The visible effect of this problem is a huge memory usage and a high
concurrent connection count during benchmarks when using such bad data
(a typical place where this easily happens).

This fix must be backported to 1.8.

diff --git a/src/mux_h2.c b/src/mux_h2.c
index de05ca04afe01fb6988331032987039f76ab15a2..1857fe3d2d3a5641df68b865c88919864eafc5b7 100644 (file)
--- a/src/mux_h2.c
+++ b/src/mux_h2.c
@@ -3030,6 +3030,9 @@ static int h2s_frt_make_resp_headers(struct h2s *h2s, struct buffer *buf)
         * body or directly end in TRL2.
         */
        if (es_now) {
+               // trim any possibly pending data (eg: inconsistent content-length)
+               bo_del(buf, buf->o);
+
                h1m->state = HTTP_MSG_DONE;
                h2s->flags |= H2_SF_ES_SENT;
                if (h2s->st == H2_SS_OPEN)
@@ -3279,8 +3282,12 @@ static int h2s_frt_make_resp_data(struct h2s *h2s, struct buffer *buf)
                else
                        h2c_stream_close(h2c, h2s);
 
-               if (!(h1m->flags & H1_MF_CHNK))
+               if (!(h1m->flags & H1_MF_CHNK)) {
+                       // trim any possibly pending data (eg: inconsistent content-length)
+                       bo_del(buf, buf->o);
+
                        h1m->state = HTTP_MSG_DONE;
+               }
 
                h2s->flags |= H2_SF_ES_SENT;
        }
@@ -3329,6 +3336,10 @@ static int h2_snd_buf(struct conn_stream *cs, struct buffer *buf, int flags)
                        }
                        total += count;
                        bo_del(buf, count);
+
+                       // trim any possibly pending data (eg: extra CR-LF, ...)
+                       bo_del(buf, buf->o);
+
                        h2s->res.state = HTTP_MSG_DONE;
                        break;
                }