BUG/MEDIUM: mux-h1: Don't set MSG_MORE on bodyless responses forwarded to client

When h1_snd_buf() inherits the CO_SFL_MSG_MORE flag from the upper layer, it
unconditionally propagates it to H1C_F_CO_MSG_MORE, which eventually sets
MSG_MORE on the sendmsg() call. For bodyless responses (HEAD, 204, 304), this
causes the kernel to cork the TCP connection for ~200ms waiting for body data
that will never be sent.

With an H1 frontend and H2 backend, this adds ~200ms of latency to many or
all bodyless responses. The 200ms corresponds to the kernel's tcp_cork_time
default. H1 backends are less affected because h1_postparse_res_hdrs() sets
HTX_FL_EOM during header parsing for bodyless responses, but H2 backends
frequently deliver the end-of-stream signal in a separate scheduling round,
leaving htx_expect_more() returning TRUE when headers are first forwarded.

The fix guards H1C_F_CO_MSG_MORE so it is only set when the connection is a
backend (H1C_F_IS_BACK) or the response is not bodyless
(!H1S_F_BODYLESS_RESP). This ensures bodyless responses on the front
connection are sent immediately without corking.

This should be backported to all stable branches.

Co-developed-by: Billy Campoli <bcampoli@meta.com>
Co-developed-by: Chandan Avdhut <cavdhut@meta.com>
Co-developed-by: Neel Raja <neelraja@meta.com

diff --git a/src/mux_h1.c b/src/mux_h1.c
index 47cdfff2aa34b3a4896d3f1eadec17a38449e6e9..d165b88d6e7a83071b4c8863fdc978a77776d2cb 100644 (file)
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -4885,8 +4885,14 @@ static size_t h1_snd_buf(struct stconn *sc, struct buffer *buf, size_t count, in
 
        /* Inherit some flags from the upper layer */
        h1c->flags &= ~(H1C_F_CO_MSG_MORE|H1C_F_CO_STREAMER);
-       if (flags & CO_SFL_MSG_MORE)
-               h1c->flags |= H1C_F_CO_MSG_MORE;
+       if (flags & CO_SFL_MSG_MORE) {
+               /* Don't set H1C_F_CO_MSG_MORE when sending a bodyless response to client.
+                * We must do that if the response is not finished, regardless it a bodyless
+                * response, to be sure to send it ASAP.
+                */
+               if ((h1c->flags & H1C_F_IS_BACK) || !(h1s->flags & H1S_F_BODYLESS_RESP))
+                       h1c->flags |= H1C_F_CO_MSG_MORE;
+       }
        if (flags & CO_SFL_STREAMER)
                h1c->flags |= H1C_F_CO_STREAMER;