From: Willy Tarreau Date: Sat, 4 Nov 2023 07:34:23 +0000 (+0100) Subject: OPTIM: mux-h2: call h2_send() directly from h2_snd_buf() X-Git-Tag: v2.9-dev9~2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=e16762f8a;p=thirdparty%2Fhaproxy.git OPTIM: mux-h2: call h2_send() directly from h2_snd_buf() This allows to eliminate full buffers very quickly and to recycle them much faster, resulting in higher transfer rates and lower memory usage at the same time. We just wake the tasklet up if it succeeded so that h2_process() and friends are called to finalize what needs to. For regular buffer sizes, the performance level becomes quite close to the one obtained with the zero-copy mechanism (zero-copy remains much faster with non-default buffer sizes). The memory savings are huge with default buffer size: at 64c * 100 streams on a single thread, we used to forward 4.4 Gbps of traffic using 10400 buffers. After the change, the performance reaches 5.9 Gbps with only 22-24 buffers, since they are quickly recycled. That's asaving of 160 MB of RAM. A concern was an increase in the number of syscalls but this is not the case, the numbers remained exactly the same before and after. Some experimentations were made to try to cork data and not send incomplete buffers, and that always voided these changes. One explanation might be that keeping a first buffer with only headers frames is sufficient to prevent a zero-copy of the data coming in a next snd_buf() call. This still needs to be studied anyway. --- diff --git a/src/mux_h2.c b/src/mux_h2.c index d0ab3c8a01..95003cfb3e 100644 --- a/src/mux_h2.c +++ b/src/mux_h2.c @@ -6873,7 +6873,8 @@ static size_t h2_snd_buf(struct stconn *sc, struct buffer *buf, size_t count, in if (total > 0) { if (!(h2s->h2c->wait_event.events & SUB_RETRY_SEND)) { TRACE_DEVEL("data queued, waking up h2c sender", H2_EV_H2S_SEND|H2_EV_H2C_SEND, h2s->h2c->conn, h2s); - tasklet_wakeup(h2s->h2c->wait_event.tasklet); + if (h2_send(h2s->h2c)) + tasklet_wakeup(h2s->h2c->wait_event.tasklet); } }