BUG/MEDIUM: quic: consume contig space on requeue datagram

When handling UDP datagram reception, it is possible to receive a QUIC
packet for one connection to the socket attached to another connection.
To protect against this, an explicit comparison is done against the
packet DCID and the quic-conn CID. On no match, the datagram is requeued
and dispatched via rxbuf and will be treated as if it arrived on the
listener socket.

One reason for this wrong reception is explained by the small race
condition that exists between bind() and connect() syscalls during
connection socket initialization. However, one other reason which was
not thought initially is when clients reuse the same IP:PORT for
different connections. In this case the current FD attribution is not
optimal and this can cause a substantial number of requeuing.

This situation has revealed a bug during requeuing. If rxbuf contig
space is not big enough for the datagram, the incoming datagram was
dropped, even if there is space at buffer origin. This can cause several
datagrams to be dropped in a series until eventually buffer head is
moved when passing through the listener FD.

To fix this, allocate a fake datagram to consume contig space. This is
similar to the handling of datagrams on the listener FD. This allows
then to store the datagram to requeue on buffer head and continue.

This can be reproduced by starting a lot of connections. To increase the
phenomena, POST are used to increase the number of datagram dropping :

$ while true; do curl -F "a=@~/50k" -k --http3-only -o /dev/null https://127.0.0.1:20443/; done

diff --git a/src/quic_sock.c b/src/quic_sock.c
index 887568663b0656a782786c9b1f91d4e649c6102c..f45cd91cadd8ac1396a987b3f4d3193b35f0feb7 100644 (file)
--- a/src/quic_sock.c
+++ b/src/quic_sock.c
@@ -734,14 +734,31 @@ int qc_rcv_buf(struct quic_conn *qc)
                        struct quic_receiver_buf *rxbuf;
                        struct quic_dgram *tmp_dgram;
                        unsigned char *rxbuf_tail;
+                       size_t cspace;
 
                        TRACE_STATE("datagram for other connection on quic-conn socket, requeue it", QUIC_EV_CONN_RCV, qc);
 
                        rxbuf = MT_LIST_POP(&l->rx.rxbuf_list, typeof(rxbuf), rxbuf_el);
+                       cspace = b_contig_space(&rxbuf->buf);
 
                        tmp_dgram = quic_rxbuf_purge_dgrams(rxbuf);
                        pool_free(pool_head_quic_dgram, tmp_dgram);
 
+                       /* Insert a fake datagram if space wraps to consume it. */
+                       if (cspace < new_dgram->len && b_space_wraps(&rxbuf->buf)) {
+                               struct quic_dgram *fake_dgram = pool_alloc(pool_head_quic_dgram);
+                               if (!fake_dgram) {
+                                       /* TODO count lost datagrams */
+                                       continue;
+                               }
+
+                               fake_dgram->buf = NULL;
+                               fake_dgram->len = cspace;
+                               LIST_APPEND(&rxbuf->dgram_list, &fake_dgram->recv_list);
+                               b_add(&rxbuf->buf, cspace);
+                       }
+
+                       /* Recheck contig space after fake datagram insert. */
                        if (b_contig_space(&rxbuf->buf) < new_dgram->len) {
                                /* TODO count lost datagrams */
                                MT_LIST_APPEND(&l->rx.rxbuf_list, &rxbuf->rxbuf_el);