buffers: fix handshake_compare when sequence numbers match

The comparator function used for ordering DTLS packets
by sequence numbers did not follow qsort comparator contracts
in case of packets with duplicate sequence numbers,
which could lead to unstable ordering or undefined behaviour.
Returning 0 in such cases makes the sorting stable.

Reported-by: Joshua Rogers of AISLE Research Team <joshua@joshua.hu>
Fixes: #1848
Fixes: CVE-2026-42009
Fixes: GNUTLS-SA-2026-04-29-2
CVSS: 7.5 High CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
Signed-off-by: Joshua Rogers <joshua@joshua.hu>

diff --git a/lib/buffers.c b/lib/buffers.c
index 48f4a3210c0693793639239aa0c29ce2c2521c60..09779a8f31fb67442bec8df0dba13ca816afa8cf 100644 (file)
--- a/lib/buffers.c
+++ b/lib/buffers.c
@@ -844,11 +844,7 @@ static int handshake_compare(const void *_e1, const void *_e2)
 {
        const handshake_buffer_st *e1 = _e1;
        const handshake_buffer_st *e2 = _e2;
-
-       if (e1->sequence <= e2->sequence)
-               return 1;
-       else
-               return -1;
+       return (e1->sequence < e2->sequence) - (e1->sequence > e2->sequence);
 }
 
 #define SSL2_HEADERS 1