Defer the check that the record fits in our buffer

Previously we confirmed that the record we received from the peer
actually fits in our buffer before validating the record header. However,
this interferes with the checks that the "any" method does for accidental
use of HTTP because the record length will be wrong in this case. To solve
this we simply defer the check until after the record header has been
validated.

Fixes #30196

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
MergeDate: Fri Mar 13 15:12:41 2026
(Merged from https://github.com/openssl/openssl/pull/30204)

(cherry picked from commit 69d0f3febe446c61b6ef395bc3372a58899f5171)

diff --git a/ssl/record/methods/tls_common.c b/ssl/record/methods/tls_common.c
index 64a53b0cb0b4282b67682c6fd2e38a5236dfb2be..0ffdfe500a2786590cf34d34ff9b6107b5b1edb5 100644 (file)
--- a/ssl/record/methods/tls_common.c
+++ b/ssl/record/methods/tls_common.c
@@ -608,14 +608,14 @@ int tls_get_more_records(OSSL_RECORD_LAYER *rl)
             if (rl->msg_callback != NULL)
                 rl->msg_callback(0, version, SSL3_RT_HEADER, p, 5, rl->cbarg);
 
-            if (thisrr->length > TLS_BUFFER_get_len(rbuf) - SSL3_RT_HEADER_LENGTH) {
-                RLAYERfatal(rl, SSL_AD_RECORD_OVERFLOW,
-                    SSL_R_PACKET_LENGTH_TOO_LONG);
+            if (!rl->funcs->validate_record_header(rl, thisrr)) {
+                /* RLAYERfatal already called */
                 return OSSL_RECORD_RETURN_FATAL;
             }
 
-            if (!rl->funcs->validate_record_header(rl, thisrr)) {
-                /* RLAYERfatal already called */
+            if (thisrr->length > TLS_BUFFER_get_len(rbuf) - SSL3_RT_HEADER_LENGTH) {
+                RLAYERfatal(rl, SSL_AD_RECORD_OVERFLOW,
+                    SSL_R_PACKET_LENGTH_TOO_LONG);
                 return OSSL_RECORD_RETURN_FATAL;
             }