set SSLfatal if tls1_set_shared_sigalgs has a malloc failure

Detected another memfail failure

https://github.com/openssl/openssl/actions/runs/16926186604/job/47962169870

Tracking it back, it occurs because tls1_set_server_sigalgs attempts to
preform an allocation, and in the event of failure, returns 0 without
setting SSLfatal, like the other failure paths in this function do when
returning 0, which translates to a return of WORK_ERROR higher up the
stack

The result is that on the next call to check_fatal in
read_state_machine, we fail the assert when deubg is enabled (as it is
in the coverage tests).

Fix it by calling SSLfatal when the call to OPENSSL_calloc fails in this
function.

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28250)

diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index aa41a341ac1ebe735031a7d82f757ad198bcf249..e556bff26318b6230eaed7a8f1a46ffa0dce9136 100644 (file)
--- a/ssl/t1_lib.c
+++ b/ssl/t1_lib.c
@@ -2960,8 +2960,10 @@ int tls1_set_server_sigalgs(SSL_CONNECTION *s)
         memset(s->s3.tmp.valid_flags, 0, s->ssl_pkey_num * sizeof(uint32_t));
     else
         s->s3.tmp.valid_flags = OPENSSL_calloc(s->ssl_pkey_num, sizeof(uint32_t));
-    if (s->s3.tmp.valid_flags == NULL)
+    if (s->s3.tmp.valid_flags == NULL) {
+        SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR);
         return 0;
+    }
     /*
      * If peer sent no signature algorithms check to see if we support
      * the default algorithm for each certificate type