From: Neil Horman <nhorman@openssl.org>
Date: Mon, 15 Jul 2024 18:30:16 +0000 (-0400)
Subject: Fix coverity-1604666
X-Git-Tag: openssl-3.3.2~52
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=1d37b00b28962095fba1baa1590d05539dcdc6db;p=thirdparty%2Fopenssl.git

Fix coverity-1604666

Coverity recently flaged an error in which the return value for
EVP_MD_get_size wasn't checked for negative values prior to use, which
can cause underflow later in the function.

Just add the check and error out if get_size returns an error.

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/24896)

(cherry picked from commit 22e08c7cdc596d4f16749811d1022fb8b07a8e41)
---

diff --git a/crypto/rsa/rsa_oaep.c b/crypto/rsa/rsa_oaep.c
index b9030440c45..52a0e8f0522 100644
--- a/crypto/rsa/rsa_oaep.c
+++ b/crypto/rsa/rsa_oaep.c
@@ -184,7 +184,7 @@ int RSA_padding_check_PKCS1_OAEP_mgf1(unsigned char *to, int tlen,
 
     mdlen = EVP_MD_get_size(md);
 
-    if (tlen <= 0 || flen <= 0)
+    if (tlen <= 0 || flen <= 0 || mdlen <= 0)
         return -1;
     /*
      * |num| is the length of the modulus; |flen| is the length of the