From: slontis <shane.lontis@oracle.com>
Date: Fri, 1 Jul 2022 03:47:11 +0000 (+1000)
Subject: Fix bn_gcd code to check return value when calling BN_one()
X-Git-Tag: openssl-3.2.0-alpha1~2441
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=7fe7cc57af3db1e497877f0329ba17609b2efc8b;p=thirdparty%2Fopenssl.git

Fix bn_gcd code to check return value when calling BN_one()

BN_one() uses the expand function which calls malloc which may fail.
All other places that reference BN_one() check the return value.

The issue is triggered by a memory allocation failure.
Detected by PR #18355

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18697)
---

diff --git a/crypto/bn/bn_gcd.c b/crypto/bn/bn_gcd.c
index 6d709811ac0..2b42c7df97f 100644
--- a/crypto/bn/bn_gcd.c
+++ b/crypto/bn/bn_gcd.c
@@ -47,7 +47,8 @@ BIGNUM *bn_mod_inverse_no_branch(BIGNUM *in,
     if (R == NULL)
         goto err;
 
-    BN_one(X);
+    if (!BN_one(X))
+        goto err;
     BN_zero(Y);
     if (BN_copy(B, a) == NULL)
         goto err;
@@ -235,7 +236,8 @@ BIGNUM *int_bn_mod_inverse(BIGNUM *in,
     if (R == NULL)
         goto err;
 
-    BN_one(X);
+    if (!BN_one(X))
+        goto err;
     BN_zero(Y);
     if (BN_copy(B, a) == NULL)
         goto err;