The rsa_validate_keypair_multiprime() function return is not boolean

A -ve return value from this function indicates an error which we should
treat as a failure to validate.

Fixes #18538

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18651)

diff --git a/crypto/rsa/rsa_chk.c b/crypto/rsa/rsa_chk.c
index 01fe9ead69b684b08e7e5200bd2d761a7692daa3..8ab116687427f5e1f179710732f696adfc1d462e 100644 (file)
--- a/crypto/rsa/rsa_chk.c
+++ b/crypto/rsa/rsa_chk.c
@@ -245,7 +245,7 @@ int ossl_rsa_validate_pairwise(const RSA *key)
 #ifdef FIPS_MODULE
     return ossl_rsa_sp800_56b_check_keypair(key, NULL, -1, RSA_bits(key));
 #else
-    return rsa_validate_keypair_multiprime(key, NULL);
+    return rsa_validate_keypair_multiprime(key, NULL) > 0;
 #endif
 }