Fix incomplete error check on RSA_public_decrypt()

According to the documentation and my analysis tool RSA_public_decrypt()
can return -1 on error, but this is not checked. Fix it by changing the
error condition.

CLA: trivial

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20250)

diff --git a/providers/implementations/signature/rsa_sig.c b/providers/implementations/signature/rsa_sig.c
index 7463efbc0fdf952f2ecdfcd9215e50a5914d0ed7..e0faf1c1ad3c511c787d8f4854cb5fd4a92e6671 100644 (file)
--- a/providers/implementations/signature/rsa_sig.c
+++ b/providers/implementations/signature/rsa_sig.c
@@ -838,7 +838,7 @@ static int rsa_verify(void *vprsactx, const unsigned char *sig, size_t siglen,
             return 0;
         rslen = RSA_public_decrypt(siglen, sig, prsactx->tbuf, prsactx->rsa,
                                    prsactx->pad_mode);
-        if (rslen == 0) {
+        if (rslen <= 0) {
             ERR_raise(ERR_LIB_PROV, ERR_R_RSA_LIB);
             return 0;
         }