rsa_sig: reject short buffers in raw verify_recover

The md==NULL path of rsa_verify_recover passed the caller buffer to
RSA_public_decrypt without checking routsize, while the X9.31 and PKCS#1
paths already reject undersized output buffers. RSA_public_decrypt writes
up to RSA_size() bytes, so a short rout overflows. Validate routsize
against RSA_size() before the call.

Fixes: 6f4b7663150e "PROV: add RSA signature implementation"
Reviewed-by: Tomas Mraz <tomas@openssl.foundation>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
MergeDate: Tue Jun  2 11:55:00 2026
(Merged from https://github.com/openssl/openssl/pull/31340)

diff --git a/providers/implementations/signature/rsa_sig.c b/providers/implementations/signature/rsa_sig.c
index 43f648e2d4268e9837e22a49989cc734e806d58f..f2be3fd94c18b7427eb166a2374e8cd33bcc7daf 100644 (file)
--- a/providers/implementations/signature/rsa_sig.c
+++ b/providers/implementations/signature/rsa_sig.c
@@ -1016,6 +1016,14 @@ static int rsa_verify_recover(void *vprsactx,
             return 0;
         }
     } else {
+        int rsasize = RSA_size(prsactx->rsa);
+
+        if (routsize < (size_t)rsasize) {
+            ERR_raise_data(ERR_LIB_PROV, PROV_R_OUTPUT_BUFFER_TOO_SMALL,
+                "buffer size is %d, should be %d",
+                routsize, rsasize);
+            return 0;
+        }
         ret = RSA_public_decrypt((int)siglen, sig, rout, prsactx->rsa,
             prsactx->pad_mode);
         if (ret <= 0) {