rsa_kem: validate RSA_public_encrypt() result in RSASVE

rsa_kem: validate RSA_public_encrypt() result in RSASVE

RSA_public_encrypt() returns the number of bytes written on success and
-1 on failure. With the existing `if (ret)` check, a provider-side RSA KEM
encapsulation can incorrectly succeed when the underlying RSA public
encrypt operation fails. In that case the code reports success, returns
lengths as if encapsulation completed normally, and leaves the freshly
generated secret available instead of discarding it.

Tighten the success condition so RSASVE only succeeds when
RSA_public_encrypt() returns a positive value equal to the modulus-sized
output expected for RSA_NO_PADDING. Any other return value is treated as
failure, and the generated secret is cleansed before returning.

Fixes CVE-2026-31790
Signed-off-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.foundation>
Reviewed-by: Tomas Mraz <tomas@openssl.foundation>
MergeDate: Mon Apr  6 19:45:38 2026
(cherry picked from commit 89dde74b69debbf0c4d0a0ee925de87638bbfe16)