Don't truncate the input when decrypting in pkeyutl

The pkeyutl app was truncating the input file for decryption leading to
incorrect results. This was probably ok historically when RSA was being
used for decryption which has short maximum sizes. This is not ok with SM2.

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21272)

diff --git a/apps/lib/apps.c b/apps/lib/apps.c
index c9b509525cfef412ecb76bcc88949191b3654467..887fa74dea01203b761cb564997b7acb5491066e 100644 (file)
--- a/apps/lib/apps.c
+++ b/apps/lib/apps.c
@@ -2043,7 +2043,8 @@ int bio_to_mem(unsigned char **out, int maxlen, BIO *in)
             BIO_free(mem);
             return -1;
         }
-        maxlen -= len;
+        if (maxlen != -1)
+            maxlen -= len;
 
         if (maxlen == 0)
             break;

diff --git a/apps/pkeyutl.c b/apps/pkeyutl.c
index d8b2ad5f652d78767741b407207ad1e2d18c0d62..efd98684cb46b4e3b1f1967d905786abacbcacc9 100644 (file)
--- a/apps/pkeyutl.c
+++ b/apps/pkeyutl.c
@@ -420,7 +420,7 @@ int pkeyutl_main(int argc, char **argv)
     /* Raw input data is handled elsewhere */
     if (in != NULL && !rawin) {
         /* Read the input data */
-        buf_inlen = bio_to_mem(&buf_in, keysize * 10, in);
+        buf_inlen = bio_to_mem(&buf_in, -1, in);
         if (buf_inlen < 0) {
             BIO_printf(bio_err, "Error reading input Data\n");
             goto end;