Add some documentation to describe the encap/decap requirements

Document the fact that we now require unwrappedlen/wrappedlen to be set
to the size of the unwrapped/wrapped buffers

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25522)

(cherry picked from commit 1c1223ff535944de880a23cbf0ef9bba6092b0d9)

diff --git a/doc/man3/EVP_PKEY_decapsulate.pod b/doc/man3/EVP_PKEY_decapsulate.pod
index 833c34152ac138a5e6ecfa641038ea1de351f38b..cd6f5f0221a2bbb9954ead0615d53921202690f2 100644 (file)
--- a/doc/man3/EVP_PKEY_decapsulate.pod
+++ b/doc/man3/EVP_PKEY_decapsulate.pod
@@ -25,10 +25,13 @@ specifying the private key to use.
 The EVP_PKEY_decapsulate() function performs a private key decapsulation
 operation using I<ctx>. The data to be decapsulated is specified using the
 I<wrapped> and I<wrappedlen> parameters.
-If I<unwrapped> is NULL then the maximum size of the output secret buffer
+If I<unwrapped> is NULL then the size of the output secret buffer
 is written to I<*unwrappedlen>. If I<unwrapped> is not NULL and the
 call is successful then the decapsulated secret data is written to I<unwrapped>
-and the amount of data written to I<*unwrappedlen>.
+and the amount of data written to I<*unwrappedlen>.  Note that, if I<unwrappedlen>
+is not NULL in this call, the value it points to must be initialised to the length of
+I<unwrapped>, so that the call can validate it is of sufficient size to hold the
+result of the operation.
 
 =head1 NOTES
 

diff --git a/doc/man3/EVP_PKEY_encapsulate.pod b/doc/man3/EVP_PKEY_encapsulate.pod
index 573088c3bf58face044f6e01eb30e803c15197e4..eb51836d795122ea2d34aa8fa135b000e10fd69c 100644 (file)
--- a/doc/man3/EVP_PKEY_encapsulate.pod
+++ b/doc/man3/EVP_PKEY_encapsulate.pod
@@ -35,7 +35,10 @@ unless I<genkeylen> is NULL.
 If I<wrappedkey> is not NULL and the call is successful then the
 internally generated key is written to I<genkey> and its size is written to
 I<*genkeylen>. The encapsulated version of the generated key is written to
-I<wrappedkey> and its size is written to I<*wrappedkeylen>.
+I<wrappedkey> and its size is written to I<*wrappedkeylen>.  Note that if
+I<wrappedlen> is not NULL, then the value it points to must initially hold the size of
+the  I<unwrapped> buffer so that its size can be validated by the call, ensuring
+it is large enough to hold the result written to I<wrapped>.
 
 =head1 NOTES
 

diff --git a/providers/implementations/kem/rsa_kem.c b/providers/implementations/kem/rsa_kem.c
index 6b48c8e92b8c0b063719862b8756845eec5b2c83..d3fdb69614a668ce3b3298ccf2b08640aa8f01d1 100644 (file)
--- a/providers/implementations/kem/rsa_kem.c
+++ b/providers/implementations/kem/rsa_kem.c
@@ -266,6 +266,11 @@ static int rsasve_generate(PROV_RSA_CTX *prsactx,
         return 1;
     }
 
+    /*
+     * If outlen is specified, then it must report the length
+     * of the out buffer on input so that we can confirm
+     * its size is sufficent for encapsulation
+     */
     if (outlen != NULL && *outlen < nlen) {
         ERR_raise(ERR_LIB_PROV, PROV_R_INVALID_OUTPUT_LENGTH);
         return 0;
@@ -340,6 +345,12 @@ static int rsasve_recover(PROV_RSA_CTX *prsactx,
         ERR_raise(ERR_LIB_PROV, PROV_R_BAD_LENGTH);
         return 0;
     }
+
+    /*
+     * If outlen is specified, then it must report the length
+     * of the out buffer, so that we can confirm that it is of
+     * sufficient size to hold the output of decapsulation
+     */
     if (outlen != NULL && *outlen < nlen) {
         ERR_raise(ERR_LIB_PROV, PROV_R_INVALID_OUTPUT_LENGTH);
         return 0;