Fail serialization of finalized MD contexts

Add tests to verify that `EVP_MD_CTX_serialize` and
`EVP_MD_CTX_deserialize` fail when called on a finalized `EVP_MD_CTX`.

A finalized context is in a terminal state and should not be serializable for
resumption or have a new state deserialized into it. These tests confirm the
expected failure behavior.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

diff --git a/test/evp_extra_test2.c b/test/evp_extra_test2.c
index edff52975d878328731e6609ba2cdc8694fa9194..6552a9e6bcca11c0f95551c61550dd16c8a7e819 100644 (file)
--- a/test/evp_extra_test2.c
+++ b/test/evp_extra_test2.c
@@ -3442,6 +3442,7 @@ static int test_evp_md_ctx_serialize(int tstid)
     EVP_MD *md = NULL;
     unsigned char *buf = NULL;
     size_t buflen;
+    size_t tmplen;
     unsigned char d1[EVP_MAX_MD_SIZE], d2[EVP_MAX_MD_SIZE];
     unsigned int d1_len, d2_len;
     int ret = 0;
@@ -3481,6 +3482,11 @@ static int test_evp_md_ctx_serialize(int tstid)
         || !TEST_mem_eq(d1, d1_len, d2, d2_len))
         goto end;
 
+    /* Check that serialization fails on finalized contexts */
+    if (!TEST_false(EVP_MD_CTX_serialize(mdctx1, NULL, &tmplen))
+        || !TEST_false(EVP_MD_CTX_deserialize(mdctx1, buf, buflen)))
+        goto end;
+
     ret = 1;
 
 end: