ossl_cmp_error_new(): Fix Coverity issue 1486534, and consequently also issues 148653...

The issues are due to an integer overflow that may happen on '(ERR_SYSTEM_FLAG << 1)'.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15938)

diff --git a/crypto/cmp/cmp_msg.c b/crypto/cmp/cmp_msg.c
index fe4b64d575cbdae3cb64bfb70de18f63e8f56992..4fef006933a6bba6d44819a7e4c4a8cb8844e136 100644 (file)
--- a/crypto/cmp/cmp_msg.c
+++ b/crypto/cmp/cmp_msg.c
@@ -748,7 +748,8 @@ OSSL_CMP_MSG *ossl_cmp_error_new(OSSL_CMP_CTX *ctx, const OSSL_CMP_PKISI *si,
         goto err;
     if (!ASN1_INTEGER_set_int64(msg->body->value.error->errorCode, errorCode))
         goto err;
-    if (errorCode > 0 && errorCode < (ERR_SYSTEM_FLAG << 1)) {
+    if (errorCode > 0
+            && (uint64_t)errorCode < ((uint64_t)ERR_SYSTEM_FLAG << 1)) {
         lib = ERR_lib_error_string((unsigned long)errorCode);
         reason = ERR_reason_error_string((unsigned long)errorCode);
     }