Fix arithmetic expression overflow

If the value of a->length is large (>= 2^12), then an integer overflow will
occur for the signed type, which according to the C standard is UB.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/23274)

(cherry picked from commit 486ab0fb003d05f89620662260486d31bd3faa8c)

diff --git a/crypto/objects/obj_dat.c b/crypto/objects/obj_dat.c
index 85d30eb58ae013c5bcce8bee7556067276d9b127..d7e55d0e06bb439765f07a97c32652a4f7ec39ae 100644 (file)
--- a/crypto/objects/obj_dat.c
+++ b/crypto/objects/obj_dat.c
@@ -62,7 +62,7 @@ static unsigned long added_obj_hash(const ADDED_OBJ *ca)
     a = ca->obj;
     switch (ca->type) {
     case ADDED_DATA:
-        ret = a->length << 20L;
+        ret = (unsigned long)a->length << 20UL;
         p = (unsigned char *)a->data;
         for (i = 0; i < a->length; i++)
             ret ^= p[i] << ((i * 3) % 24);