Fix DB2 hash bitmap page count validation

In __kdb2_hash_open(), bpages is computed from the hash file header
and then used as the size argument when clearing hashp->mapp.  The
mapp array has only NCACHED entries, so a malformed hash database can
cause memset() to write past the end of the array.  Return EFTYPE if
the computed bitmap page count is negative or greater then NCACHED.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

ticket: 9215

diff --git a/src/plugins/kdb/db2/libdb2/hash/hash.c b/src/plugins/kdb/db2/libdb2/hash/hash.c
index 7c3e951aa25c0d9711b4193bb2a95828b7605fb5..f90aae0f9afcfc9695564291bf73f8a6e2b035df 100644 (file)
--- a/src/plugins/kdb/db2/libdb2/hash/hash.c
+++ b/src/plugins/kdb/db2/libdb2/hash/hash.c
@@ -170,6 +170,9 @@ __kdb2_hash_open(const char *file, int flags, int mode, const HASHINFO *info,
                    (hashp->hdr.bsize << BYTE_SHIFT) - 1) >>
                    (hashp->hdr.bshift + BYTE_SHIFT);
 
                    (hashp->hdr.bsize << BYTE_SHIFT) - 1) >>
                    (hashp->hdr.bshift + BYTE_SHIFT);
 

+               if (bpages > NCACHED || bpages < 0)
+                       RETURN_ERROR(EFTYPE, error1);
+

                hashp->nmaps = bpages;
                (void)memset(&hashp->mapp[0], 0, bpages * sizeof(u_int32_t *));
        }
                hashp->nmaps = bpages;
                (void)memset(&hashp->mapp[0], 0, bpages * sizeof(u_int32_t *));
        }