From: Nick Terrell <terrelln@meta.com>
Date: Mon, 21 Aug 2023 18:33:29 +0000 (-0700)
Subject: No longer reject dictionaries with literals maxSymbolValue < 255
X-Git-Tag: v1.5.6^2~132
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=bd02c9be6e3708c6dd53f4df1f4dc13d29441e89;p=thirdparty%2Fzstd.git

No longer reject dictionaries with literals maxSymbolValue < 255

We already have logic in our Huffman encoder to validate Huffman tables with missing symbols.
We use this for higher compression levels to re-use the previous blocks statistics, or when the dictionaries table has zero-weighted symbols.
This check was leftover as an oversight from before we added validation for Huffman tables.

I validated that the `dictionary_loader` fuzzer has coverage of every line in the `ZSTD_loadCEntropy()` function to validate that it is correctly testing this function.
---

diff --git a/lib/compress/zstd_compress.c b/lib/compress/zstd_compress.c
index 209f3b0ee..36c1f99e8 100644
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -4904,11 +4904,10 @@ size_t ZSTD_loadCEntropy(ZSTD_compressedBlockState_t* bs, void* workspace,
 
         /* We only set the loaded table as valid if it contains all non-zero
          * weights. Otherwise, we set it to check */
-        if (!hasZeroWeights)
+        if (!hasZeroWeights && maxSymbolValue == 255)
             bs->entropy.huf.repeatMode = HUF_repeat_valid;
 
         RETURN_ERROR_IF(HUF_isError(hufHeaderSize), dictionary_corrupted, "");
-        RETURN_ERROR_IF(maxSymbolValue < 255, dictionary_corrupted, "");
         dictPtr += hufHeaderSize;
     }