From: Jonathan McDowell <noodles@fb.com>
Date: Tue, 1 Feb 2022 11:20:30 +0000 (-0800)
Subject: Fix required decompression memory usage reported by -vv + --long
X-Git-Tag: v1.5.4^2~245^2
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fpull%2F3042%2Fhead;p=thirdparty%2Fzstd.git

Fix required decompression memory usage reported by -vv + --long

The use of --long alters the window size internally in the underlying
library (lib/compress/zstd_compress.c:ZSTD_getCParamsFromCCtxParams),
which changes the memory required for decompression. This means that the
reported requirement from the zstd binary when -vv is specified is
incorrect.

A full fix for this would be to add an API call to be able to retrieve
the required decompression memory from the library, but as a
lighterweight fix we can just take account of the fact we've enabled
long mode and update our verbose output appropriately.

Fixes #2968
---

diff --git a/programs/fileio.c b/programs/fileio.c
index 502f69c15..e37921ada 100644
--- a/programs/fileio.c
+++ b/programs/fileio.c
@@ -1301,8 +1301,13 @@ FIO_compressZstdFrame(FIO_ctx_t* const fCtx,
         UTIL_HumanReadableSize_t windowSize;
         CHECK(ZSTD_CCtx_getParameter(ress.cctx, ZSTD_c_windowLog, &windowLog));
         if (windowLog == 0) {
-            const ZSTD_compressionParameters cParams = ZSTD_getCParams(compressionLevel, fileSize, 0);
-            windowLog = cParams.windowLog;
+            if (prefs->ldmFlag) {
+                /* If long mode is set without a window size libzstd will set this size internally */
+                windowLog = ZSTD_WINDOWLOG_LIMIT_DEFAULT;
+            } else {
+                const ZSTD_compressionParameters cParams = ZSTD_getCParams(compressionLevel, fileSize, 0);
+                windowLog = cParams.windowLog;
+            }
         }
         windowSize = UTIL_makeHumanReadableSize(MAX(1ULL, MIN(1ULL << windowLog, pledgedSrcSize)));
         DISPLAYLEVEL(4, "Decompression will require %.*f%s of memory\n", windowSize.precision, windowSize.value, windowSize.suffix);