From: Nick Terrell <terrelln@fb.com>
Date: Tue, 2 May 2017 04:26:33 +0000 (-0700)
Subject: [cover] Make optimization faster
X-Git-Tag: v1.2.0^2~9^2^2~1
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=020b960e13499cb2cd524cc6b7ddb0ccbd76c361;p=thirdparty%2Fzstd.git

[cover] Make optimization faster
---

diff --git a/lib/dictBuilder/cover.c b/lib/dictBuilder/cover.c
index 4235f112b..0e156bd58 100644
--- a/lib/dictBuilder/cover.c
+++ b/lib/dictBuilder/cover.c
@@ -939,8 +939,8 @@ ZDICTLIB_API size_t COVER_optimizeTrainFromBuffer(void *dictBuffer,
   /* constants */
   const unsigned nbThreads = parameters->nbThreads;
   const unsigned kMinD = parameters->d == 0 ? 6 : parameters->d;
-  const unsigned kMaxD = parameters->d == 0 ? 16 : parameters->d;
-  const unsigned kMinK = parameters->k == 0 ? kMaxD : parameters->k;
+  const unsigned kMaxD = parameters->d == 0 ? 8 : parameters->d;
+  const unsigned kMinK = parameters->k == 0 ? 40 + kMaxD : parameters->k;
   const unsigned kMaxK = parameters->k == 0 ? 2048 : parameters->k;
   const unsigned kSteps = parameters->steps == 0 ? 32 : parameters->steps;
   const unsigned kStepSize = MAX((kMaxK - kMinK) / kSteps, 1);
diff --git a/programs/zstd.1 b/programs/zstd.1
index 5bd966a84..648c3365c 100644
--- a/programs/zstd.1
+++ b/programs/zstd.1
@@ -195,13 +195,13 @@ Example: \fB\-\-train \-\-cover=k=64,d=8 FILEs\fR\.
 .
 .TP
 \fB\-\-optimize\-cover[=steps=#,k=#,d=#]\fR
-If \fIsteps\fR is not specified, the default value of 32 is used\. If \fIk\fR is not specified, the \fIk\fR values in [16, 2048] are checked for each value of \fId\fR\. If \fId\fR is not specified, the values checked are [6, 8, \.\.\., 16]\.
+If \fIsteps\fR is not specified, the default value of 32 is used\. If \fIk\fR is not specified, the \fIk\fR values in [48, 2048] are checked for each value of \fId\fR\. If \fId\fR is not specified, the values checked are [6, 8]\.
 .
 .IP
 Runs the cover dictionary builder for each parameter set and saves the optimal parameters and dictionary\. Prints optimal parameters and writes optimal dictionary into output file\. Supports multithreading if \fBzstd\fR is compiled with threading support\.
 .
 .IP
-The parameter \fIk\fR is more sensitive than \fId\fR, and is faster to optimize over\. Suggested use is to run with a \fIsteps\fR <= 32 with neither \fIk\fR nor \fId\fR set\. Once it completes, use the value of \fId\fR it selects with a higher \fIsteps\fR (in the range [256, 1024])\.
+The parameter \fIk\fR is more sensitive than \fId\fR, and is faster to optimize over\.
 .
 .IP
 Examples :
@@ -210,7 +210,7 @@ Examples :
 \fBzstd \-\-train \-\-optimize\-cover FILEs\fR
 .
 .IP
-\fBzstd \-\-train \-\-optimize\-cover=d=d,steps=512 FILEs\fR
+\fBzstd \-\-train \-\-optimize\-cover=d=8,steps=512 FILEs\fR
 .
 .SH "BENCHMARK"
 .
diff --git a/programs/zstd.1.md b/programs/zstd.1.md
index 0919da702..ea346cfb7 100644
--- a/programs/zstd.1.md
+++ b/programs/zstd.1.md
@@ -194,9 +194,9 @@ Typical gains range from 10% (at 64KB) to x5 better (at <1KB).
 
 * `--optimize-cover[=steps=#,k=#,d=#]`:
     If _steps_ is not specified, the default value of 32 is used.
-    If _k_ is not specified, the _k_ values in [16, 2048] are checked for each
+    If _k_ is not specified, the _k_ values in [48, 2048] are checked for each
     value of _d_.
-    If _d_ is not specified, the values checked are [6, 8, ..., 16].
+    If _d_ is not specified, the values checked are [6, 8].
 
     Runs the cover dictionary builder for each parameter set
     and saves the optimal parameters and dictionary.
@@ -204,15 +204,12 @@ Typical gains range from 10% (at 64KB) to x5 better (at <1KB).
     Supports multithreading if `zstd` is compiled with threading support.
 
     The parameter _k_ is more sensitive than _d_, and is faster to optimize over.
-    Suggested use is to run with a _steps_ <= 32 with neither _k_ nor _d_ set.
-    Once it completes, use the value of _d_ it selects with a higher _steps_
-    (in the range [256, 1024]).
 
     Examples :
 
     `zstd --train --optimize-cover FILEs`
 
-    `zstd --train --optimize-cover=d=d,steps=512 FILEs`
+    `zstd --train --optimize-cover=d=8,steps=512 FILEs`
 
 
 BENCHMARK