reduce Max nb Workers to 64 in 32-bit mode

and restored limit to 256 when in 64-bit mode
(it was reduced to 200 to give more room for 32-bit).

This should fix test instability issues
using lot of threads in 32-bit environments.

diff --git a/CHANGELOG b/CHANGELOG
index 6f45ecfa157d559ad5ac856edf183eaccaf9ca0c..c34a11d5558eed15194cd3a19f9763dfbc664ba6 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -14,6 +14,7 @@ api: Several experimental functions have been deprecated and will emit a compile
   `ZSTD_initCStream_advanced()`
   `ZSTD_initCStream_usingCDict_advanced()`
   `ZSTD_resetCStream()`
+api: ZSTDMT_NBWORKERS_MAX reduced to 64 for 32-bit environments (@Cyan4973)
 perf: Significant speed improvements for middle compression levels (#2494, @senhuang42 @terrelln)
 perf: Block splitter to improve compression ratio, enabled by default for high compression levels (#2447, @senhuang42)
 perf: Decompression loop refactor, speed improvements on `clang` and for `--long` modes (#2614 #2630, @Cyan4973)

diff --git a/lib/compress/zstdmt_compress.h b/lib/compress/zstdmt_compress.h
index 3d9614a7eb905c7dfcf8ddd76c24be19d742e759..2fee2ec7455c2152fb5e39c5dbbb555299d85ca0 100644 (file)
--- a/lib/compress/zstdmt_compress.h
+++ b/lib/compress/zstdmt_compress.h
@@ -32,10 +32,10 @@
 
 
 /* ===   Constants   === */
-#ifndef ZSTDMT_NBWORKERS_MAX   /* can be modified at compile time */
-#  define ZSTDMT_NBWORKERS_MAX 200
+#ifndef ZSTDMT_NBWORKERS_MAX /* a different value can be selected at compile time */
+#  define ZSTDMT_NBWORKERS_MAX ((sizeof(void*)==4) /*32-bit*/ ? 64 : 256)
 #endif
-#ifndef ZSTDMT_JOBSIZE_MIN   /* can be modified at compile time */
+#ifndef ZSTDMT_JOBSIZE_MIN   /* a different value can be selected at compile time */
 #  define ZSTDMT_JOBSIZE_MIN (512 KB)
 #endif
 #define ZSTDMT_JOBLOG_MAX   (MEM_32bits() ? 29 : 30)

diff --git a/programs/README.md b/programs/README.md
index cf7f5ba46d1201e7154c37b1f83b00fb47d59838..7fd71042008561a2167848526943400bbfbf1542 100644 (file)
--- a/programs/README.md
+++ b/programs/README.md
@@ -224,7 +224,8 @@ Therefore, this avenue is intentionally restricted and only supports `ZSTD_CLEVE
 that `zstd` will use for compression, which by default is `1`.
 This functionality only exists when `zstd` is compiled with multithread support.
 `0` means "use as many threads as detected cpu cores on local system".
-The max # of threads is capped at: `ZSTDMT_NBWORKERS_MAX==200`.
+The max # of threads is capped at `ZSTDMT_NBWORKERS_MAX`,
+which is either 64 in 32-bit mode, or 256 for 64-bit environments.
 
 This functionality can be useful when `zstd` CLI is invoked in a way that doesn't allow passing arguments.
 One such scenario is `tar --zstd`.

diff --git a/programs/zstd.1 b/programs/zstd.1
index be52d303d8b4f2878a9a702602f95d1075bc26b1..861f9380b874ce5bba490ccdcf98af57fef2c856 100644 (file)
--- a/programs/zstd.1
+++ b/programs/zstd.1
@@ -105,7 +105,7 @@ Display information related to a zstd compressed file, such as size, ratio, and
 \fB\-\-fast[=#]\fR: switch to ultra\-fast compression levels\. If \fB=#\fR is not present, it defaults to \fB1\fR\. The higher the value, the faster the compression speed, at the cost of some compression ratio\. This setting overwrites compression level if one was set previously\. Similarly, if a compression level is set after \fB\-\-fast\fR, it overrides it\.
 .
 .IP "\(bu" 4
-\fB\-T#\fR, \fB\-\-threads=#\fR: Compress using \fB#\fR working threads (default: 1)\. If \fB#\fR is 0, attempt to detect and use the number of physical CPU cores\. In all cases, the nb of threads is capped to ZSTDMT_NBWORKERS_MAX==200\. This modifier does nothing if \fBzstd\fR is compiled without multithread support\.
+\fB\-T#\fR, \fB\-\-threads=#\fR: Compress using \fB#\fR working threads (default: 1)\. If \fB#\fR is 0, attempt to detect and use the number of physical CPU cores\. In all cases, the nb of threads is capped to \fBZSTDMT_NBWORKERS_MAX\fR, which is either 64 in 32\-bit mode, or 256 for 64\-bit environments\. This modifier does nothing if \fBzstd\fR is compiled without multithread support\.
 .
 .IP "\(bu" 4
 \fB\-\-single\-thread\fR: Does not spawn a thread for compression, use a single thread for both I/O and compression\. In this mode, compression is serialized with I/O, which is slightly slower\. (This is different from \fB\-T1\fR, which spawns 1 compression thread in parallel of I/O)\. This mode is the only one available when multithread support is disabled\. Single\-thread mode features lower memory usage\. Final compressed result is slightly different from \fB\-T1\fR\.

diff --git a/programs/zstd.1.md b/programs/zstd.1.md
index 9acea1b1d9fe0b30bc6c8ae3a180c24c7ab18131..ae50928667d9e8495151b5461883e850c13c927c 100644 (file)
--- a/programs/zstd.1.md
+++ b/programs/zstd.1.md
@@ -115,7 +115,8 @@ the last one takes effect.
 * `-T#`, `--threads=#`:
     Compress using `#` working threads (default: 1).
     If `#` is 0, attempt to detect and use the number of physical CPU cores.
-    In all cases, the nb of threads is capped to ZSTDMT_NBWORKERS_MAX==200.
+    In all cases, the nb of threads is capped to `ZSTDMT_NBWORKERS_MAX`,
+    which is either 64 in 32-bit mode, or 256 for 64-bit environments.
     This modifier does nothing if `zstd` is compiled without multithread support.
 * `--single-thread`:
     Does not spawn a thread for compression, use a single thread for both I/O and compression.