- **Zstd**, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios, and featuring zlib-level compression ratio.
+ **Zstd**, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level compression ratio.
It is provided as a BSD-license package, hosted on Github.
For a taste of its performance, here are a few benchmark numbers from a number of compression codecs suitable for real-time. The test was completed on a Core i7-5600U @ 2.6 GHz, using [fsbench 0.14.3](http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029), an open-source benchmark program by m^2.
-|Name | Ratio | C.speed | D.speed |
-|----------------|-------|--------:|--------:|
-| | | MB/s | MB/s |
-| **zstd 0.2** |**2.871**|**255**| **670** |
-| [zlib 1.2.8] -1| 2.730 | 70 | 300 |
-| QuickLZ 1.5.1b6| 2.237 | 370 | 415 |
-| LZO 2.06 | 2.106 | 400 | 580 |
-| [LZ4] r131 | 2.101 | 450 | 2100 |
-| Snappy 1.1.0 | 2.091 | 330 | 1100 |
-| LZF 3.6 | 2.077 | 200 | 560 |
+|Name | Ratio | C.speed | D.speed |
+|-----------------|-------|--------:|--------:|
+| | | MB/s | MB/s |
+| **zstd 0.3** |**2.858**|**280**| **670** |
+| [zlib 1.2.8] -1 | 2.730 | 70 | 300 |
+| QuickLZ 1.5.1b6 | 2.237 | 370 | 415 |
+| LZO 2.06 | 2.106 | 400 | 580 |
+| [LZ4] r131 | 2.101 | 450 | 2100 |
+| Snappy 1.1.0 | 2.091 | 330 | 1100 |
+| LZF 3.6 | 2.077 | 200 | 560 |
[zlib 1.2.8]:http://www.zlib.net/
[LZ4]:http://www.lz4.org/
-Zstd strong feature is its very high decompression speed, at more than >600 MB/s per core.
-Obviously, your exact mileage will vary depending on target system.
+Zstd can also offer stronger compression ratio at the cost of compression speed, but preserving its decompression speed. In the following test, a few compressors suitable for this scenario are selected (they offer very asymetric performance, useful when compression time has little importance). The test was completed on a Core i7-5600U @ 2.6 GHz, using [benchmark 0.6.1](http://encode.ru/threads/1266-In-memory-benchmark-with-fastest-LZSS-(QuickLZ-Snappy)-compressors?p=45217&viewfull=1#post45217), an open-source benchmark program by inikep.
-Zstd compression speed will be configurable to fit different situations.
-The first available version is the fast one, at ~250 MB/s per core, which is suitable for a few real-time scenarios.
-But similar to [LZ4], zstd can offer derivatives trading compression time for compression ratio, keeping decompression properties intact. "Offline compression", where compression time is of little importance because the content is only compressed once and decompressed many times, will likely prefer this setup.
+|Name | Ratio | C.speed | D.speed |
+|-----------------|-------|--------:|--------:|
+| | | MB/s | MB/s |
+| brotli -9 | 3.729 | 4 | 340 |
+| **zstd 0.3 -9** |**3.447**|**30** | **640** |
+| [zlib 1.2.8] -9 | 3.133 | 10 | 300 |
+| LZO 2.06 -999 | 2.790 | 1 | 560 |
+| [LZ4] r131 -9 | 2.720 | 25 | 2100 |
-Note that high compression derivatives still have to be developed.
-It's a complex area which will require time and benefit from contributions.
+[lzma]:http://www.7-zip.org/
-
-Another property zstd is developed for is configurable memory requirement, with the objective to fit into low-memory configurations, or servers handling many connections in parallel.
+Zstd compression speed is highly configurable, by small increment, to fit different situations. Its memory requirement can also be configured to fit into low-memory hardware configurations, or servers handling multiple connections/contexts in parallel.
Zstd entropy stage is provided by [Huff0 and FSE, from Finite State Entrop library](https://github.com/Cyan4973/FiniteStateEntropy).
-
-Zstd has not yet reached "stable" status. Specifically, it doesn't guarantee yet that its current compressed format will remain stable and supported in future versions. It may still change to adapt further optimizations still being investigated. However, the library starts to be pretty robust, able to withstand hazards situations, including invalid input. The library reliability has been tested using [Fuzz Testing](https://en.wikipedia.org/wiki/Fuzz_testing), using both [internal tools](programs/fuzzer.c) and [external ones](http://lcamtuf.coredump.cx/afl). Therefore, you can now safely test zstd, even within production environments.
+Zstd has not yet reached "stable" status. Specifically, it doesn't guarantee yet that its current compressed format will remain stable and supported in future versions. It may still change to adapt further optimizations still being investigated. That being said, the library is now pretty robust, able to withstand hazards situations, including invalid input. The library reliability has been tested using [Fuzz Testing](https://en.wikipedia.org/wiki/Fuzz_testing), with both [internal tools](programs/fuzzer.c) and [external ones](http://lcamtuf.coredump.cx/afl). Therefore, it's now safe to test Zstandard even within production environments.
"Stable Format" is projected sometimes early 2016.
*/
+/* *******************************************************
+* Compiler specifics
+*********************************************************/
+#ifdef _MSC_VER /* Visual Studio */
+# define FORCE_INLINE static __forceinline
+# include <intrin.h> /* For Visual 2005 */
+# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
+# pragma warning(disable : 4324) /* disable: C4324: padded structure */
+#else
+# define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
+# ifdef __GNUC__
+# define FORCE_INLINE static inline __attribute__((always_inline))
+# else
+# define FORCE_INLINE static inline
+# endif
+#endif
+
+
/* *************************************
* Includes
***************************************/
ZSTD_HC_parameters params)
{
/* validate params */
- if (params.windowLog > ZSTD_HC_WINDOWLOG_MAX) params.windowLog = ZSTD_HC_WINDOWLOG_MAX;
- if (params.windowLog < ZSTD_HC_WINDOWLOG_MIN) params.windowLog = ZSTD_HC_WINDOWLOG_MIN;
- if (params.chainLog > params.windowLog) params.chainLog = params.windowLog; /* <= ZSTD_HC_CHAINLOG_MAX */
- if (params.chainLog < ZSTD_HC_CHAINLOG_MIN) params.chainLog = ZSTD_HC_CHAINLOG_MIN;
- if (params.hashLog > ZSTD_HC_HASHLOG_MAX) params.hashLog = ZSTD_HC_HASHLOG_MAX;
- if (params.hashLog < ZSTD_HC_HASHLOG_MIN) params.hashLog = ZSTD_HC_HASHLOG_MIN;
- if (params.searchLog > ZSTD_HC_SEARCHLOG_MAX) params.searchLog = ZSTD_HC_SEARCHLOG_MAX;
- if (params.searchLog < ZSTD_HC_SEARCHLOG_MIN) params.searchLog = ZSTD_HC_SEARCHLOG_MIN;
+ if (params.windowLog > ZSTD_HC_WINDOWLOG_MAX) params.windowLog = ZSTD_HC_WINDOWLOG_MAX;
+ if (params.windowLog < ZSTD_HC_WINDOWLOG_MIN) params.windowLog = ZSTD_HC_WINDOWLOG_MIN;
+ if (params.chainLog > params.windowLog) params.chainLog = params.windowLog; /* <= ZSTD_HC_CHAINLOG_MAX */
+ if (params.chainLog < ZSTD_HC_CHAINLOG_MIN) params.chainLog = ZSTD_HC_CHAINLOG_MIN;
+ if (params.hashLog > ZSTD_HC_HASHLOG_MAX) params.hashLog = ZSTD_HC_HASHLOG_MAX;
+ if (params.hashLog < ZSTD_HC_HASHLOG_MIN) params.hashLog = ZSTD_HC_HASHLOG_MIN;
+ if (params.searchLog > ZSTD_HC_SEARCHLOG_MAX) params.searchLog = ZSTD_HC_SEARCHLOG_MAX;
+ if (params.searchLog < ZSTD_HC_SEARCHLOG_MIN) params.searchLog = ZSTD_HC_SEARCHLOG_MIN;
+ if (params.searchLength> ZSTD_HC_SEARCHLENGTH_MAX) params.searchLength = ZSTD_HC_SEARCHLENGTH_MAX;
+ if (params.searchLength< ZSTD_HC_SEARCHLENGTH_MIN) params.searchLength = ZSTD_HC_SEARCHLENGTH_MIN;
/* reserve table memory */
{
/* *************************************
-* Local Macros
+* Inline functions and Macros
***************************************/
-#define KNUTH 2654435761U
-static U32 ZSTD_HC_hash(U32 u, U32 h) { return (u * KNUTH) >> (32-h) ; }
-static U32 ZSTD_HC_hashPtr(const void* ptr, U32 h) { return ZSTD_HC_hash(MEM_read32(ptr), h); }
+static const U32 prime4bytes = 2654435761U;
+static U32 ZSTD_HC_hash4(U32 u, U32 h) { return (u * prime4bytes) >> (32-h) ; }
+static size_t ZSTD_HC_hash4Ptr(const void* ptr, U32 h) { return ZSTD_HC_hash4(MEM_read32(ptr), h); }
-//static const U64 prime5bytes = 889523592379ULL;
-//static U32 ZSTD_HC_hashPtr(const void* p, U32 h) { return (U32)((MEM_read64(p) * prime5bytes) << (64-40)) >> (64-h); }
+static const U64 prime5bytes = 889523592379ULL;
+static size_t ZSTD_HC_hash5(U64 u, U32 h) { return (size_t)((u * prime5bytes) << (64-40) >> (64-h)) ; }
+static size_t ZSTD_HC_hash5Ptr(const void* p, U32 h) { return ZSTD_HC_hash5(MEM_read64(p), h); }
-#define NEXT_IN_CHAIN(d) chainTable[(d) & chainMask] /* flexible, CHAINSIZE dependent */
+static const U64 prime6bytes = 227718039650203ULL;
+static size_t ZSTD_HC_hash6(U64 u, U32 h) { return (size_t)((u * prime6bytes) << (64-48) >> (64-h)) ; }
+static size_t ZSTD_HC_hash6Ptr(const void* p, U32 h) { return ZSTD_HC_hash6(MEM_read64(p), h); }
+static size_t ZSTD_HC_hashPtr(const void* p, U32 h, U32 mls)
+{
+ switch(mls)
+ {
+ default:
+ case 4: return ZSTD_HC_hash4Ptr(p,h);
+ case 5: return ZSTD_HC_hash5Ptr(p,h);
+ case 6: return ZSTD_HC_hash6Ptr(p,h);
+ }
+}
+
+#define NEXT_IN_CHAIN(d) chainTable[(d) & chainMask] /* flexible, CHAINSIZE dependent */
/* *************************************
* HC Compression
***************************************/
/* Update chains up to ip (excluded) */
-static void ZSTD_HC_insert (ZSTD_HC_CCtx* zc, const BYTE* ip)
+static void ZSTD_HC_insert (ZSTD_HC_CCtx* zc, const BYTE* ip, U32 mls)
{
U32* const hashTable = zc->hashTable;
const U32 hashLog = zc->params.hashLog;
while(idx < target)
{
- U32 h = ZSTD_HC_hashPtr(base+idx, hashLog);
+ size_t h = ZSTD_HC_hashPtr(base+idx, hashLog, mls);
NEXT_IN_CHAIN(idx) = hashTable[h];
hashTable[h] = idx;
idx++;
}
-static size_t ZSTD_HC_insertAndFindBestMatch (
+FORCE_INLINE /* inlining is important to hardwire a hot branch (template emulation) */
+size_t ZSTD_HC_insertAndFindBestMatch (
ZSTD_HC_CCtx* zc, /* Index table will be updated */
const BYTE* ip, const BYTE* const iLimit,
const BYTE** matchpos,
- const U32 maxNbAttempts)
+ const U32 maxNbAttempts, const U32 matchLengthSearch)
{
U32* const hashTable = zc->hashTable;
const U32 hashLog = zc->params.hashLog;
size_t ml=0;
/* HC4 match finder */
- ZSTD_HC_insert(zc, ip);
- matchIndex = hashTable[ZSTD_HC_hashPtr(ip, hashLog)];
+ ZSTD_HC_insert(zc, ip, matchLengthSearch);
+ matchIndex = hashTable[ZSTD_HC_hashPtr(ip, hashLog, matchLengthSearch)];
while ((matchIndex>=lowLimit) && (nbAttempts))
{
}
+static size_t ZSTD_HC_insertAndFindBestMatch_selectMLS (
+ ZSTD_HC_CCtx* zc, /* Index table will be updated */
+ const BYTE* ip, const BYTE* const iLimit,
+ const BYTE** matchpos,
+ const U32 maxNbAttempts, const U32 matchLengthSearch)
+{
+ switch(matchLengthSearch)
+ {
+ default :
+ case 4 : return ZSTD_HC_insertAndFindBestMatch(zc, ip, iLimit, matchpos, maxNbAttempts, 4);
+ case 5 : return ZSTD_HC_insertAndFindBestMatch(zc, ip, iLimit, matchpos, maxNbAttempts, 5);
+ case 6 : return ZSTD_HC_insertAndFindBestMatch(zc, ip, iLimit, matchpos, maxNbAttempts, 6);
+ }
+}
+
+
static size_t ZSTD_HC_compressBlock(ZSTD_HC_CCtx* ctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize)
{
seqStore_t* seqStorePtr = &(ctx->seqStore);
size_t offset_2=REPCODE_STARTVALUE, offset_1=REPCODE_STARTVALUE;
const U32 maxSearches = 1 << ctx->params.searchLog;
+ const U32 mls = ctx->params.searchLength;
/* init */
ZSTD_resetSeqStore(seqStorePtr);
/* search */
{
- size_t matchLength = ZSTD_HC_insertAndFindBestMatch(ctx, ip, iend, &match, maxSearches);
+ size_t matchLength = ZSTD_HC_insertAndFindBestMatch_selectMLS(ctx, ip, iend, &match, maxSearches, mls);
if (!matchLength) { ip++; continue; }
/* store sequence */
{
***************************************/
typedef struct
{
- U32 windowLog; /* largest match distance : impact decompression buffer size */
- U32 chainLog; /* full search distance : larger == more compression, slower, more memory*/
- U32 hashLog; /* dispatch table : larger == more memory, faster*/
- U32 searchLog; /* nb of searches : larger == more compression, slower*/
+ U32 windowLog; /* largest match distance : impact decompression buffer size */
+ U32 chainLog; /* full search distance : larger == more compression, slower, more memory*/
+ U32 hashLog; /* dispatch table : larger == more memory, faster*/
+ U32 searchLog; /* nb of searches : larger == more compression, slower*/
+ U32 searchLength; /* size of matches : larger == faster decompression */
} ZSTD_HC_parameters;
/* parameters boundaries */
#define ZSTD_HC_HASHLOG_MIN 4
#define ZSTD_HC_SEARCHLOG_MAX (ZSTD_HC_CHAINLOG_MAX-1)
#define ZSTD_HC_SEARCHLOG_MIN 1
+#define ZSTD_HC_SEARCHLENGTH_MAX 6
+#define ZSTD_HC_SEARCHLENGTH_MIN 4
/* *************************************
#define ZSTD_HC_MAX_CLEVEL 26
static const ZSTD_HC_parameters ZSTD_HC_defaultParameters[ZSTD_HC_MAX_CLEVEL+1] = {
/* W, C, H, S */
- { 18, 12, 14, 1 }, /* level 0 - never used */
- { 18, 12, 14, 1 }, /* real level 1 - all levels below are +1 */
- { 18, 12, 15, 2 }, /* level 1 */
- { 19, 14, 16, 3 }, /* level 2 */
- { 20, 18, 18, 3 }, /* level 3 */
- { 20, 19, 19, 3 }, /* level 4 */
- { 20, 19, 19, 4 }, /* level 5 */
- { 20, 20, 19, 4 }, /* level 6 */
- { 20, 19, 19, 5 }, /* level 7 */
- { 20, 19, 19, 6 }, /* level 8 */
- { 20, 20, 20, 6 }, /* level 9 */
- { 21, 20, 21, 6 }, /* level 10 */
- { 21, 20, 21, 7 }, /* level 11 */
- { 21, 20, 22, 7 }, /* level 12 */
- { 21, 21, 23, 7 }, /* level 13 */
- { 21, 21, 23, 7 }, /* level 14 */
- { 21, 21, 23, 8 }, /* level 15 */
- { 21, 21, 23, 9 }, /* level 16 */
- { 21, 21, 23, 9 }, /* level 17 */
- { 21, 21, 23, 10 }, /* level 18 */
- { 22, 22, 23, 9 }, /* level 19 */
- { 22, 22, 23, 9 }, /* level 20 */
- { 22, 22, 23, 10 }, /* level 21 */
- { 22, 22, 23, 10 }, /* level 22 */
- { 22, 22, 23, 11 }, /* level 23 */
- { 22, 22, 23, 12 }, /* level 24 */
- { 23, 23, 23, 11 }, /* level 25 */
+ { 18, 12, 14, 1, 4 }, /* level 0 - never used */
+ { 18, 12, 14, 1, 4 }, /* level 1 - in fact redirected towards zstd fast */
+ { 18, 12, 15, 2, 4 }, /* level 2 */
+ { 19, 14, 16, 3, 4 }, /* level 3 */
+ { 20, 15, 17, 4, 5 }, /* level 4 */
+ { 20, 17, 19, 4, 5 }, /* level 5 */
+ { 20, 19, 19, 4, 5 }, /* level 6 */
+ { 20, 19, 19, 5, 5 }, /* level 7 */
+ { 20, 20, 20, 5, 5 }, /* level 8 */
+ { 20, 20, 20, 6, 5 }, /* level 9 */
+ { 21, 21, 20, 5, 5 }, /* level 10 */
+ { 22, 21, 22, 6, 5 }, /* level 11 */
+ { 23, 21, 22, 6, 5 }, /* level 12 */
+ { 23, 21, 22, 7, 5 }, /* level 13 */
+ { 22, 22, 23, 7, 5 }, /* level 14 */
+ { 22, 22, 23, 7, 5 }, /* level 15 */
+ { 22, 22, 23, 8, 5 }, /* level 16 */
+ { 22, 22, 23, 8, 5 }, /* level 17 */
+ { 22, 22, 23, 9, 5 }, /* level 18 */
+ { 22, 22, 23, 9, 5 }, /* level 19 */
+ { 23, 23, 23, 9, 5 }, /* level 20 */
+ { 23, 23, 23, 9, 5 }, /* level 21 */
+ { 23, 23, 23, 10, 5 }, /* level 22 */
+ { 23, 23, 23, 10, 5 }, /* level 23 */
+ { 23, 23, 23, 11, 5 }, /* level 24 */
+ { 23, 23, 23, 12, 5 }, /* level 25 */
+ { 23, 23, 23, 13, 5 }, /* level 26 */ /* ZSTD_HC_MAX_CLEVEL */
};
-
#if defined (__cplusplus)
}
#endif
/**************************************
* Benchmark Parameters
**************************************/
-static U32 nbIterations = NBLOOPS;
+static U32 g_nbIterations = NBLOOPS;
static double g_compressibility = COMPRESSIBILITY_DEFAULT;
static U32 g_blockSize = 0;
static U32 g_rand = 1;
static U32 g_singleRun = 0;
static U32 g_target = 0;
static U32 g_noSeed = 0;
-static ZSTD_HC_parameters g_params = { 0, 0, 0, 0 };
+static ZSTD_HC_parameters g_params = { 0, 0, 0, 0, 0 };
void BMK_SetNbIterations(int nbLoops)
{
- nbIterations = nbLoops;
- DISPLAY("- %u iterations -\n", nbIterations);
+ g_nbIterations = nbLoops;
+ DISPLAY("- %u iterations -\n", g_nbIterations);
}
U32 Clog = params.chainLog;
U32 Hlog = params.hashLog;
U32 Slog = params.searchLog;
+ U32 Slength = params.searchLength;
U64 crcOrig;
/* Memory allocation & restrictions */
const int startTime =BMK_GetMilliStart();
DISPLAY("\r%79s\r", "");
- for (loopNb = 1; loopNb <= nbIterations; loopNb++)
+ for (loopNb = 1; loopNb <= g_nbIterations; loopNb++)
{
int nbLoops;
int milliTime;
if (totalTime > g_maxParamTime) break;
/* Compression */
- DISPLAY("%1u-W%02uC%02uH%02uS%02u : %9u ->\r", loopNb, Wlog, Clog, Hlog, Slog, (U32)srcSize);
+ DISPLAY("%1u-W%02uC%02uH%02uS%02uL%02u : %9u ->\r", loopNb, Wlog, Clog, Hlog, Slog, Slength, (U32)srcSize);
memset(compressedBuffer, 0xE5, maxCompressedSize);
nbLoops = 0;
cSize += blockTable[blockNb].cSize;
if ((double)milliTime < fastestC*nbLoops) fastestC = (double)milliTime / nbLoops;
ratio = (double)srcSize / (double)cSize;
- DISPLAY("%1u-W%02uC%02uH%02uS%02u : %9u ->", loopNb, Wlog, Clog, Hlog, Slog, (U32)srcSize);
+ DISPLAY("%1u-W%02uC%02uH%02uS%02uL%02u : %9u ->", loopNb, Wlog, Clog, Hlog, Slog, Slength, (U32)srcSize);
DISPLAY(" %9u (%4.3f),%7.1f MB/s\r", (U32)cSize, ratio, (double)srcSize / fastestC / 1000.);
resultPtr->cSize = cSize;
resultPtr->cSpeed = (U32)((double)srcSize / fastestC);
milliTime = BMK_GetMilliSpan(milliTime);
if ((double)milliTime < fastestD*nbLoops) fastestD = (double)milliTime / nbLoops;
- DISPLAY("%1u-W%02uC%02uH%02uS%02u : %9u -> ", loopNb, Wlog, Clog, Hlog, Slog, (U32)srcSize);
+ DISPLAY("%1u-W%02uC%02uH%02uS%02uL%02u : %9u -> ", loopNb, Wlog, Clog, Hlog, Slog, Slength, (U32)srcSize);
DISPLAY("%9u (%4.3f),%7.1f MB/s, ", (U32)cSize, ratio, (double)srcSize / fastestC / 1000.);
DISPLAY("%7.1f MB/s\r", (double)srcSize / fastestD / 1000.);
resultPtr->dSpeed = (U32)((double)srcSize / fastestD);
static void BMK_printWinner(FILE* f, U32 cLevel, BMK_result_t result, ZSTD_HC_parameters params, size_t srcSize)
{
DISPLAY("\r%79s\r", "");
- fprintf(f," {%3u,%3u,%3u,%3u }, ", params.windowLog, params.chainLog, params.hashLog, params.searchLog);
+ fprintf(f," {%3u,%3u,%3u,%3u,%3u }, ", params.windowLog, params.chainLog, params.hashLog, params.searchLog, params.searchLength);
fprintf(f,
- "/* level %2u */ /* R:%5.3f at %5.1f MB/s */ \n",
- cLevel, (double)srcSize / result.cSize, (double)result.cSpeed / 1000.);
+ "/* level %2u */ /* R:%5.3f at %5.1f MB/s - %5.1f MB/s */ \n",
+ cLevel, (double)srcSize / result.cSize, (double)result.cSpeed / 1000., (double)result.dSpeed / 1000.);
}
-static U32 g_cSpeedTarget[ZSTD_HC_MAX_CLEVEL+1] = {
- 300000, 200000, 150000, 100000, 70000, 50000, 35000, 25000, 15000, 10000, /* 0 - 9 */
- 7000, 5000, 3500, 2500, 1500, 1000, 700, 500, 350, 250, /* 10 - 19 */
- 0 }; /* 20 */
+static U32 g_cSpeedTarget[ZSTD_HC_MAX_CLEVEL+1] = { 0 };
typedef struct {
BMK_result_t result;
fprintf(f, "\n /* Selected configurations : */ \n");
fprintf(f, "#define ZSTD_HC_MAX_CLEVEL %2u \n", ZSTD_HC_MAX_CLEVEL);
fprintf(f, "static const ZSTD_HC_parameters ZSTD_HC_defaultParameters[ZSTD_HC_MAX_CLEVEL+1] = {\n");
- fprintf(f, " /* W, C, H, S */ \n");
+ fprintf(f, " /* W, C, H, S, L */ \n");
for (cLevel=0; cLevel <= ZSTD_HC_MAX_CLEVEL; cLevel++)
BMK_printWinner(f, cLevel, winners[cLevel].result, winners[cLevel].params, srcSize);
BMK_benchParam(&testResult, srcBuffer, srcSize, ctx, params);
- for (cLevel = 1; cLevel <= ZSTD_HC_MAX_CLEVEL; cLevel++)
+ for (cLevel = 2; cLevel <= ZSTD_HC_MAX_CLEVEL; cLevel++)
{
if (testResult.cSpeed < g_cSpeedTarget[cLevel])
continue; /* not fast enough for this level */
double W_CMemUsed_note = W_ratioNote * ( 50 + 13*cLevel) - log((double)W_CMemUsed);
double O_CMemUsed_note = O_ratioNote * ( 50 + 13*cLevel) - log((double)O_CMemUsed);
- double W_CSpeed_note = W_ratioNote * ( 30 + 8*cLevel) + log((double)testResult.cSpeed);
- double O_CSpeed_note = O_ratioNote * ( 30 + 8*cLevel) + log((double)winners[cLevel].result.cSpeed);
+ double W_CSpeed_note = W_ratioNote * ( 30 + 10*cLevel) + log((double)testResult.cSpeed);
+ double O_CSpeed_note = O_ratioNote * ( 30 + 10*cLevel) + log((double)winners[cLevel].result.cSpeed);
double W_DSpeed_note = W_ratioNote * ( 20 + 2*cLevel) + log((double)testResult.dSpeed);
double O_DSpeed_note = O_ratioNote * ( 20 + 2*cLevel) + log((double)winners[cLevel].result.dSpeed);
for (; nbChanges; nbChanges--)
{
- const U32 changeID = FUZ_rand(&g_rand) & 7;
+ const U32 changeID = FUZ_rand(&g_rand) % 9;
switch(changeID)
{
case 0:
p.windowLog++; break;
case 7:
p.windowLog--; break;
+ case 8:
+ p.searchLength++; break;
+ case 9:
+ p.searchLength--; break;
}
}
if (p.hashLog < ZSTD_HC_HASHLOG_MIN) continue;
if (p.searchLog > p.chainLog) continue;
if (p.searchLog < ZSTD_HC_SEARCHLOG_MIN) continue;
+ if (p.searchLength > ZSTD_HC_SEARCHLENGTH_MAX) continue;
+ if (p.searchLength < ZSTD_HC_SEARCHLENGTH_MIN) continue;
/* exclude faster if already played params */
if (FUZ_rand(&g_rand) & ((1 << NB_TESTS_PLAYED(p))-1)) continue;
{
/* totally random entry */
ZSTD_HC_parameters p;
- p.chainLog = FUZ_rand(&g_rand) % (ZSTD_HC_CHAINLOG_MAX+1 - ZSTD_HC_CHAINLOG_MIN) + ZSTD_HC_CHAINLOG_MIN;
- p.hashLog = FUZ_rand(&g_rand) % (ZSTD_HC_HASHLOG_MAX+1 - ZSTD_HC_HASHLOG_MIN) + ZSTD_HC_HASHLOG_MIN;
- p.searchLog = FUZ_rand(&g_rand) % (ZSTD_HC_SEARCHLOG_MAX+1 - ZSTD_HC_SEARCHLOG_MIN) + ZSTD_HC_SEARCHLOG_MIN;
- p.windowLog = FUZ_rand(&g_rand) % (ZSTD_HC_WINDOWLOG_MAX+1 - ZSTD_HC_WINDOWLOG_MIN) + ZSTD_HC_WINDOWLOG_MIN;
+ p.chainLog = FUZ_rand(&g_rand) % (ZSTD_HC_CHAINLOG_MAX+1 - ZSTD_HC_CHAINLOG_MIN) + ZSTD_HC_CHAINLOG_MIN;
+ p.hashLog = FUZ_rand(&g_rand) % (ZSTD_HC_HASHLOG_MAX+1 - ZSTD_HC_HASHLOG_MIN) + ZSTD_HC_HASHLOG_MIN;
+ p.searchLog = FUZ_rand(&g_rand) % (ZSTD_HC_SEARCHLOG_MAX+1 - ZSTD_HC_SEARCHLOG_MIN) + ZSTD_HC_SEARCHLOG_MIN;
+ p.windowLog = FUZ_rand(&g_rand) % (ZSTD_HC_WINDOWLOG_MAX+1 - ZSTD_HC_WINDOWLOG_MIN) + ZSTD_HC_WINDOWLOG_MIN;
+ p.searchLength=FUZ_rand(&g_rand) % (ZSTD_HC_SEARCHLENGTH_MAX+1 - ZSTD_HC_SEARCHLENGTH_MIN) + ZSTD_HC_SEARCHLENGTH_MIN;
playAround(f, winners, p, srcBuffer, srcSize, ctx);
}
else
}
-static const ZSTD_HC_parameters* seedParams = ZSTD_HC_defaultParameters;
+static const ZSTD_HC_parameters* g_seedParams = ZSTD_HC_defaultParameters;
static void BMK_benchMem(void* srcBuffer, size_t srcSize)
{
if (f==NULL) { DISPLAY("error opening %s \n", rfName); exit(1); }
if (g_target)
- g_cSpeedTarget[1] = g_target * 1000;
+ g_cSpeedTarget[2] = g_target * 1000;
else
{
/* baseline config for level 2 */
BMK_result_t testResult;
- params = seedParams[2];
+ params = g_seedParams[2];
params.windowLog = MIN(srcLog, params.windowLog);
params.chainLog = MIN(params.windowLog, params.chainLog);
params.searchLog = MIN(params.chainLog, params.searchLog);
const int maxSeeds = g_noSeed ? 2 : ZSTD_HC_MAX_CLEVEL;
for (i=2; i<=maxSeeds; i++)
{
- params = seedParams[i];
+ params = g_seedParams[i];
params.windowLog = MIN(srcLog, params.windowLog);
params.chainLog = MIN(params.windowLog, params.chainLog);
params.searchLog = MIN(params.chainLog, params.searchLog);
case 'i':
argument++;
if ((argument[0] >='0') && (argument[0] <='9'))
- {
- int iters = argument[0] - '0';
- BMK_SetNbIterations(iters);
- argument++;
- }
+ g_nbIterations = *argument++ - '0';
break;
/* Sample compressibility (when no file provided) */
/* Run Single conf */
case 'S':
+ g_singleRun = 1;
+ argument++;
+ g_params = g_seedParams[2];
+ for ( ; ; )
{
- if (argument[ 1]!='w') return badusage(exename);
- if (argument[ 4]!='c') return badusage(exename);
- if (argument[ 7]!='h') return badusage(exename);
- if (argument[10]!='s') return badusage(exename);
- g_singleRun = 1;
- g_params.windowLog = (argument[ 2] - '0') * 10 + (argument[ 3] - '0');
- g_params.chainLog = (argument[ 5] - '0') * 10 + (argument[ 6] - '0');
- g_params.hashLog = (argument[ 8] - '0') * 10 + (argument[ 9] - '0');
- g_params.searchLog = (argument[11] - '0') * 10 + (argument[12] - '0');
- argument += 13;
+ switch(*argument)
+ {
+ case 'w':
+ g_params.windowLog = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ g_params.windowLog *= 10, g_params.windowLog += *argument++ - '0';
+ continue;
+ case 'c':
+ g_params.chainLog = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ g_params.chainLog *= 10, g_params.chainLog += *argument++ - '0';
+ continue;
+ case 'h':
+ g_params.hashLog = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ g_params.hashLog *= 10, g_params.hashLog += *argument++ - '0';
+ continue;
+ case 's':
+ g_params.searchLog = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ g_params.searchLog *= 10, g_params.searchLog += *argument++ - '0';
+ continue;
+ case 'l':
+ g_params.searchLength = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ g_params.searchLength *= 10, g_params.searchLength += *argument++ - '0';
+ continue;
+ case 'L':
+ {
+ int cLevel = 0;
+ argument++;
+ while ((*argument>= '0') && (*argument<='9'))
+ cLevel *= 10, cLevel += *argument++ - '0';
+ if (cLevel < 2) cLevel = 2;
+ if (cLevel > ZSTD_HC_MAX_CLEVEL) cLevel = ZSTD_HC_MAX_CLEVEL;
+ g_params = g_seedParams[cLevel];
+ continue;
+ }
+ default : ;
+ }
break;
}
+ break;
- /* target level1 speed objective, in MB/s */
+ /* target level2 speed objective, in MB/s */
case 'T':
argument++;
g_target = 0;
projects / thirdparty / zstd.git / commitdiff
