</ol>
<hr>
<a name="Chapter1"></a><h2>Introduction</h2><pre>
- zstd, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios
- at zlib-level and better compression ratios. The zstd compression library provides in-memory compression and
- decompression functions. The library supports compression levels from 1 up to ZSTD_maxCLevel() which is 22.
+ zstd, short for Zstandard, is a fast lossless compression algorithm,
+ targeting real-time compression scenarios at zlib-level and better compression ratios.
+ The zstd compression library provides in-memory compression and decompression functions.
+ The library supports compression levels from 1 up to ZSTD_maxCLevel() which is currently 22.
Levels >= 20, labeled `--ultra`, should be used with caution, as they require more memory.
Compression can be done in:
- a single step (described as Simple API)
</p></pre><BR>
<pre><b>unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);
-</b><p> NOTE: This function is planned to be obsolete, in favour of ZSTD_getFrameContentSize.
- ZSTD_getFrameContentSize functions the same way, returning the decompressed size of a single
- frame, but distinguishes empty frames from frames with an unknown size, or errors.
-
- Additionally, ZSTD_findDecompressedSize can be used instead. It can handle multiple
- concatenated frames in one buffer, and so is more general.
- As a result however, it requires more computation and entire frames to be passed to it,
- as opposed to ZSTD_getFrameContentSize which requires only a single frame's header.
+</b><p> NOTE: This function is planned to be obsolete, in favor of ZSTD_getFrameContentSize().
+ ZSTD_getFrameContentSize() works the same way,
+ returning the decompressed size of a single frame,
+ but distinguishes empty frames from frames with an unknown size, or errors.
'src' is the start of a zstd compressed frame.
@return : content size to be decompressed, as a 64-bits value _if known_, 0 otherwise.
- note 1 : decompressed size is an optional field, that may not be present, especially in streaming mode.
+ note 1 : decompressed size is an optional field, it may not be present, typically in streaming mode.
When `return==0`, data to decompress could be any size.
In which case, it's necessary to use streaming mode to decompress data.
- Optionally, application can still use ZSTD_decompress() while relying on implied limits.
+ Optionally, application can use ZSTD_decompress() while relying on implied limits.
(For example, data may be necessarily cut into blocks <= 16 KB).
note 2 : decompressed size is always present when compression is done with ZSTD_compress()
note 3 : decompressed size can be very large (64-bits value),
ZSTD_CCtx* ZSTD_createCCtx(void);
size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx);
</pre></b><BR>
-<pre><b>size_t ZSTD_compressCCtx(ZSTD_CCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize, int compressionLevel);
+<pre><b>size_t ZSTD_compressCCtx(ZSTD_CCtx* ctx,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize,
+ int compressionLevel);
</b><p> Same as ZSTD_compress(), requires an allocated ZSTD_CCtx (see ZSTD_createCCtx()).
</p></pre><BR>
<h3>Decompression context</h3><pre> When decompressing many times,
- it is recommended to allocate a context just once, and re-use it for each successive compression operation.
+ it is recommended to allocate a context only once,
+ and re-use it for each successive compression operation.
This will make workload friendlier for system's memory.
- Use one context per thread for parallel execution in multi-threaded environments.
+ Use one context per thread for parallel execution.
</pre><b><pre>typedef struct ZSTD_DCtx_s ZSTD_DCtx;
ZSTD_DCtx* ZSTD_createDCtx(void);
size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx);
</pre></b><BR>
-<pre><b>size_t ZSTD_decompressDCtx(ZSTD_DCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
-</b><p> Same as ZSTD_decompress(), requires an allocated ZSTD_DCtx (see ZSTD_createDCtx()).
+<pre><b>size_t ZSTD_decompressDCtx(ZSTD_DCtx* ctx,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize);
+</b><p> Same as ZSTD_decompress(), requires an allocated ZSTD_DCtx (see ZSTD_createDCtx())
</p></pre><BR>
<a name="Chapter5"></a><h2>Simple dictionary API</h2><pre></pre>
<a name="Chapter6"></a><h2>Fast dictionary API</h2><pre></pre>
-<pre><b>ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize, int compressionLevel);
+<pre><b>ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize,
+ int compressionLevel);
</b><p> When compressing multiple messages / blocks with the same dictionary, it's recommended to load it just once.
ZSTD_createCDict() will create a digested dictionary, ready to start future compression operations without startup delay.
ZSTD_CDict can be created once and used by multiple threads concurrently, as its usage is read-only.
<a name="Chapter11"></a><h2>Advanced types</h2><pre></pre>
-<pre><b>typedef enum { ZSTD_fast, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2, ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra } ZSTD_strategy; </b>/* from faster to stronger */<b>
+<pre><b>typedef enum { ZSTD_fast=1, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2,
+ ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra } ZSTD_strategy; </b>/* from faster to stronger */<b>
</b></pre><BR>
<pre><b>typedef struct {
unsigned windowLog; </b>/**< largest match distance : larger == more compression, more memory needed during decompression */<b>
ZSTD_p_compressionStrategy, </b>/* See ZSTD_strategy enum definition.<b>
* Cast selected strategy as unsigned for ZSTD_CCtx_setParameter() compatibility.
* The higher the value of selected strategy, the more complex it is,
- * resulting in stronger and slower compression. */
+ * resulting in stronger and slower compression.
+ * Special: value 0 means "do not change strategy". */
#if 0
ZSTD_p_windowSize, </b>/* Maximum allowed back-reference distance.<b>
* Can be set to a more precise value than windowLog.
return 0;
case ZSTD_p_compressionStrategy :
- if (value > (unsigned)ZSTD_btultra) return ERROR(compressionParameter_unsupported);
+ if (value == 0) return 0; /* special value : 0 means "don't change anything" */
+ CLAMPCHECK(value, (unsigned)ZSTD_fast, (unsigned)ZSTD_btultra);
ZSTD_cLevelToCParams(cctx);
cctx->params.cParams.strategy = (ZSTD_strategy)value;
return 0;
static ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, int extDict)
{
- static const ZSTD_blockCompressor blockCompressor[2][8] = {
- { ZSTD_compressBlock_fast, ZSTD_compressBlock_doubleFast, ZSTD_compressBlock_greedy,
+ static const ZSTD_blockCompressor blockCompressor[2][(unsigned)ZSTD_btultra+1] = {
+ { NULL,
+ ZSTD_compressBlock_fast, ZSTD_compressBlock_doubleFast, ZSTD_compressBlock_greedy,
ZSTD_compressBlock_lazy, ZSTD_compressBlock_lazy2, ZSTD_compressBlock_btlazy2,
ZSTD_compressBlock_btopt, ZSTD_compressBlock_btultra },
- { ZSTD_compressBlock_fast_extDict, ZSTD_compressBlock_doubleFast_extDict, ZSTD_compressBlock_greedy_extDict,
+ { NULL,
+ ZSTD_compressBlock_fast_extDict, ZSTD_compressBlock_doubleFast_extDict, ZSTD_compressBlock_greedy_extDict,
ZSTD_compressBlock_lazy_extDict,ZSTD_compressBlock_lazy2_extDict, ZSTD_compressBlock_btlazy2_extDict,
ZSTD_compressBlock_btopt_extDict, ZSTD_compressBlock_btultra_extDict }
};
+ ZSTD_STATIC_ASSERT((unsigned)ZSTD_fast == 1);
return blockCompressor[extDict][(U32)strat];
}
/*******************************************************************************************************
Introduction
- zstd, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios
- at zlib-level and better compression ratios. The zstd compression library provides in-memory compression and
- decompression functions. The library supports compression levels from 1 up to ZSTD_maxCLevel() which is 22.
+ zstd, short for Zstandard, is a fast lossless compression algorithm,
+ targeting real-time compression scenarios at zlib-level and better compression ratios.
+ The zstd compression library provides in-memory compression and decompression functions.
+ The library supports compression levels from 1 up to ZSTD_maxCLevel() which is currently 22.
Levels >= 20, labeled `--ultra`, should be used with caution, as they require more memory.
Compression can be done in:
- a single step (described as Simple API)
const void* src, size_t compressedSize);
/*! ZSTD_getDecompressedSize() :
- * NOTE: This function is planned to be obsolete, in favour of ZSTD_getFrameContentSize.
- * ZSTD_getFrameContentSize functions the same way, returning the decompressed size of a single
- * frame, but distinguishes empty frames from frames with an unknown size, or errors.
- *
- * Additionally, ZSTD_findDecompressedSize can be used instead. It can handle multiple
- * concatenated frames in one buffer, and so is more general.
- * As a result however, it requires more computation and entire frames to be passed to it,
- * as opposed to ZSTD_getFrameContentSize which requires only a single frame's header.
+ * NOTE: This function is planned to be obsolete, in favor of ZSTD_getFrameContentSize().
+ * ZSTD_getFrameContentSize() works the same way,
+ * returning the decompressed size of a single frame,
+ * but distinguishes empty frames from frames with an unknown size, or errors.
*
* 'src' is the start of a zstd compressed frame.
* @return : content size to be decompressed, as a 64-bits value _if known_, 0 otherwise.
- * note 1 : decompressed size is an optional field, that may not be present, especially in streaming mode.
+ * note 1 : decompressed size is an optional field, it may not be present, typically in streaming mode.
* When `return==0`, data to decompress could be any size.
* In which case, it's necessary to use streaming mode to decompress data.
- * Optionally, application can still use ZSTD_decompress() while relying on implied limits.
+ * Optionally, application can use ZSTD_decompress() while relying on implied limits.
* (For example, data may be necessarily cut into blocks <= 16 KB).
* note 2 : decompressed size is always present when compression is done with ZSTD_compress()
* note 3 : decompressed size can be very large (64-bits value),
/*! ZSTD_compressCCtx() :
* Same as ZSTD_compress(), requires an allocated ZSTD_CCtx (see ZSTD_createCCtx()). */
-ZSTDLIB_API size_t ZSTD_compressCCtx(ZSTD_CCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize, int compressionLevel);
+ZSTDLIB_API size_t ZSTD_compressCCtx(ZSTD_CCtx* ctx,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize,
+ int compressionLevel);
/*= Decompression context
* When decompressing many times,
- * it is recommended to allocate a context just once, and re-use it for each successive compression operation.
+ * it is recommended to allocate a context only once,
+ * and re-use it for each successive compression operation.
* This will make workload friendlier for system's memory.
- * Use one context per thread for parallel execution in multi-threaded environments. */
+ * Use one context per thread for parallel execution. */
typedef struct ZSTD_DCtx_s ZSTD_DCtx;
ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx(void);
ZSTDLIB_API size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx);
/*! ZSTD_decompressDCtx() :
- * Same as ZSTD_decompress(), requires an allocated ZSTD_DCtx (see ZSTD_createDCtx()). */
-ZSTDLIB_API size_t ZSTD_decompressDCtx(ZSTD_DCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ * Same as ZSTD_decompress(), requires an allocated ZSTD_DCtx (see ZSTD_createDCtx()) */
+ZSTDLIB_API size_t ZSTD_decompressDCtx(ZSTD_DCtx* ctx,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize);
/**************************
* ZSTD_createCDict() will create a digested dictionary, ready to start future compression operations without startup delay.
* ZSTD_CDict can be created once and used by multiple threads concurrently, as its usage is read-only.
* `dictBuffer` can be released after ZSTD_CDict creation, as its content is copied within CDict */
-ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize, int compressionLevel);
+ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize,
+ int compressionLevel);
/*! ZSTD_freeCDict() :
* Function frees memory allocated by ZSTD_createCDict(). */
/*--- Advanced types ---*/
-typedef enum { ZSTD_fast, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2, ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra } ZSTD_strategy; /* from faster to stronger */
+typedef enum { ZSTD_fast=1, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2,
+ ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra } ZSTD_strategy; /* from faster to stronger */
typedef struct {
unsigned windowLog; /**< largest match distance : larger == more compression, more memory needed during decompression */
ZSTD_p_compressionStrategy, /* See ZSTD_strategy enum definition.
* Cast selected strategy as unsigned for ZSTD_CCtx_setParameter() compatibility.
* The higher the value of selected strategy, the more complex it is,
- * resulting in stronger and slower compression. */
+ * resulting in stronger and slower compression.
+ * Special: value 0 means "do not change strategy". */
#if 0
ZSTD_p_windowSize, /* Maximum allowed back-reference distance.
* Can be set to a more precise value than windowLog.
if (comprParams->searchLog) params.cParams.searchLog = comprParams->searchLog;
if (comprParams->searchLength) params.cParams.searchLength = comprParams->searchLength;
if (comprParams->targetLength) params.cParams.targetLength = comprParams->targetLength;
- if (comprParams->strategy) params.cParams.strategy = (ZSTD_strategy)(comprParams->strategy - 1); /* 0 means : do not change */
+ if (comprParams->strategy) params.cParams.strategy = (ZSTD_strategy) comprParams->strategy;
#ifdef ZSTD_MULTITHREAD
{ size_t const errorCode = ZSTDMT_initCStream_advanced(ress.cctx, dictBuffer, dictBuffSize, params, srcSize);
if (ZSTD_isError(errorCode)) EXM_THROW(33, "Error initializing CStream : %s", ZSTD_getErrorName(errorCode));
* Compiler specific
**************************************/
#ifdef _MSC_VER /* Visual Studio */
-# define _CRT_SECURE_NO_WARNINGS /* fgets */
-# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
-# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */
+# define _CRT_SECURE_NO_WARNINGS /* fgets */
+# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
+# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */
#endif
#include <stdio.h> /* fgets, sscanf */
#include <string.h> /* strcmp */
#include <time.h> /* clock_t */
-#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_compressContinue, ZSTD_compressBlock */
+#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_compressContinue, ZSTD_compressBlock */
#include "zstd.h" /* ZSTD_VERSION_STRING */
#include "zstd_errors.h" /* ZSTD_getErrorCode */
#include "zstdmt_compress.h"
* Compiler specific
**************************************/
#ifdef _MSC_VER /* Visual Studio */
-# define _CRT_SECURE_NO_WARNINGS /* fgets */
-# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
-# pragma warning(disable : 4146) /* disable: C4146: minus unsigned expression */
+# define _CRT_SECURE_NO_WARNINGS /* fgets */
+# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
+# pragma warning(disable : 4146) /* disable: C4146: minus unsigned expression */
#endif
#include <time.h> /* clock_t, clock() */
#include <string.h> /* strcmp */
#include "mem.h"
-#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_maxCLevel, ZSTD_customMem, ZSTD_getDictID_fromFrame */
+#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_maxCLevel, ZSTD_customMem, ZSTD_getDictID_fromFrame */
#include "zstd.h" /* ZSTD_compressBound */
#include "zstd_errors.h" /* ZSTD_error_srcSize_wrong */
#include "zstdmt_compress.h"
projects / thirdparty / zstd.git / commitdiff
