A ZSTD_CStream object is required to track streaming operation.
Use ZSTD_createCStream() and ZSTD_freeCStream() to create/release resources.
ZSTD_CStream objects can be reused multiple times on consecutive compression operations.
- It is recommended to re-use ZSTD_CStream in situations where many streaming operations will be achieved consecutively,
- since it will play nicer with system's memory, by re-using already allocated memory.
- Use one separate ZSTD_CStream per thread for parallel execution.
+ It is recommended to re-use ZSTD_CStream since it will play nicer with system's memory, by re-using already allocated memory.
- Start a new compression by initializing ZSTD_CStream context.
- Use ZSTD_initCStream() to start a new compression operation.
- Use variants ZSTD_initCStream_usingDict() or ZSTD_initCStream_usingCDict() for streaming with dictionary (experimental section)
+ For parallel execution, use one separate ZSTD_CStream per thread.
+
+ note : since v1.3.0, ZSTD_CStream and ZSTD_CCtx are the same thing.
+
+ Parameters are sticky : when starting a new compression on the same context,
+ it will re-use the same sticky parameters as previous compression session.
+ It's recommended to initialize the context before every usage.
+ Use ZSTD_initCStream() to set the parameter to a selected compression level.
+ Use advanced API (ZSTD_CCtx_setParameter(), etc.) to set more detailed parameters.
Use ZSTD_compressStream() as many times as necessary to consume input stream.
The function will automatically update both `pos` fields within `input` and `output`.
If not, the caller must make some room to receive more compressed data,
typically by emptying output buffer, or allocating a new output buffer,
and then present again remaining input data.
- @return : a size hint, preferred nb of bytes to use as input for next function call
- or an error code, which can be tested using ZSTD_isError().
- Note 1 : it's just a hint, to help latency a little, any other value will work fine.
- Note 2 : size hint is guaranteed to be <= ZSTD_CStreamInSize()
+ @return : a size hint, preferred nb of bytes to use as input for next function call
+ or an error code, which can be tested using ZSTD_isError().
+ Note 1 : it's just a hint, to help latency a little, any value will work fine.
+ Note 2 : size hint is guaranteed to be <= ZSTD_CStreamInSize()
At any moment, it's possible to flush whatever data might remain stuck within internal buffer,
using ZSTD_flushStream(). `output->pos` will be updated.
ZSTD_btlazy2=6,
ZSTD_btopt=7,
ZSTD_btultra=8
- </b>/* note : new strategies might be added in the future */<b>
+ </b>/* note : new strategies might be added in the future<b>
+ at this stage, only the order (from fast to strong) is guaranteed.
+ new strategy names may be introduced, pushing the maximum number upward */
} ZSTD_strategy;
</b></pre><BR>
<pre><b>typedef enum {
* ZSTD_p_forceMaxWindow
* ZSTD_p_forceAttachDict
* Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
- * note : never use experimentalParam names directly
+ * note : never ever use experimentalParam? names directly
*/
ZSTD_p_experimentalParam1=500,
ZSTD_p_experimentalParam2=10,
const void* src, size_t srcSize);
</b><p> Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced API.
- Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
- - The function is always blocking.
+ - The function is always blocking, returns when compression is completed.
Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`.
@return : compressed size written into `dst` (<= `dstCapacity),
or an error code if it fails (which can be tested using ZSTD_isError()).
ZSTD_e_flush=1, </b>/* flush any data provided so far,<b>
* it creates (at least) one new block, that can be decoded immediately on reception;
* frame will continue: any future data can still reference previously compressed data, improving compression. */
- ZSTD_e_end=2 </b>/* flush any remaining data and close current frame.<b>
- * any additional data starts a new frame.
- * each frame is independent (does not reference any content from previous frame). */
+ ZSTD_e_end=2 </b>/* flush any remaining data _and_ close current frame.<b>
+ * note that frame is only closed after compressed data is fully flushed (return value == 0).
+ * After that point, any additional data starts a new frame.
+ * note : each frame is independent (does not reference any content from previous frame). */
} ZSTD_EndDirective;
</b></pre><BR>
<pre><b>size_t ZSTD_compressStream2( ZSTD_CCtx* cctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input,
ZSTD_EndDirective endOp);
-</b><p> Behave about the same as ZSTD_compressStream, with additional control on end directive.
+</b><p> Behaves about the same as ZSTD_compressStream, with additional control on end directive.
- Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
- Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode)
- outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
- outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
- - In single-thread mode (default), function is blocking : it completes its job before returning to caller.
- - In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
- and then immediately returns, just indicating that there is some data remaining to be flushed.
- The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
+ - When nbWorkers==0 (default), function is blocking : it completes its job before returning to caller.
+ - When nbWorkers>=1, function is non-blocking : it just acquires a copy of input, and distributes jobs to internal worker threads,
+ and then immediately returns, just indicating that there is some data remaining to be flushed.
+ The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
- Exception : if the first call requests a ZSTD_e_end directive, the function delegates to ZSTD_compress2() which is always blocking.
- @return provides a minimum amount of data remaining to be flushed from internal buffers
or an error code, which can be tested using ZSTD_isError().
</p></pre><BR>
-<pre><b>size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
-</b><p> Refuses allocating internal buffers for frames requiring a window size larger than provided limit.
- This protects a decoder context from reserving too much memory for itself (potential attack scenario).
- This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
- By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT)
- @return : 0, or an error code (which can be tested using ZSTD_isError()).
+<pre><b>typedef enum {
+
+ ZSTD_d_windowLogMax=100, </b>/* Used to select a limit beyond which<b>
+ * the streaming API will refuse to allocate memory buffer
+ * in order to protect the host from unreasonable memory requirements.
+ * This parameter is only useful in streaming mode, as no internal buffer is allocated in single-pass mode.
+ * By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT) */
+
+ </b>/* note : additional experimental parameters are also available<b>
+ * within the experimental section of the API.
+ * At the time of this writing, they include :
+ * ZSTD_p_format
+ * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
+ * note : never ever use experimentalParam? names directly
+ */
+ ZSTD_d_experimentalParam1=1000
+
+} ZSTD_dParameter;
+</b></pre><BR>
+<pre><b>ZSTD_bounds ZSTD_dParam_getBounds(ZSTD_dParameter dParam); </b>/* not implemented yet */<b>
+</b><p> All parameters must belong to an interval with lower and upper bounds,
+ otherwise they will either trigger an error or be automatically clamped.
+ @return : a structure, ZSTD_bounds, which contains
+ - an error status field, which must be tested using ZSTD_isError()
+ - both lower and upper bounds, inclusive
+
+</p></pre><BR>
+
+<pre><b>size_t ZSTD_DCtx_setParameter(ZSTD_DCtx* dctx, ZSTD_dParameter param, int value); </b>/* not implemented yet */<b>
+</b><p> Set one compression parameter, selected by enum ZSTD_dParameter.
+ All parameters have valid bounds. Bounds can be queried using ZSTD_dParam_getBounds().
+ Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter).
+ Setting a parameter is only possible during frame initialization (before starting decompression).
+ @return : an error code (which can be tested using ZSTD_isError()).
</p></pre><BR>
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Note 1 : Currently, only one dictionary can be managed.
Referencing a new dictionary effectively "discards" any previous one.
- Special : adding a NULL DDict means "return to no-dictionary mode".
+ Special: referencing a NULL DDict means "return to no-dictionary mode".
Note 2 : DDict is just referenced, its lifetime must outlive its usage from DCtx.
</p></pre><BR>
This is the reverse operation of ZSTD_CCtx_refPrefix(),
and must use the same prefix as the one used during compression.
Prefix is **only used once**. Reference is discarded at end of frame.
- End of frame is reached when ZSTD_DCtx_decompress_generic() returns 0.
+ End of frame is reached when ZSTD_decompressStream() returns 0.
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Note 1 : Adding any prefix (including NULL) invalidates any previously set prefix or dictionary
Note 2 : Prefix buffer is referenced. It **must** outlive decompression job.
Prefix buffer must remain unmodified up to the end of frame,
- reached when ZSTD_DCtx_decompress_generic() returns 0.
+ reached when ZSTD_decompressStream() returns 0.
Note 3 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent).
- Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode.
+ Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode (Experimental section)
Note 4 : Referencing a raw content prefix has almost no cpu nor memory cost.
- A fulldict prefix requires building tables, hence is more costly.
+ A full dictionary is more costly, as it requires building tables.
</p></pre><BR>
</p></pre><BR>
-<pre><b>size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx,
- ZSTD_outBuffer* output,
- ZSTD_inBuffer* input);
-</b><p> Behave the same as ZSTD_decompressStream.
- Decompression parameters cannot be changed once decompression is started.
- @return : an error code, which can be tested using ZSTD_isError()
- if >0, a hint, nb of expected input bytes for next invocation.
- `0` means : a frame has just been fully decoded and flushed.
-
-</p></pre><BR>
-
<a name="Chapter14"></a><h2>experimental API (static linking only)</h2><pre>
The following symbols and constants
are not planned to join "stable API" status in the near future.
how to interpret prefix content (automatic ? force raw mode (default) ? full mode only ?)
</p></pre><BR>
+<pre><b>size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
+</b><p> Refuses allocating internal buffers for frames requiring a window size larger than provided limit.
+ This protects a decoder context from reserving too much memory for itself (potential attack scenario).
+ This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
+ By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT)
+ @return : 0, or an error code (which can be tested using ZSTD_isError()).
+
+</p></pre><BR>
+
<pre><b>size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
</b><p> Instruct the decoder context about what kind of data to decode next.
This instruction is mandatory to decode data without a fully-formed header,
@return : 0, or an error code (which can be tested using ZSTD_isError()).
</p></pre><BR>
-<pre><b>size_t ZSTD_decompress_generic_simpleArgs (
+<pre><b>size_t ZSTD_decompressStream_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
-</b><p> Same as ZSTD_decompress_generic(),
+</b><p> Same as ZSTD_decompressStream(),
but using only integral types as arguments.
This can be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.
***************************************/
/* API design :
- * In this API, parameters are pushed one by one into an existing context,
+ * Parameters are pushed one by one into an existing context,
* using ZSTD_CCtx_set*() functions.
* Pushed parameters are sticky : they are applied to next job, and any subsequent job.
- * Note that "sticky" parameters are only applicable with `ZSTD_compress2()` and `ZSTD_compressStream2()` !
- * They do not apply should the context be used with a "simple" variant such as ZSTD_compressCCtx()
+ * "sticky" parameters are applicable to `ZSTD_compress2()` and `ZSTD_compressStream*()` !
+ * They do not apply to "simple" one-shot variants such as ZSTD_compressCCtx()
*
* It's possible to reset all parameters to "default" using ZSTD_CCtx_reset().
*
- * This API gives access to all advanced capabilities.
- * It supercedes all other "advanced" API entry points in the experimental section.
+ * This API supercedes all other "advanced" API entry points in the experimental section.
* In the future, we expect to remove from experimental API entry points which are redundant with this API.
*/
* ZSTD_p_forceMaxWindow
* ZSTD_p_forceAttachDict
* Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
- * note : never use experimentalParam names directly
+ * note : never ever use experimentalParam? names directly
*/
ZSTD_p_experimentalParam1=500,
ZSTD_p_experimentalParam2=10,
/* Advanced decompression API */
/* ============================== */
-/* The following API works the same way as the advanced compression API :
- * a context is created, parameters are pushed into it one by one,
- * then the context can be used to decompress data
- * using ZSTD_decompress_generic(), similar to the streaming API.
- * Note that sticky parameters only apply to ZSTD_decompress_generic() and ZSTD_decompress_generic_simpleArgs().
- * They are not valid if a "simple" function is used on the context (like `ZSTD_decompressDCtx()`).
+/* The advanced API pushes parameters one by one into an existing DCtx context.
+ * Parameters are "sticky, and remain valid for all futures decompression jobs
+ * started with the same DCtx context.
+ * It's possible to reset parameters to default values using ZSTD_DCtx_reset().
+ * Note : No new decompression function is provided,
+ * use existing ZSTD_decompressDCtx() and ZSTD_decompressStream().
*/
* At the time of this writing, they include :
* ZSTD_p_format
* Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
- * note : never use experimentalParam names directly
+ * note : never ever use experimentalParam? names directly
*/
ZSTD_d_experimentalParam1=1000
* @result : 0, or an error code (which can be tested with ZSTD_isError()).
* Note 1 : Currently, only one dictionary can be managed.
* Referencing a new dictionary effectively "discards" any previous one.
- * Special : adding a NULL DDict means "return to no-dictionary mode".
+ * Special: referencing a NULL DDict means "return to no-dictionary mode".
* Note 2 : DDict is just referenced, its lifetime must outlive its usage from DCtx.
*/
ZSTDLIB_API size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict);
* This is the reverse operation of ZSTD_CCtx_refPrefix(),
* and must use the same prefix as the one used during compression.
* Prefix is **only used once**. Reference is discarded at end of frame.
- * End of frame is reached when ZSTD_DCtx_decompress_generic() returns 0.
+ * End of frame is reached when ZSTD_decompressStream() returns 0.
* @result : 0, or an error code (which can be tested with ZSTD_isError()).
* Note 1 : Adding any prefix (including NULL) invalidates any previously set prefix or dictionary
* Note 2 : Prefix buffer is referenced. It **must** outlive decompression job.
* Prefix buffer must remain unmodified up to the end of frame,
- * reached when ZSTD_DCtx_decompress_generic() returns 0.
+ * reached when ZSTD_decompressStream() returns 0.
* Note 3 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent).
- * Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode.
+ * Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode (Experimental section)
* Note 4 : Referencing a raw content prefix has almost no cpu nor memory cost.
- * A fulldict prefix requires building tables, hence is more costly.
+ * A full dictionary is more costly, as it requires building tables.
*/
ZSTDLIB_API size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx,
const void* prefix, size_t prefixSize);
ZSTDLIB_API size_t ZSTD_DCtx_reset(ZSTD_DCtx* dctx, ZSTD_ResetDirective reset);
-/*! ZSTD_decompress_generic() :
- * Behave the same as ZSTD_decompressStream.
- * Decompression parameters cannot be changed once decompression is started.
- * @return : an error code, which can be tested using ZSTD_isError()
- * if >0, a hint, nb of expected input bytes for next invocation.
- * `0` means : a frame has just been fully decoded and flushed.
- */
-ZSTDLIB_API size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx,
- ZSTD_outBuffer* output,
- ZSTD_inBuffer* input);
-
/****************************************************************************************
* experimental API (static linking only)
* @return : 0, or an error code (which can be tested using ZSTD_isError()). */
ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
-/*! ZSTD_decompress_generic_simpleArgs() :
- * Same as ZSTD_decompress_generic(),
+/*! ZSTD_decompressStream_simpleArgs() :
+ * Same as ZSTD_decompressStream(),
* but using only integral types as arguments.
* This can be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
-ZSTDLIB_API size_t ZSTD_decompress_generic_simpleArgs (
+ZSTDLIB_API size_t ZSTD_decompressStream_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
projects / thirdparty / zstd.git / commitdiff
