From: Yann Collet Date: Tue, 28 Mar 2023 23:43:33 +0000 (-0700) Subject: updated version number to v1.5.5 X-Git-Tag: v1.5.5~2^2~1 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=9f58241dcc9f0f7882347ec5dc5560e41727b8c4;p=thirdparty%2Fzstd.git updated version number to v1.5.5 also : updated man pages --- diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html index c33f0e262..dcc10208d 100644 --- a/doc/zstd_manual.html +++ b/doc/zstd_manual.html @@ -1,10 +1,10 @@ -zstd 1.5.4 Manual +zstd 1.5.5 Manual -

zstd 1.5.4 Manual

+

zstd 1.5.5 Manual

Contents

    @@ -22,15 +22,15 @@
  1. Dictionary helper functions
  2. Advanced dictionary and prefix API (Requires v1.4.0+)
  3. experimental API (static linking only)
    4. -
  4. Frame size functions
    5. +
  5. Frame header and size functions
  6. Memory management
  7. Advanced compression functions
  8. Advanced decompression functions
  9. Advanced streaming functions
    10. -
  10. Buffer-less and synchronous inner streaming functions
    11. +
  11. Buffer-less and synchronous inner streaming functions (DEPRECATED)
  12. Buffer-less streaming compression (synchronous mode)
  13. Buffer-less streaming decompression (synchronous mode)
    14. -
  14. Block level API
    15. +
  15. Block level API (DEPRECATED)

Introduction

@@ -80,7 +80,8 @@
                 const void* src, size_t srcSize,
                       int compressionLevel);
 
  Compresses `src` content as a single zstd compressed frame into already allocated `dst`.
-  Hint : compression runs faster if `dstCapacity` >=  `ZSTD_compressBound(srcSize)`.
+  NOTE: Providing `dstCapacity >= ZSTD_compressBound(srcSize)` guarantees that zstd will have
+        enough space to successfully compress the data.
   @return : compressed size written into `dst` (<= `dstCapacity),
             or an error code if it fails (which can be tested using ZSTD_isError()).

@@ -486,7 +487,8 @@ size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx); /* accept NULL pointer */ Should cctx hold data from a previously unfinished frame, everything about it is forgotten. - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*() - The function is always blocking, returns when compression is completed. - Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`. + NOTE: Providing `dstCapacity >= ZSTD_compressBound(srcSize)` guarantees that zstd will have + enough space to successfully compress the data, though it is possible it fails for other reasons. @return : compressed size written into `dst` (<= `dstCapacity), or an error code if it fails (which can be tested using ZSTD_isError()). @@ -866,9 +868,11 @@ size_t ZSTD_freeDStream(ZSTD_DStream* zds); /* accept NULL pointer */

Advanced dictionary and prefix API (Requires v1.4.0+)

  This API allows dictionaries to be used with ZSTD_compress2(),
- ZSTD_compressStream2(), and ZSTD_decompressDCtx(). Dictionaries are sticky, and
- only reset with the context is reset with ZSTD_reset_parameters or
- ZSTD_reset_session_and_parameters. Prefixes are single-use.
+ ZSTD_compressStream2(), and ZSTD_decompressDCtx().
+ Dictionaries are sticky, they remain valid when same context is re-used,
+ they only reset when the context is reset
+ with ZSTD_reset_parameters or ZSTD_reset_session_and_parameters.
+ In contrast, Prefixes are single-use.
 
size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
@@ -888,7 +892,11 @@ size_t ZSTD_freeDStream(ZSTD_DStream* zds);  /* accept NULL pointer */
            Use experimental ZSTD_CCtx_loadDictionary_byReference() to reference content instead.
            In such a case, dictionary buffer must outlive its users.
   Note 4 : Use ZSTD_CCtx_loadDictionary_advanced()
-           to precisely select how dictionary content must be interpreted. 
+           to precisely select how dictionary content must be interpreted.
+  Note 5 : This method does not benefit from LDM (long distance mode).
+           If you want to employ LDM on some large dictionary content,
+           prefer employing ZSTD_CCtx_refPrefix() described below.
+ 
 

size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict);
@@ -912,6 +920,7 @@ size_t ZSTD_freeDStream(ZSTD_DStream* zds);  /* accept NULL pointer */
   Decompression will need same prefix to properly regenerate data.
   Compressing with a prefix is similar in outcome as performing a diff and compressing it,
   but performs much faster, especially during decompression (compression speed is tunable with compression level).
+  This method is compatible with LDM (long distance mode).
  @result : 0, or an error code (which can be tested with ZSTD_isError()).
   Special: Adding any prefix (including NULL) invalidates any previous prefix or dictionary
   Note 1 : Prefix buffer is referenced. It **must** outlive compression.
@@ -1146,7 +1155,7 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
   ZSTD_ps_disable = 2       /* Do not use the feature */
 } ZSTD_paramSwitch_e;

-

Frame size functions


+
Frame header and size functions

 
 
ZSTDLIB_STATIC_API unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize);
 
  `src` should point to the start of a series of ZSTD encoded and/or skippable frames
@@ -1192,6 +1201,31 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
            or an error code (if srcSize is too small) 
 


 
+
typedef enum { ZSTD_frame, ZSTD_skippableFrame } ZSTD_frameType_e;
+


+
typedef struct {
+    unsigned long long frameContentSize; /* if == ZSTD_CONTENTSIZE_UNKNOWN, it means this field is not available. 0 means "empty" */
+    unsigned long long windowSize;       /* can be very large, up to <= frameContentSize */
+    unsigned blockSizeMax;
+    ZSTD_frameType_e frameType;          /* if == ZSTD_skippableFrame, frameContentSize is the size of skippable content */
+    unsigned headerSize;
+    unsigned dictID;
+    unsigned checksumFlag;
+    unsigned _reserved1;
+    unsigned _reserved2;
+} ZSTD_frameHeader;
+


+
ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize);   /**< doesn't consume input */
+/*! ZSTD_getFrameHeader_advanced() :
+ *  same as ZSTD_getFrameHeader(),
+ *  with added capability to select a format (like ZSTD_f_zstd1_magicless) */
+ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format);
+
  decode Frame Header, or requires larger `srcSize`.
+ @return : 0, `zfhPtr` is correctly filled,
+          >0, `srcSize` is too small, value is wanted `srcSize` amount,
+           or an error code, which can be tested using ZSTD_isError() 
+


+
 
ZSTDLIB_STATIC_API size_t ZSTD_decompressionMargin(const void* src, size_t srcSize);
 
 Zstd supports in-place decompression, where the input and output buffers overlap.
  In this case, the output buffer must be at least (Margin + Output_Size) bytes large,
@@ -1505,10 +1539,24 @@ ZSTDLIB_STATIC_API size_t ZSTD_CCtx_refThreadPool(ZSTD_CCtx* cctx, ZSTD_threadPo
 


 
 
ZSTDLIB_STATIC_API size_t ZSTD_CCtx_setCParams(ZSTD_CCtx* cctx, ZSTD_compressionParameters cparams);
-
  Set all parameters provided within @cparams into the working @cctx.
+
  Set all parameters provided within @p cparams into the working @p cctx.
   Note : if modifying parameters during compression (MT mode only),
          note that changes to the .windowLog parameter will be ignored.
- @return 0 on success, or an error code (can be checked with ZSTD_isError()) 
+ @return 0 on success, or an error code (can be checked with ZSTD_isError()).
+         On failure, no parameters are updated.
+ 
+


+
+
ZSTDLIB_STATIC_API size_t ZSTD_CCtx_setFParams(ZSTD_CCtx* cctx, ZSTD_frameParameters fparams);
+
  Set all parameters provided within @p fparams into the working @p cctx.
+ @return 0 on success, or an error code (can be checked with ZSTD_isError()).
+ 
+


+
+
ZSTDLIB_STATIC_API size_t ZSTD_CCtx_setParams(ZSTD_CCtx* cctx, ZSTD_parameters params);
+
  Set all parameters provided within @p params into the working @p cctx.
+ @return 0 on success, or an error code (can be checked with ZSTD_isError()).
+ 
 


 
 
ZSTD_DEPRECATED("use ZSTD_compress2")
@@ -1754,12 +1802,9 @@ size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
         const void* dict, size_t dictSize,
               ZSTD_parameters params,
               unsigned long long pledgedSrcSize);
-
 This function is DEPRECATED, and is approximately equivalent to:
+
 This function is DEPRECATED, and is equivalent to:
      ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
-     // Pseudocode: Set each zstd parameter and leave the rest as-is.
-     for ((param, value) : params) {
-         ZSTD_CCtx_setParameter(zcs, param, value);
-     }
+     ZSTD_CCtx_setParams(zcs, params);
      ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize);
      ZSTD_CCtx_loadDictionary(zcs, dict, dictSize);
 
@@ -1788,12 +1833,9 @@ size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs,
                    const ZSTD_CDict* cdict,
                          ZSTD_frameParameters fParams,
                          unsigned long long pledgedSrcSize);
-
   This function is DEPRECATED, and is approximately equivalent to:
+
   This function is DEPRECATED, and is equivalent to:
      ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
-     // Pseudocode: Set each zstd frame parameter and leave the rest as-is.
-     for ((fParam, value) : fParams) {
-         ZSTD_CCtx_setParameter(zcs, fParam, value);
-     }
+     ZSTD_CCtx_setFParams(zcs, fParams);
      ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize);
      ZSTD_CCtx_refCDict(zcs, cdict);
 
@@ -1880,10 +1922,42 @@ ZSTDLIB_STATIC_API size_t ZSTD_resetDStream(ZSTD_DStream* zds);
  
 


 
-
Buffer-less and synchronous inner streaming functions
-  This is an advanced API, giving full control over buffer management, for users which need direct control over memory.
-  But it's also a complex one, with several restrictions, documented below.
-  Prefer normal streaming API for an easier experience.
+
ZSTDLIB_STATIC_API void
+ZSTD_registerSequenceProducer(
+  ZSTD_CCtx* cctx,
+  void* sequenceProducerState,
+  ZSTD_sequenceProducer_F* sequenceProducer
+);
+
 Instruct zstd to use a block-level external sequence producer function.
+
+ The sequenceProducerState must be initialized by the caller, and the caller is
+ responsible for managing its lifetime. This parameter is sticky across
+ compressions. It will remain set until the user explicitly resets compression
+ parameters.
+
+ Sequence producer registration is considered to be an "advanced parameter",
+ part of the "advanced API". This means it will only have an effect on compression
+ APIs which respect advanced parameters, such as compress2() and compressStream2().
+ Older compression APIs such as compressCCtx(), which predate the introduction of
+ "advanced parameters", will ignore any external sequence producer setting.
+
+ The sequence producer can be "cleared" by registering a NULL function pointer. This
+ removes all limitations described above in the "LIMITATIONS" section of the API docs.
+
+ The user is strongly encouraged to read the full API documentation (above) before
+ calling this function. 
+


+
+
Buffer-less and synchronous inner streaming functions (DEPRECATED)
+  This API is deprecated, and will be removed in a future version.
+  It allows streaming (de)compression with user allocated buffers.
+  However, it is hard to use, and not as well tested as the rest of
+  our API.
+
+  Please use the normal streaming API instead: ZSTD_compressStream2,
+  and ZSTD_decompressStream.
+  If there is functionality that you need, but it doesn't provide,
+  please open an issue on our GitHub.
  
 

 
@@ -1914,8 +1988,11 @@ ZSTDLIB_STATIC_API size_t ZSTD_resetDStream(ZSTD_DStream* zds);
   `ZSTD_CCtx` object can be re-used (ZSTD_compressBegin()) to compress again.
 

 
-
Buffer-less streaming compression functions
ZSTDLIB_STATIC_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
+
Buffer-less streaming compression functions
ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
+ZSTDLIB_STATIC_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
+ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
 ZSTDLIB_STATIC_API size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
+ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
 ZSTDLIB_STATIC_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /**< note: fails if cdict==NULL */
 


 
size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**<  note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
@@ -1993,36 +2070,25 @@ ZSTDLIB_STATIC_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const Z
   For skippable frames ZSTD_decompressContinue() always returns 0 : it only skips the content.
 

 
-
Buffer-less streaming decompression functions
typedef enum { ZSTD_frame, ZSTD_skippableFrame } ZSTD_frameType_e;
-typedef struct {
-    unsigned long long frameContentSize; /* if == ZSTD_CONTENTSIZE_UNKNOWN, it means this field is not available. 0 means "empty" */
-    unsigned long long windowSize;       /* can be very large, up to <= frameContentSize */
-    unsigned blockSizeMax;
-    ZSTD_frameType_e frameType;          /* if == ZSTD_skippableFrame, frameContentSize is the size of skippable content */
-    unsigned headerSize;
-    unsigned dictID;
-    unsigned checksumFlag;
-    unsigned _reserved1;
-    unsigned _reserved2;
-} ZSTD_frameHeader;
-


-
ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize);   /**< doesn't consume input */
-/*! ZSTD_getFrameHeader_advanced() :
- *  same as ZSTD_getFrameHeader(),
- *  with added capability to select a format (like ZSTD_f_zstd1_magicless) */
-ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format);
-ZSTDLIB_STATIC_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize);  /**< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */
-
  decode Frame Header, or requires larger `srcSize`.
- @return : 0, `zfhPtr` is correctly filled,
-          >0, `srcSize` is too small, value is wanted `srcSize` amount,
-           or an error code, which can be tested using ZSTD_isError() 
-


-
+
Buffer-less streaming decompression functions


+
ZSTDLIB_STATIC_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize);  /**< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */
+


 
typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e;
 


-
Block level API

+
Block level API (DEPRECATED)

 
-
    Frame metadata cost is typically ~12 bytes, which can be non-negligible for very small blocks (< 100 bytes).
+
    You can get the frame header down to 2 bytes by setting:
+      - ZSTD_c_format = ZSTD_f_zstd1_magicless
+      - ZSTD_c_contentSizeFlag = 0
+      - ZSTD_c_checksumFlag = 0
+      - ZSTD_c_dictIDFlag = 0
+
+    This API is not as well tested as our normal API, so we recommend not using it.
+    We will be removing it in a future version. If the normal API doesn't provide
+    the functionality you need, please open a GitHub issue.
+
+    Block functions produce and decode raw zstd blocks, without frame metadata.
+    Frame metadata cost is typically ~12 bytes, which can be non-negligible for very small blocks (< 100 bytes).
     But users will have to take in charge needed metadata to regenerate data, such as compressed and content sizes.
 
     A few rules to respect :
@@ -2046,36 +2112,14 @@ ZSTDLIB_STATIC_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowS
         Use ZSTD_insertBlock() for such a case.
 


 
-
Raw zstd block functions
ZSTDLIB_STATIC_API size_t ZSTD_getBlockSize   (const ZSTD_CCtx* cctx);
+
Raw zstd block functions
ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
+ZSTDLIB_STATIC_API size_t ZSTD_getBlockSize   (const ZSTD_CCtx* cctx);
+ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
 ZSTDLIB_STATIC_API size_t ZSTD_compressBlock  (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
 ZSTDLIB_STATIC_API size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
 ZSTDLIB_STATIC_API size_t ZSTD_insertBlock    (ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize);  /**< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression. */
 


-
ZSTDLIB_STATIC_API void
-ZSTD_registerSequenceProducer(
-  ZSTD_CCtx* cctx,
-  void* sequenceProducerState,
-  ZSTD_sequenceProducer_F* sequenceProducer
-);
-
 Instruct zstd to use a block-level external sequence producer function.
-
- The sequenceProducerState must be initialized by the caller, and the caller is
- responsible for managing its lifetime. This parameter is sticky across
- compressions. It will remain set until the user explicitly resets compression
- parameters.
-
- Sequence producer registration is considered to be an "advanced parameter",
- part of the "advanced API". This means it will only have an effect on compression
- APIs which respect advanced parameters, such as compress2() and compressStream2().
- Older compression APIs such as compressCCtx(), which predate the introduction of
- "advanced parameters", will ignore any external sequence producer setting.
-
- The sequence producer can be "cleared" by registering a NULL function pointer. This
- removes all limitations described above in the "LIMITATIONS" section of the API docs.
-
- The user is strongly encouraged to read the full API documentation (above) before
- calling this function. 
-


-
 
 
diff --git a/lib/zstd.h b/lib/zstd.h
index bdca92512..e5c3f8b68 100644
--- a/lib/zstd.h
+++ b/lib/zstd.h
@@ -106,7 +106,7 @@ extern "C" {
 /*------   Version   ------*/
 #define ZSTD_VERSION_MAJOR    1
 #define ZSTD_VERSION_MINOR    5
-#define ZSTD_VERSION_RELEASE  4
+#define ZSTD_VERSION_RELEASE  5
 #define ZSTD_VERSION_NUMBER  (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
 
 /*! ZSTD_versionNumber() :
diff --git a/programs/zstd.1 b/programs/zstd.1
index edc128fd1..383d99470 100644
--- a/programs/zstd.1
+++ b/programs/zstd.1
@@ -1,4 +1,4 @@
-.TH "ZSTD" "1" "February 2023" "zstd 1.5.4" "User Commands"
+.TH "ZSTD" "1" "March 2023" "zstd 1.5.5" "User Commands"
 .SH "NAME"
 \fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files
 .SH "SYNOPSIS"
@@ -105,7 +105,7 @@ Note: for all levels, you can use \fB\-\-patch\-from\fR in \fB\-\-single\-thread
 .IP
 Note: for level 19, you can get increased compression ratio at the cost of speed by specifying \fB\-\-zstd=targetLength=\fR to be something large (i\.e\. 4096), and by setting a large \fB\-\-zstd=chainLog=\fR\.
 .IP "\[ci]" 4
-\fB\-\-rsyncable\fR: \fBzstd\fR will periodically synchronize the compression state to make the compressed file more rsync\-friendly\. There is a negligible impact to compression ratio, and the faster compression levels will see a small compression speed hit\. This feature does not work with \fB\-\-single\-thread\fR\. You probably don\'t want to use it with long range mode, since it will decrease the effectiveness of the synchronization points, but your mileage may vary\.
+\fB\-\-rsyncable\fR: \fBzstd\fR will periodically synchronize the compression state to make the compressed file more rsync\-friendly\. There is a negligible impact to compression ratio, and a potential impact to compression speed, perceptible at higher speeds, for example when combining \fB\-\-rsyncable\fR with many parallel worker threads\. This feature does not work with \fB\-\-single\-thread\fR\. You probably don\'t want to use it with long range mode, since it will decrease the effectiveness of the synchronization points, but your mileage may vary\.
 .IP "\[ci]" 4
 \fB\-C\fR, \fB\-\-[no\-]check\fR: add integrity check computed from uncompressed data (default: enabled)
 .IP "\[ci]" 4
diff --git a/programs/zstdgrep.1 b/programs/zstdgrep.1
index 77d29b4bf..1204e5bb6 100644
--- a/programs/zstdgrep.1
+++ b/programs/zstdgrep.1
@@ -1,4 +1,4 @@
-.TH "ZSTDGREP" "1" "February 2023" "zstd 1.5.4" "User Commands"
+.TH "ZSTDGREP" "1" "March 2023" "zstd 1.5.5" "User Commands"
 .SH "NAME"
 \fBzstdgrep\fR \- print lines matching a pattern in zstandard\-compressed files
 .SH "SYNOPSIS"
diff --git a/programs/zstdless.1 b/programs/zstdless.1
index 6c82b9c9e..bc019b263 100644
--- a/programs/zstdless.1
+++ b/programs/zstdless.1
@@ -1,4 +1,4 @@
-.TH "ZSTDLESS" "1" "February 2023" "zstd 1.5.4" "User Commands"
+.TH "ZSTDLESS" "1" "March 2023" "zstd 1.5.5" "User Commands"
 .SH "NAME"
 \fBzstdless\fR \- view zstandard\-compressed files
 .SH "SYNOPSIS"