]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
4 years agoDocument the ZSTD_c_enableDedicatedDictSearch Parameter
W. Felix Handte [Tue, 18 Aug 2020 20:58:11 +0000 (16:58 -0400)] 
Document the ZSTD_c_enableDedicatedDictSearch Parameter

4 years agoRename and Move DD_BLOG Constant to ZSTD_LAZY_DDSS_BUCKET_LOG
W. Felix Handte [Tue, 18 Aug 2020 19:20:12 +0000 (15:20 -0400)] 
Rename and Move DD_BLOG Constant to ZSTD_LAZY_DDSS_BUCKET_LOG

4 years agoPrefetch First Batch of Match Positions; +11% Speed in Level 5 w/ 1 Dict
W. Felix Handte [Tue, 18 Aug 2020 17:30:41 +0000 (13:30 -0400)] 
Prefetch First Batch of Match Positions; +11% Speed in Level 5 w/ 1 Dict

4 years agoFix Off-By-One Error in Counting DDS Search Attempts
W. Felix Handte [Mon, 17 Aug 2020 21:43:01 +0000 (17:43 -0400)] 
Fix Off-By-One Error in Counting DDS Search Attempts

This caused us to double-search the first position and fail to search the
last position in the chain, slowing down search and making it less effective.

4 years agoEnable Dedicated Dict Search in the CLI
W. Felix Handte [Mon, 17 Aug 2020 16:37:58 +0000 (12:37 -0400)] 
Enable Dedicated Dict Search in the CLI

4 years agoInit CCtx's Local Dict with CCtxParams
W. Felix Handte [Mon, 17 Aug 2020 16:35:50 +0000 (12:35 -0400)] 
Init CCtx's Local Dict with CCtxParams

4 years agoDecision for Whether to Attach Should be Based on CDict Config, not CCtx
W. Felix Handte [Thu, 13 Aug 2020 20:52:07 +0000 (16:52 -0400)] 
Decision for Whether to Attach Should be Based on CDict Config, not CCtx

4 years agoMove Prefetch Before Insertion; Speed Up ~6%
W. Felix Handte [Thu, 13 Aug 2020 18:54:10 +0000 (14:54 -0400)] 
Move Prefetch Before Insertion; Speed Up ~6%

4 years agoMisc Refactor of DDS Search Code
W. Felix Handte [Thu, 13 Aug 2020 15:57:31 +0000 (11:57 -0400)] 
Misc Refactor of DDS Search Code

4 years agoRename enableDedicatedDictSearch to dedicatedDictSearch in MatchState
W. Felix Handte [Wed, 12 Aug 2020 20:50:44 +0000 (16:50 -0400)] 
Rename enableDedicatedDictSearch to dedicatedDictSearch in MatchState

This makes it clear that not only is the feature allowed here, we're actually
using it, as opposed to the CCtxParam field, in which it's enabled, but we may
or may not be using it.

4 years agoAlways Init CDict's enableDedicatedDictSearch Field
W. Felix Handte [Tue, 11 Aug 2020 22:57:12 +0000 (18:57 -0400)] 
Always Init CDict's enableDedicatedDictSearch Field

4 years agoAdd a ZSTD_dedicatedDictSearch ZSTD_dictMode_e to Allow Const Propagation
W. Felix Handte [Tue, 11 Aug 2020 22:48:22 +0000 (18:48 -0400)] 
Add a ZSTD_dedicatedDictSearch ZSTD_dictMode_e to Allow Const Propagation

Speed +1.5%.

4 years agoFix ZSTD_c_forceAttachDict Bounds
W. Felix Handte [Tue, 11 Aug 2020 22:24:11 +0000 (18:24 -0400)] 
Fix ZSTD_c_forceAttachDict Bounds

4 years agoMake ZSTD_c_enableDedicatedDictSearch an Experimental Param
W. Felix Handte [Tue, 11 Aug 2020 22:03:38 +0000 (18:03 -0400)] 
Make ZSTD_c_enableDedicatedDictSearch an Experimental Param

4 years agoAdd Fuzzer Test for the Various Dict Attachment Strategies
W. Felix Handte [Tue, 11 Aug 2020 19:29:12 +0000 (15:29 -0400)] 
Add Fuzzer Test for the Various Dict Attachment Strategies

4 years agoTODO: Comment
W. Felix Handte [Fri, 7 Aug 2020 20:41:16 +0000 (16:41 -0400)] 
TODO: Comment

4 years agoFix Compilation Warnings
W. Felix Handte [Fri, 7 Aug 2020 16:02:18 +0000 (12:02 -0400)] 
Fix Compilation Warnings

4 years agoupdating cold benchmark
Bimba Shrestha [Fri, 12 Jun 2020 15:53:58 +0000 (08:53 -0700)] 
updating cold benchmark

4 years agocreating ZSTD_createCDict_advanced_internal
Bimba Shrestha [Fri, 12 Jun 2020 15:01:18 +0000 (08:01 -0700)] 
creating ZSTD_createCDict_advanced_internal

4 years agochanging to int for consistency
Bimba Shrestha [Fri, 12 Jun 2020 05:57:53 +0000 (22:57 -0700)] 
changing to int for consistency

4 years agousing dict mls instead of src mls
Bimba Shrestha [Fri, 12 Jun 2020 05:55:32 +0000 (22:55 -0700)] 
using dict mls instead of src mls

4 years agoadd hashtable head prefetching
Bimba Shrestha [Fri, 12 Jun 2020 05:54:36 +0000 (22:54 -0700)] 
add hashtable head prefetching

4 years agochange method name for consistency
Bimba Shrestha [Fri, 12 Jun 2020 05:48:48 +0000 (22:48 -0700)] 
change method name for consistency

4 years agopass correct cparams
Bimba Shrestha [Fri, 12 Jun 2020 05:46:16 +0000 (22:46 -0700)] 
pass correct cparams

4 years agoremoving wrong comment
Bimba Shrestha [Fri, 12 Jun 2020 05:42:57 +0000 (22:42 -0700)] 
removing wrong comment

4 years agomaking cctxParams a pointer
Bimba Shrestha [Fri, 12 Jun 2020 05:42:11 +0000 (22:42 -0700)] 
making cctxParams a pointer

4 years agoloading dict using new algorithm
Bimba Shrestha [Fri, 12 Jun 2020 02:41:03 +0000 (19:41 -0700)] 
loading dict using new algorithm

4 years agoadding dedicatedDictSearch algorithm
Bimba Shrestha [Fri, 12 Jun 2020 01:54:44 +0000 (18:54 -0700)] 
adding dedicatedDictSearch algorithm

4 years agoadding enableDedicatedDictSearch to matchState_t
Bimba Shrestha [Fri, 12 Jun 2020 01:34:14 +0000 (18:34 -0700)] 
adding enableDedicatedDictSearch to matchState_t

4 years agoadding dedicated dict load method to lazy
Bimba Shrestha [Fri, 12 Jun 2020 01:27:07 +0000 (18:27 -0700)] 
adding dedicated dict load method to lazy

4 years agoadding ZSTD_createCDict_advanced2 to zstd.h
Bimba Shrestha [Fri, 12 Jun 2020 01:12:48 +0000 (18:12 -0700)] 
adding ZSTD_createCDict_advanced2 to zstd.h

4 years agoalways attach dict when using dedicatedDictSearch
Bimba Shrestha [Thu, 11 Jun 2020 21:19:12 +0000 (14:19 -0700)] 
always attach dict when using dedicatedDictSearch

4 years agoadding dedicatedDictSearch cParams helper methods
Bimba Shrestha [Thu, 11 Jun 2020 20:48:57 +0000 (13:48 -0700)] 
adding dedicatedDictSearch cParams helper methods

4 years agoadding ZSTD_dedicatedDictSearch_defaultCParameters variable
Bimba Shrestha [Thu, 11 Jun 2020 20:33:52 +0000 (13:33 -0700)] 
adding ZSTD_dedicatedDictSearch_defaultCParameters variable

4 years agoAdd ZSTD_c_enableDedicatedDictSearch Param
Bimba Shrestha [Thu, 11 Jun 2020 20:21:28 +0000 (13:21 -0700)] 
Add ZSTD_c_enableDedicatedDictSearch Param

4 years agoMerge pull request #2301 from terrelln/prepare-for-release
Nick Terrell [Thu, 10 Sep 2020 22:42:16 +0000 (15:42 -0700)] 
Merge pull request #2301 from terrelln/prepare-for-release

Prepare library for a release

4 years ago[lib] Bump version number to 1.4.6 2301/head
Nick Terrell [Thu, 10 Sep 2020 00:13:45 +0000 (17:13 -0700)] 
[lib] Bump version number to 1.4.6

4 years agoUpdate CHANGELOG for the next release
Nick Terrell [Thu, 10 Sep 2020 00:13:34 +0000 (17:13 -0700)] 
Update CHANGELOG for the next release

4 years ago[doc] Document new build macros in lib/README.md
Nick Terrell [Thu, 10 Sep 2020 00:13:16 +0000 (17:13 -0700)] 
[doc] Document new build macros in lib/README.md

4 years agoMerge pull request #2289 from terrelln/zstd-kernel-2
Nick Terrell [Wed, 9 Sep 2020 22:38:02 +0000 (15:38 -0700)] 
Merge pull request #2289 from terrelln/zstd-kernel-2

[linux-kernel] Prepare zstd for the Linux Kernel

4 years ago[linux-kernel] Update README with initial instructions on how to update zstd in the... 2289/head
Nick Terrell [Wed, 9 Sep 2020 19:45:16 +0000 (12:45 -0700)] 
[linux-kernel] Update README with initial instructions on how to update zstd in the kernel

4 years ago[linux-kernel] Add test that checks the ifdef hardwiring
Nick Terrell [Wed, 9 Sep 2020 19:36:58 +0000 (12:36 -0700)] 
[linux-kernel] Add test that checks the ifdef hardwiring

4 years agoAdd linux-kernel freestanding
Nick Terrell [Tue, 11 Aug 2020 06:11:20 +0000 (23:11 -0700)] 
Add linux-kernel freestanding

4 years ago[contrib] Add preprocessor hardwiring to freestanding.py
Nick Terrell [Tue, 11 Aug 2020 06:09:59 +0000 (23:09 -0700)] 
[contrib] Add preprocessor hardwiring to freestanding.py

4 years ago[lib] Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions
Nick Terrell [Fri, 28 Aug 2020 00:10:04 +0000 (17:10 -0700)] 
[lib] Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions

The unused function definitions are hidden behind a
`#ifndef ZSTD_NO_UNUSED_FUNCTIONS` check.

Initially hiding all functions which are unused and take up more than
2KB of stack space, because these will show up as warnings in the
Linux Kernel build system.

4 years ago[lib] Replace 64-bit divisions with ZSTD_div64()
Nick Terrell [Thu, 27 Aug 2020 22:01:18 +0000 (15:01 -0700)] 
[lib] Replace 64-bit divisions with ZSTD_div64()

4 years ago[lib] Reduce zstd stack usage by 1KB
Nick Terrell [Thu, 27 Aug 2020 00:52:54 +0000 (17:52 -0700)] 
[lib] Reduce zstd stack usage by 1KB

4 years agoFix ZSTD_initCStream_advanced() with no dictionary and static allocation
Nick Terrell [Fri, 14 Aug 2020 19:42:06 +0000 (12:42 -0700)] 
Fix ZSTD_initCStream_advanced() with no dictionary and static allocation

4 years agoAdd ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics
Nick Terrell [Fri, 14 Aug 2020 19:41:28 +0000 (12:41 -0700)] 
Add ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics

4 years ago[lib] s/current/curr because it collides with Linux Kernel macro
Nick Terrell [Tue, 11 Aug 2020 21:31:09 +0000 (14:31 -0700)] 
[lib] s/current/curr because it collides with Linux Kernel macro

4 years agoMerge pull request #2291 from i-do-cpp/fix-compression-level-default
Nick Terrell [Tue, 8 Sep 2020 23:42:34 +0000 (16:42 -0700)] 
Merge pull request #2291 from i-do-cpp/fix-compression-level-default

Fix setParameter not falling back to default compression level

4 years agoMerge pull request #2294 from felixhandte/makefile-lib-fix-var-order
Felix Handte [Fri, 4 Sep 2020 14:58:57 +0000 (10:58 -0400)] 
Merge pull request #2294 from felixhandte/makefile-lib-fix-var-order

Fix Makefile Variable Concatenation Order

4 years agoFix Makefile Variable Concatenation Order 2294/head
W. Felix Handte [Thu, 3 Sep 2020 21:28:37 +0000 (17:28 -0400)] 
Fix Makefile Variable Concatenation Order

Previously, this construct would add `-O3` onto the end of the compiler flags
variable, **after** `MOREFLAGS`, which meant that it was impossible to over-
ride. This commit fixes this order and should otherwise be a no-op.

4 years agoMerge pull request #2293 from allanjude/coverity
Nick Terrell [Thu, 3 Sep 2020 20:58:45 +0000 (13:58 -0700)] 
Merge pull request #2293 from allanjude/coverity

Resolve Coverity 1432392 Unintentional integer overflow

4 years agoResolve Coverity 1432392 Unintentional integer overflow 2293/head
Allan Jude [Thu, 3 Sep 2020 19:31:50 +0000 (19:31 +0000)] 
Resolve Coverity 1432392 Unintentional integer overflow

Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
overflow_before_widen: Potentially overflowing expression:
cdict->dictContentSize * 6U
with type unsigned int (32 bits, unsigned) is evaluated using 32-bit
arithmetic, and then used in a context that expects an expression of
type U64 (64 bits, unsigned).

4 years agoMerge pull request #2279 from senhuang42/show_warnings_with_multiple_input_files
Nick Terrell [Wed, 2 Sep 2020 18:32:11 +0000 (11:32 -0700)] 
Merge pull request #2279 from senhuang42/show_warnings_with_multiple_input_files

Add a warning whenever (de)compressing multiple files

4 years agoAdjusted extra explanation warning to always pop up if applicable 2279/head
senhuang42 [Wed, 2 Sep 2020 12:44:42 +0000 (08:44 -0400)] 
Adjusted extra explanation warning to always pop up if applicable

4 years agoFixed newline issue and adjusted wording in comment
senhuang42 [Tue, 1 Sep 2020 17:18:30 +0000 (13:18 -0400)] 
Fixed newline issue and adjusted wording in comment

4 years agoUpdate zstd_compress.c 2291/head
i-do-cpp [Mon, 31 Aug 2020 07:34:08 +0000 (09:34 +0200)] 
Update zstd_compress.c

4 years agoFix setParameter not falling back to default compression level on 0 value
i-do-cpp [Mon, 31 Aug 2020 07:25:43 +0000 (09:25 +0200)] 
Fix setParameter not falling back to default compression level on 0 value

See documentation for `ZSTD_c_compressionLevel`: `Special: value 0 means default, which is controlled by ZSTD_CLEVEL_DEFAULT`

4 years agoImprove documentation, update man
senhuang42 [Thu, 27 Aug 2020 21:57:20 +0000 (17:57 -0400)] 
Improve documentation, update man

4 years agoMerge pull request #2288 from animalize/doc_version
Yann Collet [Thu, 27 Aug 2020 19:32:07 +0000 (12:32 -0700)] 
Merge pull request #2288 from animalize/doc_version

[doc] Add ZSTD_versionString() to manual

4 years agoAdd ZSTD_versionString() function to manual. 2288/head
animalize [Thu, 27 Aug 2020 05:51:22 +0000 (13:51 +0800)] 
Add ZSTD_versionString() function to manual.

4 years agoMerge pull request #2264 from terrelln/zstd-kernel
Nick Terrell [Thu, 27 Aug 2020 01:15:25 +0000 (18:15 -0700)] 
Merge pull request #2264 from terrelln/zstd-kernel

Prepare for freestanding library and add initial translator script

4 years agoFix potential memory leak
senhuang42 [Wed, 26 Aug 2020 22:52:32 +0000 (18:52 -0400)] 
Fix potential memory leak

4 years agoFix UTIL_requireUserConfirmation() declaration
senhuang42 [Wed, 26 Aug 2020 21:20:37 +0000 (17:20 -0400)] 
Fix UTIL_requireUserConfirmation() declaration

4 years agoMove logic into new function FIO_removeMultiFilesWarning, add support for decompression
senhuang42 [Wed, 26 Aug 2020 20:50:20 +0000 (16:50 -0400)] 
Move logic into new function FIO_removeMultiFilesWarning, add support for decompression

4 years agoFix merge conflicts
senhuang42 [Wed, 26 Aug 2020 19:43:38 +0000 (15:43 -0400)] 
Fix merge conflicts

4 years ago[contrib] Add freestanding translator prototype 2264/head
Nick Terrell [Mon, 10 Aug 2020 17:42:27 +0000 (10:42 -0700)] 
[contrib] Add freestanding translator prototype

This is the idea, some of the functionality isn't yet implemented.

4 years agoZSTD_ prefix mem{cpy,move,set},malloc,calloc,free
Nick Terrell [Mon, 10 Aug 2020 19:46:38 +0000 (12:46 -0700)] 
ZSTD_ prefix mem{cpy,move,set},malloc,calloc,free

4 years agoRename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free}
Nick Terrell [Mon, 10 Aug 2020 19:42:03 +0000 (12:42 -0700)] 
Rename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free}

4 years agoMove standard includes to zstd_deps.h
Nick Terrell [Fri, 7 Aug 2020 03:18:05 +0000 (20:18 -0700)] 
Move standard includes to zstd_deps.h

4 years agoMerge pull request #2283 from senhuang42/progress_bars_for_multiple_files
Yann Collet [Wed, 26 Aug 2020 18:54:50 +0000 (11:54 -0700)] 
Merge pull request #2283 from senhuang42/progress_bars_for_multiple_files

Refreshing progress bar for processing multiple files

4 years agoMerge pull request #2282 from terrelln/ncount-fix
Nick Terrell [Wed, 26 Aug 2020 17:31:07 +0000 (10:31 -0700)] 
Merge pull request #2282 from terrelln/ncount-fix

[bug] Fix FSE_readNCount()

4 years agoRemove extraneous FIO_setNbFiles() 2283/head
senhuang42 [Wed, 26 Aug 2020 16:05:04 +0000 (12:05 -0400)] 
Remove extraneous FIO_setNbFiles()

4 years agoFix bug in user prompt where line was not flushed on negative response
senhuang42 [Wed, 26 Aug 2020 15:57:38 +0000 (11:57 -0400)] 
Fix bug in user prompt where line was not flushed on negative response

4 years agoAdjust playTests.sh refuse overwrite test to include -q
senhuang42 [Wed, 26 Aug 2020 15:40:05 +0000 (11:40 -0400)] 
Adjust playTests.sh refuse overwrite test to include -q

4 years agoFix single file decompression summary, remove escape code dependency, add currFileIdx...
senhuang42 [Wed, 26 Aug 2020 15:35:07 +0000 (11:35 -0400)] 
Fix single file decompression summary, remove escape code dependency, add currFileIdx field to FIO_prefs, general cleanups/bugfixes

4 years agoMerge pull request #2285 from terrelln/oss-fuzz-fixes
Nick Terrell [Wed, 26 Aug 2020 01:35:16 +0000 (18:35 -0700)] 
Merge pull request #2285 from terrelln/oss-fuzz-fixes

[fuzz] Fix oss-fuzz test problems

4 years agoMerge pull request #2284 from terrelln/github-actions-arm
Yann Collet [Wed, 26 Aug 2020 00:50:12 +0000 (17:50 -0700)] 
Merge pull request #2284 from terrelln/github-actions-arm

[github actions] Disable armbuild and armfuzz tests

4 years ago[fuzz] Fix stream_decompress timeouts 2285/head
Nick Terrell [Wed, 26 Aug 2020 00:10:04 +0000 (17:10 -0700)] 
[fuzz] Fix stream_decompress timeouts

4 years ago[fuzz] Disable superblock expansion test
Nick Terrell [Tue, 25 Aug 2020 23:32:00 +0000 (16:32 -0700)] 
[fuzz] Disable superblock expansion test

4 years ago[github actions] Disable armbuild and armfuzz tests 2284/head
Nick Terrell [Tue, 25 Aug 2020 23:28:41 +0000 (16:28 -0700)] 
[github actions] Disable armbuild and armfuzz tests

4 years ago[bug] Fix FSE_readNCount() 2282/head
Nick Terrell [Tue, 25 Aug 2020 18:37:41 +0000 (11:37 -0700)] 
[bug] Fix FSE_readNCount()

* Fix bug introduced in PR #2271
* Fix long-standing bug that is impossible to trigger inside of zstd
* Add a fuzzer that makes sure the normalized count always round trips
  correctly

4 years agoSmall formatting changes
senhuang42 [Tue, 25 Aug 2020 22:03:08 +0000 (18:03 -0400)] 
Small formatting changes

4 years agoRemove decompression status update
senhuang42 [Tue, 25 Aug 2020 21:54:00 +0000 (17:54 -0400)] 
Remove decompression status update

4 years agoSmall cleanups
senhuang42 [Tue, 25 Aug 2020 21:34:00 +0000 (17:34 -0400)] 
Small cleanups

4 years agoMake compatible with displaylevel = 2 and decompression
senhuang42 [Tue, 25 Aug 2020 21:23:47 +0000 (17:23 -0400)] 
Make compatible with displaylevel = 2 and decompression

4 years agoAdd initial live update with displayLevel = 2, add new field to FIO_prefs_t
senhuang42 [Tue, 25 Aug 2020 20:46:47 +0000 (16:46 -0400)] 
Add initial live update with displayLevel = 2, add new field to FIO_prefs_t

4 years agoMerge pull request #2278 from senhuang42/ignore_checksum_advanced_param
Yann Collet [Tue, 25 Aug 2020 19:08:53 +0000 (12:08 -0700)] 
Merge pull request #2278 from senhuang42/ignore_checksum_advanced_param

New advanced decompression param to ignore checksums

4 years agoAdd initial functionality to support -q
senhuang42 [Tue, 25 Aug 2020 17:50:44 +0000 (13:50 -0400)] 
Add initial functionality to support -q

4 years agoUnified warning prompts into new function UTIL_requireUserConfirmationToProceed()
senhuang42 [Tue, 25 Aug 2020 15:25:49 +0000 (11:25 -0400)] 
Unified warning prompts into new function UTIL_requireUserConfirmationToProceed()

4 years agoMerge pull request #2271 from terrelln/small-blocks
Nick Terrell [Tue, 25 Aug 2020 01:54:33 +0000 (18:54 -0700)] 
Merge pull request #2271 from terrelln/small-blocks

Small block optimizations

4 years agoOnly ask to proceed if using --rm, otherwise just display warning. -f bypasses it...
senhuang42 [Tue, 25 Aug 2020 00:19:55 +0000 (20:19 -0400)] 
Only ask to proceed if using --rm, otherwise just display warning. -f bypasses it all. More robust tests

4 years agoAdd a warning whenever (de)compressing multiple files into one source, or into stdout
senhuang42 [Mon, 24 Aug 2020 23:10:03 +0000 (19:10 -0400)] 
Add a warning whenever (de)compressing multiple files into one source, or into stdout

4 years agoFix compiler warnings 2271/head
Nick Terrell [Mon, 24 Aug 2020 23:09:45 +0000 (16:09 -0700)] 
Fix compiler warnings

4 years agoMerge pull request #2272 from terrelln/dstSize_tooSmall
Nick Terrell [Mon, 24 Aug 2020 22:01:17 +0000 (15:01 -0700)] 
Merge pull request #2272 from terrelln/dstSize_tooSmall

[fix] Always return dstSize_tooSmall when it is the case

4 years agoDocument the BMI2 default() functions
Nick Terrell [Mon, 24 Aug 2020 21:44:33 +0000 (14:44 -0700)] 
Document the BMI2 default() functions

4 years agoAdjust #ifndef logic 2278/head
senhuang42 [Mon, 24 Aug 2020 21:36:36 +0000 (17:36 -0400)] 
Adjust #ifndef logic

4 years agoAdd new DCtx param: validateChecksum and update unit tests
senhuang42 [Mon, 24 Aug 2020 21:28:00 +0000 (17:28 -0400)] 
Add new DCtx param: validateChecksum and update unit tests