]>
git.ipfire.org Git - thirdparty/zstd.git/log
W. Felix Handte [Fri, 4 Sep 2020 16:16:35 +0000 (12:16 -0400)]
Fix Bug: Only Use DDSS Insertion on CDict MatchStates
Previously, if DDSS was enabled on a CCtx and a dictionary was inserted into
the CCtx, the CCtx MatchState would be filled as a DDSS struct, causing
segfaults etc. This changes the check to use whether the MatchState is marked
as using the DDSS (which is only ever set for CDict MatchStates), rather than
looking at the CCtxParams.
W. Felix Handte [Wed, 2 Sep 2020 17:27:11 +0000 (13:27 -0400)]
Shorten Dict Mode Conditionals in Order to Improve Readability
W. Felix Handte [Wed, 2 Sep 2020 16:40:42 +0000 (12:40 -0400)]
Force-Inline ZSTD_insertAndFindFirstIndex_internal()
Without this, gcc was declining to inline the function in `ZSTD_noDict` mode,
resulting in a ~10% slowdown.
W. Felix Handte [Wed, 2 Sep 2020 16:40:25 +0000 (12:40 -0400)]
Clean Up Search Function Selection
W. Felix Handte [Fri, 28 Aug 2020 18:14:29 +0000 (14:14 -0400)]
Fix `-Wshorten-64-to-32` Error
W. Felix Handte [Fri, 28 Aug 2020 16:38:50 +0000 (12:38 -0400)]
Adjust Working Context Table Sizes Back Down
W. Felix Handte [Fri, 28 Aug 2020 16:31:01 +0000 (12:31 -0400)]
Add Warning Comment to ZSTD_createCDict_advanced2() Declaration
W. Felix Handte [Wed, 26 Aug 2020 22:33:44 +0000 (18:33 -0400)]
Permit Matching Against Lowest Valid Position
This comparison was previously faulty: the lowest valid position is itself
valid, and we should therefore be allowed to match against it.
W. Felix Handte [Wed, 26 Aug 2020 20:33:13 +0000 (16:33 -0400)]
Make ZSTD_dedicatedDictSearch_getCParams Wrap ZSTD_getCParams
Fixes up bounds-checking, and lets us clean up what is at the moment an
unnecessary duplication of the default cparams tables.
W. Felix Handte [Fri, 21 Aug 2020 21:00:26 +0000 (17:00 -0400)]
Easy: Fix Test
W. Felix Handte [Thu, 20 Aug 2020 21:17:10 +0000 (17:17 -0400)]
Remove Chain Limit on Hash Cache Entries; Slightly Improve Compression
Entries in the hashTable chain cache aren't subject to the same aliasing that
the circular chain table is subject to. As such, we don't need to stop when we
cross the chain limit. We can delve deeper. :)
W. Felix Handte [Thu, 20 Aug 2020 16:31:25 +0000 (12:31 -0400)]
Split Lookups in Hash Cache and Chain Table into Two Loops
Sliiiight speedup.
W. Felix Handte [Thu, 20 Aug 2020 15:40:47 +0000 (11:40 -0400)]
Simplify DDS Hash Table Construction
No need to walk the chainTable; we can just keep shifting the entries in the
hashTable.
W. Felix Handte [Tue, 18 Aug 2020 20:58:11 +0000 (16:58 -0400)]
Document the ZSTD_c_enableDedicatedDictSearch Parameter
W. Felix Handte [Tue, 18 Aug 2020 19:20:12 +0000 (15:20 -0400)]
Rename and Move DD_BLOG Constant to ZSTD_LAZY_DDSS_BUCKET_LOG
W. Felix Handte [Tue, 18 Aug 2020 17:30:41 +0000 (13:30 -0400)]
Prefetch First Batch of Match Positions; +11% Speed in Level 5 w/ 1 Dict
W. Felix Handte [Mon, 17 Aug 2020 21:43:01 +0000 (17:43 -0400)]
Fix Off-By-One Error in Counting DDS Search Attempts
This caused us to double-search the first position and fail to search the
last position in the chain, slowing down search and making it less effective.
W. Felix Handte [Mon, 17 Aug 2020 16:37:58 +0000 (12:37 -0400)]
Enable Dedicated Dict Search in the CLI
W. Felix Handte [Mon, 17 Aug 2020 16:35:50 +0000 (12:35 -0400)]
Init CCtx's Local Dict with CCtxParams
W. Felix Handte [Thu, 13 Aug 2020 20:52:07 +0000 (16:52 -0400)]
Decision for Whether to Attach Should be Based on CDict Config, not CCtx
W. Felix Handte [Thu, 13 Aug 2020 18:54:10 +0000 (14:54 -0400)]
Move Prefetch Before Insertion; Speed Up ~6%
W. Felix Handte [Thu, 13 Aug 2020 15:57:31 +0000 (11:57 -0400)]
Misc Refactor of DDS Search Code
W. Felix Handte [Wed, 12 Aug 2020 20:50:44 +0000 (16:50 -0400)]
Rename enableDedicatedDictSearch to dedicatedDictSearch in MatchState
This makes it clear that not only is the feature allowed here, we're actually
using it, as opposed to the CCtxParam field, in which it's enabled, but we may
or may not be using it.
W. Felix Handte [Tue, 11 Aug 2020 22:57:12 +0000 (18:57 -0400)]
Always Init CDict's enableDedicatedDictSearch Field
W. Felix Handte [Tue, 11 Aug 2020 22:48:22 +0000 (18:48 -0400)]
Add a ZSTD_dedicatedDictSearch ZSTD_dictMode_e to Allow Const Propagation
Speed +1.5%.
W. Felix Handte [Tue, 11 Aug 2020 22:24:11 +0000 (18:24 -0400)]
Fix ZSTD_c_forceAttachDict Bounds
W. Felix Handte [Tue, 11 Aug 2020 22:03:38 +0000 (18:03 -0400)]
Make ZSTD_c_enableDedicatedDictSearch an Experimental Param
W. Felix Handte [Tue, 11 Aug 2020 19:29:12 +0000 (15:29 -0400)]
Add Fuzzer Test for the Various Dict Attachment Strategies
W. Felix Handte [Fri, 7 Aug 2020 20:41:16 +0000 (16:41 -0400)]
TODO: Comment
W. Felix Handte [Fri, 7 Aug 2020 16:02:18 +0000 (12:02 -0400)]
Fix Compilation Warnings
Bimba Shrestha [Fri, 12 Jun 2020 15:53:58 +0000 (08:53 -0700)]
updating cold benchmark
Bimba Shrestha [Fri, 12 Jun 2020 15:01:18 +0000 (08:01 -0700)]
creating ZSTD_createCDict_advanced_internal
Bimba Shrestha [Fri, 12 Jun 2020 05:57:53 +0000 (22:57 -0700)]
changing to int for consistency
Bimba Shrestha [Fri, 12 Jun 2020 05:55:32 +0000 (22:55 -0700)]
using dict mls instead of src mls
Bimba Shrestha [Fri, 12 Jun 2020 05:54:36 +0000 (22:54 -0700)]
add hashtable head prefetching
Bimba Shrestha [Fri, 12 Jun 2020 05:48:48 +0000 (22:48 -0700)]
change method name for consistency
Bimba Shrestha [Fri, 12 Jun 2020 05:46:16 +0000 (22:46 -0700)]
pass correct cparams
Bimba Shrestha [Fri, 12 Jun 2020 05:42:57 +0000 (22:42 -0700)]
removing wrong comment
Bimba Shrestha [Fri, 12 Jun 2020 05:42:11 +0000 (22:42 -0700)]
making cctxParams a pointer
Bimba Shrestha [Fri, 12 Jun 2020 02:41:03 +0000 (19:41 -0700)]
loading dict using new algorithm
Bimba Shrestha [Fri, 12 Jun 2020 01:54:44 +0000 (18:54 -0700)]
adding dedicatedDictSearch algorithm
Bimba Shrestha [Fri, 12 Jun 2020 01:34:14 +0000 (18:34 -0700)]
adding enableDedicatedDictSearch to matchState_t
Bimba Shrestha [Fri, 12 Jun 2020 01:27:07 +0000 (18:27 -0700)]
adding dedicated dict load method to lazy
Bimba Shrestha [Fri, 12 Jun 2020 01:12:48 +0000 (18:12 -0700)]
adding ZSTD_createCDict_advanced2 to zstd.h
Bimba Shrestha [Thu, 11 Jun 2020 21:19:12 +0000 (14:19 -0700)]
always attach dict when using dedicatedDictSearch
Bimba Shrestha [Thu, 11 Jun 2020 20:48:57 +0000 (13:48 -0700)]
adding dedicatedDictSearch cParams helper methods
Bimba Shrestha [Thu, 11 Jun 2020 20:33:52 +0000 (13:33 -0700)]
adding ZSTD_dedicatedDictSearch_defaultCParameters variable
Bimba Shrestha [Thu, 11 Jun 2020 20:21:28 +0000 (13:21 -0700)]
Add ZSTD_c_enableDedicatedDictSearch Param
Nick Terrell [Thu, 10 Sep 2020 22:42:16 +0000 (15:42 -0700)]
Merge pull request #2301 from terrelln/prepare-for-release
Prepare library for a release
Nick Terrell [Thu, 10 Sep 2020 00:13:45 +0000 (17:13 -0700)]
[lib] Bump version number to 1.4.6
Nick Terrell [Thu, 10 Sep 2020 00:13:34 +0000 (17:13 -0700)]
Update CHANGELOG for the next release
Nick Terrell [Thu, 10 Sep 2020 00:13:16 +0000 (17:13 -0700)]
[doc] Document new build macros in lib/README.md
Nick Terrell [Wed, 9 Sep 2020 22:38:02 +0000 (15:38 -0700)]
Merge pull request #2289 from terrelln/zstd-kernel-2
[linux-kernel] Prepare zstd for the Linux Kernel
Nick Terrell [Wed, 9 Sep 2020 19:45:16 +0000 (12:45 -0700)]
[linux-kernel] Update README with initial instructions on how to update zstd in the kernel
Nick Terrell [Wed, 9 Sep 2020 19:36:58 +0000 (12:36 -0700)]
[linux-kernel] Add test that checks the ifdef hardwiring
Nick Terrell [Tue, 11 Aug 2020 06:11:20 +0000 (23:11 -0700)]
Add linux-kernel freestanding
Nick Terrell [Tue, 11 Aug 2020 06:09:59 +0000 (23:09 -0700)]
[contrib] Add preprocessor hardwiring to freestanding.py
Nick Terrell [Fri, 28 Aug 2020 00:10:04 +0000 (17:10 -0700)]
[lib] Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions
The unused function definitions are hidden behind a
`#ifndef ZSTD_NO_UNUSED_FUNCTIONS` check.
Initially hiding all functions which are unused and take up more than
2KB of stack space, because these will show up as warnings in the
Linux Kernel build system.
Nick Terrell [Thu, 27 Aug 2020 22:01:18 +0000 (15:01 -0700)]
[lib] Replace 64-bit divisions with ZSTD_div64()
Nick Terrell [Thu, 27 Aug 2020 00:52:54 +0000 (17:52 -0700)]
[lib] Reduce zstd stack usage by 1KB
Nick Terrell [Fri, 14 Aug 2020 19:42:06 +0000 (12:42 -0700)]
Fix ZSTD_initCStream_advanced() with no dictionary and static allocation
Nick Terrell [Fri, 14 Aug 2020 19:41:28 +0000 (12:41 -0700)]
Add ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics
Nick Terrell [Tue, 11 Aug 2020 21:31:09 +0000 (14:31 -0700)]
[lib] s/current/curr because it collides with Linux Kernel macro
Nick Terrell [Tue, 8 Sep 2020 23:42:34 +0000 (16:42 -0700)]
Merge pull request #2291 from i-do-cpp/fix-compression-level-default
Fix setParameter not falling back to default compression level
Felix Handte [Fri, 4 Sep 2020 14:58:57 +0000 (10:58 -0400)]
Merge pull request #2294 from felixhandte/makefile-lib-fix-var-order
Fix Makefile Variable Concatenation Order
W. Felix Handte [Thu, 3 Sep 2020 21:28:37 +0000 (17:28 -0400)]
Fix Makefile Variable Concatenation Order
Previously, this construct would add `-O3` onto the end of the compiler flags
variable, **after** `MOREFLAGS`, which meant that it was impossible to over-
ride. This commit fixes this order and should otherwise be a no-op.
Nick Terrell [Thu, 3 Sep 2020 20:58:45 +0000 (13:58 -0700)]
Merge pull request #2293 from allanjude/coverity
Resolve Coverity
1432392 Unintentional integer overflow
Allan Jude [Thu, 3 Sep 2020 19:31:50 +0000 (19:31 +0000)]
Resolve Coverity
1432392 Unintentional integer overflow
Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
overflow_before_widen: Potentially overflowing expression:
cdict->dictContentSize * 6U
with type unsigned int (32 bits, unsigned) is evaluated using 32-bit
arithmetic, and then used in a context that expects an expression of
type U64 (64 bits, unsigned).
Nick Terrell [Wed, 2 Sep 2020 18:32:11 +0000 (11:32 -0700)]
Merge pull request #2279 from senhuang42/show_warnings_with_multiple_input_files
Add a warning whenever (de)compressing multiple files
senhuang42 [Wed, 2 Sep 2020 12:44:42 +0000 (08:44 -0400)]
Adjusted extra explanation warning to always pop up if applicable
senhuang42 [Tue, 1 Sep 2020 17:18:30 +0000 (13:18 -0400)]
Fixed newline issue and adjusted wording in comment
i-do-cpp [Mon, 31 Aug 2020 07:34:08 +0000 (09:34 +0200)]
Update zstd_compress.c
i-do-cpp [Mon, 31 Aug 2020 07:25:43 +0000 (09:25 +0200)]
Fix setParameter not falling back to default compression level on 0 value
See documentation for `ZSTD_c_compressionLevel`: `Special: value 0 means default, which is controlled by ZSTD_CLEVEL_DEFAULT`
senhuang42 [Thu, 27 Aug 2020 21:57:20 +0000 (17:57 -0400)]
Improve documentation, update man
Yann Collet [Thu, 27 Aug 2020 19:32:07 +0000 (12:32 -0700)]
Merge pull request #2288 from animalize/doc_version
[doc] Add ZSTD_versionString() to manual
animalize [Thu, 27 Aug 2020 05:51:22 +0000 (13:51 +0800)]
Add ZSTD_versionString() function to manual.
Nick Terrell [Thu, 27 Aug 2020 01:15:25 +0000 (18:15 -0700)]
Merge pull request #2264 from terrelln/zstd-kernel
Prepare for freestanding library and add initial translator script
senhuang42 [Wed, 26 Aug 2020 22:52:32 +0000 (18:52 -0400)]
Fix potential memory leak
senhuang42 [Wed, 26 Aug 2020 21:20:37 +0000 (17:20 -0400)]
Fix UTIL_requireUserConfirmation() declaration
senhuang42 [Wed, 26 Aug 2020 20:50:20 +0000 (16:50 -0400)]
Move logic into new function FIO_removeMultiFilesWarning, add support for decompression
senhuang42 [Wed, 26 Aug 2020 19:43:38 +0000 (15:43 -0400)]
Fix merge conflicts
Nick Terrell [Mon, 10 Aug 2020 17:42:27 +0000 (10:42 -0700)]
[contrib] Add freestanding translator prototype
This is the idea, some of the functionality isn't yet implemented.
Nick Terrell [Mon, 10 Aug 2020 19:46:38 +0000 (12:46 -0700)]
ZSTD_ prefix mem{cpy,move,set},malloc,calloc,free
Nick Terrell [Mon, 10 Aug 2020 19:42:03 +0000 (12:42 -0700)]
Rename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free}
Nick Terrell [Fri, 7 Aug 2020 03:18:05 +0000 (20:18 -0700)]
Move standard includes to zstd_deps.h
Yann Collet [Wed, 26 Aug 2020 18:54:50 +0000 (11:54 -0700)]
Merge pull request #2283 from senhuang42/progress_bars_for_multiple_files
Refreshing progress bar for processing multiple files
Nick Terrell [Wed, 26 Aug 2020 17:31:07 +0000 (10:31 -0700)]
Merge pull request #2282 from terrelln/ncount-fix
[bug] Fix FSE_readNCount()
senhuang42 [Wed, 26 Aug 2020 16:05:04 +0000 (12:05 -0400)]
Remove extraneous FIO_setNbFiles()
senhuang42 [Wed, 26 Aug 2020 15:57:38 +0000 (11:57 -0400)]
Fix bug in user prompt where line was not flushed on negative response
senhuang42 [Wed, 26 Aug 2020 15:40:05 +0000 (11:40 -0400)]
Adjust playTests.sh refuse overwrite test to include -q
senhuang42 [Wed, 26 Aug 2020 15:35:07 +0000 (11:35 -0400)]
Fix single file decompression summary, remove escape code dependency, add currFileIdx field to FIO_prefs, general cleanups/bugfixes
Nick Terrell [Wed, 26 Aug 2020 01:35:16 +0000 (18:35 -0700)]
Merge pull request #2285 from terrelln/oss-fuzz-fixes
[fuzz] Fix oss-fuzz test problems
Yann Collet [Wed, 26 Aug 2020 00:50:12 +0000 (17:50 -0700)]
Merge pull request #2284 from terrelln/github-actions-arm
[github actions] Disable armbuild and armfuzz tests
Nick Terrell [Wed, 26 Aug 2020 00:10:04 +0000 (17:10 -0700)]
[fuzz] Fix stream_decompress timeouts
Nick Terrell [Tue, 25 Aug 2020 23:32:00 +0000 (16:32 -0700)]
[fuzz] Disable superblock expansion test
Nick Terrell [Tue, 25 Aug 2020 23:28:41 +0000 (16:28 -0700)]
[github actions] Disable armbuild and armfuzz tests
Nick Terrell [Tue, 25 Aug 2020 18:37:41 +0000 (11:37 -0700)]
[bug] Fix FSE_readNCount()
* Fix bug introduced in PR #2271
* Fix long-standing bug that is impossible to trigger inside of zstd
* Add a fuzzer that makes sure the normalized count always round trips
correctly
senhuang42 [Tue, 25 Aug 2020 22:03:08 +0000 (18:03 -0400)]
Small formatting changes
senhuang42 [Tue, 25 Aug 2020 21:54:00 +0000 (17:54 -0400)]
Remove decompression status update
senhuang42 [Tue, 25 Aug 2020 21:34:00 +0000 (17:34 -0400)]
Small cleanups