]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
4 years agoUse kNullRawSeqStore constant in zstdmt_compress.c
senhuang42 [Mon, 5 Oct 2020 14:50:49 +0000 (10:50 -0400)] 
Use kNullRawSeqStore constant in zstdmt_compress.c

4 years agoRemove bubbling down matches with longer offCode and same matchLen
senhuang42 [Mon, 5 Oct 2020 01:48:39 +0000 (21:48 -0400)] 
Remove bubbling down matches with longer offCode and same matchLen

4 years agoDisable LDM minMatch adjustment when using opt parser
senhuang42 [Mon, 5 Oct 2020 01:47:25 +0000 (21:47 -0400)] 
Disable LDM minMatch adjustment when using opt parser

4 years agoEnable inclusion of mid-flight LDMs in opt parser
senhuang42 [Mon, 5 Oct 2020 01:46:47 +0000 (21:46 -0400)] 
Enable inclusion of mid-flight LDMs in opt parser

4 years agoCorrect incorrect offcode calculation
senhuang42 [Fri, 2 Oct 2020 23:08:10 +0000 (19:08 -0400)] 
Correct incorrect offcode calculation

4 years agoFix static analyze fuzzer.c error
senhuang42 [Fri, 2 Oct 2020 19:27:06 +0000 (15:27 -0400)] 
Fix static analyze fuzzer.c error

4 years agoAdd explicit conversion of size_t to U32
senhuang42 [Fri, 2 Oct 2020 17:47:54 +0000 (13:47 -0400)] 
Add explicit conversion of size_t to U32

4 years agoAdd cSize regression test to fuzzer.c
senhuang42 [Fri, 2 Oct 2020 13:49:35 +0000 (09:49 -0400)] 
Add cSize regression test to fuzzer.c

4 years agoPrefix new static ldm helpers with ZSTD_opt
senhuang42 [Thu, 1 Oct 2020 20:14:41 +0000 (16:14 -0400)] 
Prefix new static ldm helpers with ZSTD_opt

4 years agoAdd DEBUGLOG() calls in ldm helpers
senhuang42 [Thu, 1 Oct 2020 16:07:18 +0000 (12:07 -0400)] 
Add DEBUGLOG() calls in ldm helpers

4 years agoMake function descriptions more accurate
senhuang42 [Thu, 1 Oct 2020 15:47:35 +0000 (11:47 -0400)] 
Make function descriptions more accurate

4 years agoImprove documentation of relevant structs
senhuang42 [Thu, 1 Oct 2020 15:40:52 +0000 (11:40 -0400)] 
Improve documentation of relevant structs

4 years agoCorrect matchLength calculation and remove unnecessary functions
senhuang42 [Thu, 1 Oct 2020 15:35:48 +0000 (11:35 -0400)] 
Correct matchLength calculation and remove unnecessary functions

4 years agoReset ldmSeqStore after initStats_ultra() pass for btultra2
senhuang42 [Thu, 1 Oct 2020 14:13:06 +0000 (10:13 -0400)] 
Reset ldmSeqStore after initStats_ultra() pass for btultra2

4 years agoRefactor existing functions to use posInSequence
senhuang42 [Thu, 1 Oct 2020 14:12:21 +0000 (10:12 -0400)] 
Refactor existing functions to use posInSequence

4 years agoAdjustments to ldm_calculateMatchRange() to calculate bounds correctly
senhuang42 [Wed, 30 Sep 2020 23:45:40 +0000 (19:45 -0400)] 
Adjustments to ldm_calculateMatchRange() to calculate bounds correctly

4 years agoAdd ldm_calculateMatchRange() function
senhuang42 [Wed, 30 Sep 2020 23:14:31 +0000 (19:14 -0400)] 
Add ldm_calculateMatchRange() function

4 years agoRemove rawSeqStore.base and add rawSeqStore.posInSequence
senhuang42 [Wed, 30 Sep 2020 21:20:47 +0000 (17:20 -0400)] 
Remove rawSeqStore.base and add rawSeqStore.posInSequence

4 years agoAdd unit tests into playTests.sh
senhuang42 [Wed, 30 Sep 2020 17:25:02 +0000 (13:25 -0400)] 
Add unit tests into playTests.sh

4 years agoPrevent duplicate LDMs from being inserted
senhuang42 [Wed, 30 Sep 2020 16:54:45 +0000 (12:54 -0400)] 
Prevent duplicate LDMs from being inserted

4 years agoAdd extra bounds check to prevent heap access after free ASAN error
senhuang42 [Wed, 30 Sep 2020 15:25:51 +0000 (11:25 -0400)] 
Add extra bounds check to prevent heap access after free ASAN error

4 years agoAddress mixed variables C90 warning
senhuang42 [Wed, 30 Sep 2020 15:06:44 +0000 (11:06 -0400)] 
Address mixed variables C90 warning

4 years agoldm_getNextMatch fixed return values
senhuang42 [Tue, 29 Sep 2020 22:12:32 +0000 (18:12 -0400)] 
ldm_getNextMatch fixed return values

4 years agoCleanups, add comments and explanations
senhuang42 [Tue, 29 Sep 2020 20:35:06 +0000 (16:35 -0400)] 
Cleanups, add comments and explanations

4 years agoFixed sifting algorithm
senhuang42 [Tue, 29 Sep 2020 19:35:13 +0000 (15:35 -0400)] 
Fixed sifting algorithm

4 years agoFixed end of match boundary update issues
senhuang42 [Tue, 29 Sep 2020 16:12:58 +0000 (12:12 -0400)] 
Fixed end of match boundary update issues

4 years agoAdd proper bounds check on adding ldms
senhuang42 [Mon, 28 Sep 2020 21:58:23 +0000 (17:58 -0400)] 
Add proper bounds check on adding ldms

4 years agoAdd a function ldm_voidSequences()
senhuang42 [Mon, 28 Sep 2020 14:29:17 +0000 (10:29 -0400)] 
Add a function ldm_voidSequences()

4 years agoFix function argument to getNextMatch()
senhuang42 [Mon, 28 Sep 2020 00:06:44 +0000 (20:06 -0400)] 
Fix function argument to getNextMatch()

4 years agoAdjustments to no longer segfault on nci
senhuang42 [Mon, 28 Sep 2020 00:00:14 +0000 (20:00 -0400)] 
Adjustments to no longer segfault on nci

4 years agoAdd base adjustment correction
senhuang42 [Sun, 27 Sep 2020 17:25:27 +0000 (13:25 -0400)] 
Add base adjustment correction

4 years agoAdd initial getNextMatch() in opt parser
senhuang42 [Sun, 27 Sep 2020 17:19:54 +0000 (13:19 -0400)] 
Add initial getNextMatch() in opt parser

4 years agoAdded more debugging
senhuang42 [Sun, 27 Sep 2020 17:02:46 +0000 (13:02 -0400)] 
Added more debugging

4 years agoAdd re-copying of ldmSeqStore after processing
senhuang42 [Sun, 27 Sep 2020 02:14:21 +0000 (22:14 -0400)] 
Add re-copying of ldmSeqStore after processing

4 years agoAdd debug statements, flesh out functions
senhuang42 [Sun, 27 Sep 2020 01:56:14 +0000 (21:56 -0400)] 
Add debug statements, flesh out functions

4 years agoGet zstd to build with new functions and callsites, fix arguments
senhuang42 [Sun, 27 Sep 2020 01:38:35 +0000 (21:38 -0400)] 
Get zstd to build with new functions and callsites, fix arguments

4 years agoImplement part of ldm_maybeAddLdm()
senhuang42 [Sat, 26 Sep 2020 16:28:10 +0000 (12:28 -0400)] 
Implement part of ldm_maybeAddLdm()

4 years agoImplement ldm_getNextMatch()
senhuang42 [Sat, 26 Sep 2020 16:16:15 +0000 (12:16 -0400)] 
Implement ldm_getNextMatch()

4 years agoImplement basic splitSequence and skipSequence functions
senhuang42 [Sat, 26 Sep 2020 16:09:19 +0000 (12:09 -0400)] 
Implement basic splitSequence and skipSequence functions

4 years agoFlesh out required args for ldm_handleLdm()
senhuang42 [Sat, 26 Sep 2020 15:59:07 +0000 (11:59 -0400)] 
Flesh out required args for ldm_handleLdm()

4 years agoAdd callsites to appropriate locations in ..opt_generic()
senhuang42 [Sat, 26 Sep 2020 15:24:29 +0000 (11:24 -0400)] 
Add callsites to appropriate locations in ..opt_generic()

4 years agoAdd ldm helper function declarations into opt parser
senhuang42 [Sat, 26 Sep 2020 15:22:50 +0000 (11:22 -0400)] 
Add ldm helper function declarations into opt parser

4 years agoModify codepath to use opt parser exclusively if the compression level is high enough
senhuang42 [Sat, 26 Sep 2020 15:13:39 +0000 (11:13 -0400)] 
Modify codepath to use opt parser exclusively if the compression level is high enough

4 years agoAdd rawSeqStore to match state
senhuang42 [Sat, 26 Sep 2020 15:07:52 +0000 (11:07 -0400)] 
Add rawSeqStore to match state

4 years agoMerge pull request #2330 from senhuang42/fix_stdinout_error_messages
Yann Collet [Tue, 6 Oct 2020 22:58:37 +0000 (15:58 -0700)] 
Merge pull request #2330 from senhuang42/fix_stdinout_error_messages

Improve error messages on console input/output

4 years agoMerge pull request #2338 from terrelln/comments
Nick Terrell [Fri, 2 Oct 2020 01:56:24 +0000 (18:56 -0700)] 
Merge pull request #2338 from terrelln/comments

Add comments to ZSTD_getLowest{Match,Prefix}Index()

4 years agoMerge pull request #2333 from terrelln/stable-dst
Nick Terrell [Fri, 2 Oct 2020 01:56:11 +0000 (18:56 -0700)] 
Merge pull request #2333 from terrelln/stable-dst

Reset all decompression parameters in ZSTD_DCtx_reset()

4 years agoReset all decompression parameters in ZSTD_DCtx_reset() 2333/head
Nick Terrell [Tue, 29 Sep 2020 23:25:03 +0000 (16:25 -0700)] 
Reset all decompression parameters in ZSTD_DCtx_reset()

* Reset all decompression parameters in `ZSTD_DCtx_reset()` when
  resetting parameters.
* Add a test case.

4 years agoAdd comments to ZSTD_getLowest{Match,Prefix}Index() 2338/head
Nick Terrell [Thu, 1 Oct 2020 20:21:46 +0000 (13:21 -0700)] 
Add comments to ZSTD_getLowest{Match,Prefix}Index()

Clarify how we handle dictionaries in each case.

4 years agoMerge pull request #2317 from animalize/msvc_inline
Yann Collet [Wed, 30 Sep 2020 15:27:53 +0000 (08:27 -0700)] 
Merge pull request #2317 from animalize/msvc_inline

Let MSVC force inline ZSTD_hashPtr() function

4 years agoMerge pull request #2321 from senhuang42/disallow_repcode_0_in_dict
Yann Collet [Wed, 30 Sep 2020 15:27:21 +0000 (08:27 -0700)] 
Merge pull request #2321 from senhuang42/disallow_repcode_0_in_dict

Update documentation about repcodes in dictionaries

4 years agoMerge pull request #2322 from senhuang42/guard_against_stdin_for_warning_prompts
Yann Collet [Wed, 30 Sep 2020 15:26:50 +0000 (08:26 -0700)] 
Merge pull request #2322 from senhuang42/guard_against_stdin_for_warning_prompts

Don't let warning messages consume input from stdin

4 years agoMerge pull request #2326 from terrelln/kernel-test-fix
Nick Terrell [Tue, 29 Sep 2020 20:51:23 +0000 (13:51 -0700)] 
Merge pull request #2326 from terrelln/kernel-test-fix

Fix issues and warnings exposed by Kernel Test Robot

4 years agoModify error messages on console input/output 2330/head
senhuang42 [Mon, 28 Sep 2020 16:15:18 +0000 (12:15 -0400)] 
Modify error messages on console input/output

4 years agoFix Stdin typo 2322/head
senhuang42 [Fri, 25 Sep 2020 15:51:35 +0000 (11:51 -0400)] 
Fix Stdin typo

4 years agoMerge pull request #2310 from senhuang42/fix_multifile_status_bar
Yann Collet [Fri, 25 Sep 2020 14:48:33 +0000 (07:48 -0700)] 
Merge pull request #2310 from senhuang42/fix_multifile_status_bar

Fix multifile status bar and summary - clear out extraneous characters

4 years ago[superblock] Reduce stack usage by correctly sizing header buffers 2326/head
Nick Terrell [Fri, 25 Sep 2020 01:04:44 +0000 (18:04 -0700)] 
[superblock] Reduce stack usage by correctly sizing header buffers

4 years ago[lib] Add ZSTD_COMPRESS_HEAPMODE tuning parameter
Nick Terrell [Fri, 25 Sep 2020 00:52:16 +0000 (17:52 -0700)] 
[lib] Add ZSTD_COMPRESS_HEAPMODE tuning parameter

4 years ago[freestanding] Improve macro resolution to handle #if X
Nick Terrell [Thu, 24 Sep 2020 23:25:58 +0000 (16:25 -0700)] 
[freestanding] Improve macro resolution to handle #if X

4 years agoAllow user to override ASAN/MSAN detection
Nick Terrell [Thu, 24 Sep 2020 23:04:21 +0000 (16:04 -0700)] 
Allow user to override ASAN/MSAN detection

Rename ADDRESS_SANITIZER -> ZSTD_ADDRESS_SANITIZER and same for
MEMORY_SANITIZER. Also set it to 0/1 instead of checking for defined.
This allows the user to override ASAN/MSAN detection for platforms that
don't support it.

4 years agoRemove call to memset
Nick Terrell [Thu, 24 Sep 2020 23:03:28 +0000 (16:03 -0700)] 
Remove call to memset

The previous commit fixes the test so it errors on calls to mem*()
functions from <string.h>.

4 years ago[linux-kernel] Avoid including <string.h> in the tests
Nick Terrell [Thu, 24 Sep 2020 22:22:34 +0000 (15:22 -0700)] 
[linux-kernel] Avoid including <string.h> in the tests

4 years agoReorganize zstd_deps.h and mem.h + replace mem.h for the kernel
Nick Terrell [Thu, 24 Sep 2020 21:22:56 +0000 (14:22 -0700)] 
Reorganize zstd_deps.h and mem.h + replace mem.h for the kernel

4 years agoMove __has_builtin() fallback define to compiler.h
Nick Terrell [Thu, 24 Sep 2020 22:45:08 +0000 (15:45 -0700)] 
Move __has_builtin() fallback define to compiler.h

4 years agoMove ASAN/MSAN support declarations to compiler.h
Nick Terrell [Thu, 24 Sep 2020 21:14:32 +0000 (14:14 -0700)] 
Move ASAN/MSAN support declarations to compiler.h

4 years agoRemove MEM_STATIC_ASSERT and use DEBUG_STATIC_ASSERT instead
Nick Terrell [Thu, 24 Sep 2020 21:13:16 +0000 (14:13 -0700)] 
Remove MEM_STATIC_ASSERT and use DEBUG_STATIC_ASSERT instead

4 years agoUpdate unit tests
senhuang42 [Thu, 24 Sep 2020 20:44:33 +0000 (16:44 -0400)] 
Update unit tests

4 years agoAdd more useful failure message when stdin is an input
senhuang42 [Thu, 24 Sep 2020 20:29:12 +0000 (16:29 -0400)] 
Add more useful failure message when stdin is an input

4 years agoExpand UTIL_requireUserConfirmation to include stdin input check
senhuang42 [Thu, 24 Sep 2020 19:58:06 +0000 (15:58 -0400)] 
Expand UTIL_requireUserConfirmation to include stdin input check

4 years agoAdd FIO_determineHasStdinInput() function and member to fCtx
senhuang42 [Thu, 24 Sep 2020 19:55:30 +0000 (15:55 -0400)] 
Add FIO_determineHasStdinInput() function and member to fCtx

4 years agoAdd fCtx to FIO_openDstFile()
senhuang42 [Thu, 24 Sep 2020 19:49:30 +0000 (15:49 -0400)] 
Add fCtx to FIO_openDstFile()

4 years agoMerge pull request #2315 from senhuang42/allow_zstd_suffix
Yann Collet [Thu, 24 Sep 2020 16:44:48 +0000 (09:44 -0700)] 
Merge pull request #2315 from senhuang42/allow_zstd_suffix

Support .zstd suffix only for decompression

4 years ago[linux-kernel] Add missing semicolon in zstd_deps.h
Nick Terrell [Thu, 24 Sep 2020 03:36:47 +0000 (20:36 -0700)] 
[linux-kernel] Add missing semicolon in zstd_deps.h

4 years ago[lib] Wrap customMem xor checks in parens for readability
Nick Terrell [Thu, 24 Sep 2020 03:34:44 +0000 (20:34 -0700)] 
[lib] Wrap customMem xor checks in parens for readability

This clarifies operator precedence, and quiets cppcheck in
the Kernel Test Robot. I think this is a slight bonus to
readability, so I am accepting the suggestion.

4 years agoMerge pull request #2324 from terrelln/kernel-test-fix
Nick Terrell [Wed, 23 Sep 2020 22:01:56 +0000 (15:01 -0700)] 
Merge pull request #2324 from terrelln/kernel-test-fix

Silence warnings reported by Kernel Test Robot

4 years ago[CI][linux-kernel] Enable -Wunused-const-variable and -Wunused-but-set-variable 2324/head
Nick Terrell [Wed, 23 Sep 2020 19:59:23 +0000 (12:59 -0700)] 
[CI][linux-kernel] Enable -Wunused-const-variable and -Wunused-but-set-variable

4 years ago[lib] Silence -Wunused-const-variable warnings
Nick Terrell [Wed, 23 Sep 2020 19:58:38 +0000 (12:58 -0700)] 
[lib] Silence -Wunused-const-variable warnings

4 years ago[linux-kernel] Fix unused variable warnings with malloc, calloc, and free
Nick Terrell [Wed, 23 Sep 2020 19:54:10 +0000 (12:54 -0700)] 
[linux-kernel] Fix unused variable warnings with malloc, calloc, and free

4 years agoAdd unit tests to guard against bad stdin
senhuang42 [Tue, 22 Sep 2020 18:16:26 +0000 (14:16 -0400)] 
Add unit tests to guard against bad stdin

4 years agoUse IS_CONSOLE macro to detect that we're indeed using a console
senhuang42 [Tue, 22 Sep 2020 18:15:52 +0000 (14:15 -0400)] 
Use IS_CONSOLE macro to detect that we're indeed using a console

4 years agoMerge pull request #2320 from terrelln/test-fix
Nick Terrell [Tue, 22 Sep 2020 17:58:48 +0000 (10:58 -0700)] 
Merge pull request #2320 from terrelln/test-fix

[tests] Don't write to stdout

4 years agoUpdated to repcode documentation to reflect dict content size 2321/head
senhuang42 [Tue, 22 Sep 2020 17:24:27 +0000 (13:24 -0400)] 
Updated to repcode documentation to reflect dict content size

4 years agoUpdate documentation about repcodes in dictionaries
senhuang42 [Tue, 22 Sep 2020 16:58:44 +0000 (12:58 -0400)] 
Update documentation about repcodes in dictionaries

4 years ago[tests] Don't write to stdout 2320/head
Nick Terrell [Tue, 22 Sep 2020 07:40:27 +0000 (00:40 -0700)] 
[tests] Don't write to stdout

4 years agoUse `MEM_STATIC FORCE_INLINE_ATTR` instead of `FORCE_INLINE_TEMPLATE` 2317/head
animalize [Mon, 21 Sep 2020 05:26:38 +0000 (13:26 +0800)] 
Use `MEM_STATIC FORCE_INLINE_ATTR` instead of `FORCE_INLINE_TEMPLATE`

It adds `__attribute__((unused))` for __GNUC__, to eliminate `-Werror=unused-function` error.

4 years agoLet MSVC force inline ZSTD_hashPtr() function
animalize [Mon, 21 Sep 2020 02:35:47 +0000 (10:35 +0800)] 
Let MSVC force inline ZSTD_hashPtr() function

ZSTD_hashPtr() function was not expanded by MSVC, led to low performance compared to GCC.

4 years agoMerge pull request #2311 from felixhandte/ddss-fix-cparam-derivation
Felix Handte [Fri, 18 Sep 2020 18:02:14 +0000 (14:02 -0400)] 
Merge pull request #2311 from felixhandte/ddss-fix-cparam-derivation

Fix Compression Parameter Derivation Bugs Introduced by DDSS Changes

4 years agoAdd -f to .zstd decompression CLI test 2315/head
senhuang42 [Fri, 18 Sep 2020 17:01:45 +0000 (13:01 -0400)] 
Add -f to .zstd decompression CLI test

4 years agoSupport .zstd suffix only for decompression
senhuang42 [Fri, 18 Sep 2020 16:49:51 +0000 (12:49 -0400)] 
Support .zstd suffix only for decompression

4 years agoMerge pull request #2313 from felixhandte/fuzzer-separate-long-tests
Felix Handte [Fri, 18 Sep 2020 16:03:00 +0000 (12:03 -0400)] 
Merge pull request #2313 from felixhandte/fuzzer-separate-long-tests

Separate Long Tests in Fuzzer

4 years agoRun Long Tests in GitHub CI 2313/head
W. Felix Handte [Thu, 17 Sep 2020 17:28:20 +0000 (13:28 -0400)] 
Run Long Tests in GitHub CI

4 years agoMove Last Two Long Tests in fuzzer.c into Separate --long-tests Section
W. Felix Handte [Thu, 17 Sep 2020 16:57:39 +0000 (12:57 -0400)] 
Move Last Two Long Tests in fuzzer.c into Separate --long-tests Section

4 years agoUpdate Regression Test Results 2311/head
W. Felix Handte [Thu, 17 Sep 2020 16:23:05 +0000 (12:23 -0400)] 
Update Regression Test Results

4 years agoUse ZSTD_CCtxParams_init() to Init CCtxParams, not memset()
W. Felix Handte [Thu, 17 Sep 2020 16:15:33 +0000 (12:15 -0400)] 
Use ZSTD_CCtxParams_init() to Init CCtxParams, not memset()

Even if the discrepancies are at the moment benign, it's probably better to
standardize on using the one true initializer, rather than trying (and failing)
to correctly duplicate its behavior.

4 years agoAvoid Redundancy in ZSTD_initCDict_internal() Args; Don't Take CParams + CCtxParams
W. Felix Handte [Thu, 17 Sep 2020 16:08:36 +0000 (12:08 -0400)] 
Avoid Redundancy in ZSTD_initCDict_internal() Args; Don't Take CParams + CCtxParams

4 years agoFix multifile decompression summary 2310/head
senhuang42 [Wed, 16 Sep 2020 14:38:13 +0000 (10:38 -0400)] 
Fix multifile decompression summary

4 years agoAdd filename truncations to decompression
senhuang42 [Wed, 16 Sep 2020 14:28:45 +0000 (10:28 -0400)] 
Add filename truncations to decompression

4 years agoFall Back if Derived CParams are Incompatible with DDSS; Refactor CDict Creation
W. Felix Handte [Tue, 15 Sep 2020 18:06:58 +0000 (14:06 -0400)] 
Fall Back if Derived CParams are Incompatible with DDSS; Refactor CDict Creation

Rewrite ZSTD_createCDict_advanced() as a wrapper around
ZSTD_createCDict_advanced2(). Evaluate whether to use DDSS mode *after* fully
resolving cparams. If not, fall back.

4 years agoAdd display clears
senhuang42 [Tue, 15 Sep 2020 19:53:32 +0000 (15:53 -0400)] 
Add display clears

4 years agoMake ZSTD_createCDict_advanced2() cctxParams Arg Const
W. Felix Handte [Tue, 15 Sep 2020 18:06:10 +0000 (14:06 -0400)] 
Make ZSTD_createCDict_advanced2() cctxParams Arg Const