]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
8 years agoFix typos in README.md 751/head
Igor Vuk [Thu, 6 Jul 2017 18:43:14 +0000 (20:43 +0200)] 
Fix typos in README.md

8 years agofixed fullbench, part 2
Yann Collet [Thu, 6 Jul 2017 09:48:00 +0000 (02:48 -0700)] 
fixed fullbench, part 2

8 years agofixed fullbench
Yann Collet [Thu, 6 Jul 2017 09:22:57 +0000 (02:22 -0700)] 
fixed fullbench

8 years agofixed #718, reported by @GregSlazinski, solution suggested by @mcmilk
Yann Collet [Thu, 6 Jul 2017 08:42:46 +0000 (01:42 -0700)] 
fixed #718, reported by @GregSlazinski, solution suggested by @mcmilk

8 years agominor code refactoring
Yann Collet [Thu, 6 Jul 2017 01:10:07 +0000 (18:10 -0700)] 
minor code refactoring

8 years agocli : use new advanced API by default
Yann Collet [Thu, 6 Jul 2017 00:34:15 +0000 (17:34 -0700)] 
cli : use new advanced API by default

8 years agoclarified status of zstdmt_compress.h API
Yann Collet [Thu, 6 Jul 2017 00:20:52 +0000 (17:20 -0700)] 
clarified status of zstdmt_compress.h API

8 years agofixed wrong assert() condition 750/head
Yann Collet [Wed, 5 Jul 2017 02:33:16 +0000 (19:33 -0700)] 
fixed wrong assert() condition

A single job created by ZSTDMT_compress() can be < 256KB
if data to compress is < 256 KB
(in which case it is delegated to single thread mode)

8 years agomore precise estimation of amount to flush at end of stream (single thread mode) 748/head
Yann Collet [Tue, 4 Jul 2017 19:39:26 +0000 (12:39 -0700)] 
more precise estimation of amount to flush at end of stream (single thread mode)

also : can use DEBUGLEVEL variable in /tests

8 years agofixed : 0-copy in NULL is UB
Yann Collet [Tue, 4 Jul 2017 17:36:41 +0000 (10:36 -0700)] 
fixed : 0-copy in NULL is UB

8 years agoMerge pull request #743 from facebook/fullbench
Yann Collet [Tue, 4 Jul 2017 04:26:38 +0000 (21:26 -0700)] 
Merge pull request #743 from facebook/fullbench

compress_generic() automatic optimization opportunities

8 years agoMerge pull request #747 from facebook/noExit
Yann Collet [Tue, 4 Jul 2017 01:15:13 +0000 (18:15 -0700)] 
Merge pull request #747 from facebook/noExit

No exit() on decoding errors

8 years agorefactor ZSTDMT_compress() 743/head
Yann Collet [Mon, 3 Jul 2017 23:23:36 +0000 (16:23 -0700)] 
refactor ZSTDMT_compress()

8 years agofixed comments
Yann Collet [Mon, 3 Jul 2017 22:52:19 +0000 (15:52 -0700)] 
fixed comments

8 years agofixed wrong function name in comment
Yann Collet [Mon, 3 Jul 2017 22:21:24 +0000 (15:21 -0700)] 
fixed wrong function name in comment

8 years agoMerge pull request #745 from terrelln/libfuzzer
Yann Collet [Mon, 3 Jul 2017 22:15:20 +0000 (15:15 -0700)] 
Merge pull request #745 from terrelln/libfuzzer

[fuzz] Add libFuzzer targets

8 years agominor code refactoring 747/head
Yann Collet [Mon, 3 Jul 2017 22:14:55 +0000 (15:14 -0700)] 
minor code refactoring

8 years agofixed gz error reporting
Yann Collet [Mon, 3 Jul 2017 20:47:46 +0000 (13:47 -0700)] 
fixed gz error reporting

8 years agogzip : only display inflateEnd error when there is no error before
Yann Collet [Mon, 3 Jul 2017 20:45:09 +0000 (13:45 -0700)] 
gzip : only display inflateEnd error when there is no error before

8 years agoMerge pull request #746 from terrelln/dev
Yann Collet [Mon, 3 Jul 2017 20:31:27 +0000 (13:31 -0700)] 
Merge pull request #746 from terrelln/dev

[libzstd] Fix memcpy() on potential NULL source

8 years agogzip decoding does no longer exit() on invalid input
Yann Collet [Mon, 3 Jul 2017 20:24:50 +0000 (13:24 -0700)] 
gzip decoding does no longer exit() on invalid input

8 years ago[fuzz] Move from fuzz/ to tests/fuzz/ 745/head
Nick Terrell [Mon, 3 Jul 2017 19:40:12 +0000 (12:40 -0700)] 
[fuzz] Move from fuzz/ to tests/fuzz/

8 years ago[libzstd] Fix memcpy() on potential NULL source 746/head
Nick Terrell [Mon, 3 Jul 2017 19:31:55 +0000 (12:31 -0700)] 
[libzstd] Fix memcpy() on potential NULL source

* `ZSTD_decompressStream_generic()` `ip` may be `NULL` for one of the calls
  to `memcpy()`
* Assert the source is not `NULL` for calls to `memcpy()` where I believe
  the source should not be `NULL`.

8 years agodecoding malformed lz4 frames does no longer exit()
Yann Collet [Mon, 3 Jul 2017 18:27:29 +0000 (11:27 -0700)] 
decoding malformed lz4 frames does no longer exit()

8 years agofileio : decoding malformed lzma frame does no longer exit()
Yann Collet [Mon, 3 Jul 2017 17:27:16 +0000 (10:27 -0700)] 
fileio : decoding malformed lzma frame does no longer exit()

makes it possible to continue decoding file list

8 years agofixed legacy version init bug
Yann Collet [Sat, 1 Jul 2017 16:09:34 +0000 (09:09 -0700)] 
fixed legacy version init bug

8 years agoblindfix : fullbench's one-time leak, detected by valgrind
cyan4973 [Sat, 1 Jul 2017 15:03:59 +0000 (08:03 -0700)] 
blindfix : fullbench's one-time leak, detected by valgrind

8 years agoadded --show-leak-kind=all to valgrind tests
cyan4973 [Sat, 1 Jul 2017 14:05:11 +0000 (07:05 -0700)] 
added --show-leak-kind=all to valgrind tests

8 years agoMerge branch 'dev' into fullbench
cyan4973 [Sat, 1 Jul 2017 14:01:08 +0000 (07:01 -0700)] 
Merge branch 'dev' into fullbench

8 years agofix : ZSTDMT_compressStream_generic() can accept NULL input
cyan4973 [Sat, 1 Jul 2017 13:59:24 +0000 (06:59 -0700)] 
fix : ZSTDMT_compressStream_generic() can accept NULL input

also : converge implementations towards new version of ZSTDMT_compressStream_generic()

8 years agoMerge pull request #744 from terrelln/dev
Yann Collet [Sat, 1 Jul 2017 12:58:33 +0000 (05:58 -0700)] 
Merge pull request #744 from terrelln/dev

[pzstd] Remove appveyor tests

8 years ago[fuzz] Add libFuzzer targets
Nick Terrell [Thu, 29 Jun 2017 23:53:52 +0000 (16:53 -0700)] 
[fuzz] Add libFuzzer targets

* The regression driver serves both as a regression test, and as a binary for afl-fuzz.
* Next, we want to check in a seed corpus for each target. Then we can run the regression
  test binary on them on Travis or Circle CI.

8 years ago[pzstd] Remove appveyor tests 744/head
Nick Terrell [Fri, 30 Jun 2017 23:31:11 +0000 (16:31 -0700)] 
[pzstd] Remove appveyor tests

The appveyor tests sometimes hang, and pzstd is now deprecated in favor of
zstdmt, so delete the tests.

8 years ago[man] Specify that strategies start at 1
Nick Terrell [Fri, 30 Jun 2017 23:29:37 +0000 (16:29 -0700)] 
[man] Specify that strategies start at 1

8 years agofixed : dictionary compression with new advanced API in Multi-threading mode
Yann Collet [Fri, 30 Jun 2017 23:01:02 +0000 (16:01 -0700)] 
fixed : dictionary compression with new advanced API in Multi-threading mode

8 years agoOptimized ZSTDMT single-pass mode speed on large sources
Yann Collet [Fri, 30 Jun 2017 22:44:57 +0000 (15:44 -0700)] 
Optimized ZSTDMT single-pass mode speed on large sources

by ensuring job sizes remain "not too large"

8 years agoimplemented shortcut for zstd_compress_generic() in MT mode
Yann Collet [Fri, 30 Jun 2017 21:51:01 +0000 (14:51 -0700)] 
implemented shortcut for zstd_compress_generic() in MT mode

added ZSTDMT_compress_advanced() API

8 years agoMerge pull request #742 from stellamplau/stack-space
Yann Collet [Fri, 30 Jun 2017 21:50:23 +0000 (14:50 -0700)] 
Merge pull request #742 from stellamplau/stack-space

Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2

8 years agoFix typo 742/head
Stella Lau [Fri, 30 Jun 2017 19:56:24 +0000 (12:56 -0700)] 
Fix typo

8 years agoAdd comment to HUF_DECOMPRESS_WORKSPACE_SIZE
Stella Lau [Fri, 30 Jun 2017 19:53:56 +0000 (12:53 -0700)] 
Add comment to HUF_DECOMPRESS_WORKSPACE_SIZE

8 years agoClarify typedef of rankVal_t and rankValCol_t
Stella Lau [Fri, 30 Jun 2017 16:52:20 +0000 (09:52 -0700)] 
Clarify typedef of rankVal_t and rankValCol_t

8 years agoRename ALIGN and ALIGN_MASK to HUF_ALIGN and HUF_ALIGN_MASK
Stella Lau [Fri, 30 Jun 2017 16:38:11 +0000 (09:38 -0700)] 
Rename ALIGN and ALIGN_MASK to HUF_ALIGN and HUF_ALIGN_MASK

8 years agofixed Visual compilation of fullbench-dll
cyan4973 [Fri, 30 Jun 2017 02:59:37 +0000 (19:59 -0700)] 
fixed Visual compilation of fullbench-dll

8 years agomade fullbench compatible with multi-threading
Yann Collet [Fri, 30 Jun 2017 01:56:24 +0000 (18:56 -0700)] 
made fullbench compatible with multi-threading

fullbench 61/62 measure speed of ZSTD_compress_generic with 2 threads

8 years agozstreamtest : big tests are only enabled in 64-bits mode
Yann Collet [Fri, 30 Jun 2017 01:19:09 +0000 (18:19 -0700)] 
zstreamtest : big tests are only enabled in 64-bits mode

to avoid requesting too much memory in 32-bits mode during MT tests

8 years agoDelegate HUF_decompress4X_hufOnly to workspace version
Stella Lau [Thu, 29 Jun 2017 23:22:32 +0000 (16:22 -0700)] 
Delegate HUF_decompress4X_hufOnly to workspace version

8 years agoFix bitshift error
Stella Lau [Thu, 29 Jun 2017 22:40:49 +0000 (15:40 -0700)] 
Fix bitshift error

8 years agoMerge pull request #741 from facebook/promotion
Yann Collet [Thu, 29 Jun 2017 21:48:32 +0000 (14:48 -0700)] 
Merge pull request #741 from facebook/promotion

ZSTD_getFrameContentSize() promoted to "stable" status

8 years agocompression optimization opportunity
Yann Collet [Thu, 29 Jun 2017 21:44:49 +0000 (14:44 -0700)] 
compression optimization opportunity

switch to single-pass mode directly into output buffer
when outputSize >= ZSTD_compressBound(inputSize).
Speed gains observed with fullbench (~+15% on level 1)

8 years agoFix pointer casting warning
Stella Lau [Thu, 29 Jun 2017 20:04:15 +0000 (13:04 -0700)] 
Fix pointer casting warning

8 years agoupdated fullbench to also measure ZSTD_compress_generic()
Yann Collet [Thu, 29 Jun 2017 20:03:10 +0000 (13:03 -0700)] 
updated fullbench to also measure ZSTD_compress_generic()

will make it possible to visualize
optimization opportunity for ZSTD_e_end

8 years agoFix alignment warnings with pointer casting
Stella Lau [Thu, 29 Jun 2017 19:39:34 +0000 (12:39 -0700)] 
Fix alignment warnings with pointer casting

8 years agoReduce stack usage of HUF_readDTableX4 and HUF_readDTableX2
Stella Lau [Thu, 29 Jun 2017 18:49:59 +0000 (11:49 -0700)] 
Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2

8 years agominor : fix typo 741/head
Yann Collet [Thu, 29 Jun 2017 18:31:40 +0000 (11:31 -0700)] 
minor : fix typo

8 years agominor : specify compression level in tags
Yann Collet [Thu, 29 Jun 2017 18:30:31 +0000 (11:30 -0700)] 
minor : specify compression level in tags

8 years agoMerge pull request #738 from terrelln/lk-div
Yann Collet [Thu, 29 Jun 2017 18:01:28 +0000 (11:01 -0700)] 
Merge pull request #738 from terrelln/lk-div

[linux-kernel] Update patches for v2

8 years agoZSTD_getFrameContentSize() is promoted to "stable" status
Yann Collet [Thu, 29 Jun 2017 12:18:09 +0000 (05:18 -0700)] 
ZSTD_getFrameContentSize() is promoted to "stable" status

8 years agoMerge pull request #739 from facebook/refPrefix
Yann Collet [Thu, 29 Jun 2017 11:36:03 +0000 (04:36 -0700)] 
Merge pull request #739 from facebook/refPrefix

ZSTD_refPrefix

8 years ago[linux-kernel] Update patches for v2 738/head
Nick Terrell [Tue, 27 Jun 2017 01:33:04 +0000 (18:33 -0700)] 
[linux-kernel] Update patches for v2

* Reduce stack usage of many zstd functions, none use over 388 B anymore.
* Remove an incorrect `const` in `xxhash`.

8 years agoMerge pull request #736 from terrelln/cover-default-api
Yann Collet [Thu, 29 Jun 2017 03:25:36 +0000 (20:25 -0700)] 
Merge pull request #736 from terrelln/cover-default-api

[zdict] Make COVER the default algorithm

8 years agoMerge pull request #737 from facebook/advancedAPI2
Yann Collet [Thu, 29 Jun 2017 03:25:22 +0000 (20:25 -0700)] 
Merge pull request #737 from facebook/advancedAPI2

simplified API for *initStatic*()

8 years agorefactor ZSTD_check_compressionLevel_monotonicIncrease_memoryBudget() 739/head
Yann Collet [Thu, 29 Jun 2017 03:17:22 +0000 (20:17 -0700)] 
refactor ZSTD_check_compressionLevel_monotonicIncrease_memoryBudget()

use less macro statements
the initial version was meant to work with STATIC_ASSERT
but since it doesn't work and needs assert()
it's possible to rewrite it using normally compiled code
which is better for compiler.

Downside : the error message is less precise.
There is a DEBUGLOG(3,) to compensate.

8 years agoMerge branch 'advancedAPI2' into refPrefix
Yann Collet [Wed, 28 Jun 2017 23:35:49 +0000 (16:35 -0700)] 
Merge branch 'advancedAPI2' into refPrefix

8 years agoadded a test to ensure ZSTD_refPrefix() doesn't impact the following compression job
Yann Collet [Wed, 28 Jun 2017 23:25:13 +0000 (16:25 -0700)] 
added a test to ensure ZSTD_refPrefix() doesn't impact the following compression job

8 years agofixed -Wdeclaration-after-statement 737/head
Yann Collet [Wed, 28 Jun 2017 22:40:21 +0000 (15:40 -0700)] 
fixed -Wdeclaration-after-statement

8 years agoMerge pull request #740 from paulcruz74/dev
Yann Collet [Wed, 28 Jun 2017 22:36:36 +0000 (15:36 -0700)] 
Merge pull request #740 from paulcruz74/dev

decodecorpus.c patch

8 years agoadded function to control monotonic memory budget increase of ZSTD_defaultCParameters[0]
Yann Collet [Wed, 28 Jun 2017 22:34:56 +0000 (15:34 -0700)] 
added function to control monotonic memory budget increase of ZSTD_defaultCParameters[0]

It's a runtime test, based on assert(),
played once, on first ZSTD_getCParams() usage,
when ZSTD_DEBUG is enabled.

8 years agoadjusted compression levels to guarantee a monotonically increasing memory budget
Yann Collet [Wed, 28 Jun 2017 20:22:37 +0000 (13:22 -0700)] 
adjusted compression levels to guarantee a monotonically increasing memory budget

8 years agoremoved fnum from DISPLAY statements 740/head
Paul Cruz [Wed, 28 Jun 2017 20:00:49 +0000 (13:00 -0700)] 
removed fnum from DISPLAY statements

8 years agofixed generation of buggy test, corrected DISPLAY statements for errors
Paul Cruz [Wed, 28 Jun 2017 19:19:37 +0000 (12:19 -0700)] 
fixed generation of buggy test, corrected DISPLAY statements for errors

8 years agofixed ZSTD_refPrefix with Multithread-enabled CCtx
Yann Collet [Wed, 28 Jun 2017 18:09:43 +0000 (11:09 -0700)] 
fixed ZSTD_refPrefix with Multithread-enabled CCtx

8 years agofixed zstreamtest
Yann Collet [Wed, 28 Jun 2017 16:57:59 +0000 (09:57 -0700)] 
fixed zstreamtest

modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()

8 years agoMerge remote-tracking branch 'upstream/dev' into dev
Paul Cruz [Wed, 28 Jun 2017 16:45:54 +0000 (09:45 -0700)] 
Merge remote-tracking branch 'upstream/dev' into dev

8 years agocontrolled dictMode
Yann Collet [Wed, 28 Jun 2017 00:09:12 +0000 (17:09 -0700)] 
controlled dictMode

8 years agoimplemented ZSTD_refPrefix()
Yann Collet [Tue, 27 Jun 2017 22:49:12 +0000 (15:49 -0700)] 
implemented ZSTD_refPrefix()

8 years agoexposed ZSTD_MAGIC_DICTIONARY in zstd.h
Yann Collet [Tue, 27 Jun 2017 20:50:34 +0000 (13:50 -0700)] 
exposed ZSTD_MAGIC_DICTIONARY in zstd.h

makes it easier to explain ZSTD_dictMode

8 years agoadded parameter ZSTD_p_refDictContent
Yann Collet [Tue, 27 Jun 2017 18:46:39 +0000 (11:46 -0700)] 
added parameter ZSTD_p_refDictContent

8 years agoMerge pull request #735 from terrelln/xz-default
Yann Collet [Tue, 27 Jun 2017 16:42:02 +0000 (09:42 -0700)] 
Merge pull request #735 from terrelln/xz-default

[zstdcli] Support xz by default when liblzma is available

8 years ago[tests] Fix HAVE_LZMA flag 735/head
Nick Terrell [Tue, 27 Jun 2017 01:57:04 +0000 (18:57 -0700)] 
[tests] Fix HAVE_LZMA flag

8 years ago[zdict] Make COVER the default algorithm 736/head
Nick Terrell [Tue, 27 Jun 2017 04:07:14 +0000 (21:07 -0700)] 
[zdict] Make COVER the default algorithm

8 years agofixed minor conversion warning
Yann Collet [Tue, 27 Jun 2017 01:02:23 +0000 (18:02 -0700)] 
fixed minor conversion warning

8 years agoMerge pull request #729 from paulcruz74/corpus
Yann Collet [Tue, 27 Jun 2017 00:47:28 +0000 (17:47 -0700)] 
Merge pull request #729 from paulcruz74/corpus

Corpus

8 years agorefactored ZSTD_estimateDStreamSize()
Yann Collet [Tue, 27 Jun 2017 00:44:26 +0000 (17:44 -0700)] 
refactored ZSTD_estimateDStreamSize()

now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()

8 years agoZSTD_estimateCDictSize_advanced()
Yann Collet [Mon, 26 Jun 2017 23:47:32 +0000 (16:47 -0700)] 
ZSTD_estimateCDictSize_advanced()

ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()

8 years agoZSTD_estimateCStreamSize_advanced()
Yann Collet [Mon, 26 Jun 2017 23:02:25 +0000 (16:02 -0700)] 
ZSTD_estimateCStreamSize_advanced()

8 years agoZSTD_estimateCCtx_advanced()
Yann Collet [Mon, 26 Jun 2017 22:52:39 +0000 (15:52 -0700)] 
ZSTD_estimateCCtx_advanced()

ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.

8 years ago[zstdcli] Document HAVE_LZMA and zstd-noxz
Nick Terrell [Mon, 26 Jun 2017 18:25:00 +0000 (11:25 -0700)] 
[zstdcli] Document HAVE_LZMA and zstd-noxz

8 years ago[zstdcli] Add unlzma and unxz symlinks
Nick Terrell [Mon, 26 Jun 2017 18:24:36 +0000 (11:24 -0700)] 
[zstdcli] Add unlzma and unxz symlinks

8 years ago[zstdcli] Disable lzma support for MSAN testing
Nick Terrell [Mon, 26 Jun 2017 18:23:56 +0000 (11:23 -0700)] 
[zstdcli] Disable lzma support for MSAN testing

8 years agoMerge pull request #4 from facebook/dev
Paul Cruz [Mon, 26 Jun 2017 16:24:13 +0000 (09:24 -0700)] 
Merge pull request #4 from facebook/dev

pull from facebook/zstd

8 years ago[zstdcli] Support xz by default when liblzma is available
Nick Terrell [Fri, 23 Jun 2017 23:54:16 +0000 (16:54 -0700)] 
[zstdcli] Support xz by default when liblzma is available

8 years agoadded cli tests to test-all 729/head
Paul Cruz [Sat, 24 Jun 2017 00:10:41 +0000 (17:10 -0700)] 
added cli tests to test-all

8 years agoadded additional condition so large offsets into the dictionary are not generated...
Paul Cruz [Fri, 23 Jun 2017 22:54:51 +0000 (15:54 -0700)] 
added additional condition so large offsets into the dictionary are not generated past windowSize

8 years agoMerge pull request #733 from terrelln/pzstd-deadlock
Yann Collet [Fri, 23 Jun 2017 21:18:55 +0000 (14:18 -0700)] 
Merge pull request #733 from terrelln/pzstd-deadlock

[pzstd] Fix potential deadlock in the case of errors

8 years agofixed bug detected by the API test
Paul Cruz [Fri, 23 Jun 2017 20:44:24 +0000 (13:44 -0700)] 
fixed bug detected by the API test

8 years agotype warnings
Paul Cruz [Fri, 23 Jun 2017 19:00:48 +0000 (12:00 -0700)] 
type warnings

8 years agofixed bus error bug
Paul Cruz [Fri, 23 Jun 2017 18:22:29 +0000 (11:22 -0700)] 
fixed bus error bug

8 years agoMerge pull request #725 from facebook/advancedAPI2
Yann Collet [Fri, 23 Jun 2017 16:50:47 +0000 (09:50 -0700)] 
Merge pull request #725 from facebook/advancedAPI2

New Advanced API

8 years ago--no-big-tests for msan 725/head
Yann Collet [Fri, 23 Jun 2017 07:37:15 +0000 (00:37 -0700)] 
--no-big-tests for msan

msan tests require too much RAM for 2 GB VM

8 years agoMerge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2
Yann Collet [Fri, 23 Jun 2017 07:13:03 +0000 (00:13 -0700)] 
Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2