]>
git.ipfire.org Git - thirdparty/zstd.git/log
Yann Collet [Fri, 22 Jun 2018 19:14:59 +0000 (12:14 -0700)]
changed POOL_resize() return type to int
return is now just en error code.
This guarantee that `ctx` remains valid after POOL_resize().
Gets rid of internal POOL_free() operation.
Yann Collet [Fri, 22 Jun 2018 01:04:58 +0000 (18:04 -0700)]
add a cond_broadcast after resize
to make sure all threads (notably newly available threads)
get awaken to immediately process potential items in the queue.
Yann Collet [Thu, 21 Jun 2018 21:58:59 +0000 (14:58 -0700)]
added extended POOL test
abrupt end + downsizing with running jobs remaining in queue.
also : POOL_resize() requires numThreads >= 1
Yann Collet [Thu, 21 Jun 2018 19:24:36 +0000 (12:24 -0700)]
added a test for POOL (multithreading)
ensuring all jobs in queue are nonetheless completed
when POOL is instructed to end abruptly (POOL_free())
Yann Collet [Thu, 21 Jun 2018 00:18:57 +0000 (17:18 -0700)]
fixed: bug when counting nb of active threads
when queueSize > 1
also : added a test in testpool.c
verifying resizing is effective.
Yann Collet [Wed, 20 Jun 2018 21:35:39 +0000 (14:35 -0700)]
change control of threadLimit
now limits maximum nb of active threads
even when queueSize > 1.
Yann Collet [Wed, 20 Jun 2018 03:14:03 +0000 (20:14 -0700)]
fixed wrong size in pthread struct transfer
Yann Collet [Wed, 20 Jun 2018 01:07:18 +0000 (18:07 -0700)]
reduced POOL_resize() restriction
It's not necessary to ensure that no job is ongoing.
The pool is only expanded, existing threads are preserved.
In case of error, the only option is to return NULL and terminate the thread pool anyway.
Yann Collet [Wed, 20 Jun 2018 00:28:56 +0000 (17:28 -0700)]
make zstdmt resize its context
when nbThreads change.
Technically, it only expands.
But when instructed to use less threads,
the thread pool will limit nb of concurrent threads.
Yann Collet [Tue, 19 Jun 2018 23:03:12 +0000 (16:03 -0700)]
finalized POOL_resize()
POOL_ctx* POOL_resize(POOL_ctx* ctx, size_t numThreads)
The function may fail, and returns a NULL pointer in this case.
Yann Collet [Tue, 19 Jun 2018 03:46:39 +0000 (20:46 -0700)]
introduced POOL_resize()
not complete yet :
finalize behavior in case of unfinished expansion
Yann Collet [Mon, 18 Jun 2018 20:29:44 +0000 (13:29 -0700)]
Merge pull request #1185 from felixhandte/zstd-btlazy-in-place-dict
ZSTD_btlazy2: Support Searching the Dictionary Context In-Place
Yann Collet [Fri, 15 Jun 2018 15:25:08 +0000 (11:25 -0400)]
Merge pull request #1191 from facebook/huf_rename
Rename huffman decompression functions
Yann Collet [Fri, 15 Jun 2018 00:42:10 +0000 (20:42 -0400)]
Merge branch 'dev' into huf_rename
Yann Collet [Fri, 15 Jun 2018 00:41:50 +0000 (20:41 -0400)]
grouped X2 function together
Yann Collet [Fri, 15 Jun 2018 00:39:52 +0000 (20:39 -0400)]
Merge pull request #1189 from facebook/hist
histogram module
Yann Collet [Thu, 14 Jun 2018 23:48:40 +0000 (19:48 -0400)]
added debug.h for cmake
Yann Collet [Thu, 14 Jun 2018 23:47:05 +0000 (19:47 -0400)]
avoid duplicate code comments
when a function is decribed in hist.h,
do not describe it again in hist.c
to avoid future doc synchronization issues.
Yann Collet [Thu, 14 Jun 2018 19:51:31 +0000 (15:51 -0400)]
Visual Studio project blind fix
Yann Collet [Thu, 14 Jun 2018 20:22:22 +0000 (16:22 -0400)]
Merge pull request #1187 from facebook/fix1186
fix dctx initialization within ZSTD_decompress in stack mode
Yann Collet [Thu, 14 Jun 2018 20:21:53 +0000 (16:21 -0400)]
Merge pull request #1184 from facebook/debug
Grouped debug functions into debug.h
Ryan Schmidt [Thu, 14 Jun 2018 20:05:33 +0000 (15:05 -0500)]
Add CXXFLAGS to ALL_LDFLAGS (#1178)
pzstd requires C++11, which older C++ standard libraries like libstdc++
as used on OS X 10.8 and earlier don't support. The user might address
this by setting "CXXFLAGS=-stdlib=libc++". This flag must be used both
at compile time and at link time. Asking the user to also put the flag
in LDFLAGS is undesirable because then the flag would also be used when
linking C code, which would be inappropriate.
Yann Collet [Thu, 14 Jun 2018 19:17:03 +0000 (15:17 -0400)]
renamed all HUF_decompress*X4*() functions into *X2
to underline they generate up to 2 symbols per decoding,
in preparation for a future *X3 variant.
Yann Collet [Thu, 14 Jun 2018 19:08:43 +0000 (15:08 -0400)]
renamed all HUF_decompress*X2*() functions into *X1
to underline they generate one symbol per decoding operation.
The new naming scheme will make it easier to introduce an *X3 variant.
W. Felix Handte [Thu, 14 Jun 2018 18:53:36 +0000 (14:53 -0400)]
Force Inline BtFindBestMatch
Yann Collet [Thu, 14 Jun 2018 14:22:24 +0000 (10:22 -0400)]
fix dctx initialization within ZSTD_decompress in stack mode
when ZSTD_HEAPMODE=0 (which is not default).
Also : added an associated test (test-fuzzer-stackmode)
run on travis CI
fix #1186
Yann Collet [Thu, 14 Jun 2018 00:02:33 +0000 (20:02 -0400)]
fixed g_debuglevel variable name
in debug.h
Yann Collet [Wed, 13 Jun 2018 23:49:31 +0000 (19:49 -0400)]
grouped all histogram functions into hist.c
renamed functions with HIST_* prefix
W. Felix Handte [Tue, 12 Jun 2018 22:38:10 +0000 (18:38 -0400)]
Search Dict for Matches
W. Felix Handte [Tue, 12 Jun 2018 22:32:57 +0000 (18:32 -0400)]
Attach Dicts when Using ZSTD_btlazy2
Yann Collet [Wed, 13 Jun 2018 20:05:41 +0000 (16:05 -0400)]
Merge pull request #1169 from felixhandte/zstd-lazy-in-place-dict
ZSTD_lazy: Support Searching the Dictionary Context In-Place
Yann Collet [Wed, 13 Jun 2018 18:59:26 +0000 (14:59 -0400)]
grouped debug functions into debug.h
There were 2 competing set of debug functions
within zstd_internal.h and bitstream.h.
They were mostly duplicate, and required care to avoid messing with each other.
There is now a single implementation, shared by both.
Significant change :
The macro variable ZSTD_DEBUG does no longer exist,
it has been replaced by DEBUGLEVEL,
which required modifying several source files.
W. Felix Handte [Sat, 9 Jun 2018 21:19:04 +0000 (17:19 -0400)]
Fix Cast Warning
W. Felix Handte [Fri, 8 Jun 2018 19:06:47 +0000 (15:06 -0400)]
Extend Dictionary Matches Backwards
W. Felix Handte [Thu, 7 Jun 2018 20:57:45 +0000 (16:57 -0400)]
Update Dictionary Attachment Cutoff Values Again
W. Felix Handte [Thu, 7 Jun 2018 20:55:13 +0000 (16:55 -0400)]
Do Not Inline DMS Search Function
W. Felix Handte [Wed, 6 Jun 2018 23:54:13 +0000 (19:54 -0400)]
Also Implement Depth Repcode Checks
W. Felix Handte [Wed, 6 Jun 2018 21:34:29 +0000 (17:34 -0400)]
Apply Match Continuation Bug Fix
W. Felix Handte [Thu, 31 May 2018 22:32:41 +0000 (18:32 -0400)]
Update Dictionary Attachment Cutoff Values
W. Felix Handte [Tue, 29 May 2018 20:26:23 +0000 (16:26 -0400)]
Check Dict Match State in ZSTD_HcFindBestMatch_generic
W. Felix Handte [Tue, 29 May 2018 20:06:23 +0000 (16:06 -0400)]
Focus on Non-BT Impls for the Moment
W. Felix Handte [Tue, 29 May 2018 20:05:48 +0000 (16:05 -0400)]
Remove Pre-Existing Repcode Check
W. Felix Handte [Wed, 23 May 2018 19:49:43 +0000 (15:49 -0400)]
Second Repcode Check
W. Felix Handte [Wed, 23 May 2018 19:06:24 +0000 (15:06 -0400)]
First Repcode Check
W. Felix Handte [Wed, 23 May 2018 18:13:16 +0000 (14:13 -0400)]
Disallow Too-Long Repcodes When Using an Attached Dict
W. Felix Handte [Wed, 23 May 2018 19:10:18 +0000 (15:10 -0400)]
Rename `base` -> `prefixLowest`
W. Felix Handte [Fri, 1 Jun 2018 18:23:14 +0000 (14:23 -0400)]
Go Back to Separate Intermediate Functions for Different Dict Modes
W. Felix Handte [Wed, 16 May 2018 08:32:21 +0000 (04:32 -0400)]
Further Templatize Intermediate Functions on dictMode
W. Felix Handte [Wed, 16 May 2018 08:07:09 +0000 (04:07 -0400)]
Convert Existing U32 extDict Vars to ZSTD_dictMode Enums
W. Felix Handte [Wed, 16 May 2018 06:31:30 +0000 (02:31 -0400)]
Attach Dict When Using ZSTD_lazy Strategies
W. Felix Handte [Wed, 16 May 2018 06:30:20 +0000 (02:30 -0400)]
Add _dictMatchState Stubs for ZSTD_lazy Functions
W. Felix Handte [Wed, 16 May 2018 06:26:07 +0000 (02:26 -0400)]
Add dictMode Param to ZSTD_compressBlock_lazy_generic
Yann Collet [Wed, 13 Jun 2018 15:38:29 +0000 (11:38 -0400)]
Merge pull request #1168 from GeorgeLu97/paramgrillfeatures
Have paramgrill share bench.c benchmarking function
Yann Collet [Wed, 13 Jun 2018 15:32:06 +0000 (11:32 -0400)]
Merge pull request #1175 from ryandesign/macos
Fix name of macOS
George Lu [Mon, 11 Jun 2018 14:59:05 +0000 (10:59 -0400)]
Requested changes
-Remove g_displaylevel/setNotificationLevel function
-Add extern "C"
-Remove averaging
-Reorder arguments
More fixes
-Added BMK_return_t (result + possible error)
-Correct comment'
-Nullcheck ctx, dctx when allocated
-Remove extra assert
Yann Collet [Tue, 12 Jun 2018 19:09:01 +0000 (12:09 -0700)]
Merge pull request #1174 from duc0/document_default_level
Expose ZSTD_CLEVEL_DEFAULT and update documentation
Duc Ngo [Mon, 11 Jun 2018 17:02:15 +0000 (10:02 -0700)]
Line break
Duc Ngo [Mon, 11 Jun 2018 17:01:35 +0000 (10:01 -0700)]
Address comments
Yann Collet [Mon, 11 Jun 2018 00:34:19 +0000 (17:34 -0700)]
Merge pull request #1177 from ryandesign/gen_html-CXXFLAGS
Remove duplicate CXXFLAGS
Ryan Schmidt [Sun, 10 Jun 2018 23:04:21 +0000 (18:04 -0500)]
Remove duplicate CXXFLAGS
Ryan Schmidt [Sat, 9 Jun 2018 19:31:17 +0000 (14:31 -0500)]
Fix name of macOS
Yann Collet [Fri, 8 Jun 2018 19:30:42 +0000 (12:30 -0700)]
Merge pull request #1171 from facebook/negLevels
Negative compression levels
George Lu [Mon, 4 Jun 2018 23:32:37 +0000 (16:32 -0700)]
Make paramgrill use bench.c benchmarking
Duc Ngo [Fri, 8 Jun 2018 18:29:51 +0000 (11:29 -0700)]
Expose ZSTD_CLEVEL_DEFAULT and update documentation
Yann Collet [Thu, 7 Jun 2018 23:59:33 +0000 (16:59 -0700)]
fixed minor conversion warning
Yann Collet [Thu, 7 Jun 2018 23:07:46 +0000 (16:07 -0700)]
fixed regressiontest
ZSTD_TARGETLEN_MIN no longer exists
since is would be tautological to check if an unsigned value is < 0.
Yann Collet [Thu, 7 Jun 2018 22:51:35 +0000 (15:51 -0700)]
Merge branch 'dev' into negLevels
Yann Collet [Thu, 7 Jun 2018 22:49:01 +0000 (15:49 -0700)]
support targetlen==0 with strategy==ZSTD_fast
to mean "normal compression",
targetlen >= 1 now means "disable huffman compression of literals"
Yann Collet [Thu, 7 Jun 2018 22:24:12 +0000 (15:24 -0700)]
removed literalCompression directive
in this version, literal compression is always disabled for ZSTD_fast strategy.
Performance parity between ZSTD_compress_advanced() and ZSTD_compress_generic()
Yann Collet [Thu, 7 Jun 2018 22:12:13 +0000 (15:12 -0700)]
fuzzer: make negative compression level fail
result of ZSTD_compress_advanced()
is different from ZSTD_compress_generic()
when using negative compression levels
because the disabling of huffman compression is not passed in parameters.
Yann Collet [Thu, 7 Jun 2018 21:47:29 +0000 (14:47 -0700)]
Merge branch 'zstd_compress' into negLevels
Yann Collet [Thu, 7 Jun 2018 21:46:55 +0000 (14:46 -0700)]
negative compression level test
compare results from simple and advanced AIP
Yann Collet [Thu, 7 Jun 2018 21:44:32 +0000 (14:44 -0700)]
Merge pull request #1165 from facebook/ctxSizeDown
Dynamic context downsize
Yann Collet [Thu, 7 Jun 2018 20:53:30 +0000 (13:53 -0700)]
clean ZSTD_compress() initialization
The (pretty old) code inside ZSTD_compress()
was making some pretty bold assumptions
on what's inside a CCtx and how to init it.
This is pretty fragile by design.
CCtx content evolve.
Knowledge of how to handle that should be concentrate in one place.
A side effect of this strategy
is that ZSTD_compress() wouldn't check for BMI2 capability,
and is therefore missing out some potential speed opportunity.
This patch makes ZSTD_compress() use
the same initialization and release functions
as the normal creator / destructor ones.
Measured on my laptop, with a custom version of bench
manually modified to use ZSTD_compress() (instead of the advanced API) :
This patch :
1#silesia.tar :
211984896 ->
73651053 (2.878), 312.2 MB/s , 723.8 MB/s
2#silesia.tar :
211984896 ->
70163650 (3.021), 226.2 MB/s , 649.8 MB/s
3#silesia.tar :
211984896 ->
66996749 (3.164), 169.4 MB/s , 636.7 MB/s
4#silesia.tar :
211984896 ->
65998319 (3.212), 136.7 MB/s , 619.2 MB/s
dev branch :
1#silesia.tar :
211984896 ->
73651053 (2.878), 291.7 MB/s , 727.5 MB/s
2#silesia.tar :
211984896 ->
70163650 (3.021), 216.2 MB/s , 655.7 MB/s
3#silesia.tar :
211984896 ->
66996749 (3.164), 162.2 MB/s , 633.1 MB/s
4#silesia.tar :
211984896 ->
65998319 (3.212), 130.6 MB/s , 618.6 MB/s
Yann Collet [Thu, 7 Jun 2018 19:18:47 +0000 (12:18 -0700)]
disable aarch64 test on travis CI
there's a bug on travis' ld
> collect2: error: ld terminated with signal 11 [Segmentation fault], core dumped
I cannot reproduce it.
Note that travis' ld version is 2.24
while the one in my Linux VM is 2.26.1.
ld is part of binutils, so it's pretty hard to change.
I would expect the bug to no longer be triggered after some random code change.
To be re-enabled later.
Yann Collet [Thu, 7 Jun 2018 01:15:57 +0000 (18:15 -0700)]
added ld -v to aarch64fuzz
to better understand the travis CI linking bug
Yann Collet [Wed, 6 Jun 2018 23:47:42 +0000 (16:47 -0700)]
Merge pull request #1164 from GeorgeLu97/CustomMacros
Partial Compilation Macros
Yann Collet [Wed, 6 Jun 2018 22:51:55 +0000 (15:51 -0700)]
bumped version number to v1.3.5
George Lu [Wed, 6 Jun 2018 22:16:37 +0000 (15:16 -0700)]
Rename tests
Yann Collet [Wed, 6 Jun 2018 22:04:12 +0000 (15:04 -0700)]
context can be sized down even with constant parameters
when parameters are "equivalent",
the context is re-used in continue mode,
hence needed workspace size is not recalculated.
This incidentally also evades the size-down check and action.
This patch intercepts the "continue mode"
so that the size-down check and action is actually triggered.
Yann Collet [Wed, 6 Jun 2018 22:00:27 +0000 (15:00 -0700)]
changed member name to workSpaceOversizedDuration
Yann Collet [Wed, 6 Jun 2018 21:41:21 +0000 (14:41 -0700)]
Merge pull request #1167 from facebook/seekable_gcc5
contrib/seekable_format on gcc5+
Yann Collet [Wed, 6 Jun 2018 20:10:29 +0000 (13:10 -0700)]
fixed seekable_format type mismatch
and some minor "unused variable" warnings.
Also : zstd_seekable.h is actually depending on zstd.h for ZSTDLIB_API
Yann Collet [Wed, 6 Jun 2018 19:47:16 +0000 (12:47 -0700)]
better make -j all behavior
avoid concurrent compilation of libzstd
Yann Collet [Wed, 6 Jun 2018 19:20:19 +0000 (12:20 -0700)]
refactor travis CI test
to add a `make all` target with gcc-6.
Note : should fail at this stage, due to contrib/seekable_format
Yann Collet [Wed, 6 Jun 2018 19:09:58 +0000 (12:09 -0700)]
ensure seekable_format/examples generated libzstd.a
when it's not already present in the expected directory
George Lu [Wed, 6 Jun 2018 18:33:39 +0000 (11:33 -0700)]
More Tests
George Lu [Tue, 5 Jun 2018 22:21:31 +0000 (15:21 -0700)]
Remove test from Circle
George Lu [Tue, 5 Jun 2018 22:20:34 +0000 (15:20 -0700)]
Move stuff around
test execution -> travis
logic -> partialTests.sh
Yann Collet [Tue, 5 Jun 2018 21:53:28 +0000 (14:53 -0700)]
added test case
George Lu [Tue, 5 Jun 2018 21:41:41 +0000 (14:41 -0700)]
Remove optimizations
George Lu [Tue, 5 Jun 2018 21:10:29 +0000 (14:10 -0700)]
circle test
George Lu [Tue, 5 Jun 2018 20:55:36 +0000 (13:55 -0700)]
Revert "Partial compilation test?"
This reverts commit
b2496ab60663992eb566dd221705da86fc9dafc7 .
George Lu [Tue, 5 Jun 2018 20:24:00 +0000 (13:24 -0700)]
Partial compilation test?
Yann Collet [Tue, 5 Jun 2018 18:23:18 +0000 (11:23 -0700)]
added workSpaceTooLarge and workSpaceWasteful
also :
slightly increased speed of test fuzzer.16
Yann Collet [Tue, 5 Jun 2018 13:50:29 +0000 (06:50 -0700)]
Merge pull request #1162 from facebook/corruptionFix
Fixed a corruption bug
George Lu [Tue, 5 Jun 2018 00:19:06 +0000 (17:19 -0700)]
Fix Typos
Yann Collet [Tue, 5 Jun 2018 00:10:50 +0000 (17:10 -0700)]
changed a few variable names
to unify naming convention
Yann Collet [Mon, 4 Jun 2018 21:56:23 +0000 (14:56 -0700)]
Merge pull request #1161 from GeorgeLu97/paramgrillformat
Added zstdcli-style format for compression parameters in paramgrill
George Lu [Mon, 4 Jun 2018 21:33:21 +0000 (14:33 -0700)]
Added Deprecated Dependencies