]>
git.ipfire.org Git - thirdparty/zstd.git/log
Przemyslaw Skibinski [Tue, 17 Jan 2017 11:40:06 +0000 (12:40 +0100)]
added "Makefile is validated"
Yann Collet [Tue, 17 Jan 2017 04:17:44 +0000 (20:17 -0800)]
Merge pull request #511 from indygreg/cdict-dictid
Set dictionary ID in ZSTD_initCStream_usingCDict()
Yann Collet [Tue, 17 Jan 2017 03:46:22 +0000 (19:46 -0800)]
added test checking dictID when using ZSTD_initCStream_usingCDict()
It shows that dictID is not properly added into frame header
Gregory Szorc [Sun, 15 Jan 2017 01:44:54 +0000 (17:44 -0800)]
Set dictionary ID in ZSTD_initCStream_usingCDict()
When porting python-zstandard to use ZSTD_initCStream_usingCDict()
so compression dictionaries could be reused, an automated test
failed due to compressed content changing.
I tracked this down to ZSTD_initCStream_usingCDict() not
setting the dictID field of the ZSTD_CCtx attached to the
ZSTD_CStream instance.
I'm not 100% convinced this is the correct or full solution,
as I'm still seeing one automated test failing with this change.
Yann Collet [Thu, 12 Jan 2017 21:10:04 +0000 (22:10 +0100)]
Merge pull request #510 from iburinoc/baddict
Fixed decompress_usingDict not propagating corrupted dictionary error
Sean Purcell [Thu, 12 Jan 2017 17:38:29 +0000 (09:38 -0800)]
Fix missing 'OK' logging on fuzzer testcase
Yann Collet [Thu, 12 Jan 2017 16:46:46 +0000 (17:46 +0100)]
fix gcc-arm warning "suggest braces around empty body"
Yann Collet [Thu, 12 Jan 2017 02:06:35 +0000 (03:06 +0100)]
zstdmt : fix : resources properly collected even when early fail
In previous version, main function would return early when detecting a job error.
Late threads resources were therefore not collected back into pools.
New version just register the error, but continue the collecting process.
All buffers and context should be released back to pool before leaving main function.
Sean Purcell [Thu, 12 Jan 2017 01:31:06 +0000 (17:31 -0800)]
Fixed decompress_usingDict not propagating corrupted dictionary error
Yann Collet [Thu, 12 Jan 2017 01:01:28 +0000 (02:01 +0100)]
zstdmt : correctly check for cctx and buffer allocation
Result from getBuffer and getCCtx could be NULL when allocation fails.
Now correctly checks : job creation stop and last job reports an allocation error.
releaseBuffer and releaseCCtx are now also compatible with NULL input.
Identified a new potential issue :
when early job fails, later jobs are not collected for resource retrieval.
Yann Collet [Thu, 12 Jan 2017 00:25:46 +0000 (01:25 +0100)]
zstdmt : changed internal naming from frame to chunk
Since the result of mt compression is a single frame,
changed naming, which implied the concatenation of multiple frames.
minor : ensures that content size is written in header
Yann Collet [Wed, 11 Jan 2017 17:21:25 +0000 (18:21 +0100)]
ZSTDMT_compress() creates a single frame
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()
As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().
This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
Yann Collet [Wed, 11 Jan 2017 15:08:08 +0000 (16:08 +0100)]
minor refactor (release CCtx 1st) and comment clarification
Yann Collet [Wed, 11 Jan 2017 14:58:05 +0000 (15:58 +0100)]
fixed ZSTDMT_createCCtx() : checked inner objects are properly created
Yann Collet [Wed, 11 Jan 2017 14:44:26 +0000 (15:44 +0100)]
improved ZSTD_createCCtxPool() cancellation
use ZSTD_freeCCtxPool() to release the partially created pool.
avoids to duplicate logic.
Also : identified a new difficult corner case :
when freeing the Pool, all CCtx should be previously released back to the pool.
Otherwise, it means some CCtx are still in use.
There is currently no clear policy on what to do in such a case.
Note : it's supposed to never happen.
Since pool creation/usage is static, it has no external user,
which limits risks.
Yann Collet [Wed, 11 Jan 2017 14:35:56 +0000 (15:35 +0100)]
fixed ZSTDMT_createCCtxPool() when inner CCtx creation fails
Yann Collet [Tue, 10 Jan 2017 05:30:28 +0000 (06:30 +0100)]
Merge pull request #509 from terrelln/dict-builder-32
Handle cover dictionary builder maximum input size for 32-bit mode
Nick Terrell [Tue, 10 Jan 2017 01:00:12 +0000 (17:00 -0800)]
Document memory requirements for COVER algorithm
Nick Terrell [Tue, 10 Jan 2017 00:50:00 +0000 (16:50 -0800)]
Handle large input size in 32-bit mode correctly
Nick Terrell [Tue, 10 Jan 2017 00:49:04 +0000 (16:49 -0800)]
Use COVER_MEMMULT when training with COVER.
Yann Collet [Mon, 9 Jan 2017 18:47:09 +0000 (19:47 +0100)]
minor man page update
Yann Collet [Tue, 3 Jan 2017 19:33:33 +0000 (20:33 +0100)]
Merge pull request #507 from terrelln/dict-builder-optimize
Add new dictionary builder
Nick Terrell [Tue, 3 Jan 2017 02:45:19 +0000 (18:45 -0800)]
Add COVER dictionary builder to fuzzer unit tests
Nick Terrell [Mon, 2 Jan 2017 20:40:43 +0000 (12:40 -0800)]
Simplify COVER parameters
Yann Collet [Mon, 2 Jan 2017 23:25:01 +0000 (00:25 +0100)]
separated ppc and ppc64 tests, for more regular timing
Nick Terrell [Sun, 1 Jan 2017 06:59:51 +0000 (01:59 -0500)]
Add cover cli to playtests
Nick Terrell [Sun, 1 Jan 2017 05:08:24 +0000 (21:08 -0800)]
Add COVER to the zstd cli
Nick Terrell [Sun, 1 Jan 2017 05:08:12 +0000 (21:08 -0800)]
Add cover.c to build files
Nick Terrell [Sun, 1 Jan 2017 05:07:44 +0000 (21:07 -0800)]
Add COVER dictionary builder
Yann Collet [Mon, 2 Jan 2017 02:22:18 +0000 (03:22 +0100)]
compile object files, for faster recompilation
Yann Collet [Mon, 2 Jan 2017 01:05:45 +0000 (02:05 +0100)]
fixed cmake tests
Yann Collet [Mon, 2 Jan 2017 00:43:56 +0000 (01:43 +0100)]
new Makefile target zstdmt
Yann Collet [Mon, 2 Jan 2017 00:11:55 +0000 (01:11 +0100)]
fixed linux warnings
Yann Collet [Sun, 1 Jan 2017 23:49:42 +0000 (00:49 +0100)]
minor fixes
Yann Collet [Sun, 1 Jan 2017 16:31:33 +0000 (17:31 +0100)]
use pthread_cond to send signals between threads
Yann Collet [Sun, 1 Jan 2017 14:53:51 +0000 (15:53 +0100)]
Merge pull request #505 from terrelln/pthread_cond_t
pthread_cond_t
Nick Terrell [Sun, 1 Jan 2017 03:39:32 +0000 (22:39 -0500)]
Shorten thread pool tests
Nick Terrell [Sun, 1 Jan 2017 00:10:47 +0000 (19:10 -0500)]
Fix pool for threading.h
Nick Terrell [Sun, 1 Jan 2017 00:10:29 +0000 (19:10 -0500)]
Add threading.h condition variables
Nick Terrell [Sun, 1 Jan 2017 00:10:13 +0000 (19:10 -0500)]
Switch thread pool test to threading.h
Yann Collet [Sat, 31 Dec 2016 15:32:19 +0000 (16:32 +0100)]
extended ZSTDMT code support for non-MT systems and WIN32 (preliminary)
Yann Collet [Sat, 31 Dec 2016 13:45:33 +0000 (14:45 +0100)]
simplified Buffer Pool
Yann Collet [Sat, 31 Dec 2016 05:04:25 +0000 (06:04 +0100)]
new zstdmt version using generic treadpool
Yann Collet [Sat, 31 Dec 2016 02:31:26 +0000 (03:31 +0100)]
bench correctly measures time for multi-threaded compression (posix only)
Yann Collet [Fri, 30 Dec 2016 14:31:49 +0000 (15:31 +0100)]
Merge pull request #504 from terrelln/thread-pool
[zstdmt] Add thread pool
Nick Terrell [Fri, 30 Dec 2016 07:41:03 +0000 (23:41 -0800)]
Add thread pool tests
Nick Terrell [Fri, 30 Dec 2016 07:39:44 +0000 (23:39 -0800)]
Add a thread pool for ZSTDMT and COVER
Przemyslaw Skibinski [Thu, 29 Dec 2016 14:24:18 +0000 (15:24 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Yann Collet [Thu, 29 Dec 2016 00:24:01 +0000 (01:24 +0100)]
Changed : input divided into roughly equal parts.
Debug : can measure time waiting for mutexes to unlock.
Yann Collet [Wed, 28 Dec 2016 16:08:28 +0000 (17:08 +0100)]
compression threads use ZSTD_compressCCtx()
Yann Collet [Wed, 28 Dec 2016 15:50:39 +0000 (16:50 +0100)]
Merge pull request #503 from inikep/dev11
Dev11
Yann Collet [Wed, 28 Dec 2016 15:11:09 +0000 (16:11 +0100)]
added -T command , to set nb of threads
Yann Collet [Wed, 28 Dec 2016 14:31:19 +0000 (15:31 +0100)]
protect buffer pool with a mutex
Yann Collet [Wed, 28 Dec 2016 13:38:48 +0000 (14:38 +0100)]
Merge pull request #502 from Chocobo1/cpp11
Move -std=c++11 cxxflag to PZSTD_CXXFLAGS
Przemyslaw Skibinski [Wed, 28 Dec 2016 11:32:41 +0000 (12:32 +0100)]
changed default PREFIX and MANDIR
Yann Collet [Tue, 27 Dec 2016 06:19:36 +0000 (07:19 +0100)]
first zstdmt sketch
Chocobo1 [Mon, 26 Dec 2016 15:04:59 +0000 (23:04 +0800)]
Move -std=c++11 cxxflag to PZSTD_CXXFLAGS
Fixes the problem that the compiler doesn't enable c++11 mode by default
and the package build system has its own CXXFLAGS
Yann Collet [Fri, 23 Dec 2016 21:25:31 +0000 (22:25 +0100)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Fri, 23 Dec 2016 21:25:03 +0000 (22:25 +0100)]
fixed ZSTD_sizeof_?Dict()
Yann Collet [Fri, 23 Dec 2016 20:32:03 +0000 (21:32 +0100)]
Merge pull request #499 from inikep/dev11
improved *BSD and Solaris compatibility
Yann Collet [Fri, 23 Dec 2016 10:18:10 +0000 (11:18 +0100)]
Merge pull request #497 from apjanke/zstdless-with-shebang
zstdless: add shebang and quote $@
Yann Collet [Fri, 23 Dec 2016 10:16:23 +0000 (11:16 +0100)]
Merge pull request #498 from terrelln/zdict-deprecated
Fix failing test due to deprecation warning
Przemyslaw Skibinski [Fri, 23 Dec 2016 09:05:49 +0000 (10:05 +0100)]
BSD: improved "make install"
Nick Terrell [Fri, 23 Dec 2016 02:14:57 +0000 (18:14 -0800)]
ZDICT_finalizeDictionary() flipped comparison
Nick Terrell [Fri, 23 Dec 2016 02:01:14 +0000 (18:01 -0800)]
ZDICT_finalizeDictionary() flipped comparison
`ZDICT_finalizeDictionary()` had a flipped comparison.
I also allowed `dictBufferCapacity == dictContentSize`.
It might be the case that the user wants to fill the dictionary
completely up, and then let zstd take exactly the space it needs
for the entropy tables.
Nick Terrell [Fri, 23 Dec 2016 01:28:49 +0000 (17:28 -0800)]
Fix failing test due to deprecation warning
Andrew Janke [Thu, 22 Dec 2016 22:40:10 +0000 (17:40 -0500)]
zstdless: add shebang and quote $@
Yann Collet [Thu, 22 Dec 2016 19:18:43 +0000 (20:18 +0100)]
added ZDICT_finalizeDictionary()
Przemyslaw Skibinski [Thu, 22 Dec 2016 19:14:37 +0000 (20:14 +0100)]
Solaris: working "make -C lib install"
Przemyslaw Skibinski [Thu, 22 Dec 2016 18:26:01 +0000 (19:26 +0100)]
Solaris: working "make -C programs install"
Przemyslaw Skibinski [Thu, 22 Dec 2016 17:05:07 +0000 (18:05 +0100)]
"make test" is now compatible with Solaris
Przemyslaw Skibinski [Thu, 22 Dec 2016 16:59:22 +0000 (17:59 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Yann Collet [Thu, 22 Dec 2016 14:26:33 +0000 (15:26 +0100)]
fixed zlib_wrapper conversion warnings
Yann Collet [Thu, 22 Dec 2016 11:43:00 +0000 (12:43 +0100)]
fixed allmost target
Yann Collet [Thu, 22 Dec 2016 10:31:39 +0000 (11:31 +0100)]
removed examples from standard C tests, since they contain some POSIX elements
Yann Collet [Wed, 21 Dec 2016 18:57:18 +0000 (19:57 +0100)]
fix : zlib wrapper compile in gnu90 mode
Yann Collet [Wed, 21 Dec 2016 18:25:15 +0000 (19:25 +0100)]
added ZSTD_createDDict_byReference() body
Przemyslaw Skibinski [Wed, 21 Dec 2016 16:02:06 +0000 (17:02 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Yann Collet [Wed, 21 Dec 2016 15:44:35 +0000 (16:44 +0100)]
added ZSTD_createDDict_byReference()
Yann Collet [Wed, 21 Dec 2016 15:44:24 +0000 (16:44 +0100)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Wed, 21 Dec 2016 15:42:46 +0000 (16:42 +0100)]
Merge pull request #489 from inikep/v112
improved detection of POSIX
Yann Collet [Wed, 21 Dec 2016 15:20:41 +0000 (16:20 +0100)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Wed, 21 Dec 2016 15:20:11 +0000 (16:20 +0100)]
added : ZSTD_createCDict_byReference()
Yann Collet [Wed, 21 Dec 2016 14:12:06 +0000 (15:12 +0100)]
Merge pull request #493 from inikep/fullbench
Fullbench extDict variant
Przemyslaw Skibinski [Wed, 21 Dec 2016 14:08:44 +0000 (15:08 +0100)]
util.h: restore BSD license for Facebook Open-Source
Przemyslaw Skibinski [Wed, 21 Dec 2016 13:00:41 +0000 (14:00 +0100)]
updated comments
Przemyslaw Skibinski [Wed, 21 Dec 2016 12:47:11 +0000 (13:47 +0100)]
_CRT_SECURE_NO_WARNINGS moved to util.h
Przemyslaw Skibinski [Wed, 21 Dec 2016 12:23:34 +0000 (13:23 +0100)]
platform.h: removed Compiler Options
Przemyslaw Skibinski [Wed, 21 Dec 2016 10:43:11 +0000 (11:43 +0100)]
fixed Visual Studio compilation
Yann Collet [Wed, 21 Dec 2016 10:23:04 +0000 (11:23 +0100)]
Merge pull request #492 from inikep/dev11
test CMake on Windows
Yann Collet [Wed, 21 Dec 2016 10:22:28 +0000 (11:22 +0100)]
Merge pull request #490 from jacquesg/cmake-fix
Fix cmake build (MSVC)
Przemyslaw Skibinski [Wed, 21 Dec 2016 10:18:45 +0000 (11:18 +0100)]
improved MinGW support
Yann Collet [Wed, 21 Dec 2016 08:33:48 +0000 (09:33 +0100)]
Merge pull request #494 from terrelln/msan-dict-load
Fix dictionary loading bug causing an MSAN failure
Przemyslaw Skibinski [Wed, 21 Dec 2016 08:26:00 +0000 (09:26 +0100)]
fix basic types redefinition
Przemyslaw Skibinski [Wed, 21 Dec 2016 08:19:15 +0000 (09:19 +0100)]
simplified zstdcli.c
Przemyslaw Skibinski [Wed, 21 Dec 2016 08:06:14 +0000 (09:06 +0100)]
executables use new util.h and platform.h
Przemyslaw Skibinski [Wed, 21 Dec 2016 08:04:59 +0000 (09:04 +0100)]
improved util.h and platform.h
Nick Terrell [Tue, 20 Dec 2016 19:13:45 +0000 (11:13 -0800)]
Add test for invalid offset rep codes
Nick Terrell [Tue, 20 Dec 2016 18:47:52 +0000 (10:47 -0800)]
Fix dictionary loading bug causing an MSAN failure
Offset rep codes must be in the range `[1, dictSize)`.
Fix dictionary loading to reject `0` as a offset rep code.
Przemyslaw Skibinski [Tue, 20 Dec 2016 10:21:26 +0000 (11:21 +0100)]
windres updated to v1.1.3