]>
git.ipfire.org Git - thirdparty/zstd.git/log
Yann Collet [Wed, 14 Nov 2018 00:04:35 +0000 (16:04 -0800)]
circumvent gcc-8 strict aliasing warning
because gcc considers that
`void* const *` and `void**` are 2 different types ...
Yann Collet [Tue, 13 Nov 2018 22:15:12 +0000 (14:15 -0800)]
fixed decode-only test condition
Yann Collet [Tue, 13 Nov 2018 21:27:22 +0000 (13:27 -0800)]
fix debug trace
Yann Collet [Tue, 13 Nov 2018 21:22:16 +0000 (13:22 -0800)]
fixed VS2010+ build script
Yann Collet [Tue, 13 Nov 2018 21:16:41 +0000 (13:16 -0800)]
fixed cmake build script
Yann Collet [Tue, 13 Nov 2018 21:05:39 +0000 (13:05 -0800)]
changed benchfn api
to use structure for function parameters
as it expresses much clearer than a long list of parameters,
since each parameter can now be named.
Yann Collet [Tue, 13 Nov 2018 20:01:17 +0000 (12:01 -0800)]
benchfn can provided faulty return value
with BMK_extract_returnValue()
Yann Collet [Tue, 13 Nov 2018 19:01:59 +0000 (11:01 -0800)]
separated bench module into benchfn and benchzstd
it shall be possible to use benchfn
without any dependency on zstd.
Yann Collet [Tue, 13 Nov 2018 01:05:32 +0000 (17:05 -0800)]
changed PREFETCH() macro into PREFETCH_L2()
which is more accurate
Yann Collet [Fri, 9 Nov 2018 19:31:35 +0000 (11:31 -0800)]
Merge pull request #1411 from facebook/prefetch_dict
Improves decompression speed when using cold dictionary
Yann Collet [Fri, 9 Nov 2018 02:41:58 +0000 (18:41 -0800)]
Merge pull request #1410 from facebook/prefetch_dec
improve long-range decoder speed
Yann Collet [Fri, 9 Nov 2018 01:00:23 +0000 (17:00 -0800)]
Improves decompression speed when using cold dictionary
by triggering the prefetching decoder path
(which used to be dedicated to long-range offsets only).
Figures on my laptop :
no content prefetch : ~300 MB/s (for reference)
full content prefetch : ~325 MB/s (before this patch)
new prefetch path : ~375 MB/s (after this patch)
The benchmark speed is already significant,
but another side-effect is that this version
prefetch less data into memory,
since it only prefetches what's needed, instead of the full dictionary.
This is supposed to help highly active environments
such as active databases,
that can't be properly measured in benchmark environment (too clean).
Also :
fixed the largeNbDict test program
which was working improperly when setting nbBlocks > nbFiles.
Yann Collet [Thu, 8 Nov 2018 21:43:01 +0000 (13:43 -0800)]
Merge pull request #1409 from terrelln/tsan-dict
Add TSAN dictionary tests
Yann Collet [Thu, 8 Nov 2018 20:57:34 +0000 (12:57 -0800)]
reduced assertion strength
one limit case can apparently be generated during fuzzer tests
Yann Collet [Thu, 8 Nov 2018 20:51:42 +0000 (12:51 -0800)]
added a simple test case for decode-only benchmark mode
as it was broken and no one noticed
Yann Collet [Thu, 8 Nov 2018 20:47:46 +0000 (12:47 -0800)]
improve long-range decoder speed
on enwik9 at level 22 (which is almost a worst case scenario),
speed improves by +7% on my laptop (415 -> 445 MB/s)
Yann Collet [Thu, 8 Nov 2018 20:36:39 +0000 (12:36 -0800)]
fixed decompression-only benchmark
Nick Terrell [Thu, 8 Nov 2018 18:59:01 +0000 (10:59 -0800)]
Run TSAN fuzztest
Nick Terrell [Thu, 8 Nov 2018 18:58:51 +0000 (10:58 -0800)]
Add multithreaded dictbuilder tests to fuzzer.c
Nick Terrell [Thu, 8 Nov 2018 18:45:53 +0000 (10:45 -0800)]
Signal before unlocking in pool.c
Yann Collet [Thu, 8 Nov 2018 18:43:40 +0000 (10:43 -0800)]
Merge pull request #1408 from szreder/dev
Prevent deadlock on malloc() failure.
Bartosz Szreder [Thu, 8 Nov 2018 09:29:31 +0000 (10:29 +0100)]
Prevent deadlock on malloc() failure.
Yann Collet [Tue, 6 Nov 2018 21:58:50 +0000 (13:58 -0800)]
Merge pull request #1405 from pitrou/allow_cmake_debug_builds
Allow creating debug builds with CMake
Antoine Pitrou [Tue, 6 Nov 2018 20:24:29 +0000 (21:24 +0100)]
Allow creating debug builds with CMake
Only set the build type to Release if the user didn't select any.
See discussion in https://github.com/facebook/zstd/pull/1278#issuecomment-
436327923
and below.
Yann Collet [Tue, 6 Nov 2018 19:53:20 +0000 (11:53 -0800)]
Merge pull request #1404 from facebook/T36302429
fixed T36302471
Yann Collet [Tue, 6 Nov 2018 01:57:30 +0000 (17:57 -0800)]
Merge pull request #1403 from facebook/assert_opt
just add an assert() in ZSTD_insertBtAndGetAllMatches()
Yann Collet [Tue, 6 Nov 2018 01:50:30 +0000 (17:50 -0800)]
fixed T36302429
Yann Collet [Tue, 30 Oct 2018 01:07:07 +0000 (18:07 -0700)]
just add an assert() in ZSTD_insertBtAndGetAllMatches()
to express a condition on ll0 .
May help static analyzer as in #1397
Yann Collet [Mon, 29 Oct 2018 23:28:09 +0000 (16:28 -0700)]
Merge pull request #1395 from facebook/decompressblock
created zstd_decompress_block module
Yann Collet [Mon, 29 Oct 2018 23:00:26 +0000 (16:00 -0700)]
Merge pull request #1398 from bmwiedemann/pgo
Fix building zstd-pgo
Yann Collet [Mon, 29 Oct 2018 22:03:57 +0000 (15:03 -0700)]
fixed a second memset() on NULL
not sure why it only triggers now,
this code has been around for a while.
Introduced a new error code : dstBuffer_null,
I couldn't express anything even remotely similar with existing error codes set.
Yann Collet [Mon, 29 Oct 2018 20:57:37 +0000 (13:57 -0700)]
fixed memcpy() on NULL warning
memcpy(NULL, src, 0) is undefined behavior.
Yann Collet [Mon, 29 Oct 2018 20:06:45 +0000 (13:06 -0700)]
Merge pull request #1396 from facebook/huf_refactor
refactor HUF_compress_internal for clarity
Bernhard M. Wiedemann [Mon, 29 Oct 2018 16:20:03 +0000 (17:20 +0100)]
pgo: also optimize compression
and clean *.o so that linking does not fail from code
that is partially augmented with coverage generation instructions
Bernhard M. Wiedemann [Mon, 29 Oct 2018 14:11:28 +0000 (15:11 +0100)]
zstd-pgo: do not make clean and zstd in parallel
otherwise, there might not be a ./zstd to run
Yann Collet [Fri, 26 Oct 2018 23:38:23 +0000 (16:38 -0700)]
added flag -Wc++-compat
for library and cli
Yann Collet [Fri, 26 Oct 2018 22:57:13 +0000 (15:57 -0700)]
renamed NEWS as CHANGELOG
to conform fb publication rules.
Yann Collet [Fri, 26 Oct 2018 22:51:51 +0000 (15:51 -0700)]
minor clarifications of history update rules
Yann Collet [Fri, 26 Oct 2018 22:29:07 +0000 (15:29 -0700)]
Merge pull request #1392 from coetry/dev
provide consistent spacing to enum field
Yann Collet [Fri, 26 Oct 2018 22:28:44 +0000 (15:28 -0700)]
Merge pull request #1394 from terrelln/circleci-packaging
Publish uncompressed, zstd, and gzipped packages
Yann Collet [Fri, 26 Oct 2018 22:25:01 +0000 (15:25 -0700)]
Merge branch 'dev' into huf_Refactor
Yann Collet [Fri, 26 Oct 2018 22:21:52 +0000 (15:21 -0700)]
added a few comments for clarifications
Yann Collet [Fri, 26 Oct 2018 22:00:00 +0000 (15:00 -0700)]
Merge branch 'dev' into decompressblock
Yann Collet [Fri, 26 Oct 2018 21:43:16 +0000 (14:43 -0700)]
Merge pull request #1390 from facebook/nullAsOutput
support decompressing an empty frame into NULL
Allen Hai [Fri, 26 Oct 2018 21:16:06 +0000 (16:16 -0500)]
vertically align code comment
Yann Collet [Fri, 26 Oct 2018 20:21:37 +0000 (13:21 -0700)]
refactor HUF_compress_internal for clarity
changed workspace parameter convention
to always provide workspaceSize,
so that size can be explicitly checked.
Also, use more enum to make the meaning of some parameters more explicit.
Nick Terrell [Fri, 26 Oct 2018 20:06:53 +0000 (13:06 -0700)]
Publish uncompressed, zstd, and gzipped packages
Yann Collet [Fri, 26 Oct 2018 18:25:40 +0000 (11:25 -0700)]
fixed VS2010 project
Yann Collet [Fri, 26 Oct 2018 17:39:49 +0000 (10:39 -0700)]
fixed : extraneous function prototype in fullbench.c
now provided through zstd_internal.h
Allen Hai [Thu, 25 Oct 2018 23:45:20 +0000 (18:45 -0500)]
provide consistent spacing to enum field
Yann Collet [Thu, 25 Oct 2018 23:28:41 +0000 (16:28 -0700)]
created zstd_decompress_block module
isolate all logic associated with block decompression
into its own module.
zstd_decompress is still in charge
of context creation/destruction,
frames, headers, streaming, special blocks, etc.
Compressed blocks themselves are now handled within zstd_decompress_block .
Yann Collet [Thu, 25 Oct 2018 17:20:40 +0000 (10:20 -0700)]
Merge pull request #1388 from facebook/ddict
separate DDict logic into its own module
Yann Collet [Wed, 24 Oct 2018 23:50:03 +0000 (16:50 -0700)]
added comment on public ddict functions
Yann Collet [Wed, 24 Oct 2018 23:34:35 +0000 (16:34 -0700)]
support decompressing an empty frame into NULL
fix #1385
decompressing into NULL was an automatic error.
It is now allowed, as long as the content of the frame is empty.
Seems to simplify things for `arrow`.
Maybe some other projects rely on this behavior ?
Yann Collet [Wed, 24 Oct 2018 18:32:09 +0000 (11:32 -0700)]
fix and refactored libzstd_partial_build.sh
make the test intention clearer
(easier to maintain)
also resist the existence of zstd_ddict.o within libzstd.a
Yann Collet [Wed, 24 Oct 2018 17:36:06 +0000 (10:36 -0700)]
fixed warnings in testpools
Yann Collet [Wed, 24 Oct 2018 00:55:23 +0000 (17:55 -0700)]
added simple guard macros
in case of accidental multi-includes
Yann Collet [Wed, 24 Oct 2018 00:44:43 +0000 (17:44 -0700)]
fixed VS2010 build script
Yann Collet [Wed, 24 Oct 2018 00:40:15 +0000 (17:40 -0700)]
restored cmake build tests in Travis
for some reason, these tests were no longer played on Travis.
As a consequence, I discovered some left-over from the util.c change while testing.
Yann Collet [Wed, 24 Oct 2018 00:25:49 +0000 (17:25 -0700)]
separate DDict logic into its own module
created zstd_ddict.c within lib/decompress
Yann Collet [Tue, 23 Oct 2018 21:34:58 +0000 (14:34 -0700)]
Merge pull request #1384 from ulikunitz/dev
Clarify special case of offset history update
Ulrich Kunitz [Tue, 23 Oct 2018 06:43:19 +0000 (08:43 +0200)]
Reverted removal of a trailing space.
My editor removes trailing spaces while saving. Not confusing things I
reverted that change.
Ulrich Kunitz [Tue, 23 Oct 2018 06:36:50 +0000 (08:36 +0200)]
Fixed a typo
I fixed a typo in the last commit. Many thanks to @terrelin for pointing
that out.
Ulrich Kunitz [Mon, 22 Oct 2018 21:46:43 +0000 (23:46 +0200)]
Clarify special case of offset history update
If the current sequence has literal length of zero then an offset value
of three is handled in a special manner. While I implemented a golang
decoder I had to consult the educational decoder for clarification on
the update of the offset history in that case. This commit provides the
clarification that the offset value Repeated_Offset1-1 is handled as a
new offset is added to the offset history accordingly.
Yann Collet [Mon, 22 Oct 2018 21:04:15 +0000 (14:04 -0700)]
Merge pull request #1383 from facebook/uninstall
fix : removed zstdmt symlink on uninstall
Yann Collet [Mon, 22 Oct 2018 19:51:56 +0000 (12:51 -0700)]
fix : removed zstdmt symlink on uninstall
Yann Collet [Mon, 22 Oct 2018 18:57:12 +0000 (11:57 -0700)]
Merge pull request #1380 from facebook/decodecorpus
fix decodecorpus incorrect frame generation
Nick Terrell [Mon, 22 Oct 2018 17:48:56 +0000 (10:48 -0700)]
Merge pull request #1382 from jonesz/ticket-compiler-rt-fuzzer
Fix libFuzzer location in makefile.
Ethan Jones [Mon, 22 Oct 2018 16:19:13 +0000 (11:19 -0500)]
Fix libFuzzer location in makefile.
libFuzzer was moved into compiler-rt, update the repo location
accordingly.
Yann Collet [Sun, 21 Oct 2018 01:53:02 +0000 (18:53 -0700)]
fix decodecorpus incorrect frame generation
fix #1379
decodecorpus was generating one extraneous byte when `nbSeq==0`.
This is disallowed by the specification.
The reference decoder was just skipping the extraneous byte.
It is now stricter, and flag such situation as an error.
Yann Collet [Fri, 19 Oct 2018 23:31:10 +0000 (16:31 -0700)]
Merge pull request #1368 from rkjain89/test-branch
Moving Code To util.c
Rohit Jain [Wed, 17 Oct 2018 06:33:17 +0000 (23:33 -0700)]
Made the changes in zstd vcproj file for VS2008 as requested
Yann Collet [Wed, 17 Oct 2018 03:58:52 +0000 (20:58 -0700)]
minor NEWS update for v1.3.7
Yann Collet [Tue, 16 Oct 2018 17:09:33 +0000 (10:09 -0700)]
Merge pull request #1372 from samrussell/dev
Add man pages for zstdgrep and zstdless
Sam Russell [Mon, 15 Oct 2018 19:42:02 +0000 (08:42 +1300)]
fix up zgrep/zless references
Sam Russell [Mon, 15 Oct 2018 09:20:10 +0000 (22:20 +1300)]
Add man pages for zstdgrep and zstdless
Rohit Jain [Sat, 13 Oct 2018 02:06:58 +0000 (19:06 -0700)]
Changing tests/fuzz/Makefile to move util.o to FUZZ_SRC instead
Rohit Jain [Fri, 12 Oct 2018 21:22:05 +0000 (14:22 -0700)]
Adding util.c to zstd VS file
Rohit Jain [Fri, 12 Oct 2018 21:08:13 +0000 (14:08 -0700)]
Adding chagnes for cmake and VStudio
Rohit Jain [Fri, 12 Oct 2018 18:36:02 +0000 (11:36 -0700)]
Fixing poolTests, I will understand about cmake and VStudio and push them in a later commit
Rohit Jain [Fri, 12 Oct 2018 01:05:15 +0000 (18:05 -0700)]
Address most comments
Rohit Jain [Fri, 12 Oct 2018 00:34:47 +0000 (17:34 -0700)]
Moving more function to the new C file
Rohit Jain [Fri, 12 Oct 2018 00:08:42 +0000 (17:08 -0700)]
Fixing regressiontest makefile
Rohit Jain [Thu, 11 Oct 2018 23:51:29 +0000 (16:51 -0700)]
Removing static and making extern
Yann Collet [Thu, 11 Oct 2018 23:49:55 +0000 (16:49 -0700)]
update NEWS for v1.3.7
Yann Collet [Thu, 11 Oct 2018 22:59:57 +0000 (15:59 -0700)]
Merge pull request #1369 from facebook/v137
bumped version number to v1.3.7
Rohit Jain [Thu, 11 Oct 2018 22:51:57 +0000 (15:51 -0700)]
Making changes to make it compile on my laptop
Rohit Jain [Thu, 11 Oct 2018 22:17:40 +0000 (15:17 -0700)]
Fixed makefile and removed multiple definitions from util.h (UTIL_getTime)
Rohit Jain [Thu, 11 Oct 2018 22:07:12 +0000 (15:07 -0700)]
Removed UTIL_setFileStat from util.h and move it to util.c
Yann Collet [Thu, 11 Oct 2018 21:58:00 +0000 (14:58 -0700)]
Merge pull request #1366 from facebook/fix_strncat
fixed minor gcc warning
Yann Collet [Thu, 11 Oct 2018 21:40:12 +0000 (14:40 -0700)]
bumped version number to v1.3.7
updated documentation
Rohit Jain [Thu, 11 Oct 2018 19:52:19 +0000 (12:52 -0700)]
Moving code to util.c
Yann Collet [Thu, 11 Oct 2018 19:49:25 +0000 (12:49 -0700)]
Merge pull request #1367 from atdt/dev
Enable use of bswap intrinsics in clang
Ori Livneh [Thu, 11 Oct 2018 02:32:03 +0000 (22:32 -0400)]
Enable use of bswap intrinsics in clang
Necessary because clang disguises itself as an older (__GNUC_MINOR__ = 2) GCC.
Yann Collet [Thu, 11 Oct 2018 02:16:35 +0000 (19:16 -0700)]
Merge pull request #1365 from facebook/bitspeed
Improved decompression speed
Yann Collet [Thu, 11 Oct 2018 01:26:44 +0000 (18:26 -0700)]
restored bitMask for shift values
since corrupted bitstreams can generate too large values.
This slightly reduces the benefits from clang on my laptop.
gcc results and code generation are not affected.
Yann Collet [Thu, 11 Oct 2018 00:33:04 +0000 (17:33 -0700)]
removed one assert()
that can be triggered by a corrupted bitstream.
Yann Collet [Thu, 11 Oct 2018 00:06:25 +0000 (17:06 -0700)]
fixed minor gcc warning
gcc-8 on Linux doesn't like usage of strncat :
`warning: ‘strncat’ output truncated before terminating nul copying as many bytes from a string as its length`.
Not sure what was wrong, it might be a false positive,
but the logic is simple enough to replaced by a simple `memcpy()`,
thus avoiding the shenanigans of null-terminated strings.
Yann Collet [Wed, 10 Oct 2018 23:36:11 +0000 (16:36 -0700)]
removed one assert()
which can be triggered when input is corrupted.
Yann Collet [Wed, 10 Oct 2018 22:48:43 +0000 (15:48 -0700)]
improved decompression speed
while reviewing #1364,
I found a decompression speed improvement.
On my laptop, the new code decompresses +5-6% faster on clang
and +2-3% faster on gcc.
not bad for an accidental optimization...