]>
git.ipfire.org Git - thirdparty/zstd.git/log
maxice8 [Sat, 6 Oct 2018 00:23:45 +0000 (21:23 -0300)]
don't assume __linux__ means __GLIBC__ on BACKTRACES_ENABLE
check for __GLIBC__ instead of __linux__ as musl libc doesn't provide
execinfo.h
fixes compilation on Alpine Linux and Void Linux musl arches.
Yann Collet [Fri, 5 Oct 2018 00:25:11 +0000 (17:25 -0700)]
fixed fullbench-lib target
Yann Collet [Thu, 4 Oct 2018 21:59:35 +0000 (14:59 -0700)]
Merge pull request #1351 from facebook/haikubuild
portability macro flags updates, for Haiku
Yann Collet [Thu, 4 Oct 2018 21:59:14 +0000 (14:59 -0700)]
Merge pull request #1353 from facebook/paramgrill
fixed paramgrill wrong assert() conditions
Yann Collet [Thu, 4 Oct 2018 21:39:35 +0000 (14:39 -0700)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Thu, 4 Oct 2018 21:39:11 +0000 (14:39 -0700)]
added graph for ZSTD_compress_usingCDict() in v1.3.6
Yann Collet [Thu, 4 Oct 2018 21:30:02 +0000 (14:30 -0700)]
Merge pull request #1352 from facebook/visual
fixed VS2017Community build script
Yann Collet [Thu, 4 Oct 2018 21:27:13 +0000 (14:27 -0700)]
fixed paramgrill wrong assert() conditions
and slightly refactored affected function.
Honestly, the formula calculating variance should get a second reviewing round,
it's not clear if it's correct.
Yann Collet [Thu, 4 Oct 2018 01:14:35 +0000 (18:14 -0700)]
fixed VS2017Community build script
reported by @epicabsol
Yann Collet [Wed, 3 Oct 2018 23:37:50 +0000 (16:37 -0700)]
finalized minor warnings on Haiku
Yann Collet [Wed, 3 Oct 2018 22:34:41 +0000 (15:34 -0700)]
changed ZSTD_NANOSLEEP_SUPPORT definition
to please `-Wexpansion-to-defined`
Yann Collet [Wed, 3 Oct 2018 21:54:33 +0000 (14:54 -0700)]
portability macro flags updates, for Haiku
some non-trivial changes to platform.h and util.h,
initially related to compilation for Haiku,
but I used this opportunity to make them cleaner
and add some documentation.
Noticed several tests that could be improved
(too harsh conditions, useless exception, etc.)
but I did not dare modifying too many tests just before release.
Yann Collet [Wed, 3 Oct 2018 20:36:00 +0000 (13:36 -0700)]
Merge pull request #1330 from korli/haiku
Enable building zstd on Haiku.
Yann Collet [Wed, 3 Oct 2018 19:43:59 +0000 (12:43 -0700)]
fixed wrong assert() position
could fire on invalid input.
blocking for afl tests.
Yann Collet [Wed, 3 Oct 2018 19:01:45 +0000 (12:01 -0700)]
Merge pull request #1350 from facebook/fixmsan
fixed msan warning
Jerome Duval [Sun, 6 Aug 2017 20:27:54 +0000 (22:27 +0200)]
Enable building zstd on Haiku.
Yann Collet [Wed, 3 Oct 2018 01:20:20 +0000 (18:20 -0700)]
fixed msan warning
on btlazy2 strategy with dictAttach
Yann Collet [Tue, 2 Oct 2018 23:37:58 +0000 (16:37 -0700)]
Merge pull request #1348 from facebook/donotdelete
Fix #1082
Yann Collet [Tue, 2 Oct 2018 22:59:11 +0000 (15:59 -0700)]
fixed static analyzer warnings
note : for some reason,
scan-build version on my laptop found problems within fastcover.c
that scan-build on travisCI does not flag.
They are, as usual, false positive :
the analyzer does not understand that a table (`offset`) is correctly filled before usage.
Yann Collet [Tue, 2 Oct 2018 00:50:16 +0000 (17:50 -0700)]
restored backtrace on failure
for Linux and Mac OS-X.
Note : the backtraces fires up through a trap
before the sanitizer get a chance to report.
There are situations where the sanitizer report is actually preferable.
It might be good to consider a kind of build macro
which can disable backtrace
when sanitizer is enabled.
Yann Collet [Tue, 2 Oct 2018 00:49:18 +0000 (17:49 -0700)]
updated NEWS in anticipation for v1.3.6
Yann Collet [Tue, 2 Oct 2018 00:20:12 +0000 (17:20 -0700)]
Merge pull request #1317 from felixhandte/split-logs
Independent Dictionary and Working Context Table Logs
Yann Collet [Tue, 2 Oct 2018 00:16:34 +0000 (17:16 -0700)]
./zstd -f do no longer overwrite destination file
if source file does not exist (#1082)
Yann Collet [Mon, 1 Oct 2018 21:04:00 +0000 (14:04 -0700)]
zstd -d -f do no longer erase destination file
when source file does not exist (#1082)
W. Felix Handte [Mon, 1 Oct 2018 20:28:13 +0000 (13:28 -0700)]
Revert Ability to Set HashLog and ChainLog on Context When Dict is Attached
This capability is not needed / used in the current unit of work. I'll
re-introduce it later, when we start allowing users to override the deduced
working context logs.
Yann Collet [Sat, 29 Sep 2018 01:19:23 +0000 (18:19 -0700)]
regroup name creation logic into its own function
for a cleaner main file decompression loop
Yann Collet [Sat, 29 Sep 2018 00:34:49 +0000 (17:34 -0700)]
Merge pull request #1347 from terrelln/werror
Add -Werror to *build rules
Yann Collet [Sat, 29 Sep 2018 00:34:27 +0000 (17:34 -0700)]
Merge pull request #1346 from terrelln/small-block-fse
[zstd] Fix resetting FSE tables
W. Felix Handte [Wed, 26 Sep 2018 17:56:28 +0000 (10:56 -0700)]
Restore Passing CParams to `ZSTD_insertAndFindFirstIndex_internal`
W. Felix Handte [Thu, 13 Sep 2018 18:13:08 +0000 (11:13 -0700)]
Use Working Ctx Logs when not in DMS Mode
We pre-hash the ptr for the dict match state sometimes. When that actually
happens, a hashlog of 0 can produce undefined behavior (right shift a long
long by 64). Only applies to unoptimized compilations, since when
optimizations are applied, those hash operations are dropped when we're not
actually in dms mode.
W. Felix Handte [Wed, 12 Sep 2018 19:29:53 +0000 (12:29 -0700)]
When Attaching Dictionary, Size Working Tables Based on Input Size Only
W. Felix Handte [Wed, 12 Sep 2018 19:28:53 +0000 (12:28 -0700)]
Remove Log Overriding for the Time Being
W. Felix Handte [Wed, 12 Sep 2018 19:04:04 +0000 (12:04 -0700)]
Remove Strategy-Dependency in Making Attachment Decision
W. Felix Handte [Mon, 27 Aug 2018 23:37:04 +0000 (16:37 -0700)]
Support Split Logs in ZSTD_btopt..ZSTD_btultra
W. Felix Handte [Mon, 27 Aug 2018 23:36:52 +0000 (16:36 -0700)]
Bump Split Log Support to ZSTD_btultra
W. Felix Handte [Mon, 27 Aug 2018 23:19:41 +0000 (16:19 -0700)]
Support Split Logs in ZSTD_greedy..ZSTD_btlazy2
W. Felix Handte [Mon, 27 Aug 2018 23:19:24 +0000 (16:19 -0700)]
Bump Split Log Support to ZSTD_btlazy2
W. Felix Handte [Mon, 27 Aug 2018 22:23:51 +0000 (15:23 -0700)]
Support Split Logs in ZSTD_dfast
W. Felix Handte [Mon, 27 Aug 2018 22:23:27 +0000 (15:23 -0700)]
Bump Split Log Support to ZSTD_dfast
W. Felix Handte [Thu, 23 Aug 2018 21:49:09 +0000 (14:49 -0700)]
Support a Separate Hash Log in ZSTD_fast
W. Felix Handte [Thu, 23 Aug 2018 21:21:35 +0000 (14:21 -0700)]
Stop Passing in `hashLog` and `stepSize` to `ZSTD_compressBlock_fast_generic`
W. Felix Handte [Mon, 27 Aug 2018 22:09:56 +0000 (15:09 -0700)]
Temporary Commit to Retain Requested Hash and Chain Logs During Dict Attach
W. Felix Handte [Mon, 27 Aug 2018 22:09:25 +0000 (15:09 -0700)]
Allow Setting Hash and Chain Logs on Contexts with Attached CDict
W. Felix Handte [Mon, 27 Aug 2018 20:21:01 +0000 (13:21 -0700)]
For Supported Strategies, Attach Dict Even When Params Don't Match
W. Felix Handte [Fri, 24 Aug 2018 22:54:58 +0000 (15:54 -0700)]
Split Attach and Copy Reset Strategies into Separate Implementation Functions
W. Felix Handte [Fri, 24 Aug 2018 20:13:48 +0000 (13:13 -0700)]
Pull Attachment Decision into Separate Function
W. Felix Handte [Thu, 23 Aug 2018 21:36:51 +0000 (14:36 -0700)]
And Then Avoid the Unused Parameter Warning
W. Felix Handte [Thu, 23 Aug 2018 21:09:18 +0000 (14:09 -0700)]
Move Asserts into Function to Avoid Unused Function Warning
W. Felix Handte [Thu, 23 Aug 2018 19:53:16 +0000 (12:53 -0700)]
Move Match State CParams Setting into `resetCCtx` and `continueCCtx`
W. Felix Handte [Thu, 23 Aug 2018 19:35:03 +0000 (12:35 -0700)]
Strengthen Assertion to Assert Equality
W. Felix Handte [Thu, 23 Aug 2018 19:21:27 +0000 (12:21 -0700)]
Remove CParams Arg from ZSTD_ldm_blockCompress
W. Felix Handte [Thu, 23 Aug 2018 19:17:58 +0000 (12:17 -0700)]
Stop Separately Passing CParams in ZSTD_lazy Internal Functions
W. Felix Handte [Thu, 23 Aug 2018 19:11:20 +0000 (12:11 -0700)]
Stop Separately Passing CParams in ZSTD_opt Internal Functions
W. Felix Handte [Thu, 23 Aug 2018 19:08:03 +0000 (12:08 -0700)]
Also Remove CParams from Table Filling Functions' Args
W. Felix Handte [Thu, 23 Aug 2018 18:57:54 +0000 (11:57 -0700)]
Also Assert Equivalency When Filling MatchState with Prefix
W. Felix Handte [Thu, 23 Aug 2018 18:53:34 +0000 (11:53 -0700)]
Remove CParams from Block Compressor Functions' Args
W. Felix Handte [Thu, 23 Aug 2018 18:38:35 +0000 (11:38 -0700)]
Assert `ctx` and `ms` cparams Equivalency
W. Felix Handte [Thu, 23 Aug 2018 18:33:08 +0000 (11:33 -0700)]
Remove cParams from CDict
W. Felix Handte [Thu, 23 Aug 2018 18:32:32 +0000 (11:32 -0700)]
Add ZSTD_compressionParameters to ZSTD_matchState_t
Yann Collet [Fri, 28 Sep 2018 23:04:00 +0000 (16:04 -0700)]
changed macro name
from EXIT_IF() to RETURN_IF()
EXIT could be misunderstood as exit(), which terminates program execution.
But the macro only leaves the function, not the program.
Yann Collet [Fri, 28 Sep 2018 22:57:35 +0000 (15:57 -0700)]
minor improvement in the multi-format suffix selection
Nick Terrell [Fri, 28 Sep 2018 22:28:34 +0000 (15:28 -0700)]
Fix minigzip in std=c99 mode
Nick Terrell [Fri, 28 Sep 2018 22:08:15 +0000 (15:08 -0700)]
[util] Fix lstat feature test macro
Nick Terrell [Fri, 28 Sep 2018 21:59:40 +0000 (14:59 -0700)]
Add -Werror to *build rules
Nick Terrell [Fri, 28 Sep 2018 21:24:38 +0000 (14:24 -0700)]
Fix another warning
Nick Terrell [Fri, 28 Sep 2018 19:14:24 +0000 (12:14 -0700)]
Fix warnings
Nick Terrell [Fri, 28 Sep 2018 19:09:28 +0000 (12:09 -0700)]
[zstd] Fix newly added test case
Nick Terrell [Fri, 28 Sep 2018 19:09:14 +0000 (12:09 -0700)]
[zstreamtest] Add failing test case
Yann Collet [Fri, 28 Sep 2018 16:34:16 +0000 (09:34 -0700)]
fixed unreachable section warning on Visual
Nick Terrell [Fri, 28 Sep 2018 02:29:02 +0000 (19:29 -0700)]
Merge pull request #1343 from terrelln/faster-ci
[zstreamtest] Reduce memory of newapi tests
Yann Collet [Fri, 28 Sep 2018 02:23:55 +0000 (19:23 -0700)]
Merge branch 'dev' into donotdelete
Yann Collet [Fri, 28 Sep 2018 02:23:27 +0000 (19:23 -0700)]
Merge pull request #1345 from facebook/fixLegacy
fixed legacy compilation tests
Yann Collet [Fri, 28 Sep 2018 01:29:15 +0000 (18:29 -0700)]
fixed minor Visual conversion warnings
Yann Collet [Fri, 28 Sep 2018 01:24:41 +0000 (18:24 -0700)]
fixed longmatch test too
Yann Collet [Fri, 28 Sep 2018 01:15:14 +0000 (18:15 -0700)]
fixed legacy compilation tests
for some reason, these tests started failing recently on CircleCI
Yann Collet [Fri, 28 Sep 2018 00:36:54 +0000 (17:36 -0700)]
Merge pull request #1344 from terrelln/grub-fixes
Fixes for GRUB
Yann Collet [Thu, 27 Sep 2018 23:49:08 +0000 (16:49 -0700)]
minor refactoring of --list
trying to reduce recurrent patterns.
Nick Terrell [Thu, 27 Sep 2018 23:06:02 +0000 (16:06 -0700)]
Restore ZSTD_noCompressBlock() for clarity
Nick Terrell [Thu, 27 Sep 2018 22:49:31 +0000 (15:49 -0700)]
Reset number of threads less often
Nick Terrell [Thu, 27 Sep 2018 22:23:20 +0000 (15:23 -0700)]
Change byte to value to avoid a GRUB typedef
Nick Terrell [Thu, 27 Sep 2018 22:22:34 +0000 (15:22 -0700)]
Include stddef.h for size_t
Nick Terrell [Thu, 27 Sep 2018 22:13:43 +0000 (15:13 -0700)]
[zstd] Add -Wmissing-prototypes
Nick Terrell [Thu, 27 Sep 2018 20:55:24 +0000 (13:55 -0700)]
More aggressive limitations
Nick Terrell [Thu, 27 Sep 2018 19:48:29 +0000 (12:48 -0700)]
[zstreamtest] Reduce memory of newapi tests
We could allocate up to 2^28 bytes of memory when using 2 threads with
window log = 24. Now, we limit it to 2^26 bytes of memory when not running
big tests.
I chose max window log = 22 since that is the maximum source size when
big tests are disabled. Hopefully this will be enough to reduce or
eliminate the test failures.
Yann Collet [Thu, 27 Sep 2018 17:20:14 +0000 (10:20 -0700)]
Merge pull request #1342 from facebook/fixcatyd
fix : huge (>4GB) chain of blocks
Yann Collet [Thu, 27 Sep 2018 01:06:53 +0000 (18:06 -0700)]
moved again overflow correction
cannot work from within ZSTD_compressBlock()
Yann Collet [Wed, 26 Sep 2018 22:52:23 +0000 (15:52 -0700)]
Merge pull request #1340 from terrelln/generate-artifacts
Publish artifacts with CircleCI
Yann Collet [Wed, 26 Sep 2018 22:35:38 +0000 (15:35 -0700)]
changed overflow correction place
keep one in compress_frameChunk(),
so that it's tested at every loop
in case some user simply some large mulit-GB input in a single invocation.
Add one in ZSTD_compressBlock(),
since compressBlock() explicitly skips frameChunk().
Yann Collet [Wed, 26 Sep 2018 21:39:20 +0000 (14:39 -0700)]
Merge pull request #1339 from facebook/grep_colors
fixed usage of grep in Makefile
Yann Collet [Wed, 26 Sep 2018 21:39:08 +0000 (14:39 -0700)]
Merge pull request #1327 from facebook/adapt
Adaptive compression
Yann Collet [Wed, 26 Sep 2018 21:24:28 +0000 (14:24 -0700)]
fix : huge (>4GB) stream of blocks
experimental function ZSTD_compressBlock() is designed for very small data in mind,
for situation where saving the ~12 bytes of frame header can actually make a difference.
Some systems though may have to deal with small and large data entangled.
If it's larger than a block (> 128KB), compressBlock() cannot compress them in one round.
That's why it's possible to compress in multiple rounds.
This is a chain of compressed blocks.
Some users push this capability to the limit, encoding gigantic chain of blocks.
On crossing the 4GB limit, some internal overflow occurs.
This fix moves the overflow correction mechanism higher in the call chain,
so that it's applied also to gigantic chains of blocks.
Added a test case in fuzzer.c, which crashes before the fix, and pass now.
Yann Collet [Wed, 26 Sep 2018 20:40:35 +0000 (13:40 -0700)]
Merge pull request #1338 from facebook/rfc
updated format documentation
Yann Collet [Wed, 26 Sep 2018 20:40:20 +0000 (13:40 -0700)]
Merge pull request #1337 from facebook/test_failure
fixed `!` tests
Yann Collet [Wed, 26 Sep 2018 20:39:05 +0000 (13:39 -0700)]
Merge pull request #1329 from facebook/v04isout
Changed default legacy support to v0.5+
Nick Terrell [Tue, 25 Sep 2018 23:10:17 +0000 (16:10 -0700)]
Publish artifacts with CircleCI
* Updates CircleCI to use workflows.
We can now specify any number of test jobs to run in parallel.
* Switch the image to `buildpack-deps:trusty` which is only 500 MB
instead of 7 GB, so that saves 7 minutes to download it if it isn't
already cached on the host.
* Publish the source tarball and sha256sum as artifacts.
* If the `GITHUB_TOKEN` environment variable is set, we will also
add the tarball + sha256sum to the tagged release, after manual
approval.
Yann Collet [Tue, 25 Sep 2018 23:56:53 +0000 (16:56 -0700)]
fixed usage of grep in Makefile
when terminal uses colors
as suggested by @danielshir (#1294)
Yann Collet [Tue, 25 Sep 2018 23:34:26 +0000 (16:34 -0700)]
updated format documentation
to match last edits of RFC8478.
Yann Collet [Tue, 25 Sep 2018 01:16:08 +0000 (18:16 -0700)]
bounds for --adapt mode
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
Yann Collet [Tue, 25 Sep 2018 22:57:28 +0000 (15:57 -0700)]
fixed `!` tests
Sometimes, it's necessary to test that a certain command fail, as expected.
Such failure is actually a success, and must not stop the flow of tests.
Several tests were prefixed with `!` to invert return code.
This does not work : it effectively makes the tests pass no matter what.
Use instead function die(), which is meant to trap successes, and transform them into errors.
Yann Collet [Mon, 24 Sep 2018 23:56:45 +0000 (16:56 -0700)]
Merge branch 'dev' into adapt