]>
git.ipfire.org Git - thirdparty/zstd.git/log
dependabot[bot] [Mon, 11 Nov 2024 05:27:06 +0000 (05:27 +0000)]
Bump github/codeql-action from 3.26.2 to 3.27.1
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.2 to 3.27.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/
429e1977040da7a23b6822b13c129cd1ba93dbb2 ...
4f3212b61783c3c68e8309a0f18a699764811cda )
---
updated-dependencies:
- dependency-name: github/codeql-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Yann Collet [Sun, 10 Nov 2024 20:34:05 +0000 (12:34 -0800)]
Merge pull request #4188 from facebook/qnx_include
blind fix for QNX
Yann Collet [Tue, 5 Nov 2024 08:09:13 +0000 (00:09 -0800)]
blind fix for QNX
following notification from @rainbowball.
fix #4186.
Note: there is currently no QNX compilation test in CI
so this is a "blind" fix,
and this target can be silently broken again in the future.
Yann Collet [Tue, 5 Nov 2024 05:55:56 +0000 (21:55 -0800)]
Merge pull request #4184 from facebook/ZSTD_getErrorCode
elevated ZSTD_getErrorCode() to stable status
Yann Collet [Thu, 31 Oct 2024 22:06:48 +0000 (15:06 -0700)]
inform manual users that it's automatically generated
suggested by @Eugeny1
Yann Collet [Thu, 31 Oct 2024 21:38:36 +0000 (14:38 -0700)]
fixed single file library test
copy "zstd_errors.h" alongside "zstd.h" to pass compilation test.
Yann Collet [Thu, 31 Oct 2024 21:04:11 +0000 (14:04 -0700)]
elevated ZSTD_getErrorCode() to stable status
answering #4183
Yann Collet [Thu, 31 Oct 2024 21:15:29 +0000 (14:15 -0700)]
Merge pull request #4180 from facebook/split_param
Block splitter control parameter
Yann Collet [Thu, 31 Oct 2024 20:43:40 +0000 (13:43 -0700)]
change experimental parameter name
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
Yann Collet [Tue, 29 Oct 2024 18:04:11 +0000 (11:04 -0700)]
changed variable name to ZSTD_c_blockSplitterLevel
suggested by @terrelln
Yann Collet [Mon, 28 Oct 2024 23:57:01 +0000 (16:57 -0700)]
removed trace left over
Yann Collet [Mon, 28 Oct 2024 23:47:38 +0000 (16:47 -0700)]
fixed minor conversion warning
Yann Collet [Sat, 26 Oct 2024 03:29:15 +0000 (20:29 -0700)]
added a test
test both that the new parameter works as intended,
and that the over-split protection works as intended
Yann Collet [Fri, 25 Oct 2024 23:08:22 +0000 (16:08 -0700)]
expose new parameter ZSTD_c_blockSplitter_level
Yann Collet [Fri, 25 Oct 2024 22:51:03 +0000 (15:51 -0700)]
add internal compression parameter preBlockSplitter_level
not yet exposed to the interface.
Also: renames `useBlockSplitter` to `postBlockSplitter`
to better qualify the difference between the 2 settings.
Yann Collet [Mon, 28 Oct 2024 23:30:49 +0000 (16:30 -0700)]
Merge pull request #4178 from facebook/split_fromBorders
Add fastest block-splitter variant
Yann Collet [Fri, 25 Oct 2024 23:33:00 +0000 (16:33 -0700)]
update regression results
Yann Collet [Fri, 25 Oct 2024 23:25:02 +0000 (16:25 -0700)]
update ZSTD_splitBlock() documentation
Yann Collet [Thu, 24 Oct 2024 22:57:29 +0000 (15:57 -0700)]
new block splitting variant _fromBorders
less precise but still suitable for `fast` strategy.
Yann Collet [Fri, 25 Oct 2024 23:13:17 +0000 (16:13 -0700)]
Merge pull request #4176 from facebook/sample11
Added faster block splitter variants for levels 3-7
Yann Collet [Thu, 24 Oct 2024 21:47:55 +0000 (14:47 -0700)]
update regression results
Yann Collet [Thu, 24 Oct 2024 21:37:00 +0000 (14:37 -0700)]
add faster block splitting heuristic, suitable for dfast strategy
Yann Collet [Thu, 24 Oct 2024 20:38:12 +0000 (13:38 -0700)]
fix minor visual conversion warning
Yann Collet [Thu, 24 Oct 2024 20:27:01 +0000 (13:27 -0700)]
added block splitter variant for greedy & lazy (levels 5 to 7)
Yann Collet [Thu, 24 Oct 2024 03:20:32 +0000 (20:20 -0700)]
complete sample11 with reduced fingerprint size
Yann Collet [Thu, 24 Oct 2024 02:59:03 +0000 (19:59 -0700)]
reduce splitBlock arguments
Yann Collet [Wed, 23 Oct 2024 23:43:44 +0000 (16:43 -0700)]
organize specialization at recordFingerprint level
Yann Collet [Wed, 23 Oct 2024 18:57:23 +0000 (11:57 -0700)]
new variant, sampling by 11
Yann Collet [Thu, 24 Oct 2024 20:17:36 +0000 (13:17 -0700)]
Merge pull request #4136 from facebook/preSplit
Block splitter
Yann Collet [Thu, 24 Oct 2024 18:43:16 +0000 (11:43 -0700)]
update regression results
first block is no longer splitted since adding the @savings over-split protection
Yann Collet [Thu, 24 Oct 2024 18:36:56 +0000 (11:36 -0700)]
apply limit conditions for all splitting strategies
instead of just for blind split.
This is in anticipation of adversarial input,
that would intentionally target the sampling pattern of the split detector.
Note that, even without this protection, splitting can never expand beyond ZSTD_COMPRESSBOUND(),
because this upper limit uses a 1KB block size worst case scenario,
and splitting never creates blocks thath small.
The protection is more to ensure that data is not expanded by more than 3-bytes per 128 KB full block,
which is a much stricter limit.
Yann Collet [Wed, 23 Oct 2024 22:56:56 +0000 (15:56 -0700)]
update regression results
Yann Collet [Wed, 23 Oct 2024 21:55:10 +0000 (14:55 -0700)]
stricter limits to ensure expansion factor with blind-split strategy
issue reported by @terrelln
Yann Collet [Wed, 23 Oct 2024 21:11:49 +0000 (14:11 -0700)]
split all full 128 KB blocks
this helps make the streaming behavior more consistent,
since it does no longer depend on having more data presented on the input.
suggested by @terrelln
Yann Collet [Wed, 23 Oct 2024 18:10:07 +0000 (11:10 -0700)]
rewrite fingerprint storage to no longer need 64-bit members
so that it can be stored using standard alignment requirement (sizeof(void*)).
Distance function still requires 64-bit signed multiplication though,
so it won't change the issue regarding the bug in ubsan for clang 32-bit on github ci.
Yann Collet [Wed, 23 Oct 2024 04:49:35 +0000 (21:49 -0700)]
fixed minor strict pedantic C90 issue
Yann Collet [Tue, 22 Oct 2024 23:51:52 +0000 (16:51 -0700)]
fixed extraneous return
strict C90 compliance test
Yann Collet [Tue, 22 Oct 2024 23:33:55 +0000 (16:33 -0700)]
updated compression results
due to integration of `sample5` strategy, leading to better compression ratios on a range of levels
Yann Collet [Tue, 22 Oct 2024 23:25:44 +0000 (16:25 -0700)]
fixes static state allocation check
detected by @felixhandte
Yann Collet [Tue, 22 Oct 2024 23:22:17 +0000 (16:22 -0700)]
rewrote ZSTD_cwksp_initialAllocStart() to be easier to read
following a discussion with @felixhandte
Yann Collet [Tue, 22 Oct 2024 23:06:07 +0000 (16:06 -0700)]
rewrite penalty update
suggested by @terrelln
Yann Collet [Tue, 22 Oct 2024 23:03:19 +0000 (16:03 -0700)]
changed loop exit condition so that there is no need to assert() within the loop.
Yann Collet [Tue, 22 Oct 2024 22:32:11 +0000 (15:32 -0700)]
renamed: FingerPrint => Fingerprint
suggested by @terrelln
Yann Collet [Tue, 22 Oct 2024 22:25:45 +0000 (15:25 -0700)]
conservatively estimate over-splitting in presence of incompressible loss
ensure data can never be expanded by more than 3 bytes per full block.
Yann Collet [Tue, 22 Oct 2024 22:19:06 +0000 (15:19 -0700)]
ensure `lastBlock` is correctly determined
reported by @terrelln
Yann Collet [Tue, 22 Oct 2024 22:12:46 +0000 (15:12 -0700)]
made ZSTD_isPower2() an inline function
Yann Collet [Mon, 21 Oct 2024 21:56:43 +0000 (14:56 -0700)]
added a faster block splitter variant
that samples 1 in 5 positions.
This variant is fast enough for lazy2 and btlazy2,
but it's less good in combination with post-splitter at higher levels (>= btopt).
Yann Collet [Mon, 21 Oct 2024 00:16:17 +0000 (17:16 -0700)]
minor split optimization
let's fill the initial stats directly into target fingerprint
Yann Collet [Mon, 21 Oct 2024 00:07:43 +0000 (17:07 -0700)]
updated regression test results
Yann Collet [Fri, 18 Oct 2024 23:47:41 +0000 (16:47 -0700)]
fixed workspace alignment on non 64-bit systems
Yann Collet [Fri, 18 Oct 2024 23:00:36 +0000 (16:00 -0700)]
splitter workspace is now provided by ZSTD_CCtx*
Yann Collet [Fri, 18 Oct 2024 18:20:54 +0000 (11:20 -0700)]
fix alignment test
for non 64-bit systems
Yann Collet [Fri, 18 Oct 2024 18:16:49 +0000 (11:16 -0700)]
fixed minor conversion warnings on Visual
Yann Collet [Fri, 18 Oct 2024 01:40:47 +0000 (18:40 -0700)]
ZSTD_splitBlock_4k() uses externally provided workspace
ideally, this workspace would be provided from the ZSTD_CCtx* state
Yann Collet [Fri, 18 Oct 2024 00:05:17 +0000 (17:05 -0700)]
replaced uasan32 test by asan32 test
Yann Collet [Thu, 17 Oct 2024 23:23:05 +0000 (16:23 -0700)]
fixing minor formatting issue in 32-bit mode with logs enabled
Yann Collet [Thu, 17 Oct 2024 22:44:51 +0000 (15:44 -0700)]
fixed VS2010 solution
Yann Collet [Thu, 17 Oct 2024 22:42:02 +0000 (15:42 -0700)]
new Makefile target mesonbuild
for easier local testing
Yann Collet [Thu, 17 Oct 2024 22:35:56 +0000 (15:35 -0700)]
fixed meson build
Yann Collet [Thu, 17 Oct 2024 22:16:57 +0000 (15:16 -0700)]
fixed zstreamtest
Yann Collet [Thu, 17 Oct 2024 21:52:49 +0000 (14:52 -0700)]
fixed c90 comment style
Yann Collet [Thu, 17 Oct 2024 21:50:46 +0000 (14:50 -0700)]
fix assert
Yann Collet [Thu, 17 Oct 2024 21:46:47 +0000 (14:46 -0700)]
only split full blocks
short term simplification
Yann Collet [Thu, 17 Oct 2024 21:41:26 +0000 (14:41 -0700)]
fixed single-library build
Yann Collet [Thu, 17 Oct 2024 20:26:57 +0000 (13:26 -0700)]
fixed kernel build
Yann Collet [Thu, 17 Oct 2024 20:21:55 +0000 (13:21 -0700)]
fixed RLE detection test
Yann Collet [Thu, 17 Oct 2024 19:55:08 +0000 (12:55 -0700)]
fix overlap write scenario in presence of incompressible data
Yann Collet [Tue, 3 Sep 2024 23:59:06 +0000 (16:59 -0700)]
more ZSTD_memset() to apply
Yann Collet [Tue, 3 Sep 2024 23:54:36 +0000 (16:54 -0700)]
minor C++-ism
though I really wonder if this is a property worth maintaining.
Yann Collet [Tue, 3 Sep 2024 23:52:10 +0000 (16:52 -0700)]
use ZSTD_memset()
for better portability on Linux kernel
Yann Collet [Tue, 3 Sep 2024 23:48:52 +0000 (16:48 -0700)]
do not use `new` as variable name
Yann Collet [Tue, 3 Sep 2024 23:44:30 +0000 (16:44 -0700)]
fixed strict C90 semantic
Yann Collet [Tue, 3 Sep 2024 21:35:29 +0000 (14:35 -0700)]
XP: add a pre-splitter
instead of ingesting only full blocks, make an analysis of data, and infer where to split.
Yann Collet [Wed, 23 Oct 2024 18:14:05 +0000 (11:14 -0700)]
Merge pull request #4174 from facebook/bench_loadOnce
Modify benchmark to load sources only once
Yann Collet [Wed, 23 Oct 2024 06:53:56 +0000 (23:53 -0700)]
improve man page on benchmark mode
update the man page in troff format,
and the README with latest `--help` content and complementary details about benchmark mode.
also: display level 0 when doing decompression benchmark
Yann Collet [Tue, 22 Oct 2024 09:18:48 +0000 (02:18 -0700)]
Modify benchmark to only load sources once
After a regrettable update,
the benchmark module ended up reloading sources for every compression level.
While the delay itself is likely torelable,
the main issue is that the `--quiet` mode now also displays a loading summary between each compression line.
This wasn't the original intention, which is to produce a compact view of all compressions.
This is fixed in this version,
where sources are loaded only once, for all compression levels,
and loading summary is only displayed once.
Yann Collet [Thu, 17 Oct 2024 18:39:41 +0000 (11:39 -0700)]
Merge pull request #4171 from facebook/lvl3_ratio+
Improve compression ratio of levels 3 & 4
Yann Collet [Thu, 17 Oct 2024 17:50:28 +0000 (10:50 -0700)]
updated regression tests results
Yann Collet [Thu, 17 Oct 2024 16:45:16 +0000 (09:45 -0700)]
enable regression tests at pull request time
was transferred from circleci,
but was only triggered on push into dev,
i.e. after pull request is merged.
Yann Collet [Thu, 17 Oct 2024 16:36:56 +0000 (09:36 -0700)]
rewrite code in the manner suggested by @terrelln
Yann Collet [Wed, 16 Oct 2024 19:55:52 +0000 (12:55 -0700)]
fix test
a margin of 4 is insufficient to guarantee compression success.
Yann Collet [Wed, 16 Oct 2024 19:13:57 +0000 (12:13 -0700)]
slightly improved compression ratio at levels 3 & 4
The compression ratio benefits are small but consistent, i.e. always positive.
On `silesia.tar` corpus, this modification saves ~75 KB at level 3.
The measured speed cost is negligible, i.e. below noise level, between 0 and -1%.
Yann Collet [Thu, 17 Oct 2024 00:36:49 +0000 (17:36 -0700)]
Merge pull request #4170 from facebook/dict_cSpeed
Improve dictionary compression speed
Yann Collet [Wed, 16 Oct 2024 01:44:40 +0000 (18:44 -0700)]
fix test
Yann Collet [Wed, 16 Oct 2024 00:58:33 +0000 (17:58 -0700)]
minor improvement to level 3 dictionary compression ratio
Yann Collet [Wed, 16 Oct 2024 00:46:01 +0000 (17:46 -0700)]
small dictionary compression speed improvement
not as good as small-blocks improvement,
but generally positive.
Yann Collet [Sat, 12 Oct 2024 08:57:55 +0000 (01:57 -0700)]
Merge pull request #4167 from facebook/ci_m32test_faster
attempt to make 32-bit tests faster
Yann Collet [Sat, 12 Oct 2024 01:01:28 +0000 (18:01 -0700)]
attempt parallel test running with -j
Yann Collet [Sat, 12 Oct 2024 00:30:55 +0000 (17:30 -0700)]
measure if -O2 makes the test complete faster
Yann Collet [Fri, 11 Oct 2024 23:24:25 +0000 (16:24 -0700)]
attempt to make 32-bit tests faster
this is the longest CI test, reaching ~40mn on last PR
Yann Collet [Fri, 11 Oct 2024 23:20:19 +0000 (16:20 -0700)]
Merge pull request #4165 from facebook/cspeed_cmov
Improve compression speed on small blocks
Yann Collet [Fri, 11 Oct 2024 22:38:12 +0000 (15:38 -0700)]
rename variable name
findMatch -> matchFound
since it's a test, as opposed to an active search operation.
suggested by @terrelln
Yann Collet [Fri, 11 Oct 2024 22:36:15 +0000 (15:36 -0700)]
fixed parameter ordering in `dfast`
noticed by @terrelln
Yann Collet [Thu, 10 Oct 2024 23:56:02 +0000 (16:56 -0700)]
Merge pull request #4164 from facebook/spec_043
spec update: huffman prefix code paragraph
Yann Collet [Thu, 10 Oct 2024 23:07:20 +0000 (16:07 -0700)]
minor: better variable naming
Yann Collet [Wed, 9 Oct 2024 08:06:24 +0000 (01:06 -0700)]
added ascending order example
Yann Collet [Tue, 8 Oct 2024 23:38:35 +0000 (16:38 -0700)]
make __asm__ a __GNUC__ specific
Yann Collet [Tue, 8 Oct 2024 23:02:54 +0000 (16:02 -0700)]
store dummy bytes within ZSTD_match4Found_cmov()
feels more logical, better contained
Yann Collet [Tue, 8 Oct 2024 22:54:48 +0000 (15:54 -0700)]
introduce memory barrier to force test order
suggested by @terrelln
Yann Collet [Tue, 8 Oct 2024 18:43:07 +0000 (11:43 -0700)]
made search strategy switchable
between cmov and branch
and use a simple heuristic based on wlog to select between them.
note: performance is not good on clang (yet)