]>
git.ipfire.org Git - thirdparty/zstd.git/log
Nick Terrell [Fri, 30 Apr 2021 22:02:12 +0000 (15:02 -0700)]
[1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
Nick Terrell [Fri, 30 Apr 2021 00:15:40 +0000 (17:15 -0700)]
Merge pull request #2593 from terrelln/linux-comments
[linux-kernel] Replace kernel-style comments
Nick Terrell [Thu, 29 Apr 2021 22:50:23 +0000 (15:50 -0700)]
[linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.
E.g.
```
/** Before */
/* After */
/**
* Before
*/
/*
* After
*/
/***********************************
* Before
***********************************/
/* *********************************
* After
***********************************/
```
Nick Terrell [Tue, 27 Apr 2021 17:39:12 +0000 (10:39 -0700)]
Merge pull request #2589 from terrelln/tracing
[trace] Remove default definitions of weak symbols
sen [Tue, 27 Apr 2021 16:40:07 +0000 (12:40 -0400)]
Merge pull request #2588 from senhuang42/update_regressiontest
[regressiontest] Update results.csv
Nick Terrell [Mon, 26 Apr 2021 23:05:39 +0000 (16:05 -0700)]
[trace] Remove default definitions of weak symbols
Instead of providing a default no-op implementation, check the symbols
for `NULL` before accessing them. Providing a default implementation
doesn't reliably work with dynamic linking. Depending on link order the
default implementations may not be overridden. By skipping the default
implementation, all link order issues are resolved. If the symbols
aren't provided the weak function will be `NULL`.
senhuang42 [Mon, 26 Apr 2021 19:55:23 +0000 (15:55 -0400)]
Update results.csv
sen [Mon, 26 Apr 2021 13:55:43 +0000 (09:55 -0400)]
Merge pull request #2579 from senhuang42/getcdictID_to_stable
[1.5.0] Promote ZSTD_getDictID_fromCDict() into stable API
sen [Sat, 24 Apr 2021 16:39:16 +0000 (12:39 -0400)]
Merge pull request #2586 from senhuang42/dds_fuzz
Add DDS to oss fuzzer
felixhandte [Fri, 23 Apr 2021 20:34:21 +0000 (16:34 -0400)]
Apply DDS adjustments to avoid assert failures
senhuang42 [Thu, 22 Apr 2021 22:21:43 +0000 (18:21 -0400)]
Add DDS to oss fuzzer
sen [Tue, 20 Apr 2021 21:05:32 +0000 (17:05 -0400)]
Merge pull request #2583 from senhuang42/remove_zbuff
[1.5.0] Remove ZBUFF
senhuang42 [Mon, 19 Apr 2021 21:12:51 +0000 (17:12 -0400)]
Remove ZBUFF tests
senhuang42 [Mon, 19 Apr 2021 21:12:00 +0000 (17:12 -0400)]
Remove building of ZBUFF/deprecated folder by default
Sen Huang [Fri, 16 Apr 2021 16:52:36 +0000 (09:52 -0700)]
Moved ZSTD_getDictID_fromCDict() into stable API
sen [Tue, 13 Apr 2021 03:27:43 +0000 (23:27 -0400)]
Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix
Correct the block splitter mismatched repcodes detection.
sen [Mon, 12 Apr 2021 19:38:53 +0000 (15:38 -0400)]
Merge pull request #2572 from senhuang42/row_hash_hashcache_one_off_error_fix
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
Sen Huang [Fri, 9 Apr 2021 02:54:19 +0000 (19:54 -0700)]
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
Sen Huang [Fri, 9 Apr 2021 15:01:54 +0000 (08:01 -0700)]
Correct the detection of mismatched repcodes
sen [Thu, 8 Apr 2021 14:55:14 +0000 (10:55 -0400)]
Merge pull request #2494 from senhuang42/row_hash2
SIMD Row Based Matchfinder 🚀
Sen Huang [Fri, 2 Apr 2021 15:07:10 +0000 (08:07 -0700)]
Update results.csv, add Row hash to regression test
Nick Terrell [Tue, 3 Nov 2020 01:52:29 +0000 (17:52 -0800)]
Add and integrate lazy row hash strategy
sen [Wed, 7 Apr 2021 01:05:37 +0000 (21:05 -0400)]
Move clevel override to after initLocalDict() (#2571)
sen [Tue, 6 Apr 2021 21:25:55 +0000 (17:25 -0400)]
Maintain two repcode histories for block splitting, replace invalid repcodes (#2569)
sen [Tue, 6 Apr 2021 19:35:09 +0000 (15:35 -0400)]
Fix dictionary force reloading clevel selection (#2570)
* Move cdict clevel override to before localdict init
* Update results.csv after dict load changes
Nick Terrell [Fri, 2 Apr 2021 20:01:09 +0000 (13:01 -0700)]
Merge pull request #2541 from ihsinme/patch-1
simple fix for using bit operator.
Niclas Rosenvik [Sun, 28 Mar 2021 12:36:13 +0000 (14:36 +0200)]
Stop complaining about hash tool not found
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
sen [Fri, 2 Apr 2021 00:07:19 +0000 (20:07 -0400)]
[cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546)
* Perform 64-byte alignment of wksp tables and aligneds internally
* Clean up cwskp_finalize() function to only do two allocs
* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations
* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned
* Back out aligned/table reservation order restriction
* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
sen [Wed, 31 Mar 2021 19:14:59 +0000 (15:14 -0400)]
Fix repcode-related OSS-fuzz issues in block splitter (#2560)
* Do not emit last partitions of blocks as RLE/uncompressed
* Fix repcode updates within block splitter
* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature
* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
Nick Terrell [Tue, 30 Mar 2021 00:52:28 +0000 (17:52 -0700)]
[contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel()
Nick Terrell [Mon, 29 Mar 2021 21:23:36 +0000 (14:23 -0700)]
[copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
sen [Fri, 26 Mar 2021 19:55:12 +0000 (15:55 -0400)]
Merge pull request #2561 from senhuang42/longlength_enum
Add enum for representing long length ID
Sen Huang [Fri, 26 Mar 2021 17:35:52 +0000 (10:35 -0700)]
Add enum for representing long length ID
sen [Thu, 25 Mar 2021 23:26:06 +0000 (19:26 -0400)]
Merge pull request #2559 from senhuang42/add_dict_regression_tests_backup
Add different dict modes to compression ratio regression test, update results.csv
Sen Huang [Thu, 25 Mar 2021 17:43:10 +0000 (10:43 -0700)]
Update results.csv
sen [Thu, 25 Mar 2021 17:51:43 +0000 (13:51 -0400)]
Merge pull request #2558 from senhuang42/msan_block_splitter_fix
Fix block splitter minor MSAN warning.
Sen Huang [Thu, 25 Mar 2021 17:38:56 +0000 (10:38 -0700)]
Restrict dictmode regression tests only to advanced API, fix some compiler warnings
Sen Huang [Thu, 25 Mar 2021 00:16:49 +0000 (17:16 -0700)]
Add tests to regression tests for dict
sen [Thu, 25 Mar 2021 17:07:28 +0000 (13:07 -0400)]
Merge pull request #2555 from senhuang42/default_clevel_func
Add ZSTD_defaultCLevel() function to public API
Sen Huang [Thu, 25 Mar 2021 16:11:43 +0000 (09:11 -0700)]
Add clang msan fuzz test to github actions
Sen Huang [Thu, 25 Mar 2021 14:30:53 +0000 (07:30 -0700)]
Move lastCountSize into a returned struct, fix MSAN error
Sen Huang [Wed, 24 Mar 2021 17:43:48 +0000 (10:43 -0700)]
Add ZSTD_defaultCLevel() function to public API
Nick Terrell [Wed, 24 Mar 2021 17:34:29 +0000 (10:34 -0700)]
Merge pull request #2539 from terrelln/linux-kernel-fixes
Fixes for the next linux kernel patch version
sen [Wed, 24 Mar 2021 16:27:22 +0000 (12:27 -0400)]
Merge pull request #2447 from senhuang42/block_splitter_v2
Recursive block splitting
Yann Collet [Wed, 24 Mar 2021 15:55:33 +0000 (08:55 -0700)]
Merge pull request #2554 from facebook/fix2553
fix man page typo
Sen Huang [Mon, 22 Mar 2021 22:25:57 +0000 (15:25 -0700)]
Rename *compressSequences*() functions for clarity
Sen Huang [Mon, 22 Mar 2021 21:38:21 +0000 (14:38 -0700)]
Add a fallback in case the total blocksize of split blocks exceeds raw block size
Sen Huang [Mon, 22 Mar 2021 15:54:24 +0000 (08:54 -0700)]
Enable block splitter by default when applicable
Sen Huang [Tue, 16 Mar 2021 21:15:56 +0000 (14:15 -0700)]
Clean up various functions, add debuglogging for estimate vs. actual sizes
Sen Huang [Wed, 24 Feb 2021 21:21:49 +0000 (13:21 -0800)]
Fix various fuzzer failures: repcode history, superblocks
senhuang42 [Mon, 11 Jan 2021 13:30:16 +0000 (08:30 -0500)]
Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression
senhuang42 [Fri, 8 Jan 2021 20:50:19 +0000 (15:50 -0500)]
Refactor buildSequencesStatistics() to avoid pointer increment for superblocks
senhuang42 [Tue, 29 Dec 2020 20:56:13 +0000 (15:56 -0500)]
Add unit tests and fuzzer param
senhuang42 [Thu, 10 Dec 2020 17:18:43 +0000 (12:18 -0500)]
Add recursive block split algorithm
senhuang42 [Tue, 8 Dec 2020 18:06:18 +0000 (13:06 -0500)]
Update function names for consistency
senhuang42 [Mon, 7 Dec 2020 22:11:23 +0000 (17:11 -0500)]
Add block splitter to experimental params
senhuang42 [Mon, 7 Dec 2020 18:12:41 +0000 (13:12 -0500)]
Refactor block compression logic into single function
senhuang42 [Mon, 30 Nov 2020 18:41:03 +0000 (13:41 -0500)]
Centralize entropy statistics calculations to zstd_compress.c
Yann Collet [Wed, 24 Mar 2021 12:55:04 +0000 (05:55 -0700)]
fix man page typo
fix #2553 detected by @jwilk
sen [Mon, 22 Mar 2021 20:54:34 +0000 (16:54 -0400)]
Merge pull request #2538 from senhuang42/monotonicity_test
Add memory monotonicity test over srcSize
Nick Terrell [Mon, 22 Mar 2021 19:37:54 +0000 (12:37 -0700)]
Merge pull request #2524 from terrelln/huf-stack-reduction
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
Nick Terrell [Fri, 19 Mar 2021 23:04:59 +0000 (16:04 -0700)]
[FSE] Clean up workspace using dynamically sized struct
Yann Collet [Mon, 22 Mar 2021 17:44:44 +0000 (10:44 -0700)]
Merge pull request #2550 from facebook/vvv_srcSize
fix #2549
Sen Huang [Fri, 19 Mar 2021 23:56:58 +0000 (16:56 -0700)]
Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well
Yann Collet [Sun, 21 Mar 2021 00:29:41 +0000 (17:29 -0700)]
fix #2549
Yann Collet [Fri, 19 Mar 2021 22:20:14 +0000 (15:20 -0700)]
Merge pull request #2548 from SupervisedThinking/build_fix
meson: fix build by adding missing files
SupervisedThinking [Fri, 19 Mar 2021 18:52:45 +0000 (19:52 +0100)]
meson: fix build by adding missing files
fixes https://github.com/facebook/zstd/issues/2519
sen [Thu, 18 Mar 2021 03:25:53 +0000 (23:25 -0400)]
Merge pull request #2540 from senhuang42/fix_dds_supported
Fix dedicated dict search isSupported() requirements.
Nick Terrell [Wed, 17 Mar 2021 23:50:37 +0000 (16:50 -0700)]
[huf][fse] Clean up workspaces
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
ihsinme [Wed, 17 Mar 2021 08:37:42 +0000 (11:37 +0300)]
simple fix for using bit operator.
good day.
It seems to me that the developer intended to use a logical operator.
so I suggest a simple fix.
Sen Huang [Wed, 17 Mar 2021 00:36:05 +0000 (17:36 -0700)]
Fix ZSTD_dedicatedDictSearch_isSupported() requirements
senhuang42 [Mon, 12 Oct 2020 22:33:11 +0000 (18:33 -0400)]
Add a nbSeq argument to compressSequences()
Refactor ZSTD_compressBlock_internal() to do the block header write within and add nbSeq argument to compressSequences()
senhuang42 [Mon, 12 Oct 2020 22:27:54 +0000 (18:27 -0400)]
Move block header write into compressBlock_internal()
Nick Terrell [Tue, 16 Mar 2021 00:48:56 +0000 (17:48 -0700)]
[lib] Bump zstd version number
Nick Terrell [Mon, 15 Mar 2021 21:46:26 +0000 (14:46 -0700)]
[lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
Nick Terrell [Mon, 15 Mar 2021 18:19:21 +0000 (11:19 -0700)]
[contrib][freestanding] Remove tracing support
Remove tracing support from `freestanding.py` to keep things simple.
Nick Terrell [Thu, 3 Dec 2020 20:37:30 +0000 (12:37 -0800)]
[contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
Nick Terrell [Fri, 12 Mar 2021 23:31:38 +0000 (15:31 -0800)]
[contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
Nick Terrell [Fri, 5 Mar 2021 05:13:52 +0000 (21:13 -0800)]
[contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
sen [Tue, 16 Mar 2021 17:09:00 +0000 (13:09 -0400)]
Merge pull request #2534 from foxeng/fix-seek-descriptor-check
Fix seek table descriptor check when loading
Sen Huang [Tue, 16 Mar 2021 15:22:13 +0000 (08:22 -0700)]
Add mem monotonicity test over srcSize
Felix Handte [Mon, 15 Mar 2021 17:58:27 +0000 (13:58 -0400)]
Merge pull request #2535 from felixhandte/gha-release-artifacts
Add GitHub Action to Automatically Publish Release Tarballs
W. Felix Handte [Mon, 15 Mar 2021 16:11:53 +0000 (12:11 -0400)]
Allow a Passphrase on the Key
W. Felix Handte [Mon, 15 Mar 2021 15:59:31 +0000 (11:59 -0400)]
Maintain Artifact Name Backwards Compatibility
When the tag is `v1.2.3`, name the artifacts `zstd-1.2.3.tar*` rather than
`zstd-v1.2.3.tar*`. When the tag doesn't match, use the full tag.
Fotis Xenakis [Sat, 13 Mar 2021 09:42:47 +0000 (11:42 +0200)]
Run generic-dev:gcc-8-asan-ubsan-testzstd on latest Ubuntu
W. Felix Handte [Fri, 12 Mar 2021 22:35:11 +0000 (17:35 -0500)]
Remove CircleCI Artifact Generation
Fotis Xenakis [Fri, 12 Mar 2021 22:00:08 +0000 (00:00 +0200)]
Fix seekable test to provide valid descriptor
Fotis Xenakis [Fri, 12 Mar 2021 21:07:15 +0000 (23:07 +0200)]
Fix seek table descriptor check when loading
W. Felix Handte [Thu, 11 Mar 2021 21:06:25 +0000 (16:06 -0500)]
Add GitHub Action to Automatically Publish Release Tarballs
This commit introduces a GitHub action that is triggered on release creation,
which creates the release tarball, compresses it, hashes it, signs it, and
attaches all of those files to the release.
sen [Mon, 8 Mar 2021 16:14:34 +0000 (11:14 -0500)]
Merge pull request #2517 from PaulBone/num_cores
Make the number of physical CPU cores detection more robust
Yann Collet [Sun, 7 Mar 2021 05:33:28 +0000 (21:33 -0800)]
Merge pull request #2521 from animalize/doc_free
doc: ZSTD_free*() functions accept NULL pointer
Nick Terrell [Fri, 5 Mar 2021 21:14:06 +0000 (13:14 -0800)]
[fse] Reduce stack usage of FSE_decompress_wksp() by 512 bytes
* Move `counting` into the workspace
* Inrease `HUF_DECOMPRESS_WORKSPACE_SIZE` by 512 bytes
Nick Terrell [Fri, 5 Mar 2021 20:36:50 +0000 (12:36 -0800)]
[huf] Reduce stack usage of HUF_readDTableX2 by ~460 bytes
* Use `HUF_readStats_wksp()`
* Use workspace in `HUF_fillDTableX2*()`
* Clean up workspace usage to use a workspace struct
Nick Terrell [Fri, 5 Mar 2021 20:35:09 +0000 (12:35 -0800)]
Merge pull request #2523 from terrelln/huf-stack-reduction
Add HUF_writeCTable_wksp() function
Nick Terrell [Mon, 21 Dec 2020 18:23:08 +0000 (10:23 -0800)]
Add HUF_writeCTable_wksp() function
This saves ~700 bytes of stack space in HUF_writeCTable.
Nick Terrell [Fri, 5 Mar 2021 04:55:58 +0000 (20:55 -0800)]
Merge pull request #2522 from terrelln/stack-reduction
Reduce stack usage of ZSTD_buildCTable()
Nick Terrell [Thu, 4 Mar 2021 18:50:34 +0000 (10:50 -0800)]
Reduce stack usage of ZSTD_buildCTable()
It is a stack high-point for some compression strategies and has an easy
fix. This moves the normalized count into the entropy workspace.
Yann Collet [Thu, 4 Mar 2021 23:29:23 +0000 (15:29 -0800)]
Merge pull request #2518 from facebook/seekTable
New direct seekTable access methods
Yann Collet [Thu, 4 Mar 2021 06:54:04 +0000 (22:54 -0800)]
added code comments for new API ZSTD_seekTable
Yann Collet [Thu, 4 Mar 2021 02:56:54 +0000 (18:56 -0800)]
Merge branch 'seekTable' of github.com:facebook/zstd into seekTable