]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
6 years agoAccept `ZSTD_indexResetPolicy_e` Param in `ZSTD_reset_matchState()`
W. Felix Handte [Tue, 10 Sep 2019 21:28:52 +0000 (17:28 -0400)] 
Accept `ZSTD_indexResetPolicy_e` Param in `ZSTD_reset_matchState()`

6 years agoIntroduce `ZSTD_indexResetPolicy_e` Enum
W. Felix Handte [Tue, 10 Sep 2019 21:24:02 +0000 (17:24 -0400)] 
Introduce `ZSTD_indexResetPolicy_e` Enum

6 years agoTrack Usable Table Space in Compression Workspace
W. Felix Handte [Tue, 3 Sep 2019 17:08:24 +0000 (13:08 -0400)] 
Track Usable Table Space in Compression Workspace

6 years agoMerge pull request #1712 from felixhandte/workspace-efficiency-2
Yann Collet [Tue, 10 Sep 2019 22:20:29 +0000 (15:20 -0700)] 
Merge pull request #1712 from felixhandte/workspace-efficiency-2

Allocate Internal Buffers via Workspace Abstraction

6 years agoAddressing comments on parsing
Bimba Shrestha [Tue, 10 Sep 2019 22:10:50 +0000 (15:10 -0700)] 
Addressing comments on parsing

6 years agoCleaning up parsing per suggestion
Bimba Shrestha [Tue, 10 Sep 2019 20:18:59 +0000 (13:18 -0700)] 
Cleaning up parsing per suggestion

6 years agoRemove Empty lib/compress/zstd_cwksp.c 1712/head
W. Felix Handte [Tue, 10 Sep 2019 20:03:13 +0000 (16:03 -0400)] 
Remove Empty lib/compress/zstd_cwksp.c

6 years agoMerge pull request #1775 from facebook/edufix
Yann Collet [Tue, 10 Sep 2019 19:12:08 +0000 (12:12 -0700)] 
Merge pull request #1775 from facebook/edufix

fix educational decoder

6 years agoMerge pull request #1777 from darxsys/improvDataGen
Nick Terrell [Tue, 10 Sep 2019 18:51:23 +0000 (11:51 -0700)] 
Merge pull request #1777 from darxsys/improvDataGen

[Fuzz] Improve data generation #1723

6 years agoMerge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix
Yann Collet [Tue, 10 Sep 2019 18:17:29 +0000 (11:17 -0700)] 
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix

Removing redundant condition in decompression, making first block rle…

6 years agoFizing litLength == 0 case
Bimba Shrestha [Tue, 10 Sep 2019 17:06:02 +0000 (10:06 -0700)] 
Fizing litLength == 0 case

6 years agofix education decoder test 1775/head
Yann Collet [Tue, 10 Sep 2019 16:36:02 +0000 (09:36 -0700)] 
fix education decoder test

when `zstd` in not installed on local system
by allowing `ZSTD` variable to hold a custom location for the binary

6 years agoUsing a number instead of M prefix in head call 1773/head
Bimba Shrestha [Tue, 10 Sep 2019 16:30:37 +0000 (09:30 -0700)] 
Using a number instead of M prefix in head call

6 years agoAddressing comments
Bimba Shrestha [Tue, 10 Sep 2019 03:04:46 +0000 (20:04 -0700)] 
Addressing comments

6 years agoAddressing nits 1777/head
Dario Pavlovic [Mon, 9 Sep 2019 23:13:24 +0000 (16:13 -0700)] 
Addressing nits

6 years agoForward Declare `ZSTD_cwksp_available_space` to Fix Build
W. Felix Handte [Mon, 9 Sep 2019 23:10:09 +0000 (19:10 -0400)] 
Forward Declare `ZSTD_cwksp_available_space` to Fix Build

6 years agoInline Workspace Functions
W. Felix Handte [Mon, 9 Sep 2019 22:30:53 +0000 (18:30 -0400)] 
Inline Workspace Functions

6 years agoFixing issues with double usage of data.
Dario Pavlovic [Mon, 9 Sep 2019 22:39:04 +0000 (15:39 -0700)] 
Fixing issues with double usage of data.

6 years agoForward resetCCtx Errors when Using CDict
W. Felix Handte [Tue, 3 Sep 2019 17:13:16 +0000 (13:13 -0400)] 
Forward resetCCtx Errors when Using CDict

6 years agoFix sizeof_CCtx and sizeof_CDict Calculations for Statically Init'ed Objects
W. Felix Handte [Mon, 9 Sep 2019 20:45:17 +0000 (16:45 -0400)] 
Fix sizeof_CCtx and sizeof_CDict Calculations for Statically Init'ed Objects

6 years agoFix workspaceTooSmall Calculation
W. Felix Handte [Mon, 9 Sep 2019 19:12:14 +0000 (15:12 -0400)] 
Fix workspaceTooSmall Calculation

6 years agoChanging test file directory names to be more descriptive
Bimba Shrestha [Mon, 9 Sep 2019 19:08:33 +0000 (12:08 -0700)] 
Changing test file directory names to be more descriptive

6 years agoShorten `&zc->workspace` -> `ws` in `ZSTD_resetCCtx_internal()`
W. Felix Handte [Mon, 9 Sep 2019 18:59:09 +0000 (14:59 -0400)] 
Shorten `&zc->workspace` -> `ws` in `ZSTD_resetCCtx_internal()`

6 years agoTesting first block decompression cli
Bimba Shrestha [Mon, 9 Sep 2019 18:54:43 +0000 (11:54 -0700)] 
Testing first block decompression cli

6 years agoClean Up TODOs and Comments pt. II
W. Felix Handte [Mon, 9 Sep 2019 18:04:39 +0000 (14:04 -0400)] 
Clean Up TODOs and Comments pt. II

6 years agoNit
W. Felix Handte [Tue, 3 Sep 2019 17:00:59 +0000 (13:00 -0400)] 
Nit

6 years agoPull Phase Advance Logic Out into Internal Function
W. Felix Handte [Tue, 3 Sep 2019 16:59:10 +0000 (12:59 -0400)] 
Pull Phase Advance Logic Out into Internal Function

6 years agoHide Workspace Movement Behind Helper Function
W. Felix Handte [Tue, 3 Sep 2019 16:48:45 +0000 (12:48 -0400)] 
Hide Workspace Movement Behind Helper Function

6 years agoFix DEBUGLOG Statement Levels
W. Felix Handte [Tue, 3 Sep 2019 16:41:35 +0000 (12:41 -0400)] 
Fix DEBUGLOG Statement Levels

6 years agoFix Rescale Continue Special Case
W. Felix Handte [Thu, 29 Aug 2019 20:33:45 +0000 (16:33 -0400)] 
Fix Rescale Continue Special Case

6 years agoRemove Unused noRealloc CRP Value
W. Felix Handte [Tue, 27 Aug 2019 19:21:59 +0000 (15:21 -0400)] 
Remove Unused noRealloc CRP Value

6 years agoRe-Implement Workspace Shrinking when Oversized
W. Felix Handte [Thu, 22 Aug 2019 20:09:00 +0000 (16:09 -0400)] 
Re-Implement Workspace Shrinking when Oversized

6 years agoAdd New File to Visual Studio Projects
W. Felix Handte [Tue, 20 Aug 2019 19:19:58 +0000 (15:19 -0400)] 
Add New File to Visual Studio Projects

6 years agoAdd New File to Meson Build
W. Felix Handte [Tue, 20 Aug 2019 19:14:12 +0000 (15:14 -0400)] 
Add New File to Meson Build

6 years agoCleanup from Move
W. Felix Handte [Tue, 20 Aug 2019 18:15:06 +0000 (14:15 -0400)] 
Cleanup from Move

6 years agoMove Workspace Functions to Their Own File
W. Felix Handte [Tue, 20 Aug 2019 18:02:50 +0000 (14:02 -0400)] 
Move Workspace Functions to Their Own File

6 years agoRename
W. Felix Handte [Thu, 15 Aug 2019 16:51:24 +0000 (12:51 -0400)] 
Rename

6 years agoClean Up TODOs and Comments
W. Felix Handte [Wed, 14 Aug 2019 21:11:16 +0000 (17:11 -0400)] 
Clean Up TODOs and Comments

6 years agoImprove Comments a Bit
W. Felix Handte [Wed, 14 Aug 2019 20:55:12 +0000 (16:55 -0400)] 
Improve Comments a Bit

6 years agoAllocate CDict in Workspace (Rather than in Separate Allocation)
W. Felix Handte [Wed, 14 Aug 2019 20:48:01 +0000 (16:48 -0400)] 
Allocate CDict in Workspace (Rather than in Separate Allocation)

6 years agoRewrite ZSTD_initStaticCCtx to Alloc CCtx in Workspace
W. Felix Handte [Wed, 14 Aug 2019 18:44:17 +0000 (14:44 -0400)] 
Rewrite ZSTD_initStaticCCtx to Alloc CCtx in Workspace

6 years agoOnly the CCtx Itself Needs to be Cleared during Static CCtx Init
W. Felix Handte [Wed, 14 Aug 2019 15:35:53 +0000 (11:35 -0400)] 
Only the CCtx Itself Needs to be Cleared during Static CCtx Init

6 years agoAlign Alloc Pointer When Transitioning from Buffers to Aligned Allocs
W. Felix Handte [Tue, 13 Aug 2019 15:57:20 +0000 (11:57 -0400)] 
Align Alloc Pointer When Transitioning from Buffers to Aligned Allocs

6 years agoRemove Overly-Restrictive Assert
W. Felix Handte [Tue, 13 Aug 2019 00:01:57 +0000 (20:01 -0400)] 
Remove Overly-Restrictive Assert

6 years agoWhen Loading Dict By Copy, Always Put it in the Workspace
W. Felix Handte [Mon, 12 Aug 2019 23:24:24 +0000 (19:24 -0400)] 
When Loading Dict By Copy, Always Put it in the Workspace

6 years agoAlloc Tables Separately
W. Felix Handte [Mon, 12 Aug 2019 22:36:48 +0000 (18:36 -0400)] 
Alloc Tables Separately

6 years agoBegin Introducing Phases
W. Felix Handte [Mon, 12 Aug 2019 18:22:54 +0000 (14:22 -0400)] 
Begin Introducing Phases

6 years agoTMP
W. Felix Handte [Mon, 15 Jul 2019 19:08:49 +0000 (15:08 -0400)] 
TMP

6 years agoDisambiguate 'workspace' and 'entropyWorkspace'
W. Felix Handte [Mon, 15 Jul 2019 21:14:51 +0000 (17:14 -0400)] 
Disambiguate 'workspace' and 'entropyWorkspace'

6 years agoNormalize Case 'workSpace' -> 'workspace'
W. Felix Handte [Mon, 15 Jul 2019 19:10:33 +0000 (15:10 -0400)] 
Normalize Case 'workSpace' -> 'workspace'

6 years ago[Fuzz] Improve data generation #1723
Dario Pavlovic [Mon, 9 Sep 2019 15:43:22 +0000 (08:43 -0700)] 
[Fuzz] Improve data generation #1723

6 years agoMerge pull request #1771 from vnair81/dev
Yann Collet [Sat, 7 Sep 2019 00:02:53 +0000 (17:02 -0700)] 
Merge pull request #1771 from vnair81/dev

Passing ZSTD_CCtx_params by const pointer

6 years agofix minor compilation condition
Yann Collet [Fri, 6 Sep 2019 23:51:16 +0000 (16:51 -0700)] 
fix minor compilation condition

harness.c is not designed to pass  -Wdeclaration-after-statement

6 years agoMentioning cli only in the comment as suggested
Bimba Shrestha [Fri, 6 Sep 2019 21:48:41 +0000 (14:48 -0700)] 
Mentioning cli only in the comment as suggested

6 years agofix eductional decoder
Yann Collet [Fri, 6 Sep 2019 21:30:13 +0000 (14:30 -0700)] 
fix eductional decoder

fix #1774
also :
- fix minor compilation warnings
- make sure the `test` is run during CI tests

6 years agoPut back omission for first rle block and updated comment as suggested
Bimba Shrestha [Fri, 6 Sep 2019 20:44:25 +0000 (13:44 -0700)] 
Put back omission for first rle block and updated comment as suggested

6 years agofixes for windows compilation
Sen Huang [Fri, 6 Sep 2019 20:20:50 +0000 (13:20 -0700)] 
fixes for windows compilation

6 years agofixes for tests and segfault
Sen Huang [Fri, 6 Sep 2019 18:10:53 +0000 (11:10 -0700)] 
fixes for tests and segfault

6 years agoRemoving redundant condition in decompression, making first block rles valid to deocm...
Bimba Shrestha [Fri, 6 Sep 2019 17:46:19 +0000 (10:46 -0700)] 
Removing redundant condition in decompression, making first block rles valid to deocmpress

6 years agokill memory leaks, cleanup, fix some dumb bugs
Sen Huang [Fri, 6 Sep 2019 17:17:04 +0000 (10:17 -0700)] 
kill memory leaks, cleanup, fix some dumb bugs

6 years agoadded decompression support
Sen Huang [Fri, 6 Sep 2019 00:56:24 +0000 (17:56 -0700)] 
added decompression support

6 years agoadding support for -O flag: multiple files into one directory for compressions (decom...
Sen Huang [Thu, 5 Sep 2019 23:03:35 +0000 (16:03 -0700)] 
adding support for -O flag: multiple files into one directory for compressions (decompression to come)

6 years agoFixing assert and DEBUGLOG due to ZSTD_CCtx_params parameter change to const pointer 1771/head
Varun S Nair [Thu, 5 Sep 2019 10:17:17 +0000 (15:47 +0530)] 
Fixing assert and DEBUGLOG due to ZSTD_CCtx_params parameter change to const pointer

6 years agoPassing ZSTD_CCtx_params by const pointer
Varun S Nair [Thu, 5 Sep 2019 09:58:30 +0000 (15:28 +0530)] 
Passing ZSTD_CCtx_params by const pointer

6 years agoMerge pull request #1770 from cwoffenden/amalgamate-correctness
Yann Collet [Wed, 4 Sep 2019 21:51:23 +0000 (14:51 -0700)] 
Merge pull request #1770 from cwoffenden/amalgamate-correctness

Single file decoder correctness and tidy

6 years ago[programs] don’t do chmod when coming from stdin or multiple files 1731/head
Mike Swanson [Mon, 19 Aug 2019 18:11:04 +0000 (11:11 -0700)] 
[programs] don’t do chmod when coming from stdin or multiple files

Fixes #1729

6 years agoMinor: documented sizes smaller 1770/head
Carl Woffenden [Mon, 2 Sep 2019 16:15:31 +0000 (18:15 +0200)] 
Minor: documented sizes smaller

6 years agoCorrectness and tidy
Carl Woffenden [Mon, 2 Sep 2019 16:02:50 +0000 (18:02 +0200)] 
Correctness and tidy

Test compilation performed with warnings. Author and license added. Test for failing grep on ancient OSX versions. Replaced the test image with something less noisy (which compresses better).

6 years agoMerge pull request #1769 from cybojanek/update_streaming_examples
Yann Collet [Mon, 2 Sep 2019 01:11:54 +0000 (18:11 -0700)] 
Merge pull request #1769 from cybojanek/update_streaming_examples

Fix streaming compression/decompression examples

6 years agoFix streaming compression/decompression examples 1769/head
Jan Kasiak [Sun, 1 Sep 2019 19:35:53 +0000 (15:35 -0400)] 
Fix streaming compression/decompression examples

* Handle compression of empty file
* Error in decompression in case of trailing data

6 years agoMerge pull request #1761 from terrelln/regression
Yann Collet [Fri, 30 Aug 2019 17:48:02 +0000 (10:48 -0700)] 
Merge pull request #1761 from terrelln/regression

[regression] small improvement from RLE blocks

6 years agoChanging api to get sequences across all blocks
Bimba Shrestha [Fri, 30 Aug 2019 16:18:44 +0000 (09:18 -0700)] 
Changing api to get sequences across all blocks

6 years ago[regression] small improvement from RLE blocks 1761/head
Nick Terrell [Fri, 30 Aug 2019 01:05:58 +0000 (18:05 -0700)] 
[regression] small improvement from RLE blocks

6 years agoMerge pull request #1744 from bimbashrestha/dev
Yann Collet [Thu, 29 Aug 2019 22:19:10 +0000 (15:19 -0700)] 
Merge pull request #1744 from bimbashrestha/dev

Generate RLE blocks in the encoder

6 years agoFixing ci-circle test complaints
Bimba Shrestha [Thu, 29 Aug 2019 20:09:42 +0000 (13:09 -0700)] 
Fixing ci-circle test complaints

6 years agofix warning: always_inline function might not be inlinable 1756/head
mgrice [Thu, 29 Aug 2019 19:32:15 +0000 (12:32 -0700)] 
fix warning: always_inline function might not be inlinable

6 years agoAdding api for extracting sequences from seqstore
Bimba Shrestha [Thu, 29 Aug 2019 19:29:39 +0000 (12:29 -0700)] 
Adding api for extracting sequences from seqstore

6 years agoImprovements in zstd decode performance
mgrice [Tue, 27 Aug 2019 21:49:23 +0000 (14:49 -0700)] 
Improvements in zstd decode performance

Summary: The idea behind wildcopy is that it can be cheaper to copy more bytes (say 8) than it is to copy less (say, 3).  This change takes that further by exploiting some properties:
1. it's almost always OK to copy 16 bytes instead of 8, which means fewer copy instructions, and fewer branches
2. A 16 byte chunk size means that ~90% of wildcopy invocations will have a trip count of 1, so branch prediction will be improved.

Speedup on Xeon E5-2680v4 is in the range of 3-5%.

Measured wildcopy length distributions on silesia.tar:

level <=8 <=16 <=24 >24
1 78.05% 11.49% 3.52% 6.94%
3 82.14% 8.99% 2.44% 6.43%
6 85.81% 6.51% 2.92% 4.76%
8 83.02% 7.31% 3.64% 6.03%
10 84.13% 6.67% 3.29% 5.91%
15 77.58% 7.55% 5.21% 9.66%
16 80.07% 7.20% 3.98% 8.75%

Test Plan: benchmark silesia, make check

6 years agoMerge pull request #1754 from facebook/decTest
Yann Collet [Thu, 29 Aug 2019 19:09:09 +0000 (12:09 -0700)] 
Merge pull request #1754 from facebook/decTest

Tests for single-file decompressor

6 years agoUndoing the last commit (that was an accident) 1744/head
Bimba Shrestha [Thu, 29 Aug 2019 19:05:47 +0000 (12:05 -0700)] 
Undoing the last commit (that was an accident)

6 years agoAdding method for extracting sequences.
bimbashrestha [Thu, 29 Aug 2019 18:55:12 +0000 (11:55 -0700)] 
Adding method for extracting sequences.

6 years agoMerge branch 'dev' into decTest 1754/head
Yann Collet [Thu, 29 Aug 2019 16:48:12 +0000 (09:48 -0700)] 
Merge branch 'dev' into decTest

6 years agoMerge pull request #1755 from cwoffenden/amalgamate-c99-fixes
Yann Collet [Thu, 29 Aug 2019 16:47:28 +0000 (09:47 -0700)] 
Merge pull request #1755 from cwoffenden/amalgamate-c99-fixes

C99 and older GCC fixes

6 years agoC99 and older GCC fixes 1755/head
Carl Woffenden [Thu, 29 Aug 2019 09:16:57 +0000 (11:16 +0200)] 
C99 and older GCC fixes

6 years agoupdate test script
Yann Collet [Wed, 28 Aug 2019 20:46:18 +0000 (13:46 -0700)] 
update test script

6 years agofix create_ script for sh
Yann Collet [Wed, 28 Aug 2019 20:23:48 +0000 (13:23 -0700)] 
fix create_ script for sh

6 years agoMerge pull request #1749 from facebook/rmadapt
Yann Collet [Wed, 28 Aug 2019 19:26:29 +0000 (12:26 -0700)] 
Merge pull request #1749 from facebook/rmadapt

removed adaptive-compression

6 years agoMerge branch 'dev' into decTest
Yann Collet [Wed, 28 Aug 2019 19:17:29 +0000 (12:17 -0700)] 
Merge branch 'dev' into decTest

6 years agoMerge pull request #1750 from cwoffenden/amalgamate-bash2sh
Yann Collet [Wed, 28 Aug 2019 19:14:29 +0000 (12:14 -0700)] 
Merge pull request #1750 from cwoffenden/amalgamate-bash2sh

Rewrote the scripts to use sh instead of bash

6 years agoRewrote the scripts to sh instead of bash 1750/head
Carl Woffenden [Wed, 28 Aug 2019 17:20:42 +0000 (19:20 +0200)] 
Rewrote the scripts to sh instead of bash

6 years agoAdded test for multiple blocks of zeros and fixed nit about comments
bimbashrestha [Wed, 28 Aug 2019 15:32:34 +0000 (08:32 -0700)] 
Added test for multiple blocks of zeros and fixed nit about comments

6 years agotry to debug issues on CirrusCI
Yann Collet [Tue, 27 Aug 2019 23:38:21 +0000 (16:38 -0700)] 
try to debug issues on CirrusCI

6 years agoadded single_file_decoder to contrib/ tests
Yann Collet [Tue, 27 Aug 2019 23:04:12 +0000 (16:04 -0700)] 
added single_file_decoder to contrib/ tests

6 years agominor script renaming, for clarity
Yann Collet [Tue, 27 Aug 2019 23:01:39 +0000 (16:01 -0700)] 
minor script renaming, for clarity

6 years agochanged contrib project name for clarity
Yann Collet [Tue, 27 Aug 2019 22:50:47 +0000 (15:50 -0700)] 
changed contrib project name for clarity

6 years agoremoved associated test 1749/head
Yann Collet [Tue, 27 Aug 2019 21:50:18 +0000 (14:50 -0700)] 
removed associated test

6 years agoremoved adaptive-compression
Yann Collet [Tue, 27 Aug 2019 21:47:40 +0000 (14:47 -0700)] 
removed adaptive-compression

the functionality is already integrated into `zstd` through `--adapt` command

6 years agoMerge pull request #1748 from terrelln/cover-deadlock
Nick Terrell [Tue, 27 Aug 2019 17:17:28 +0000 (10:17 -0700)] 
Merge pull request #1748 from terrelln/cover-deadlock

[dictBuilder] Fix deadlock in *COVER error case

6 years agoMerge pull request #1739 from cwoffenden/amalgamate
Yann Collet [Tue, 27 Aug 2019 17:05:26 +0000 (10:05 -0700)] 
Merge pull request #1739 from cwoffenden/amalgamate

Tweaks to create a single-file decompressor

6 years agoFixed test failure when Emscripten not present 1739/head
Carl Woffenden [Tue, 27 Aug 2019 15:12:57 +0000 (17:12 +0200)] 
Fixed test failure when Emscripten not present