]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
6 years agomerge
Sen Huang [Mon, 21 Oct 2019 22:46:17 +0000 (18:46 -0400)] 
merge

6 years agoAdded multiplier, renamed new enum to something more useful
Sen Huang [Mon, 21 Oct 2019 19:29:55 +0000 (15:29 -0400)] 
Added multiplier, renamed new enum to something more useful

6 years agoUpdated forceAttachDict param bounds
Sen Huang [Mon, 21 Oct 2019 02:01:08 +0000 (22:01 -0400)] 
Updated forceAttachDict param bounds

6 years agoAdd enum to decision process
Sen Huang [Sun, 20 Oct 2019 23:02:47 +0000 (19:02 -0400)] 
Add enum to decision process

6 years agoAdded support for forcing new CDict behavior and updated enum
Sen Huang [Sun, 20 Oct 2019 18:03:09 +0000 (14:03 -0400)] 
Added support for forcing new CDict behavior and updated enum

6 years agoChanged to int from BYTE
Sen Huang [Wed, 16 Oct 2019 19:06:02 +0000 (15:06 -0400)] 
Changed to int from BYTE

6 years agoChanged to int from BYTE
Sen Huang [Wed, 16 Oct 2019 19:05:29 +0000 (15:05 -0400)] 
Changed to int from BYTE

6 years agoTrailing whitespace
Sen Huang [Wed, 16 Oct 2019 14:31:27 +0000 (10:31 -0400)] 
Trailing whitespace

6 years agoFix test
Sen Huang [Wed, 16 Oct 2019 14:29:31 +0000 (10:29 -0400)] 
Fix test

6 years agoCast to BYTE to appease appveyor
Sen Huang [Tue, 15 Oct 2019 17:58:44 +0000 (13:58 -0400)] 
Cast to BYTE to appease appveyor

6 years agomerge conflicts round 2
Sen Huang [Tue, 15 Oct 2019 17:10:05 +0000 (13:10 -0400)] 
merge conflicts round 2

6 years agoformatting
Sen Huang [Tue, 15 Oct 2019 16:44:48 +0000 (12:44 -0400)] 
formatting

6 years agoFix error
Sen Huang [Tue, 15 Oct 2019 16:28:23 +0000 (12:28 -0400)] 
Fix error

6 years agoAdd compressionlevel to cdict
Sen Huang [Tue, 15 Oct 2019 14:22:06 +0000 (10:22 -0400)] 
Add compressionlevel to cdict

6 years agoUpdated forceAttachDict param bounds
Sen Huang [Mon, 21 Oct 2019 02:01:08 +0000 (22:01 -0400)] 
Updated forceAttachDict param bounds

6 years agoAdd enum to decision process
Sen Huang [Sun, 20 Oct 2019 23:02:47 +0000 (19:02 -0400)] 
Add enum to decision process

6 years agoAdded support for forcing new CDict behavior and updated enum
Sen Huang [Sun, 20 Oct 2019 18:03:09 +0000 (14:03 -0400)] 
Added support for forcing new CDict behavior and updated enum

6 years agoChanged to int from BYTE
Sen Huang [Wed, 16 Oct 2019 19:06:02 +0000 (15:06 -0400)] 
Changed to int from BYTE

6 years agoChanged to int from BYTE
Sen Huang [Wed, 16 Oct 2019 19:05:29 +0000 (15:05 -0400)] 
Changed to int from BYTE

6 years agoTrailing whitespace
Sen Huang [Wed, 16 Oct 2019 14:31:27 +0000 (10:31 -0400)] 
Trailing whitespace

6 years agoFix test
Sen Huang [Wed, 16 Oct 2019 14:29:31 +0000 (10:29 -0400)] 
Fix test

6 years agoCast to BYTE to appease appveyor
Sen Huang [Tue, 15 Oct 2019 17:58:44 +0000 (13:58 -0400)] 
Cast to BYTE to appease appveyor

6 years agomerge conflicts round 2
Sen Huang [Tue, 15 Oct 2019 17:10:05 +0000 (13:10 -0400)] 
merge conflicts round 2

6 years agomerge conflict
Sen Huang [Tue, 15 Oct 2019 16:58:50 +0000 (12:58 -0400)] 
merge conflict

6 years agoformatting
Sen Huang [Tue, 15 Oct 2019 16:44:48 +0000 (12:44 -0400)] 
formatting

6 years agoFix error
Sen Huang [Tue, 15 Oct 2019 16:28:23 +0000 (12:28 -0400)] 
Fix error

6 years agoAdd compressionlevel to cdict
Sen Huang [Tue, 15 Oct 2019 14:22:06 +0000 (10:22 -0400)] 
Add compressionlevel to cdict

6 years agoMerge pull request #1731 from chungy/stdin_nochmod
Yann Collet [Mon, 14 Oct 2019 16:34:56 +0000 (09:34 -0700)] 
Merge pull request #1731 from chungy/stdin_nochmod

[programs] don’t do chmod when coming from stdin or multiple files

6 years agoMerge pull request #1792 from dmitri-shubin/dev
Yann Collet [Mon, 14 Oct 2019 16:27:25 +0000 (09:27 -0700)] 
Merge pull request #1792 from dmitri-shubin/dev

Don't hardcode installation directories in CMakeLists.txt

6 years agoMerge pull request #1812 from senhuang42/output-dir-flat
Yann Collet [Mon, 14 Oct 2019 16:25:05 +0000 (09:25 -0700)] 
Merge pull request #1812 from senhuang42/output-dir-flat

Support for --output-dir-flat: output into a directory

6 years agoMerge pull request #1823 from vangyzen/fwrite_errno
Yann Collet [Mon, 14 Oct 2019 16:20:18 +0000 (09:20 -0700)] 
Merge pull request #1823 from vangyzen/fwrite_errno

Include errno in all fwrite error messages in fileio.c

6 years agoMerge pull request #1760 from bimbashrestha/extract_sequences_api
Yann Collet [Thu, 10 Oct 2019 20:11:18 +0000 (13:11 -0700)] 
Merge pull request #1760 from bimbashrestha/extract_sequences_api

Adding api for extracting sequences from seqstore

6 years agoInclude errno in all fwrite error messages in fileio.c 1823/head
Eric van Gyzen [Thu, 10 Oct 2019 14:21:29 +0000 (09:21 -0500)] 
Include errno in all fwrite error messages in fileio.c

6 years agoOnly check for filename collisions when using --output-dir-flat 1812/head
Sen Huang [Wed, 9 Oct 2019 13:39:52 +0000 (09:39 -0400)] 
Only check for filename collisions when using --output-dir-flat

6 years agoMerge pull request #1820 from mhlakhani/dev
Nick Terrell [Tue, 8 Oct 2019 21:58:16 +0000 (17:58 -0400)] 
Merge pull request #1820 from mhlakhani/dev

Add OSS-Fuzz fuzzing badge

6 years agoAdd OSS-Fuzz fuzzing badge 1820/head
Hasnain Lakhani [Tue, 8 Oct 2019 21:13:45 +0000 (14:13 -0700)] 
Add OSS-Fuzz fuzzing badge

This is basically redoing https://github.com/facebook/zstd/pull/1818 by @inferno-chromium from the OSS-Fuzz team

6 years agoCasting to void* to avoid C4090 warnings
Sen Huang [Tue, 8 Oct 2019 13:54:59 +0000 (09:54 -0400)] 
Casting to void* to avoid C4090 warnings

6 years agoSuppress C4090 warning
Sen Huang [Tue, 8 Oct 2019 13:44:24 +0000 (09:44 -0400)] 
Suppress C4090 warning

6 years agoHopefully fix VS compiler warning
Sen Huang [Tue, 8 Oct 2019 04:15:28 +0000 (00:15 -0400)] 
Hopefully fix VS compiler warning

6 years agoChanged function name, fix test
Sen Huang [Thu, 3 Oct 2019 19:23:49 +0000 (15:23 -0400)] 
Changed function name, fix test

6 years agoNow constructs final destination path without allocating new table
Sen Huang [Thu, 3 Oct 2019 17:53:04 +0000 (13:53 -0400)] 
Now constructs final destination path without allocating new table

6 years agoManually moving instead of memcpy on decoder and using genBuffer() 1760/head
Bimba Shrestha [Thu, 3 Oct 2019 16:26:51 +0000 (09:26 -0700)] 
Manually moving instead of memcpy on decoder and using genBuffer()

6 years agoCleaning sequence parsing logic
Bimba Shrestha [Thu, 3 Oct 2019 13:42:40 +0000 (06:42 -0700)] 
Cleaning sequence parsing logic

6 years agoAddressed comments on malloc
Sen Huang [Wed, 2 Oct 2019 22:34:42 +0000 (18:34 -0400)] 
Addressed comments on malloc

6 years agoStatic analyze fix
Sen Huang [Wed, 2 Oct 2019 19:30:24 +0000 (15:30 -0400)] 
Static analyze fix

6 years agoCI Tests fix
Sen Huang [Wed, 2 Oct 2019 15:29:34 +0000 (11:29 -0400)] 
CI Tests fix

6 years agoRemoved strdup() dependency
Sen Huang [Wed, 2 Oct 2019 15:18:14 +0000 (11:18 -0400)] 
Removed strdup() dependency

6 years agoModified message
Sen Huang [Wed, 2 Oct 2019 15:11:35 +0000 (11:11 -0400)] 
Modified message

6 years agoAdd support for --output-dir-flat
Sen Huang [Wed, 2 Oct 2019 15:08:20 +0000 (11:08 -0400)] 
Add support for --output-dir-flat

New flag to specify output directory destination for multiple files.

6 years agoTypo fix
Bimba Shrestha [Fri, 27 Sep 2019 14:20:20 +0000 (07:20 -0700)] 
Typo fix

6 years agoFixing appveyor test
Bimba Shrestha [Thu, 26 Sep 2019 23:21:57 +0000 (16:21 -0700)] 
Fixing appveyor test

6 years agoFixing shortest failure
Bimba Shrestha [Thu, 26 Sep 2019 23:07:34 +0000 (16:07 -0700)] 
Fixing shortest failure

6 years agoAdding more realistic test for get sequences
Bimba Shrestha [Thu, 26 Sep 2019 22:38:31 +0000 (15:38 -0700)] 
Adding more realistic test for get sequences

6 years agobump version number to v1.4.4
Yann Collet [Tue, 24 Sep 2019 22:15:33 +0000 (15:15 -0700)] 
bump version number to v1.4.4

so that future reports on `dev` branch use this number instead

6 years agoReplacing assert with memory_allocation error code throw
Bimba Shrestha [Mon, 23 Sep 2019 22:42:16 +0000 (15:42 -0700)] 
Replacing assert with memory_allocation error code throw

6 years agoAdding test and null check for malloc
Bimba Shrestha [Mon, 23 Sep 2019 22:08:18 +0000 (15:08 -0700)] 
Adding test and null check for malloc

6 years agoMerge pull request #1808 from davidbolvansky/highset-codegen
Yann Collet [Mon, 23 Sep 2019 20:44:27 +0000 (13:44 -0700)] 
Merge pull request #1808 from davidbolvansky/highset-codegen

Improve ZSTD_highbit32's codegen

6 years agoFixed one more place 1808/head
Dávid Bolvanský [Mon, 23 Sep 2019 19:32:56 +0000 (21:32 +0200)] 
Fixed one more place

6 years agoUse clz ^ 31 instead of 31 - clz; better codegen for GCC
Dávid Bolvanský [Mon, 23 Sep 2019 19:23:09 +0000 (21:23 +0200)] 
Use clz ^ 31 instead of 31 - clz; better codegen for GCC

6 years agoMerge pull request #1804 from terrelln/wild-and-fast
Nick Terrell [Sun, 22 Sep 2019 00:04:36 +0000 (17:04 -0700)] 
Merge pull request #1804 from terrelln/wild-and-fast

Optimize (de)compression and fix wildcopy overread

6 years agoAdd UNUSED_ATTR to ZSTD_storeSeq() 1804/head
Nick Terrell [Sat, 21 Sep 2019 04:37:13 +0000 (21:37 -0700)] 
Add UNUSED_ATTR to ZSTD_storeSeq()

6 years agoHINT_INLINE ZSTD_storeSeq()
Nick Terrell [Fri, 20 Sep 2019 23:39:27 +0000 (16:39 -0700)] 
HINT_INLINE ZSTD_storeSeq()

Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.

6 years agoPassing in dummy dst buffer of compressbound(srcSize)
Bimba Shrestha [Fri, 20 Sep 2019 22:50:58 +0000 (15:50 -0700)] 
Passing in dummy dst buffer of compressbound(srcSize)

6 years agoMerge pull request #1806 from felixhandte/estimate-cctx-doc
Felix Handte [Fri, 20 Sep 2019 19:36:00 +0000 (15:36 -0400)] 
Merge pull request #1806 from felixhandte/estimate-cctx-doc

Update Comment on `ZSTD_estimateCCtxSize()`

6 years agoRemove literals overread in ZSTD_storeSeq() for ~neutral perf
Nick Terrell [Fri, 20 Sep 2019 19:23:25 +0000 (12:23 -0700)] 
Remove literals overread in ZSTD_storeSeq() for ~neutral perf

6 years agoUpdate Comment on `ZSTD_estimateCCtxSize()` 1806/head
W. Felix Handte [Fri, 20 Sep 2019 18:11:29 +0000 (14:11 -0400)] 
Update Comment on `ZSTD_estimateCCtxSize()`

6 years agoFix bounds check in ZSTD_storeSeq()
Nick Terrell [Fri, 20 Sep 2019 15:25:12 +0000 (08:25 -0700)] 
Fix bounds check in ZSTD_storeSeq()

6 years agoFix too strict assert
Nick Terrell [Fri, 20 Sep 2019 08:23:35 +0000 (01:23 -0700)] 
Fix too strict assert

6 years ago[tests] Fix decodecorpus
Nick Terrell [Fri, 20 Sep 2019 08:09:47 +0000 (01:09 -0700)] 
[tests] Fix decodecorpus

6 years agoPass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()
Nick Terrell [Fri, 20 Sep 2019 07:52:55 +0000 (00:52 -0700)] 
Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()

6 years agoWiden ZSTD_wildcopy to 32 bytes
Nick Terrell [Fri, 20 Sep 2019 07:52:15 +0000 (00:52 -0700)] 
Widen ZSTD_wildcopy to 32 bytes

6 years agoOptimize decompression and fix wildcopy overread
Nick Terrell [Thu, 19 Sep 2019 20:25:03 +0000 (13:25 -0700)] 
Optimize decompression and fix wildcopy overread

* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.

6 years agoAddressing comments
Bimba Shrestha [Thu, 19 Sep 2019 22:25:20 +0000 (15:25 -0700)] 
Addressing comments

6 years agoMerge pull request #1801 from terrelln/int-max
Nick Terrell [Thu, 19 Sep 2019 18:10:13 +0000 (11:10 -0700)] 
Merge pull request #1801 from terrelln/int-max

[test] Test the bounds of ZSTD_c_srcSizeHint

6 years agoMerge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2
Yann Collet [Wed, 18 Sep 2019 23:32:37 +0000 (16:32 -0700)] 
Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2

Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…

6 years agoadding bit container size 1802/head
Bimba Shrestha [Wed, 18 Sep 2019 20:49:45 +0000 (13:49 -0700)] 
adding bit container size

6 years agoAdding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week)
Bimba Shrestha [Wed, 18 Sep 2019 20:29:05 +0000 (13:29 -0700)] 
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week)

6 years agoMerge pull request #1756 from mgrice/dev
Yann Collet [Wed, 18 Sep 2019 18:35:50 +0000 (11:35 -0700)] 
Merge pull request #1756 from mgrice/dev

Improvements in zstd decode performance

6 years ago[test] Test the bounds of ZSTD_c_srcSizeHint 1801/head
Nick Terrell [Wed, 18 Sep 2019 18:05:08 +0000 (11:05 -0700)] 
[test] Test the bounds of ZSTD_c_srcSizeHint

6 years agoCasting void pointer to ZSTD_Sequence pointer
Bimba Shrestha [Wed, 18 Sep 2019 00:44:08 +0000 (17:44 -0700)] 
Casting void pointer to ZSTD_Sequence pointer

6 years agoMerge pull request #1798 from facebook/refac_fast
Yann Collet [Tue, 17 Sep 2019 21:54:23 +0000 (14:54 -0700)] 
Merge pull request #1798 from facebook/refac_fast

minor refactor of ZSTD_fast

6 years agominor refactor of ZSTD_fast 1798/head
Yann Collet [Tue, 17 Sep 2019 21:02:57 +0000 (14:02 -0700)] 
minor refactor of ZSTD_fast

- reduced variables lifetime
- more accurate code comments

6 years agoMerge pull request #1783 from felixhandte/mtime-nsec
Felix Handte [Tue, 17 Sep 2019 17:30:21 +0000 (13:30 -0400)] 
Merge pull request #1783 from felixhandte/mtime-nsec

Set Mod Time Nanoseconds

6 years agoResolving appveyor test failure implicit conversion
Bimba Shrestha [Mon, 16 Sep 2019 21:02:23 +0000 (14:02 -0700)] 
Resolving appveyor test failure implicit conversion

6 years agoMerge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into...
Bimba Shrestha [Mon, 16 Sep 2019 20:35:49 +0000 (13:35 -0700)] 
Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api

6 years agoHandling memory leak and potential side effect
Bimba Shrestha [Mon, 16 Sep 2019 20:35:45 +0000 (13:35 -0700)] 
Handling memory leak and potential side effect

6 years agoMerge branch 'dev' into extract_sequences_api
Bimba Shrestha [Mon, 16 Sep 2019 20:29:59 +0000 (13:29 -0700)] 
Merge branch 'dev' into extract_sequences_api

6 years agoMerge pull request #1780 from felixhandte/workspace-efficiency-3
Felix Handte [Mon, 16 Sep 2019 18:37:05 +0000 (14:37 -0400)] 
Merge pull request #1780 from felixhandte/workspace-efficiency-3

Avoid Clearing Tables Even When Changing CParams

6 years agoDon't Include `sanitizer/msan_interface.h`, Since Not All Platforms Provide It 1780/head
W. Felix Handte [Mon, 16 Sep 2019 16:08:03 +0000 (12:08 -0400)] 
Don't Include `sanitizer/msan_interface.h`, Since Not All Platforms Provide It

Instead, explicitly declare the functions we use.

6 years agoBailing early when collecting sequences and documentation
Bimba Shrestha [Mon, 16 Sep 2019 15:26:21 +0000 (08:26 -0700)] 
Bailing early when collecting sequences and documentation

6 years agoDon't hardcode installation directories in CMakeLists.txt 1792/head
Dmitri Shubin [Mon, 16 Sep 2019 08:36:37 +0000 (11:36 +0300)] 
Don't hardcode installation directories in CMakeLists.txt

Use paths provided by GNUInstallDirs module instead.

6 years agoMerge pull request #1791 from terrelln/doc-up
Nick Terrell [Sun, 15 Sep 2019 21:50:55 +0000 (14:50 -0700)] 
Merge pull request #1791 from terrelln/doc-up

[libzstd] Improve advanced API docs

6 years ago[libzstd] Improve advanced API docs 1791/head
Nick Terrell [Sun, 15 Sep 2019 19:41:24 +0000 (12:41 -0700)] 
[libzstd] Improve advanced API docs

6 years agoMerge pull request #1789 from terrelln/larger-fuzz
Nick Terrell [Fri, 13 Sep 2019 21:13:34 +0000 (14:13 -0700)] 
Merge pull request #1789 from terrelln/larger-fuzz

[fuzz] Fix leak in block_round_trip

6 years ago[fuzz] Fix leak in block_round_trip 1789/head
Nick Terrell [Fri, 13 Sep 2019 17:32:38 +0000 (10:32 -0700)] 
[fuzz] Fix leak in block_round_trip

6 years agoMerge pull request #1784 from bimbashrestha/fse_block_bound_err
Yann Collet [Fri, 13 Sep 2019 02:09:27 +0000 (19:09 -0700)] 
Merge pull request #1784 from bimbashrestha/fse_block_bound_err

Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()

6 years agoMerge pull request #1785 from terrelln/larger-fuzz
Nick Terrell [Fri, 13 Sep 2019 00:21:10 +0000 (17:21 -0700)] 
Merge pull request #1785 from terrelln/larger-fuzz

[fuzz] Generate seed data up to 256KB

6 years agoAdded assert to BIT_flushBits() 1784/head
Bimba Shrestha [Thu, 12 Sep 2019 22:35:27 +0000 (15:35 -0700)] 
Added assert to BIT_flushBits()

6 years ago[fuzz] Generate seed data up to 256KB 1785/head
Nick Terrell [Thu, 12 Sep 2019 22:02:01 +0000 (15:02 -0700)] 
[fuzz] Generate seed data up to 256KB

6 years agoRearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
Bimba Shrestha [Thu, 12 Sep 2019 21:43:50 +0000 (14:43 -0700)] 
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()