]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
5 years agoCast to BYTE to appease appveyor
Sen Huang [Tue, 15 Oct 2019 17:58:44 +0000 (13:58 -0400)] 
Cast to BYTE to appease appveyor

5 years agomerge conflicts round 2
Sen Huang [Tue, 15 Oct 2019 17:10:05 +0000 (13:10 -0400)] 
merge conflicts round 2

5 years agomerge conflict
Sen Huang [Tue, 15 Oct 2019 16:58:50 +0000 (12:58 -0400)] 
merge conflict

5 years agoformatting
Sen Huang [Tue, 15 Oct 2019 16:44:48 +0000 (12:44 -0400)] 
formatting

5 years agoFix error
Sen Huang [Tue, 15 Oct 2019 16:28:23 +0000 (12:28 -0400)] 
Fix error

5 years agoAdd compressionlevel to cdict
Sen Huang [Tue, 15 Oct 2019 14:22:06 +0000 (10:22 -0400)] 
Add compressionlevel to cdict

5 years agoMerge pull request #1731 from chungy/stdin_nochmod
Yann Collet [Mon, 14 Oct 2019 16:34:56 +0000 (09:34 -0700)] 
Merge pull request #1731 from chungy/stdin_nochmod

[programs] don’t do chmod when coming from stdin or multiple files

5 years agoMerge pull request #1792 from dmitri-shubin/dev
Yann Collet [Mon, 14 Oct 2019 16:27:25 +0000 (09:27 -0700)] 
Merge pull request #1792 from dmitri-shubin/dev

Don't hardcode installation directories in CMakeLists.txt

5 years agoMerge pull request #1812 from senhuang42/output-dir-flat
Yann Collet [Mon, 14 Oct 2019 16:25:05 +0000 (09:25 -0700)] 
Merge pull request #1812 from senhuang42/output-dir-flat

Support for --output-dir-flat: output into a directory

5 years agoMerge pull request #1823 from vangyzen/fwrite_errno
Yann Collet [Mon, 14 Oct 2019 16:20:18 +0000 (09:20 -0700)] 
Merge pull request #1823 from vangyzen/fwrite_errno

Include errno in all fwrite error messages in fileio.c

5 years agoMerge pull request #1760 from bimbashrestha/extract_sequences_api
Yann Collet [Thu, 10 Oct 2019 20:11:18 +0000 (13:11 -0700)] 
Merge pull request #1760 from bimbashrestha/extract_sequences_api

Adding api for extracting sequences from seqstore

5 years agoInclude errno in all fwrite error messages in fileio.c 1823/head
Eric van Gyzen [Thu, 10 Oct 2019 14:21:29 +0000 (09:21 -0500)] 
Include errno in all fwrite error messages in fileio.c

5 years agoOnly check for filename collisions when using --output-dir-flat 1812/head
Sen Huang [Wed, 9 Oct 2019 13:39:52 +0000 (09:39 -0400)] 
Only check for filename collisions when using --output-dir-flat

5 years agoMerge pull request #1820 from mhlakhani/dev
Nick Terrell [Tue, 8 Oct 2019 21:58:16 +0000 (17:58 -0400)] 
Merge pull request #1820 from mhlakhani/dev

Add OSS-Fuzz fuzzing badge

5 years agoAdd OSS-Fuzz fuzzing badge 1820/head
Hasnain Lakhani [Tue, 8 Oct 2019 21:13:45 +0000 (14:13 -0700)] 
Add OSS-Fuzz fuzzing badge

This is basically redoing https://github.com/facebook/zstd/pull/1818 by @inferno-chromium from the OSS-Fuzz team

5 years agoCasting to void* to avoid C4090 warnings
Sen Huang [Tue, 8 Oct 2019 13:54:59 +0000 (09:54 -0400)] 
Casting to void* to avoid C4090 warnings

5 years agoSuppress C4090 warning
Sen Huang [Tue, 8 Oct 2019 13:44:24 +0000 (09:44 -0400)] 
Suppress C4090 warning

5 years agoHopefully fix VS compiler warning
Sen Huang [Tue, 8 Oct 2019 04:15:28 +0000 (00:15 -0400)] 
Hopefully fix VS compiler warning

5 years agoChanged function name, fix test
Sen Huang [Thu, 3 Oct 2019 19:23:49 +0000 (15:23 -0400)] 
Changed function name, fix test

5 years agoNow constructs final destination path without allocating new table
Sen Huang [Thu, 3 Oct 2019 17:53:04 +0000 (13:53 -0400)] 
Now constructs final destination path without allocating new table

5 years agoManually moving instead of memcpy on decoder and using genBuffer() 1760/head
Bimba Shrestha [Thu, 3 Oct 2019 16:26:51 +0000 (09:26 -0700)] 
Manually moving instead of memcpy on decoder and using genBuffer()

5 years agoCleaning sequence parsing logic
Bimba Shrestha [Thu, 3 Oct 2019 13:42:40 +0000 (06:42 -0700)] 
Cleaning sequence parsing logic

5 years agoAddressed comments on malloc
Sen Huang [Wed, 2 Oct 2019 22:34:42 +0000 (18:34 -0400)] 
Addressed comments on malloc

5 years agoStatic analyze fix
Sen Huang [Wed, 2 Oct 2019 19:30:24 +0000 (15:30 -0400)] 
Static analyze fix

5 years agoCI Tests fix
Sen Huang [Wed, 2 Oct 2019 15:29:34 +0000 (11:29 -0400)] 
CI Tests fix

5 years agoRemoved strdup() dependency
Sen Huang [Wed, 2 Oct 2019 15:18:14 +0000 (11:18 -0400)] 
Removed strdup() dependency

5 years agoModified message
Sen Huang [Wed, 2 Oct 2019 15:11:35 +0000 (11:11 -0400)] 
Modified message

5 years agoAdd support for --output-dir-flat
Sen Huang [Wed, 2 Oct 2019 15:08:20 +0000 (11:08 -0400)] 
Add support for --output-dir-flat

New flag to specify output directory destination for multiple files.

5 years agoTypo fix
Bimba Shrestha [Fri, 27 Sep 2019 14:20:20 +0000 (07:20 -0700)] 
Typo fix

5 years agoFixing appveyor test
Bimba Shrestha [Thu, 26 Sep 2019 23:21:57 +0000 (16:21 -0700)] 
Fixing appveyor test

5 years agoFixing shortest failure
Bimba Shrestha [Thu, 26 Sep 2019 23:07:34 +0000 (16:07 -0700)] 
Fixing shortest failure

5 years agoAdding more realistic test for get sequences
Bimba Shrestha [Thu, 26 Sep 2019 22:38:31 +0000 (15:38 -0700)] 
Adding more realistic test for get sequences

5 years agobump version number to v1.4.4
Yann Collet [Tue, 24 Sep 2019 22:15:33 +0000 (15:15 -0700)] 
bump version number to v1.4.4

so that future reports on `dev` branch use this number instead

5 years agoReplacing assert with memory_allocation error code throw
Bimba Shrestha [Mon, 23 Sep 2019 22:42:16 +0000 (15:42 -0700)] 
Replacing assert with memory_allocation error code throw

5 years agoAdding test and null check for malloc
Bimba Shrestha [Mon, 23 Sep 2019 22:08:18 +0000 (15:08 -0700)] 
Adding test and null check for malloc

5 years agoMerge pull request #1808 from davidbolvansky/highset-codegen
Yann Collet [Mon, 23 Sep 2019 20:44:27 +0000 (13:44 -0700)] 
Merge pull request #1808 from davidbolvansky/highset-codegen

Improve ZSTD_highbit32's codegen

5 years agoFixed one more place 1808/head
Dávid Bolvanský [Mon, 23 Sep 2019 19:32:56 +0000 (21:32 +0200)] 
Fixed one more place

5 years agoUse clz ^ 31 instead of 31 - clz; better codegen for GCC
Dávid Bolvanský [Mon, 23 Sep 2019 19:23:09 +0000 (21:23 +0200)] 
Use clz ^ 31 instead of 31 - clz; better codegen for GCC

5 years agoMerge pull request #1804 from terrelln/wild-and-fast
Nick Terrell [Sun, 22 Sep 2019 00:04:36 +0000 (17:04 -0700)] 
Merge pull request #1804 from terrelln/wild-and-fast

Optimize (de)compression and fix wildcopy overread

5 years agoAdd UNUSED_ATTR to ZSTD_storeSeq() 1804/head
Nick Terrell [Sat, 21 Sep 2019 04:37:13 +0000 (21:37 -0700)] 
Add UNUSED_ATTR to ZSTD_storeSeq()

5 years agoHINT_INLINE ZSTD_storeSeq()
Nick Terrell [Fri, 20 Sep 2019 23:39:27 +0000 (16:39 -0700)] 
HINT_INLINE ZSTD_storeSeq()

Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.

5 years agoPassing in dummy dst buffer of compressbound(srcSize)
Bimba Shrestha [Fri, 20 Sep 2019 22:50:58 +0000 (15:50 -0700)] 
Passing in dummy dst buffer of compressbound(srcSize)

5 years agoMerge pull request #1806 from felixhandte/estimate-cctx-doc
Felix Handte [Fri, 20 Sep 2019 19:36:00 +0000 (15:36 -0400)] 
Merge pull request #1806 from felixhandte/estimate-cctx-doc

Update Comment on `ZSTD_estimateCCtxSize()`

5 years agoRemove literals overread in ZSTD_storeSeq() for ~neutral perf
Nick Terrell [Fri, 20 Sep 2019 19:23:25 +0000 (12:23 -0700)] 
Remove literals overread in ZSTD_storeSeq() for ~neutral perf

5 years agoUpdate Comment on `ZSTD_estimateCCtxSize()` 1806/head
W. Felix Handte [Fri, 20 Sep 2019 18:11:29 +0000 (14:11 -0400)] 
Update Comment on `ZSTD_estimateCCtxSize()`

5 years agoFix bounds check in ZSTD_storeSeq()
Nick Terrell [Fri, 20 Sep 2019 15:25:12 +0000 (08:25 -0700)] 
Fix bounds check in ZSTD_storeSeq()

5 years agoFix too strict assert
Nick Terrell [Fri, 20 Sep 2019 08:23:35 +0000 (01:23 -0700)] 
Fix too strict assert

5 years ago[tests] Fix decodecorpus
Nick Terrell [Fri, 20 Sep 2019 08:09:47 +0000 (01:09 -0700)] 
[tests] Fix decodecorpus

5 years agoPass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()
Nick Terrell [Fri, 20 Sep 2019 07:52:55 +0000 (00:52 -0700)] 
Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()

5 years agoWiden ZSTD_wildcopy to 32 bytes
Nick Terrell [Fri, 20 Sep 2019 07:52:15 +0000 (00:52 -0700)] 
Widen ZSTD_wildcopy to 32 bytes

5 years agoOptimize decompression and fix wildcopy overread
Nick Terrell [Thu, 19 Sep 2019 20:25:03 +0000 (13:25 -0700)] 
Optimize decompression and fix wildcopy overread

* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.

5 years agoAddressing comments
Bimba Shrestha [Thu, 19 Sep 2019 22:25:20 +0000 (15:25 -0700)] 
Addressing comments

5 years agoMerge pull request #1801 from terrelln/int-max
Nick Terrell [Thu, 19 Sep 2019 18:10:13 +0000 (11:10 -0700)] 
Merge pull request #1801 from terrelln/int-max

[test] Test the bounds of ZSTD_c_srcSizeHint

5 years agoMerge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2
Yann Collet [Wed, 18 Sep 2019 23:32:37 +0000 (16:32 -0700)] 
Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2

Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…

5 years agoadding bit container size 1802/head
Bimba Shrestha [Wed, 18 Sep 2019 20:49:45 +0000 (13:49 -0700)] 
adding bit container size

5 years agoAdding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week)
Bimba Shrestha [Wed, 18 Sep 2019 20:29:05 +0000 (13:29 -0700)] 
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week)

5 years agoMerge pull request #1756 from mgrice/dev
Yann Collet [Wed, 18 Sep 2019 18:35:50 +0000 (11:35 -0700)] 
Merge pull request #1756 from mgrice/dev

Improvements in zstd decode performance

5 years ago[test] Test the bounds of ZSTD_c_srcSizeHint 1801/head
Nick Terrell [Wed, 18 Sep 2019 18:05:08 +0000 (11:05 -0700)] 
[test] Test the bounds of ZSTD_c_srcSizeHint

5 years agoCasting void pointer to ZSTD_Sequence pointer
Bimba Shrestha [Wed, 18 Sep 2019 00:44:08 +0000 (17:44 -0700)] 
Casting void pointer to ZSTD_Sequence pointer

5 years agoMerge pull request #1798 from facebook/refac_fast
Yann Collet [Tue, 17 Sep 2019 21:54:23 +0000 (14:54 -0700)] 
Merge pull request #1798 from facebook/refac_fast

minor refactor of ZSTD_fast

5 years agominor refactor of ZSTD_fast 1798/head
Yann Collet [Tue, 17 Sep 2019 21:02:57 +0000 (14:02 -0700)] 
minor refactor of ZSTD_fast

- reduced variables lifetime
- more accurate code comments

5 years agoMerge pull request #1783 from felixhandte/mtime-nsec
Felix Handte [Tue, 17 Sep 2019 17:30:21 +0000 (13:30 -0400)] 
Merge pull request #1783 from felixhandte/mtime-nsec

Set Mod Time Nanoseconds

5 years agoResolving appveyor test failure implicit conversion
Bimba Shrestha [Mon, 16 Sep 2019 21:02:23 +0000 (14:02 -0700)] 
Resolving appveyor test failure implicit conversion

5 years agoMerge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into...
Bimba Shrestha [Mon, 16 Sep 2019 20:35:49 +0000 (13:35 -0700)] 
Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api

5 years agoHandling memory leak and potential side effect
Bimba Shrestha [Mon, 16 Sep 2019 20:35:45 +0000 (13:35 -0700)] 
Handling memory leak and potential side effect

5 years agoMerge branch 'dev' into extract_sequences_api
Bimba Shrestha [Mon, 16 Sep 2019 20:29:59 +0000 (13:29 -0700)] 
Merge branch 'dev' into extract_sequences_api

5 years agoMerge pull request #1780 from felixhandte/workspace-efficiency-3
Felix Handte [Mon, 16 Sep 2019 18:37:05 +0000 (14:37 -0400)] 
Merge pull request #1780 from felixhandte/workspace-efficiency-3

Avoid Clearing Tables Even When Changing CParams

5 years agoDon't Include `sanitizer/msan_interface.h`, Since Not All Platforms Provide It 1780/head
W. Felix Handte [Mon, 16 Sep 2019 16:08:03 +0000 (12:08 -0400)] 
Don't Include `sanitizer/msan_interface.h`, Since Not All Platforms Provide It

Instead, explicitly declare the functions we use.

5 years agoBailing early when collecting sequences and documentation
Bimba Shrestha [Mon, 16 Sep 2019 15:26:21 +0000 (08:26 -0700)] 
Bailing early when collecting sequences and documentation

5 years agoDon't hardcode installation directories in CMakeLists.txt 1792/head
Dmitri Shubin [Mon, 16 Sep 2019 08:36:37 +0000 (11:36 +0300)] 
Don't hardcode installation directories in CMakeLists.txt

Use paths provided by GNUInstallDirs module instead.

5 years agoMerge pull request #1791 from terrelln/doc-up
Nick Terrell [Sun, 15 Sep 2019 21:50:55 +0000 (14:50 -0700)] 
Merge pull request #1791 from terrelln/doc-up

[libzstd] Improve advanced API docs

5 years ago[libzstd] Improve advanced API docs 1791/head
Nick Terrell [Sun, 15 Sep 2019 19:41:24 +0000 (12:41 -0700)] 
[libzstd] Improve advanced API docs

5 years agoMerge pull request #1789 from terrelln/larger-fuzz
Nick Terrell [Fri, 13 Sep 2019 21:13:34 +0000 (14:13 -0700)] 
Merge pull request #1789 from terrelln/larger-fuzz

[fuzz] Fix leak in block_round_trip

5 years ago[fuzz] Fix leak in block_round_trip 1789/head
Nick Terrell [Fri, 13 Sep 2019 17:32:38 +0000 (10:32 -0700)] 
[fuzz] Fix leak in block_round_trip

5 years agoMerge pull request #1784 from bimbashrestha/fse_block_bound_err
Yann Collet [Fri, 13 Sep 2019 02:09:27 +0000 (19:09 -0700)] 
Merge pull request #1784 from bimbashrestha/fse_block_bound_err

Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()

5 years agoMerge pull request #1785 from terrelln/larger-fuzz
Nick Terrell [Fri, 13 Sep 2019 00:21:10 +0000 (17:21 -0700)] 
Merge pull request #1785 from terrelln/larger-fuzz

[fuzz] Generate seed data up to 256KB

5 years agoAdded assert to BIT_flushBits() 1784/head
Bimba Shrestha [Thu, 12 Sep 2019 22:35:27 +0000 (15:35 -0700)] 
Added assert to BIT_flushBits()

5 years ago[fuzz] Generate seed data up to 256KB 1785/head
Nick Terrell [Thu, 12 Sep 2019 22:02:01 +0000 (15:02 -0700)] 
[fuzz] Generate seed data up to 256KB

5 years agoRearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
Bimba Shrestha [Thu, 12 Sep 2019 21:43:50 +0000 (14:43 -0700)] 
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()

5 years agoMerge pull request #1781 from darxsys/improvDataGen
Nick Terrell [Thu, 12 Sep 2019 21:27:58 +0000 (14:27 -0700)] 
Merge pull request #1781 from darxsys/improvDataGen

Improve data generation

5 years agoFormatting and Clean Up 1783/head
W. Felix Handte [Thu, 12 Sep 2019 20:27:05 +0000 (16:27 -0400)] 
Formatting and Clean Up

5 years agoMerge branch 'improvDataGen' of github.com:darxsys/zstd into improvDataGen 1781/head
Dario Pavlovic [Thu, 12 Sep 2019 20:11:02 +0000 (13:11 -0700)] 
Merge branch 'improvDataGen' of github.com:darxsys/zstd into improvDataGen

5 years agoIt's time for all of rng seed code to go. Goodbye
Dario Pavlovic [Thu, 12 Sep 2019 20:10:34 +0000 (13:10 -0700)] 
It's time for all of rng seed code to go. Goodbye

5 years agoUpdate tests/fuzz/fuzz_data_producer.h
Dario Pavlovic [Thu, 12 Sep 2019 19:45:28 +0000 (12:45 -0700)] 
Update tests/fuzz/fuzz_data_producer.h

5 years agoUse range instead of the generic uint32 method to use less bytes when generating...
Dario Pavlovic [Thu, 12 Sep 2019 19:40:12 +0000 (12:40 -0700)] 
Use range instead of the generic uint32 method to use less bytes when generating necessary numbers.

5 years agoignore dictionary artifacts
Yann Collet [Thu, 12 Sep 2019 16:39:15 +0000 (09:39 -0700)] 
ignore dictionary artifacts

5 years agoSet M-Time Nanoseconds
W. Felix Handte [Thu, 12 Sep 2019 15:48:46 +0000 (11:48 -0400)] 
Set M-Time Nanoseconds

5 years agoMerge pull request #1708 from neheb/dev
Felix Handte [Thu, 12 Sep 2019 15:44:31 +0000 (11:44 -0400)] 
Merge pull request #1708 from neheb/dev

zstd: Don't use utime on Linux

5 years agoShrink Table Valid End During Alloc Alignment / Phase Change
W. Felix Handte [Wed, 11 Sep 2019 21:03:09 +0000 (17:03 -0400)] 
Shrink Table Valid End During Alloc Alignment / Phase Change

5 years agoAdd Assertions to Confirm that Workspace Pointers are Correctly Ordered
W. Felix Handte [Wed, 11 Sep 2019 20:41:41 +0000 (16:41 -0400)] 
Add Assertions to Confirm that Workspace Pointers are Correctly Ordered

5 years ago`__msan_poison()` Workspace When Preparing for Re-Use
W. Felix Handte [Wed, 11 Sep 2019 20:40:29 +0000 (16:40 -0400)] 
`__msan_poison()` Workspace When Preparing for Re-Use

5 years agoFix Memory Leak in Test
W. Felix Handte [Wed, 11 Sep 2019 18:25:30 +0000 (14:25 -0400)] 
Fix Memory Leak in Test

5 years agoFix Compilation Error (`uint32_t` -> `size_t`)
W. Felix Handte [Wed, 11 Sep 2019 17:59:09 +0000 (13:59 -0400)] 
Fix Compilation Error (`uint32_t` -> `size_t`)

5 years agoSpeed Up Test a Little
W. Felix Handte [Wed, 11 Sep 2019 17:23:59 +0000 (13:23 -0400)] 
Speed Up Test a Little

5 years agoAdd Fuzzer Test Case for Index Reduction
W. Felix Handte [Wed, 11 Sep 2019 17:17:19 +0000 (13:17 -0400)] 
Add Fuzzer Test Case for Index Reduction

5 years agoZero `h3size` When `h3log` is 0
W. Felix Handte [Wed, 11 Sep 2019 17:14:26 +0000 (13:14 -0400)] 
Zero `h3size` When `h3log` is 0

This led to a nasty edgecase, where index reduction for modes that don't use
the h3 table would have a degenerate table (size 4) allocated and marked clean,
but which would not be re-indexed.

5 years agoCombining fuzz_data_producer restrict calls into a single function
Dario Pavlovic [Wed, 11 Sep 2019 17:09:29 +0000 (10:09 -0700)] 
Combining fuzz_data_producer restrict calls into a single function

5 years agoAlso Shrink Clean Table Area When Reducing Indices
W. Felix Handte [Wed, 11 Sep 2019 15:40:57 +0000 (11:40 -0400)] 
Also Shrink Clean Table Area When Reducing Indices

5 years agoUpdate DEBUGLOG Statements
W. Felix Handte [Wed, 11 Sep 2019 15:21:00 +0000 (11:21 -0400)] 
Update DEBUGLOG Statements

5 years agoShrink Clean Table Area When Copying Table Contents into Context
W. Felix Handte [Wed, 11 Sep 2019 15:18:45 +0000 (11:18 -0400)] 
Shrink Clean Table Area When Copying Table Contents into Context

The source matchState is potentially at a lower current index, which means
that any extra table space not overwritten by the copy may now contain
invalid indices. The simple solution is to unconditionally shrink the valid
table area to just the area overwritten.