]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
4 years agoAdd seekable tests to CI 2408/head
senhuang42 [Thu, 3 Dec 2020 14:25:45 +0000 (09:25 -0500)] 
Add seekable tests to CI

4 years agoClean up makefile for seekable tests
senhuang42 [Thu, 3 Dec 2020 13:54:21 +0000 (08:54 -0500)] 
Clean up makefile for seekable tests

4 years agoAdd unit tests to seekable
senhuang42 [Wed, 2 Dec 2020 20:27:39 +0000 (15:27 -0500)] 
Add unit tests to seekable

4 years agoAdd a forward progress requirement bound to seekable streaming decompression
senhuang42 [Wed, 2 Dec 2020 17:24:16 +0000 (12:24 -0500)] 
Add a forward progress requirement bound to seekable streaming decompression

4 years agoMerge pull request #2348 from dscheg/dev
Yann Collet [Wed, 2 Dec 2020 01:59:56 +0000 (17:59 -0800)] 
Merge pull request #2348 from dscheg/dev

Fix dll path in case of cross-compilation

4 years agoMerge pull request #2405 from senhuang42/sequence_compression_minmatch_fix
sen [Tue, 1 Dec 2020 22:34:01 +0000 (17:34 -0500)] 
Merge pull request #2405 from senhuang42/sequence_compression_minmatch_fix

Sequence Compression API - make validator use cctx minMatch instead of global MINMATCH

4 years agoAdd sequence nb validation to compressSequences(), adjust minMatch comparisons 2405/head
senhuang42 [Tue, 1 Dec 2020 15:53:30 +0000 (10:53 -0500)] 
Add sequence nb validation to compressSequences(), adjust minMatch comparisons

4 years agoMerge pull request #2399 from brawner/brawner/update-cmake-minimum-version
Yann Collet [Mon, 30 Nov 2020 22:13:42 +0000 (14:13 -0800)] 
Merge pull request #2399 from brawner/brawner/update-cmake-minimum-version

Update cmake_minimum_required to 2.8.12

4 years agoUse cctx's minMatch instead of global MINMATCH, make fuzzer use validation
senhuang42 [Mon, 30 Nov 2020 20:38:12 +0000 (15:38 -0500)] 
Use cctx's minMatch instead of global MINMATCH, make fuzzer use validation

4 years agoMerge pull request #2397 from dscheg/fixtrainfrombuffer
Nick Terrell [Mon, 30 Nov 2020 18:38:45 +0000 (10:38 -0800)] 
Merge pull request #2397 from dscheg/fixtrainfrombuffer

Pass dictBufferCapacity to COVER_selectDict()

4 years agoMerge pull request #2404 from facebook/lz4check
Yann Collet [Mon, 30 Nov 2020 18:38:08 +0000 (10:38 -0800)] 
Merge pull request #2404 from facebook/lz4check

fix lz4 compatibility test broken in docker

4 years agofix lz4 test messed by console detection 2404/head
Yann Collet [Mon, 30 Nov 2020 14:47:16 +0000 (06:47 -0800)] 
fix lz4 test messed by console detection

4 years agoMerge pull request #2403 from facebook/no_strncpy
Yann Collet [Mon, 30 Nov 2020 14:30:26 +0000 (06:30 -0800)] 
Merge pull request #2403 from facebook/no_strncpy

fix gcc10 warnings

4 years agoadded tests presumed to break lz4 console detection
Yann Collet [Mon, 30 Nov 2020 13:32:43 +0000 (05:32 -0800)] 
added tests presumed to break lz4 console detection

4 years agofix gcc10 warnings 2403/head
Yann Collet [Mon, 30 Nov 2020 12:44:37 +0000 (04:44 -0800)] 
fix gcc10 warnings

gcc10 doesn't like its own strncpy

4 years agoUpdate cmake_minimum_required to 2.8.12 2399/head
Stephen Brawner [Tue, 24 Nov 2020 20:05:42 +0000 (12:05 -0800)] 
Update cmake_minimum_required to 2.8.12

Signed-off-by: Stephen Brawner <brawner@gmail.com>
4 years agoMerge pull request #2398 from senhuang42/fix_seq_compression_oss_fuzz
sen [Tue, 24 Nov 2020 02:54:16 +0000 (21:54 -0500)] 
Merge pull request #2398 from senhuang42/fix_seq_compression_oss_fuzz

[OSS-Fuzz] Make sequence compression fuzzer's generated minmatch to be same as CCtx's minmatch

4 years agoForce CCtx minmatch to be same as generated minmatch 2398/head
senhuang42 [Mon, 23 Nov 2020 18:29:20 +0000 (13:29 -0500)] 
Force CCtx minmatch to be same as generated minmatch

4 years agoPass dictBufferCapacity to COVER_selectDict() 2397/head
Dmitriy Titarenko [Sun, 22 Nov 2020 18:45:18 +0000 (23:45 +0500)] 
Pass dictBufferCapacity to COVER_selectDict()

closes #2371

4 years agoMerge pull request #2394 from senhuang42/seq_compress_fuzzer
sen [Sun, 22 Nov 2020 18:07:46 +0000 (13:07 -0500)] 
Merge pull request #2394 from senhuang42/seq_compress_fuzzer

Fuzzer for sequence compression API

4 years agoFix generating 1 too few characters in random string generator 2394/head
senhuang42 [Thu, 19 Nov 2020 22:34:39 +0000 (17:34 -0500)] 
Fix generating 1 too few characters in random string generator

4 years agoAdd appropriate bound to matchlengths, and reduce srcSize max
senhuang42 [Wed, 18 Nov 2020 15:25:48 +0000 (10:25 -0500)] 
Add appropriate bound to matchlengths, and reduce srcSize max

4 years agoAdd a bound for matchlength dependent on window size
senhuang42 [Tue, 17 Nov 2020 14:57:10 +0000 (09:57 -0500)] 
Add a bound for matchlength dependent on window size

4 years agoVariety of minor fixups, reduce allocation, make deterministic
senhuang42 [Mon, 16 Nov 2020 22:53:38 +0000 (17:53 -0500)] 
Variety of minor fixups, reduce allocation, make deterministic

4 years agoAdd built binary to .gitignore
senhuang42 [Mon, 16 Nov 2020 15:55:07 +0000 (10:55 -0500)] 
Add built binary to .gitignore

4 years agoAdd new fuzzer to build targets
senhuang42 [Mon, 16 Nov 2020 15:54:55 +0000 (10:54 -0500)] 
Add new fuzzer to build targets

4 years agoImplement new fuzzer for sequence compression
senhuang42 [Mon, 16 Nov 2020 15:54:19 +0000 (10:54 -0500)] 
Implement new fuzzer for sequence compression

4 years agoMerge pull request #2387 from senhuang42/compress_sequence_API
sen [Fri, 20 Nov 2020 21:54:20 +0000 (16:54 -0500)] 
Merge pull request #2387 from senhuang42/compress_sequence_API

[RFC] New sequence compression API

4 years agoAdd experimental param for sequence validation 2387/head
senhuang42 [Fri, 20 Nov 2020 16:23:22 +0000 (11:23 -0500)] 
Add experimental param for sequence validation

4 years agoRemove unnecessary repcode backup, apply style choices, use function pointer
senhuang42 [Fri, 20 Nov 2020 15:07:55 +0000 (10:07 -0500)] 
Remove unnecessary repcode backup, apply style choices, use function pointer

4 years agoMerge pull request #2395 from senhuang42/is_rle_speedup
sen [Thu, 19 Nov 2020 23:26:42 +0000 (18:26 -0500)] 
Merge pull request #2395 from senhuang42/is_rle_speedup

10x speedup for ZSTD_isRLE()

4 years agoExplicit cast for visual warnings 2395/head
sen [Thu, 19 Nov 2020 22:32:40 +0000 (17:32 -0500)] 
Explicit cast for visual warnings

Github has automatic commits now! Cool

Co-authored-by: Nick Terrell <nickrterrell@gmail.com>
4 years agoUnroll isRLE loop
senhuang42 [Thu, 19 Nov 2020 16:56:16 +0000 (11:56 -0500)] 
Unroll isRLE loop

4 years agoClean up visual conversion warnings
senhuang42 [Wed, 18 Nov 2020 15:01:30 +0000 (10:01 -0500)] 
Clean up visual conversion warnings

4 years agoImprove documentation on ZSTD_compressSequences()
senhuang42 [Wed, 18 Nov 2020 14:52:24 +0000 (09:52 -0500)] 
Improve documentation on ZSTD_compressSequences()

4 years agoModification to offset validation to include entire sequence
senhuang42 [Tue, 17 Nov 2020 15:13:22 +0000 (10:13 -0500)] 
Modification to offset validation to include entire sequence

4 years agoFix assert edge case, improve documentation in zstd.h
senhuang42 [Mon, 16 Nov 2020 23:05:35 +0000 (18:05 -0500)] 
Fix assert edge case, improve documentation in zstd.h

4 years agoFix warnings and make validation enabled by default
senhuang42 [Mon, 16 Nov 2020 16:47:27 +0000 (11:47 -0500)] 
Fix warnings and make validation enabled by default

4 years agoFix unit tests to agree with new changes
senhuang42 [Mon, 16 Nov 2020 15:47:26 +0000 (10:47 -0500)] 
Fix unit tests to agree with new changes

4 years agoAdd new sequence format as an experimental CCtx param
senhuang42 [Mon, 16 Nov 2020 15:44:57 +0000 (10:44 -0500)] 
Add new sequence format as an experimental CCtx param

4 years agoOverhaul logic to simplify, add in proper validations, fix match splitting
senhuang42 [Mon, 16 Nov 2020 15:36:06 +0000 (10:36 -0500)] 
Overhaul logic to simplify, add in proper validations, fix match splitting

4 years agoAdd new sequence compress api params to cctx
senhuang42 [Wed, 11 Nov 2020 21:28:17 +0000 (16:28 -0500)] 
Add new sequence compress api params to cctx

4 years agoFix srcSize=0 edge case
senhuang42 [Tue, 10 Nov 2020 20:33:33 +0000 (15:33 -0500)] 
Fix srcSize=0 edge case

4 years agoFix literals length calculation
senhuang42 [Tue, 10 Nov 2020 18:48:02 +0000 (13:48 -0500)] 
Fix literals length calculation

4 years agoAdjust unit tests to agree with new sequence generation API
senhuang42 [Fri, 6 Nov 2020 18:24:43 +0000 (13:24 -0500)] 
Adjust unit tests to agree with new sequence generation API

4 years agoRemove dstCapacity error check
senhuang42 [Fri, 6 Nov 2020 18:09:15 +0000 (13:09 -0500)] 
Remove dstCapacity error check

4 years agoRemove extraneous function in this API
senhuang42 [Fri, 6 Nov 2020 16:35:10 +0000 (11:35 -0500)] 
Remove extraneous function in this API

4 years agoAdd check comparing offset to window size
senhuang42 [Thu, 5 Nov 2020 17:36:08 +0000 (12:36 -0500)] 
Add check comparing offset to window size

4 years agoFix MSAN errors
senhuang42 [Wed, 4 Nov 2020 17:42:19 +0000 (12:42 -0500)] 
Fix MSAN errors

4 years agoAddress edge case with endPosInSequence
senhuang42 [Wed, 4 Nov 2020 16:05:41 +0000 (11:05 -0500)] 
Address edge case with endPosInSequence

4 years agoChange debug levels to appropriate ones
senhuang42 [Wed, 4 Nov 2020 15:43:35 +0000 (10:43 -0500)] 
Change debug levels to appropriate ones

4 years agoAdd RLE support
senhuang42 [Wed, 4 Nov 2020 14:36:36 +0000 (09:36 -0500)] 
Add RLE support

4 years agoFix various build warnings
senhuang42 [Wed, 4 Nov 2020 00:00:02 +0000 (19:00 -0500)] 
Fix various build warnings

4 years agoAdd test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()
senhuang42 [Tue, 3 Nov 2020 23:53:44 +0000 (18:53 -0500)] 
Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()

4 years agoAdd documentation for new api functions
senhuang42 [Tue, 3 Nov 2020 18:52:21 +0000 (13:52 -0500)] 
Add documentation for new api functions

4 years agoRefactor for enhanced code clarity
senhuang42 [Tue, 3 Nov 2020 18:41:20 +0000 (13:41 -0500)] 
Refactor for enhanced code clarity

4 years agoRename internal function compressSequences(), and promote new *_ext() functions to...
senhuang42 [Tue, 3 Nov 2020 18:37:50 +0000 (13:37 -0500)] 
Rename internal function compressSequences(), and promote new *_ext() functions to their actual name

4 years agoAdd another API function to compress with existing CCTX
senhuang42 [Tue, 3 Nov 2020 18:31:07 +0000 (13:31 -0500)] 
Add another API function to compress with existing CCTX

4 years agoMore adjustments to improve code clarity
senhuang42 [Tue, 3 Nov 2020 18:05:57 +0000 (13:05 -0500)] 
More adjustments to improve code clarity

4 years agoPull compressStream2() transparent initialization into its own function
senhuang42 [Tue, 3 Nov 2020 16:59:07 +0000 (11:59 -0500)] 
Pull compressStream2() transparent initialization into its own function

4 years agoAdd initial support for new ZSTD_Sequence mode
senhuang42 [Mon, 2 Nov 2020 18:29:56 +0000 (13:29 -0500)] 
Add initial support for new ZSTD_Sequence mode

4 years agoAdd sequence compression format param
senhuang42 [Mon, 2 Nov 2020 15:01:18 +0000 (10:01 -0500)] 
Add sequence compression format param

4 years agoAlways ensure sequenceRange updates properly, add more error forwarding
senhuang42 [Mon, 2 Nov 2020 14:52:52 +0000 (09:52 -0500)] 
Always ensure sequenceRange updates properly, add more error forwarding

4 years agoVarious minor logical refactors to improve clarity
senhuang42 [Mon, 2 Nov 2020 14:40:26 +0000 (09:40 -0500)] 
Various minor logical refactors to improve clarity

4 years agoFix cSize calculation for noCompressBlocks
senhuang42 [Fri, 30 Oct 2020 15:55:20 +0000 (11:55 -0400)] 
Fix cSize calculation for noCompressBlocks

4 years agoRebased, roundtrips silesia.tar
senhuang42 [Thu, 29 Oct 2020 18:58:06 +0000 (14:58 -0400)] 
Rebased, roundtrips silesia.tar

4 years agoRefactor for better debugging info
senhuang42 [Thu, 29 Oct 2020 18:47:36 +0000 (14:47 -0400)] 
Refactor for better debugging info

4 years agoCorrections and edge-case fixes to be able to roundtrip dickens
senhuang42 [Thu, 29 Oct 2020 16:52:58 +0000 (12:52 -0400)] 
Corrections and edge-case fixes to be able to roundtrip dickens

4 years agoMulti-block compression scaffolding - works on single-block files
senhuang42 [Thu, 29 Oct 2020 15:01:04 +0000 (11:01 -0400)] 
Multi-block compression scaffolding - works on single-block files

4 years agoAdd support for uncompressible blocks
senhuang42 [Thu, 29 Oct 2020 14:24:45 +0000 (10:24 -0400)] 
Add support for uncompressible blocks

4 years agoEnable usage of ZSTD_sequenceRange for single-block compression
senhuang42 [Wed, 28 Oct 2020 17:40:37 +0000 (13:40 -0400)] 
Enable usage of ZSTD_sequenceRange for single-block compression

4 years agoAdd logic to handle ZSTD_sequenceRange
senhuang42 [Wed, 28 Oct 2020 17:28:27 +0000 (13:28 -0400)] 
Add logic to handle ZSTD_sequenceRange

4 years agoAdd last literals handling like getSequences()
senhuang42 [Wed, 28 Oct 2020 15:57:21 +0000 (11:57 -0400)] 
Add last literals handling like getSequences()

4 years agoPull block compression out of main compressSequences() function
senhuang42 [Wed, 28 Oct 2020 15:50:38 +0000 (11:50 -0400)] 
Pull block compression out of main compressSequences() function

4 years agoImplement ZSTD_updateSequenceRange
senhuang42 [Wed, 28 Oct 2020 15:28:12 +0000 (11:28 -0400)] 
Implement ZSTD_updateSequenceRange

4 years agoModify SequenceRange to have posInSequence
senhuang42 [Wed, 28 Oct 2020 15:07:36 +0000 (11:07 -0400)] 
Modify SequenceRange to have posInSequence

4 years agoAdd function definition for sequenceRange updater
senhuang42 [Wed, 28 Oct 2020 15:04:44 +0000 (11:04 -0400)] 
Add function definition for sequenceRange updater

4 years agoAdd ZSTD_SequenceRange to count ranges in array of ZSTD_Sequence
senhuang42 [Wed, 28 Oct 2020 15:04:18 +0000 (11:04 -0400)] 
Add ZSTD_SequenceRange to count ranges in array of ZSTD_Sequence

4 years agoAdd support for repcodes
senhuang42 [Tue, 27 Oct 2020 15:02:58 +0000 (11:02 -0400)] 
Add support for repcodes

4 years agoCode cleanup, add debuglog statments
senhuang42 [Mon, 26 Oct 2020 16:33:58 +0000 (12:33 -0400)] 
Code cleanup, add debuglog statments

4 years agoImplement first pass at compressSequences()
senhuang42 [Wed, 30 Sep 2020 21:18:20 +0000 (17:18 -0400)] 
Implement first pass at compressSequences()

4 years agoAdd initial function prototype for ZSTD_compressSequences_ext (to be renamed later)
senhuang42 [Fri, 13 Nov 2020 14:55:05 +0000 (09:55 -0500)] 
Add initial function prototype for ZSTD_compressSequences_ext (to be renamed later)

4 years agoMerge pull request #2393 from senhuang42/fix_sequence_extractions_api
sen [Sun, 15 Nov 2020 23:29:52 +0000 (18:29 -0500)] 
Merge pull request #2393 from senhuang42/fix_sequence_extractions_api

Improve repcode handling in sequence extraction API

4 years agoReduce number of memcpy() calls 2393/head
senhuang42 [Fri, 13 Nov 2020 14:41:44 +0000 (09:41 -0500)] 
Reduce number of memcpy() calls

4 years agoUse existing repcode update function to implement updates
senhuang42 [Thu, 12 Nov 2020 21:38:23 +0000 (16:38 -0500)] 
Use existing repcode update function to implement updates

4 years agoAdd in proper block repcode histories
senhuang42 [Thu, 12 Nov 2020 19:37:47 +0000 (14:37 -0500)] 
Add in proper block repcode histories

4 years agoLet block reps persist
senhuang42 [Thu, 12 Nov 2020 17:22:58 +0000 (12:22 -0500)] 
Let block reps persist

4 years agoFix incorrect repcode setting
senhuang42 [Thu, 12 Nov 2020 16:57:01 +0000 (11:57 -0500)] 
Fix incorrect repcode setting

4 years agoImprove unit test
senhuang42 [Thu, 12 Nov 2020 16:09:01 +0000 (11:09 -0500)] 
Improve unit test

4 years agoOverhaul repcode handling logic
senhuang42 [Thu, 12 Nov 2020 15:59:35 +0000 (10:59 -0500)] 
Overhaul repcode handling logic

4 years agoMerge pull request #2388 from facebook/fix2386
Yann Collet [Fri, 6 Nov 2020 19:38:08 +0000 (11:38 -0800)] 
Merge pull request #2388 from facebook/fix2386

fix incorrect assert

4 years agoMerge pull request #2381 from senhuang42/expand_sequence_extraction_api
sen [Fri, 6 Nov 2020 18:00:31 +0000 (13:00 -0500)] 
Merge pull request #2381 from senhuang42/expand_sequence_extraction_api

Add enum to define ZSTD_Sequence type and update sequence extraction API

4 years agofix multiple minor conversion warnings 2388/head
Yann Collet [Fri, 6 Nov 2020 17:57:05 +0000 (09:57 -0800)] 
fix multiple minor conversion warnings

unrelated to #2386, just cleaning up while I'm updating this file ...

4 years agofix incorrect assert
Yann Collet [Fri, 6 Nov 2020 17:44:04 +0000 (09:44 -0800)] 
fix incorrect assert

fix #2386, reported by @Neumann-A

4 years agoUpdate unit tests 2381/head
senhuang42 [Fri, 6 Nov 2020 15:56:56 +0000 (10:56 -0500)] 
Update unit tests

4 years agoImplement mergeGeneratedSequences()
senhuang42 [Fri, 6 Nov 2020 15:55:46 +0000 (10:55 -0500)] 
Implement mergeGeneratedSequences()

4 years agoRename getSequences() to generateSequences()
senhuang42 [Fri, 6 Nov 2020 15:53:22 +0000 (10:53 -0500)] 
Rename getSequences() to generateSequences()

4 years agoAdd new mergeGeneratedSequences() function
senhuang42 [Fri, 6 Nov 2020 15:52:34 +0000 (10:52 -0500)] 
Add new mergeGeneratedSequences() function

4 years agoMerge pull request #2385 from LuAPi/add-ZSTD_getDictID_fromCDict-single-commit
Nick Terrell [Thu, 5 Nov 2020 18:36:13 +0000 (10:36 -0800)] 
Merge pull request #2385 from LuAPi/add-ZSTD_getDictID_fromCDict-single-commit

Add ZSTD_getDictID_fromCDict function to experimental section

4 years agoAdd ZSTD_getDictID_fromCDict function to experimental section 2385/head
Luke Pitt [Wed, 4 Nov 2020 11:37:37 +0000 (11:37 +0000)] 
Add ZSTD_getDictID_fromCDict function to experimental section