]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
6 years agoSimplify logic by setting default value for MESON_INSTALL_DESTDIR_PREFIX 1447/head
Lzu Tao [Thu, 13 Dec 2018 11:07:01 +0000 (18:07 +0700)] 
Simplify logic by setting default value for MESON_INSTALL_DESTDIR_PREFIX

6 years agomeson: Update man1 extension on meson 0.49.0
Lzu Tao [Thu, 13 Dec 2018 07:57:33 +0000 (14:57 +0700)] 
meson: Update man1 extension on meson 0.49.0

6 years agomeson: Update meson symlink script usage
Lzu Tao [Thu, 13 Dec 2018 07:57:08 +0000 (14:57 +0700)] 
meson: Update meson symlink script usage

6 years agomeson: Correct support for building on Windows
Lzu Tao [Tue, 4 Dec 2018 18:12:11 +0000 (01:12 +0700)] 
meson: Correct support for building on Windows

Let soversion base on version if not set. For example, if version is 3.6.0
and soversion is not defined, it is set to 3.

6 years agoReplace many os.path methods with pathlib one's
Lzu Tao [Mon, 3 Dec 2018 15:13:29 +0000 (22:13 +0700)] 
Replace many os.path methods with pathlib one's

Use MESON_INSTALL_DESTDIR_PREFIX variable instead of DESTDIR.

6 years agoMerge pull request #1444 from facebook/btultra2
Yann Collet [Thu, 13 Dec 2018 05:09:01 +0000 (21:09 -0800)] 
Merge pull request #1444 from facebook/btultra2

btultra2

6 years agoupdated compression results.csv 1444/head
Yann Collet [Thu, 13 Dec 2018 04:30:09 +0000 (20:30 -0800)] 
updated compression results.csv

and fixed nit

6 years agoMerge branch 'dev' into btultra2
Yann Collet [Thu, 13 Dec 2018 04:18:27 +0000 (20:18 -0800)] 
Merge branch 'dev' into btultra2

6 years agoMerge pull request #1445 from terrelln/regression
Nick Terrell [Wed, 12 Dec 2018 19:16:54 +0000 (11:16 -0800)] 
Merge pull request #1445 from terrelln/regression

[regression] add more methods

6 years agoReset the cctx for documentation/safety 1445/head
Nick Terrell [Tue, 11 Dec 2018 23:57:56 +0000 (15:57 -0800)] 
Reset the cctx for documentation/safety

6 years ago[regression] add more methods
Nick Terrell [Tue, 4 Dec 2018 04:06:26 +0000 (20:06 -0800)] 
[regression] add more methods

6 years agoadd clarification for ZSTD_CCtx_setPledgedSrcSize()
Yann Collet [Tue, 11 Dec 2018 20:08:21 +0000 (12:08 -0800)] 
add clarification for ZSTD_CCtx_setPledgedSrcSize()

as requested in #1391

6 years agoMerge pull request #1417 from facebook/advancedAPI
Yann Collet [Tue, 11 Dec 2018 02:48:15 +0000 (18:48 -0800)] 
Merge pull request #1417 from facebook/advancedAPI

Advanced API

6 years agofixed btultra2 usage with prefix
Yann Collet [Tue, 11 Dec 2018 02:45:03 +0000 (18:45 -0800)] 
fixed btultra2 usage with prefix

notably while using multi-threading

6 years agofix assert position within ZSTD_compress2() 1417/head
Yann Collet [Tue, 11 Dec 2018 01:42:35 +0000 (17:42 -0800)] 
fix assert position within ZSTD_compress2()

6 years agoclarified experimentalParam
Yann Collet [Tue, 11 Dec 2018 01:36:20 +0000 (17:36 -0800)] 
clarified experimentalParam

6 years agofixed ZSTD_compress2()
Yann Collet [Tue, 11 Dec 2018 01:33:49 +0000 (17:33 -0800)] 
fixed ZSTD_compress2()

as suggested by @terrelln

6 years agomake test can accept DEBUGLEVEL argument
Yann Collet [Mon, 10 Dec 2018 07:20:51 +0000 (23:20 -0800)] 
make test can accept DEBUGLEVEL argument

6 years agoupdated clevel table for large inputs
Yann Collet [Mon, 10 Dec 2018 06:38:05 +0000 (22:38 -0800)] 
updated clevel table for large inputs

6 years agoupdate clevel table for 256K
Yann Collet [Sun, 9 Dec 2018 05:40:08 +0000 (21:40 -0800)] 
update clevel table for 256K

6 years agoupdate clevel table for 128K
Yann Collet [Sat, 8 Dec 2018 18:42:55 +0000 (10:42 -0800)] 
update clevel table for 128K

6 years agoupdated clevel table for 16K
Yann Collet [Sat, 8 Dec 2018 04:12:43 +0000 (20:12 -0800)] 
updated clevel table for 16K

to introduce btultra2

6 years agoparamgrill: add status line
Yann Collet [Sat, 8 Dec 2018 00:02:24 +0000 (16:02 -0800)] 
paramgrill: add status line

get information on which config is currently tested
so that console get animated during long tests.

6 years agoadded tests for strategy=9 (btultra2)
Yann Collet [Fri, 7 Dec 2018 22:19:50 +0000 (14:19 -0800)] 
added tests for strategy=9 (btultra2)

6 years agorefactor paramgrill for clarity
Yann Collet [Fri, 7 Dec 2018 22:07:54 +0000 (14:07 -0800)] 
refactor paramgrill for clarity

restored ability to copy/paste the resulting compression level table into zstd_compress.c .

6 years agoMerge pull request #1440 from hungptit/dev
Yann Collet [Fri, 7 Dec 2018 04:56:19 +0000 (20:56 -0800)] 
Merge pull request #1440 from hungptit/dev

Refactor examples to avoid code duplication.

6 years agoRefactor examples to avoid code duplication. 1440/head
Hung Dang [Fri, 7 Dec 2018 03:42:19 +0000 (22:42 -0500)] 
Refactor examples to avoid code duplication.

6 years agolinked btultra2 as strategy9
Yann Collet [Fri, 7 Dec 2018 03:27:37 +0000 (19:27 -0800)] 
linked btultra2 as strategy9

and ensure zstdbench detects out-of-bound parameters

6 years agoupdated documentation
Yann Collet [Fri, 7 Dec 2018 01:22:19 +0000 (17:22 -0800)] 
updated documentation

introducing ZSTD_btultra2

6 years agominor bound refinements
Yann Collet [Fri, 7 Dec 2018 00:51:17 +0000 (16:51 -0800)] 
minor bound refinements

6 years agointroduced constants ZSTD_STRATEGY_MIN and ZSTD_STRATEGY_MAX
Yann Collet [Fri, 7 Dec 2018 00:16:16 +0000 (16:16 -0800)] 
introduced constants ZSTD_STRATEGY_MIN and ZSTD_STRATEGY_MAX

6 years agofixed c++ assignment to enum
Yann Collet [Thu, 6 Dec 2018 23:57:55 +0000 (15:57 -0800)] 
fixed c++ assignment to enum

6 years agoMerge pull request #1439 from obelisk/buck_readme
Yann Collet [Thu, 6 Dec 2018 23:42:33 +0000 (15:42 -0800)] 
Merge pull request #1439 from obelisk/buck_readme

Add buck build instructions to README.md

6 years agochanged ZSTD_c_compressionStrategy into ZSTD_c_strategy
Yann Collet [Thu, 6 Dec 2018 23:00:52 +0000 (15:00 -0800)] 
changed ZSTD_c_compressionStrategy into ZSTD_c_strategy

also : fixed paramgrill, and limit conditions

6 years agoAdd buck build instruction to the readme 1439/head
Mitchell Grenier [Thu, 6 Dec 2018 22:46:48 +0000 (14:46 -0800)] 
Add buck build instruction to the readme

6 years agointroduced strategy btultra2
Yann Collet [Thu, 6 Dec 2018 21:38:09 +0000 (13:38 -0800)] 
introduced strategy btultra2

note : not yet applied on any compression level

6 years agomoved ZSTD_WINDOWLOG_LIMIT_DEFAULT into static-linking-only area
Yann Collet [Thu, 6 Dec 2018 18:57:19 +0000 (10:57 -0800)] 
moved ZSTD_WINDOWLOG_LIMIT_DEFAULT into static-linking-only area

6 years agoclarified usage of word "job"
Yann Collet [Thu, 6 Dec 2018 18:14:34 +0000 (10:14 -0800)] 
clarified usage of word "job"

only applies in MT / async context now.

6 years agochanged parameter names from ZSTD_p_* to ZSTD_c_*
Yann Collet [Thu, 6 Dec 2018 01:26:02 +0000 (17:26 -0800)] 
changed parameter names from ZSTD_p_* to ZSTD_c_*

for naming consistency

6 years agoupdated API documentation
Yann Collet [Thu, 6 Dec 2018 00:23:00 +0000 (16:23 -0800)] 
updated API documentation

6 years agoZSTD_decompressDCtx() is compatible with sticky parameters
Yann Collet [Wed, 5 Dec 2018 01:30:58 +0000 (17:30 -0800)] 
ZSTD_decompressDCtx() is compatible with sticky parameters

6 years agomerge dedicated dParam setters
Yann Collet [Wed, 5 Dec 2018 01:06:48 +0000 (17:06 -0800)] 
merge dedicated dParam setters

6 years agowrite the switch()case: differently
Yann Collet [Wed, 5 Dec 2018 00:59:26 +0000 (16:59 -0800)] 
write the switch()case: differently

so that it please both
compilers which warn for dead code after the switch
and
compilers which do not detect that all branches terminate.

6 years agofixed silent conversion warning
Yann Collet [Tue, 4 Dec 2018 23:57:16 +0000 (15:57 -0800)] 
fixed silent conversion warning

6 years agofixed declaration-after-statement warnings
Yann Collet [Tue, 4 Dec 2018 23:54:01 +0000 (15:54 -0800)] 
fixed declaration-after-statement warnings

6 years agoimplemented ZSTD_dParam_getBounds()
Yann Collet [Tue, 4 Dec 2018 23:35:37 +0000 (15:35 -0800)] 
implemented ZSTD_dParam_getBounds()

and ZSTD_DCtx_setParameter()

6 years agoadvanced decompression function replaces by normal streaming one
Yann Collet [Tue, 4 Dec 2018 18:28:36 +0000 (10:28 -0800)] 
advanced decompression function replaces by normal streaming one

advanced parameters compatible with ZSTD_decompressStream().

6 years agoMerge pull request #1427 from lzutao/minor-fix-meson
Yann Collet [Tue, 4 Dec 2018 18:01:50 +0000 (10:01 -0800)] 
Merge pull request #1427 from lzutao/minor-fix-meson

Update meson build and add Travis test for it

6 years agoMerge pull request #1437 from facebook/grep
Yann Collet [Tue, 4 Dec 2018 01:43:34 +0000 (17:43 -0800)] 
Merge pull request #1437 from facebook/grep

check availability of --color=never command on grep and egrep

6 years agocheck availability of --color=never command on grep and egrep 1437/head
Yann Collet [Mon, 3 Dec 2018 23:46:55 +0000 (15:46 -0800)] 
check availability of --color=never command on grep and egrep

before applying them.
Fixes #1436

6 years agojoined normal streaming API with advanced one
Yann Collet [Mon, 3 Dec 2018 22:22:38 +0000 (14:22 -0800)] 
joined normal streaming API with advanced one

6 years agoMerge pull request #1434 from terrelln/regression
Nick Terrell [Mon, 3 Dec 2018 18:39:12 +0000 (10:39 -0800)] 
Merge pull request #1434 from terrelln/regression

[regression] Add dictionary support

6 years agomeson: Update usage of InstallSymlink helper 1427/head
Lzu Tao [Mon, 3 Dec 2018 04:02:42 +0000 (11:02 +0700)] 
meson: Update usage of InstallSymlink helper

6 years agomeson: Update tests timeout to run properly
Lzu Tao [Sun, 2 Dec 2018 20:23:09 +0000 (03:23 +0700)] 
meson: Update tests timeout to run properly

6 years agomeson: Use clang for faster build
Lzu Tao [Sun, 2 Dec 2018 17:57:10 +0000 (00:57 +0700)] 
meson: Use clang for faster build

6 years agoUse -Dlegacy_level build option to control ZSTD_LEGACY_SUPPORT macro in test
Lzu Tao [Sun, 2 Dec 2018 17:36:40 +0000 (00:36 +0700)] 
Use -Dlegacy_level build option to control ZSTD_LEGACY_SUPPORT macro in test

6 years agomeson helper: Use Python conventional name for naming function [skip ci]
Lzu Tao [Sun, 2 Dec 2018 15:48:11 +0000 (22:48 +0700)] 
meson helper: Use Python conventional name for naming function [skip ci]

6 years agomeson: Fix soversion
Lzu Tao [Sun, 2 Dec 2018 15:45:46 +0000 (22:45 +0700)] 
meson: Fix soversion

6 years agomeson: Use -werror build option instead of adding -Werror [skip ci]
Lzu Tao [Sun, 2 Dec 2018 15:33:25 +0000 (22:33 +0700)] 
meson: Use -werror build option instead of adding -Werror [skip ci]

6 years agoUse argparse instead of manually parsing [skip ci]
Lzu Tao [Sun, 2 Dec 2018 15:31:18 +0000 (22:31 +0700)] 
Use argparse instead of manually parsing [skip ci]

6 years agoAdd missed .1 extension for manpage [skip ci]
Lzu Tao [Sat, 1 Dec 2018 17:18:00 +0000 (00:18 +0700)] 
Add missed .1 extension for manpage [skip ci]

6 years agomeson: Cleanup installing symlinks
Lzu Tao [Sat, 1 Dec 2018 06:07:08 +0000 (13:07 +0700)] 
meson: Cleanup installing symlinks

6 years agoUpdate README
Lzu Tao [Sat, 1 Dec 2018 02:11:26 +0000 (09:11 +0700)] 
Update README

6 years agomeson: Remove redundant join_paths function
Lzu Tao [Fri, 30 Nov 2018 14:31:58 +0000 (21:31 +0700)] 
meson: Remove redundant join_paths function

6 years agoci: Use new meson auto_features option
Lzu Tao [Fri, 30 Nov 2018 14:10:06 +0000 (21:10 +0700)] 
ci: Use new meson auto_features option

* Install lz4 dep from Ubuntu repo
* Use curl instead of wget
* Use xenial for meson and make meson build first
* Unset Travis preset CC and CXX value for meson to use ccache
* Be more verbose with "set -x"

6 years agomeson: Change build options' name
Lzu Tao [Fri, 30 Nov 2018 14:05:03 +0000 (21:05 +0700)] 
meson: Change build options' name

6 years agoAdd almost all test cases in tests/Makefile
Lzu Tao [Fri, 30 Nov 2018 10:23:23 +0000 (17:23 +0700)] 
Add almost all test cases in tests/Makefile

6 years agoAdd clang cflags
Lzu Tao [Fri, 30 Nov 2018 09:15:33 +0000 (16:15 +0700)] 
Add clang cflags

6 years agoInitial support for Windows build
Lzu Tao [Fri, 30 Nov 2018 08:30:45 +0000 (15:30 +0700)] 
Initial support for Windows build

6 years agoChange tabsize
Lzu Tao [Fri, 30 Nov 2018 04:01:19 +0000 (11:01 +0700)] 
Change tabsize

6 years agoBuild both shared and static lib on CI
Lzu Tao [Thu, 29 Nov 2018 10:48:18 +0000 (17:48 +0700)] 
Build both shared and static lib on CI

Rename INSTALL_SYSTEM to BUILD_SYSTEM

6 years agoAdd comment about looking for dependencies
Lzu Tao [Thu, 29 Nov 2018 06:15:18 +0000 (13:15 +0700)] 
Add comment about looking for dependencies

6 years agoRename some meson build options to match autoconf convention
Lzu Tao [Thu, 29 Nov 2018 05:52:35 +0000 (12:52 +0700)] 
Rename some meson build options to match autoconf convention

6 years agoEnable debugging flag
Lzu Tao [Thu, 29 Nov 2018 05:24:43 +0000 (12:24 +0700)] 
Enable debugging flag

6 years agoUpdate relevant files after moving meson to build/
Lzu Tao [Thu, 29 Nov 2018 04:49:33 +0000 (11:49 +0700)] 
Update relevant files after moving meson to build/

Update meson build instructions

6 years agoMove contrib/meson to build/meson
Lzu Tao [Thu, 29 Nov 2018 04:45:57 +0000 (11:45 +0700)] 
Move contrib/meson to build/meson

6 years agoAdd enable_lz4 build option and fix lzma dependency
Lzu Tao [Thu, 29 Nov 2018 04:05:40 +0000 (11:05 +0700)] 
Add enable_lz4 build option and fix lzma dependency

6 years agoAdd pedantic flag
Lzu Tao [Thu, 29 Nov 2018 03:54:29 +0000 (10:54 +0700)] 
Add pedantic flag

6 years agoNo install zstd_manual.html
Lzu Tao [Thu, 29 Nov 2018 03:54:01 +0000 (10:54 +0700)] 
No install zstd_manual.html

6 years agoInstall missed zstdgrep and zstdless
Lzu Tao [Wed, 28 Nov 2018 18:50:24 +0000 (01:50 +0700)] 
Install missed zstdgrep and zstdless

6 years agoInstall zstdmt.1 manpage [skip ci]
Lzu Tao [Wed, 28 Nov 2018 18:27:20 +0000 (01:27 +0700)] 
Install zstdmt.1 manpage [skip ci]

6 years agoRename meson variables
Lzu Tao [Wed, 28 Nov 2018 18:26:41 +0000 (01:26 +0700)] 
Rename meson variables

6 years agoAdd manpage install warning [skip ci]
Lzu Tao [Wed, 28 Nov 2018 17:09:24 +0000 (00:09 +0700)] 
Add manpage install warning [skip ci]

We link new manpages with gz compressed format of the target manpage.
I have not tested it on Windows. So just place a warning here.

6 years agoFix manpage symlinks [skip ci]
Lzu Tao [Wed, 28 Nov 2018 16:59:42 +0000 (23:59 +0700)] 
Fix manpage symlinks [skip ci]

6 years agoApply new InstallSymlink script
Lzu Tao [Wed, 28 Nov 2018 15:32:37 +0000 (22:32 +0700)] 
Apply new InstallSymlink script

6 years agoHelper script to install symlink in meson
Lzu Tao [Wed, 28 Nov 2018 15:31:16 +0000 (22:31 +0700)] 
Helper script to install symlink in meson

6 years agoNo need these helpers
Lzu Tao [Wed, 28 Nov 2018 14:59:35 +0000 (21:59 +0700)] 
No need these helpers

6 years agoInit CI test for meson build
Lzu Tao [Wed, 28 Nov 2018 06:06:18 +0000 (13:06 +0700)] 
Init CI test for meson build

Do not use ccache: Build error in gcc-8

6 years agoFix lib soversion and no install cover.h header
Lzu Tao [Wed, 28 Nov 2018 07:26:57 +0000 (14:26 +0700)] 
Fix lib soversion and no install cover.h header

6 years agoFix meson tests build
Lzu Tao [Wed, 28 Nov 2018 05:16:09 +0000 (12:16 +0700)] 
Fix meson tests build

6 years agoBump to 1.3.8 and fix run_command function
Lzu Tao [Wed, 28 Nov 2018 04:57:53 +0000 (11:57 +0700)] 
Bump to 1.3.8 and fix run_command function

The run_command is run from an unspecified directory. Therefore we cannot assume
which directory it is running our command.

6 years agoUpdate legacy support to 5
Lzu Tao [Wed, 28 Nov 2018 04:49:52 +0000 (11:49 +0700)] 
Update legacy support to 5

6 years agoMinor fix for meson build
Lzu Tao [Tue, 27 Nov 2018 19:47:04 +0000 (02:47 +0700)] 
Minor fix for meson build

Use files function instead of constructing path with meson.current_source_dir()

6 years ago[regression] Add dictionary support 1434/head
Nick Terrell [Sat, 1 Dec 2018 01:16:19 +0000 (17:16 -0800)] 
[regression] Add dictionary support

Dictionaries are prebuilt and saved as part of the data object.
The config decides whether or not to use the dictionary if it is
available. Configs that require dictionaries are only run with
data that have dictionaries. The method will skip configs that are
irrelevant, so for example ZSTD_compress() will skip configs with
dictionaries.

I've also trimmed the silesia source to 1MB per file (12 MB total),
and added 500 samples from the github data set with a dictionary.

I've intentionally added an extra line to the `results.csv` to make
the nightly build fail, so that we can see how CircleCI reports it.

Full list of changes:

* Add pre-built dictionaries to the data.
* Add `use_dictionary` and `no_pledged_src_size` flags to the config.
* Add a config using a dictionary for every level.
* Add a config that specifies no pledged source size.
* Support dictionaries and streaming in the `zstdcli` method.
* Add a context-reuse method using `ZSTD_compressCCtx()`.
* Clean up the formatting of the `results.csv` file to align columns.
* Add `--data`, `--config`, and `--method` flags to constrain each
  to a particular value. This is useful for debugging a failure
  or debugging a particular config/method/data.

6 years agopreparative for ZSTD_DCtx_setParameter()
Yann Collet [Fri, 30 Nov 2018 23:59:50 +0000 (15:59 -0800)] 
preparative for ZSTD_DCtx_setParameter()

6 years agocreated ZSTD_compress2() and ZSTD_compressStream2()
Yann Collet [Fri, 30 Nov 2018 19:16:26 +0000 (11:16 -0800)] 
created ZSTD_compress2() and ZSTD_compressStream2()

ZSTD_compress_generic() is renamed ZSTD_compressStream2().

Note that, for the time being,
the "stable" API and advanced one use different parameter planes :
setting parameters using the advanced API does not influence ZSTD_compressStream()
and using ZSTD_initCStream() does not influence parameters for ZSTD_compressStream2().

6 years agoMerge pull request #1433 from obelisk/fixed_buck
Yann Collet [Fri, 30 Nov 2018 19:21:09 +0000 (11:21 -0800)] 
Merge pull request #1433 from obelisk/fixed_buck

Small changes for buck build

6 years agoFix buck for lib 1433/head
Mitchell Grenier [Fri, 30 Nov 2018 13:20:16 +0000 (13:20 +0000)] 
Fix buck for lib

6 years agoMerge pull request #1432 from terrelln/regression
Yann Collet [Fri, 30 Nov 2018 00:06:40 +0000 (16:06 -0800)] 
Merge pull request #1432 from terrelln/regression

[regression] Add initial regression test framework

6 years ago[regression] Add initial regression test framework 1432/head
Nick Terrell [Tue, 20 Nov 2018 20:37:43 +0000 (12:37 -0800)] 
[regression] Add initial regression test framework

The regression tests run nightly or on the `regression`
branch for convenience. The results get uploaded as the
artifacts of the job. If they change, check the diff
printed in the job. If all is well, download the new
results and commit them to the repo.

This code will only run on a UNIX like platform. It
could be made to run on Windows, but I don't think that
it is necessary. It also uses C99.

* data: This module defines the data to run tests on.
  It downloads data from a URL into a cache directory,
  checks it against a checksum, and unpacks it. It also
  provides helpers for accessing the data.
* config: This module defines the configs to run tests
  with. A config is a set of API parameters and a set of
  CLI flags.
* result: This module is a helper for method that defines
  the result type.
* method: This module defines the compression methods
  to test. It is what runs the regression test using the
  data and the config. It reports the total compressed
  size, or an error/skip.
* test: This is the test binary that runs the tests for
  every (data, config, method) tuple, and prints the
  results to the output file and stderr.
* results.csv: The results that the current commit is
  expected to produce.