]>
git.ipfire.org Git - thirdparty/zstd.git/log
Antonio Bueno [Wed, 19 Aug 2020 10:36:28 +0000 (12:36 +0200)]
Fixed Markdown warnings. No visible changes.
Yann Collet [Wed, 19 Aug 2020 02:08:01 +0000 (19:08 -0700)]
Merge pull request #2268 from bimbashrestha/actions-icc
[actions] removing broken icc test from travis and adding working version to actions
Felix Handte [Sat, 15 Aug 2020 01:44:56 +0000 (21:44 -0400)]
Merge pull request #2270 from felixhandte/fix-doc-cctx-set-param
Fix Documentation for ZSTD_CCtxParams_setParameter()
W. Felix Handte [Fri, 14 Aug 2020 18:44:08 +0000 (14:44 -0400)]
Fix Documentation for ZSTD_CCtxParams_setParameter()
It does not only return 0 on success.
Bimba Shrestha [Thu, 13 Aug 2020 17:47:04 +0000 (12:47 -0500)]
removing icc from travis
Bimba Shrestha [Thu, 13 Aug 2020 17:45:28 +0000 (12:45 -0500)]
adding icc to actions
Yann Collet [Thu, 13 Aug 2020 17:00:57 +0000 (10:00 -0700)]
Merge pull request #2267 from cwoffenden/clang-comma
Fix clang -Wcomma warning (single file decoder)
Carl Woffenden [Thu, 13 Aug 2020 14:11:22 +0000 (16:11 +0200)]
Fix clang -Wcomma warning
Nick Terrell [Tue, 11 Aug 2020 17:07:53 +0000 (10:07 -0700)]
Merge pull request #2265 from terrelln/prefetch-fix
Prefer __builtin_prefetch over inline asm
Nick Terrell [Tue, 11 Aug 2020 05:15:46 +0000 (22:15 -0700)]
Prefer __builtin_prefetch over inline asm
Reorder the ifdefs for the PREFETCH macros so that the compiler builtin is
favored over the inline assembly for aarch64.
Yann Collet [Tue, 11 Aug 2020 01:20:13 +0000 (18:20 -0700)]
Merge pull request #2262 from felixhandte/fewer-stat-syscalls
Share stat() Calls Where Convenient
W. Felix Handte [Mon, 10 Aug 2020 21:28:34 +0000 (17:28 -0400)]
Fix MS Build
W. Felix Handte [Mon, 10 Aug 2020 19:39:14 +0000 (15:39 -0400)]
Clean Up Redundant Checks, Rename FIO_remove() -> FIO_removeFile()
W. Felix Handte [Mon, 10 Aug 2020 19:28:02 +0000 (15:28 -0400)]
Remove UTIL_statFile() and UTIL_statDir(); Decompose Former Call-Sites
W. Felix Handte [Mon, 10 Aug 2020 19:22:29 +0000 (15:22 -0400)]
Remove Unused Function UTIL_fileExist()
W. Felix Handte [Mon, 10 Aug 2020 19:16:14 +0000 (15:16 -0400)]
Re-Organize and Document Prototypes in util.h
W. Felix Handte [Wed, 5 Aug 2020 16:10:42 +0000 (12:10 -0400)]
Share stat() Calls in Uses of UTIL_chmod()
W. Felix Handte [Wed, 5 Aug 2020 16:00:12 +0000 (12:00 -0400)]
Use stat() to Check that File Should be chmod()-ed
Rather than special-casing a check for `/dev/null`, this uses `stat()` to
avoid `chmod()`-ing any non-regular file. I believe this is the desirable
behavior. `UTIL_chmod()` is never called on directories at the moment, only
output files.
W. Felix Handte [Wed, 5 Aug 2020 05:08:34 +0000 (01:08 -0400)]
Deduplicate Some Low-Hanging Fruit of Redundant Stat Calls
W. Felix Handte [Wed, 5 Aug 2020 05:00:06 +0000 (01:00 -0400)]
Introduce Variants of Various UTIL Functions that Take Pre-Populated stat_t Structs
Instead of calling `stat()`, these functions accept the result of a previous
`stat()` call on the file in question, which will allow us to make multiple
decisions around a file without redundant `stat()` calls.
W. Felix Handte [Wed, 5 Aug 2020 04:40:16 +0000 (00:40 -0400)]
Rename UTIL_getFileStat() -> UTIL_statFile() and UTIL_getDirectoryStat() -> UTIL_statDir()
I want to introduce versions of many of these functions that take pre-
populated `stat_t` objects and use those rather than doing their own redundant
`stat()` internally. These functions will have `...Stat()` suffixes. So this
commit renames these existing functions into the active voice, to avoid
confusion.
W. Felix Handte [Wed, 5 Aug 2020 04:35:21 +0000 (00:35 -0400)]
Mark stat_t Arg to UTIL_setFileStat() const
W. Felix Handte [Wed, 5 Aug 2020 04:31:48 +0000 (00:31 -0400)]
Additionally Convert UTIL_getFileStat() Calls to UTIL_stat() Where Appropriate
W. Felix Handte [Wed, 5 Aug 2020 04:21:41 +0000 (00:21 -0400)]
Use New Stat Helper
W. Felix Handte [Wed, 5 Aug 2020 04:09:29 +0000 (00:09 -0400)]
Introduce Dedicated Helper to Call stat()
Yann Collet [Tue, 4 Aug 2020 16:43:59 +0000 (09:43 -0700)]
Merge pull request #2258 from Niadb/dev
Added STATIC_BMI2 for compile time detection of BMI2 on MSVC, when enabled various intrinsics are used
Yann Collet [Sun, 2 Aug 2020 21:08:42 +0000 (14:08 -0700)]
Merge pull request #2253 from facebook/histvec
optimized histogram
Nick Terrell [Wed, 29 Jul 2020 18:57:49 +0000 (11:57 -0700)]
Merge pull request #2256 from helloguo/dev
Optimize ZSTD_wildcopy
Yann Collet [Wed, 29 Jul 2020 16:21:03 +0000 (09:21 -0700)]
Merge pull request #2259 from cwoffenden/vs-warning
Fixed VS variable shadowing warning (single file lib)
Carl Woffenden [Wed, 29 Jul 2020 10:33:39 +0000 (12:33 +0200)]
Fixed VS variable shadowing warning (and added test)
helloguo [Tue, 28 Jul 2020 18:58:46 +0000 (11:58 -0700)]
Use ZSTD_copy16 instead of memcpy
Niadb [Tue, 28 Jul 2020 17:17:04 +0000 (11:17 -0600)]
Update bitstream.h
Profiler showed some of these not being inlined on MSVC
Niadb [Tue, 28 Jul 2020 08:52:52 +0000 (02:52 -0600)]
Add files via upload
Niadb [Tue, 28 Jul 2020 08:52:15 +0000 (02:52 -0600)]
Add files via upload
helloguo [Tue, 28 Jul 2020 05:08:52 +0000 (22:08 -0700)]
Optimize ZSTD_wildcopy
Nick Terrell [Mon, 27 Jul 2020 19:59:10 +0000 (12:59 -0700)]
Merge pull request #2252 from helloguo/dev
Fix preprocessor in ZSTD_wildcopy
Yann Collet [Mon, 27 Jul 2020 05:40:21 +0000 (22:40 -0700)]
fixed overlapping count & workspace special case
Yann Collet [Mon, 27 Jul 2020 05:24:22 +0000 (22:24 -0700)]
optimized histogram
new version easier to vectorize
leads to smaller code and faster execution
notably at the last recombination stage
(basically, fixed cost per block).
Assembly inspected with godbolt
On my laptop, with `clang` and `-mavx2` :
2K block : 1280 MB/s -> 1550 MB/s
8K block : 1750 MB/s -> 1860 MB/s
helloguo [Fri, 24 Jul 2020 17:53:58 +0000 (10:53 -0700)]
fix preprocessor in ZSTD_wildcopy
Yann Collet [Thu, 23 Jul 2020 19:10:57 +0000 (12:10 -0700)]
updated documentation
Yann Collet [Wed, 22 Jul 2020 16:49:34 +0000 (09:49 -0700)]
Merge pull request #2250 from facebook/twoFieldsArgs
CLI: simplified handling of separated-fields arguments
Yann Collet [Tue, 21 Jul 2020 15:23:37 +0000 (08:23 -0700)]
Merge pull request #2249 from facebook/fsefix
import some minor fixes from FSE project
Yann Collet [Tue, 21 Jul 2020 00:41:32 +0000 (17:41 -0700)]
improved 2-args macro to support both syntaxes
both `--long-commmand=arg` and `--long-command arg`
are supported by macro `NEXT_FIELD()`
so that the long-command only needs to be listed once.
This extends the syntax to support new syntaxes like
`-o=FILE` or `-D=dict`,
though there is no need to advertise this capability for the time being.
Also : added `NEXT_UINT32()`,
which is wrapper around `NEXT_FIELD()`
to read integer parameters.
Use the wrapper to new field, such as `--memlimit`
which can now support both syntaxes too.
Yann Collet [Fri, 17 Jul 2020 20:31:15 +0000 (13:31 -0700)]
added tests for newly enabled syntax
for --patch-from origin
and --filelist list
Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.
For example :
zstd file -of dest
used to be disallowed.
It's now allowed, and understood as:
zstd file -o dest -f
Yann Collet [Fri, 17 Jul 2020 20:09:23 +0000 (13:09 -0700)]
extended 2-fields macro usage to number arguments
leading to further simplications
Yann Collet [Fri, 17 Jul 2020 19:46:36 +0000 (12:46 -0700)]
CLI: simplified handling of separated-fields arguments
this patch reduces complexity associated with
commands requiring a separated arguments
such as :
-o filename
-D dictionary
--output-dir-flat dir
--output-dir-mirror dir
It used to be a multi-stage process with explicit context,
it's now simplified as a single macro.
Thanks to this simplification,
separated arguments logic has also been extended to
--patch-from XXX
--filelist XXX
which extends existing capability using =XXX
--patch-from=XXX
--filelist=XXX
Separated argument is useful for filenames and directories,
as it benefits from shell expansion
such as ~/dir/file
where the ~ is automatically translated by the shell.
In contrast --long-command=FILE does not interpret FILE,
so ~/ is transmitted as is to the main() function,
generally resulting in incorrect file name.
Yann Collet [Fri, 17 Jul 2020 16:16:11 +0000 (09:16 -0700)]
Merge pull request #2197 from neheb/hj
meson: remove build requirement for distutils
Yann Collet [Fri, 17 Jul 2020 16:14:20 +0000 (09:14 -0700)]
Merge pull request #2243 from facebook/uclibc
fix uclibc's st_mtim
Yann Collet [Fri, 17 Jul 2020 03:33:50 +0000 (20:33 -0700)]
ensure workspace is large enough
even when MAX_TABLELOG is reduced
Yann Collet [Fri, 17 Jul 2020 03:25:15 +0000 (20:25 -0700)]
import some minor fixes from FSE project
Felix Handte [Thu, 16 Jul 2020 16:40:43 +0000 (12:40 -0400)]
Merge pull request #2246 from ffontaine/dev
programs/util.c: fix build without st_mtime
Yann Collet [Wed, 15 Jul 2020 19:42:31 +0000 (12:42 -0700)]
Merge pull request #2247 from facebook/revert-2244-dev
Revert "Fix -Wunused-variable under FUZZING_BUILD_MODE..."
Yann Collet [Wed, 15 Jul 2020 19:42:18 +0000 (12:42 -0700)]
Revert "Fix -Wunused-variable under FUZZING_BUILD_MODE..."
Yann Collet [Wed, 15 Jul 2020 19:40:17 +0000 (12:40 -0700)]
Merge pull request #2244 from hctim/dev
Fix -Wunused-variable under FUZZING_BUILD_MODE...
Fabrice Fontaine [Wed, 15 Jul 2020 19:19:14 +0000 (21:19 +0200)]
programs/util.c: fix build without st_mtime
Since version 1.4.5 and commit
5af8cb7aea8d890b4801e50e5274371510f2cf33 , if st_mtime is not defined,
programs/util.c uses utime without including utime.h which will raise
the following build failure on some of the buildroot autobuilders:
util.c: In function 'UTIL_setFileStat':
util.c:161:24: error: storage size of 'timebuf' isn't known
struct utimbuf timebuf;
^~~~~~~
Fixes:
- http://autobuild.buildroot.org/results/
be902c5d110f37bce622a2215191f155b7d3e7e0
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Mitch Phillips [Tue, 14 Jul 2020 15:49:43 +0000 (08:49 -0700)]
Fix -Wunused-variable under FUZZING_BUILD_MODE...
Fuzzing build modes (FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) doesn't
necessarily imply that assert() is enabled, according to the manual.
When the current do-nothing is expanded under -Wunused-variable (-Wall),
it results in unused variables in some of the FUZZING_BUILD_MODE...
blocks.
This patch extends the do-nothing to avoid the unused variable.
Yann Collet [Mon, 13 Jul 2020 21:16:33 +0000 (14:16 -0700)]
fix uclibc's st_mtim
reported and suggested by @ewildgoose (#1872)
Nick Terrell [Mon, 13 Jul 2020 18:26:13 +0000 (14:26 -0400)]
Merge pull request #2242 from evverx/cifuzz-ubsan-msan
cifuzz: turn on UBsan and MSan
Evgeny Vereshchagin [Mon, 13 Jul 2020 01:20:11 +0000 (01:20 +0000)]
cifuzz: turn on UBsan and MSan
Yann Collet [Sat, 11 Jul 2020 09:16:52 +0000 (02:16 -0700)]
minor doc clarification regarding MT parameters
Nick Terrell [Fri, 10 Jul 2020 15:26:15 +0000 (11:26 -0400)]
Merge pull request #2239 from evverx/evverx-cifuzz
cifuzz: don't upload artifacts when CIFuzz fails to build fuzz targets
Yann Collet [Fri, 10 Jul 2020 08:21:47 +0000 (01:21 -0700)]
Merge pull request #2219 from xxie24/output-dir-mirror
Add output-dir-mirror option
Yann Collet [Fri, 10 Jul 2020 08:20:59 +0000 (01:20 -0700)]
Merge pull request #2183 from tonytheodore/patch-1
Install pkg-config file on MINGW also
Yann Collet [Thu, 9 Jul 2020 15:48:36 +0000 (08:48 -0700)]
Merge pull request #2237 from marxin/gcda-to-gitignore
Add GCC PGO data files to .gitignore.
Yann Collet [Thu, 9 Jul 2020 15:48:04 +0000 (08:48 -0700)]
Merge pull request #2236 from marxin/pgo-fix-comment
Update comment about PGO build.
Yann Collet [Thu, 9 Jul 2020 15:41:44 +0000 (08:41 -0700)]
Merge pull request #2231 from marxin/use-threads-in-example
Use threads in streaming_compression.c example.
Martin Liska [Thu, 9 Jul 2020 07:32:54 +0000 (09:32 +0200)]
Add GCC PGO data files to .gitignore.
Martin Liska [Thu, 9 Jul 2020 07:31:51 +0000 (09:31 +0200)]
Update comment about PGO build.
Martin Liska [Wed, 8 Jul 2020 10:09:00 +0000 (12:09 +0200)]
Use threads in streaming_compression.c example.
Fixes #1584.
Evgeny Vereshchagin [Thu, 9 Jul 2020 04:22:40 +0000 (04:22 +0000)]
cifuzz: don't upload artifacts when CIFuzz fails to build fuzz targets
It should address https://github.com/google/oss-fuzz/issues/3982
Yann Collet [Wed, 8 Jul 2020 18:49:40 +0000 (11:49 -0700)]
Merge pull request #2230 from cwoffenden/minor-tidy
Remove superfluous build-time macros (single-file libs)
Yann Collet [Wed, 8 Jul 2020 18:49:04 +0000 (11:49 -0700)]
Merge pull request #2235 from bimbashrestha/decom-macros
[actions] removing flakey min-decomp
Bimba Shrestha [Wed, 8 Jul 2020 18:11:45 +0000 (11:11 -0700)]
removing min decomp
Carl Woffenden [Wed, 8 Jul 2020 09:39:17 +0000 (11:39 +0200)]
Remove superfluous build-time options
Yann Collet [Tue, 7 Jul 2020 15:51:20 +0000 (08:51 -0700)]
Merge pull request #2227 from yoshihitoh/single-file-dict-emscripten
[contrib] Fix single-file compilation error on Emscripten build.
yoshihitoh [Tue, 7 Jul 2020 10:10:12 +0000 (19:10 +0900)]
Change Emscripten docker image to the official one which is used for testing the single-file feature.
Yann Collet [Mon, 6 Jul 2020 23:06:49 +0000 (16:06 -0700)]
Merge pull request #2228 from marxin/rename-ZSTDMT_NBTHREADS_MAX
Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX.
Martin Liska [Mon, 6 Jul 2020 22:18:20 +0000 (00:18 +0200)]
Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX.
Rename it in zstd man page.
yoshihitoh [Tue, 30 Jun 2020 16:40:15 +0000 (01:40 +0900)]
Enable docker based Emscripten build on single file library test if emcc is not available.
Nick Terrell [Tue, 30 Jun 2020 00:19:24 +0000 (17:19 -0700)]
Merge pull request #2220 from dkcasset/fix-lib-Makefile-older-sed-version
Replace -E option with equivalent -r for older versions of sed
Nick Terrell [Tue, 30 Jun 2020 00:14:50 +0000 (17:14 -0700)]
Merge pull request #2221 from BioDataAnalysis/bda_install_dll
Install MSVC runtime shared libraries (DLL) with cmake
dkcasset [Mon, 29 Jun 2020 20:44:23 +0000 (13:44 -0700)]
Add variable for sed extended RE option (defaults to -E)
Mario Emmenlauer [Mon, 29 Jun 2020 09:09:31 +0000 (11:09 +0200)]
build/cmake/lib/CMakeLists.txt: Install shared library (dll) for Windows
yoshihitoh [Mon, 29 Jun 2020 01:51:50 +0000 (10:51 +0900)]
Rename static vars to avoid redefinition error.
dkcasset [Fri, 26 Jun 2020 17:43:28 +0000 (10:43 -0700)]
Replace -E option with equivalent -r for older versions of sed
Xin Xie [Sat, 20 Jun 2020 02:35:51 +0000 (19:35 -0700)]
Add output-dir-mirror option
Nick Terrell [Tue, 23 Jun 2020 00:41:18 +0000 (17:41 -0700)]
Merge pull request #2218 from terrelln/assert-seq
Fix unused variable warnings in fuzzing build mode without asserts
Nick Terrell [Tue, 23 Jun 2020 00:41:13 +0000 (17:41 -0700)]
Merge pull request #2217 from terrelln/cover-redundant
[cover] Remove unnecessary mask and dedup hash functions
Nick Terrell [Tue, 23 Jun 2020 00:41:03 +0000 (17:41 -0700)]
Merge pull request #2209 from Niadb/dev
Explicitly use __cdecl for qsort, to avoid warning when default calling convention is not __cdecl
Nick Terrell [Mon, 22 Jun 2020 19:56:57 +0000 (12:56 -0700)]
Fix unused variable warnings in fuzzing build mode without asserts
Fix unused vairable warnings when `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined but asserts are disabled.
Fixes #2210.
Nick Terrell [Mon, 22 Jun 2020 19:52:13 +0000 (12:52 -0700)]
[cover] Remove unnecessary mask and dedup hash functions
* Remove the unnecessary mask, since `ZSTD_hash*()` already ensures
the output is mod 2^h.
* Dedup the hash functions and use `ZSTD_hash*()` directly.
Nick Terrell [Mon, 22 Jun 2020 19:46:54 +0000 (12:46 -0700)]
Merge pull request #2212 from cwoffenden/single-file-dict
Single-file libs now include dictBuilder
Nick Terrell [Mon, 22 Jun 2020 19:27:58 +0000 (12:27 -0700)]
Merge pull request #2214 from terrelln/superblock-cross-entropy
[superblock] Add defensive assert and bounds check
Nick Terrell [Mon, 22 Jun 2020 17:21:38 +0000 (10:21 -0700)]
[superblock] Add defensive assert and bounds check
The bound check condition should always be met because we selected `set_basic` as
our encoding type. But that code is very far away, so assert it is true so if it is
ever false we can catch it, and add a bounds check.
Fixes #2213.
Carl Woffenden [Mon, 22 Jun 2020 09:54:24 +0000 (11:54 +0200)]
Renamed cover and fast cover hash functions/vars
Carl Woffenden [Mon, 22 Jun 2020 08:31:36 +0000 (10:31 +0200)]
Initial implementation (files added, macros fixed)
Hashing functions still to fix.
Niadb [Fri, 19 Jun 2020 15:51:00 +0000 (09:51 -0600)]
Update compiler.h
clean wording
Niadb [Fri, 19 Jun 2020 15:48:35 +0000 (09:48 -0600)]
Update compiler.h
Added a comment explaining the purpose of the WIN_CDECL macro
Niadb [Fri, 19 Jun 2020 09:34:05 +0000 (03:34 -0600)]
Add files via upload
Niadb [Fri, 19 Jun 2020 09:32:11 +0000 (03:32 -0600)]
Add files via upload