]>
git.ipfire.org Git - thirdparty/zstd.git/log
Dima Krasner [Sat, 18 Feb 2017 21:30:57 +0000 (23:30 +0200)]
Moved to contrib
Dima Krasner [Fri, 17 Feb 2017 10:32:16 +0000 (12:32 +0200)]
Added a message when multhread=true
Dima Krasner [Fri, 17 Feb 2017 10:29:23 +0000 (12:29 +0200)]
Updated the README
Dima Krasner [Fri, 17 Feb 2017 10:27:13 +0000 (12:27 +0200)]
Added multi-threaded library support
Dima Krasner [Thu, 16 Feb 2017 17:35:56 +0000 (19:35 +0200)]
Added a Meson project
Yann Collet [Wed, 15 Feb 2017 20:00:50 +0000 (12:00 -0800)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Wed, 15 Feb 2017 20:00:03 +0000 (12:00 -0800)]
updated fse version
feature minor refactoring (removing FSE_abs())
also : fix a few minor issues recently introduced in examples
Yann Collet [Wed, 15 Feb 2017 18:20:15 +0000 (10:20 -0800)]
Merge pull request #545 from terrelln/force-window
[zstdmt] Fix MSAN failure with ZSTD_p_forceWindow
Yann Collet [Tue, 14 Feb 2017 20:29:00 +0000 (12:29 -0800)]
Merge pull request #547 from inikep/dev11
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
Yann Collet [Tue, 14 Feb 2017 18:45:19 +0000 (10:45 -0800)]
preset behavior for gzip, gunzip and gzcat
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
Yann Collet [Tue, 14 Feb 2017 18:30:22 +0000 (10:30 -0800)]
Merge pull request #543 from inikep/gz_compress
Gz compress
Nick Terrell [Tue, 14 Feb 2017 18:08:14 +0000 (10:08 -0800)]
Don't run test-pool with QEMU
> make test -n
...
./pool
> make test -n QEMU_SYS=valgrind
...
./legacy
# ./pool not run
Yann Collet [Tue, 14 Feb 2017 08:54:55 +0000 (00:54 -0800)]
Merge pull request #544 from ds77/avoid-empty
Portable way to avoid empty unit warning in threading.c
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:52:52 +0000 (09:52 +0100)]
fix LONG_SEEK
Yann Collet [Tue, 14 Feb 2017 08:49:06 +0000 (00:49 -0800)]
Merge pull request #546 from zefanxu/dev
fix memory leak
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:47:29 +0000 (09:47 +0100)]
fixed function name (2)
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:45:33 +0000 (09:45 +0100)]
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:39:09 +0000 (09:39 +0100)]
Windows resources updated to v1.1.4
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:38:51 +0000 (09:38 +0100)]
removed UTIL_doesFileExists (replaced with UTIL_isRegFile)
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:26:03 +0000 (09:26 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Przemyslaw Skibinski [Tue, 14 Feb 2017 08:23:32 +0000 (09:23 +0100)]
fixed function name
zefanxu2 [Tue, 14 Feb 2017 03:12:59 +0000 (21:12 -0600)]
fix memory leak
Nick Terrell [Tue, 14 Feb 2017 02:27:34 +0000 (18:27 -0800)]
[zstdmt] Fix MSAN failure with ZSTD_p_forceWindow
Reproduction steps:
```
make zstreamtest CC=clang CFLAGS="-O3 -g -fsanitize=memory -fsanitize-memory-track-origins"
./zstreamtest -vv -t4178 -i4178 -s4531
```
How to get to the error in gdb (may be a more efficient way):
* 2 breaks at zstd_compress.c:2418 -- in ZSTD_compressContinue_internal()
* 2 breaks at zstd_compress.c:2276 -- in ZSTD_compressBlock_internal()
* 1 break at zstd_compress.c:1547
Why the error occurred:
When `zc->forceWindow == 1`, after calling `ZSTD_loadDictionaryContent()` we
have `zc->loadedDictEnd == zc->nextToUpdate == 0`. But, we've really loaded up
to `iend` into the dictionary. Then in `ZSTD_compressBlock_internal()` we see
that `current > zc->nextToUpdate + 384`, so we load the last 192 bytes a second
time. In this case the bytes we are loading are a block of all 0s, starting in
the previous block. So when we are loading the last 192 bytes, we find a `match`
in the future, 183 bytes beyond `ip`. Since the block is all 0s, the match
extends to the end of the block. But in `ZSTD_count()` we only check that
`pIn < pInLoopLimit`, but since `pMatch > pIn`, `pMatch` eventually points past
the end of the buffer, causing the MSAN failure.
The fix:
The line changed sets sets `zc->nextToUpdate` to the end of the dictionary.
This is the behavior that existed before `ZSTD_p_forceWindow` was introduced.
This fixes the exposing test case. Since the code doesn't fail without
`zc->forceWindow`, it makes sense that this works. I've run the command
`./zstreamtest -T2mn` 64 times without failures. CI should also verify nothing
obvious broke.
Yann Collet [Tue, 14 Feb 2017 02:32:12 +0000 (18:32 -0800)]
push version and NEWS to v1.1.4
ds77 [Mon, 13 Feb 2017 23:14:24 +0000 (00:14 +0100)]
avoid empty translation unit warning without #pragma
Przemyslaw Skibinski [Mon, 13 Feb 2017 21:56:31 +0000 (22:56 +0100)]
fixed unitialized variable warning
Przemyslaw Skibinski [Mon, 13 Feb 2017 20:35:39 +0000 (21:35 +0100)]
Resolve conflict with branch 'refs/remotes/facebook/dev'
# Conflicts:
# programs/fileio.c
Yann Collet [Mon, 13 Feb 2017 20:18:16 +0000 (12:18 -0800)]
Merge pull request #536 from iburinoc/multiframe
Simple API multiframe decoding
Przemyslaw Skibinski [Mon, 13 Feb 2017 20:00:41 +0000 (21:00 +0100)]
limit zlib compression level to Z_BEST_COMPRESSION
Yann Collet [Mon, 13 Feb 2017 19:57:35 +0000 (11:57 -0800)]
Merge pull request #542 from ds77/large-sparse-fix
zstdcli: fix writing 2GB+ sparse files under Windows
Przemyslaw Skibinski [Mon, 13 Feb 2017 19:47:01 +0000 (20:47 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress
# Conflicts:
# programs/Makefile
Przemyslaw Skibinski [Mon, 13 Feb 2017 12:57:29 +0000 (13:57 +0100)]
MinGW-w64 requires _FILE_OFFSET_BITS 64
Przemyslaw Skibinski [Mon, 13 Feb 2017 11:45:53 +0000 (12:45 +0100)]
__builtin_bswap requires gcc 4.3+
Przemyslaw Skibinski [Mon, 13 Feb 2017 11:15:10 +0000 (12:15 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
ds77 [Mon, 13 Feb 2017 11:00:59 +0000 (12:00 +0100)]
fix indentation in previous commit
Przemyslaw Skibinski [Mon, 13 Feb 2017 08:48:15 +0000 (09:48 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
ds77 [Sun, 12 Feb 2017 09:27:18 +0000 (10:27 +0100)]
fix seeking 2GB+ files under Windows
Replace fseek() in FIO_fwriteSparse() and FIO_fwriteSparseEnd() with macro expanding to 64-bit fseek version provided by the platform (includes fallback workaround using Win32 API).
Yann Collet [Sun, 12 Feb 2017 05:08:08 +0000 (21:08 -0800)]
Merge pull request #540 from ds77/dev-stat64-fix
zstdcli: Fix reporting incorrect sizes of large flies on MinGW
Yann Collet [Sun, 12 Feb 2017 05:07:53 +0000 (21:07 -0800)]
Merge pull request #541 from inikep/mingw
fixed "mingw32" AppVeyor target
Przemyslaw Skibinski [Fri, 10 Feb 2017 20:15:49 +0000 (21:15 +0100)]
revert last commit
Sean Purcell [Fri, 10 Feb 2017 19:38:57 +0000 (11:38 -0800)]
Expose frameSrcSize to experimental API
Przemyslaw Skibinski [Fri, 10 Feb 2017 19:09:28 +0000 (20:09 +0100)]
fix for original MinGW
Sean Purcell [Fri, 10 Feb 2017 18:08:55 +0000 (10:08 -0800)]
Merge branch 'dev' into multiframe
ds77 [Fri, 10 Feb 2017 17:37:57 +0000 (18:37 +0100)]
use _stati64() in UTIL_getFileSize() when compiling with mingw, get rid of introduces previously preprocessor checks.
Przemyslaw Skibinski [Fri, 10 Feb 2017 12:38:54 +0000 (13:38 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into mingw
# Conflicts:
# programs/platform.h
Yann Collet [Fri, 10 Feb 2017 12:35:43 +0000 (04:35 -0800)]
Merge pull request #537 from terrelln/small-bugs
Fix small bugs
- [Fri, 10 Feb 2017 12:27:43 +0000 (13:27 +0100)]
fix previous commit
* struct _stat64 is not defined by (non-w64) MinGW releases, __stat64 should be everywhere
* proper detection of _stat64() availability (as in MinGW sys/stat.h)
Przemyslaw Skibinski [Fri, 10 Feb 2017 12:10:25 +0000 (13:10 +0100)]
appveyor.yml: fixed clang test
Yann Collet [Fri, 10 Feb 2017 11:59:56 +0000 (03:59 -0800)]
Merge pull request #538 from iburinoc/errorstring
Fix ZSTD_getErrorString and add tests
Yann Collet [Fri, 10 Feb 2017 11:50:08 +0000 (03:50 -0800)]
Merge pull request #533 from inikep/dev11
--priority=rt
Przemyslaw Skibinski [Fri, 10 Feb 2017 11:41:31 +0000 (12:41 +0100)]
appveyor.yml: reordering of tests
Przemyslaw Skibinski [Fri, 10 Feb 2017 11:32:30 +0000 (12:32 +0100)]
fix missing "
Przemyslaw Skibinski [Fri, 10 Feb 2017 11:01:14 +0000 (12:01 +0100)]
appveyor.yml: add clang target
Przemyslaw Skibinski [Fri, 10 Feb 2017 10:01:52 +0000 (11:01 +0100)]
fixed "mingw32" AppVeyor target
- [Fri, 10 Feb 2017 09:56:45 +0000 (10:56 +0100)]
use _stat64 only when targetting Win2k or later
Przemyslaw Skibinski [Fri, 10 Feb 2017 09:36:44 +0000 (10:36 +0100)]
fix 64-bit file support for MinGW
Nick Terrell [Thu, 9 Feb 2017 22:20:52 +0000 (14:20 -0800)]
Improvement from @inikep
ds77 [Thu, 9 Feb 2017 17:12:00 +0000 (18:12 +0100)]
use _stat64() on MinGW
On MinGW, use _stat64() and struct _stat64 instead of stat() and struct stat_t. This fixes reporting incorrect sizes for large files.
Sean Purcell [Thu, 9 Feb 2017 21:25:30 +0000 (13:25 -0800)]
Documentation updates
Sean Purcell [Thu, 9 Feb 2017 20:27:32 +0000 (12:27 -0800)]
Fix failing unit test
Sean Purcell [Thu, 9 Feb 2017 19:54:22 +0000 (11:54 -0800)]
Merge branch 'dev' into multiframe
Sean Purcell [Thu, 9 Feb 2017 18:50:43 +0000 (10:50 -0800)]
Make pledgedSrcSize meaning clear for other functions
- Added tests
- Moved new size functions to static link only
Przemyslaw Skibinski [Thu, 9 Feb 2017 16:01:17 +0000 (17:01 +0100)]
echo replaced with printf
Przemyslaw Skibinski [Thu, 9 Feb 2017 10:32:14 +0000 (11:32 +0100)]
don't use "echo -e"
Przemyslaw Skibinski [Thu, 9 Feb 2017 08:42:16 +0000 (09:42 +0100)]
Merge branch 'dev' into dev11
Yann Collet [Thu, 9 Feb 2017 08:36:03 +0000 (00:36 -0800)]
Merge pull request #535 from terrelln/tests
Remove test-longmatch from test target and only run it once
Nick Terrell [Thu, 9 Feb 2017 01:38:17 +0000 (17:38 -0800)]
Fix deprecation warnings for clang with C++14
Sean Purcell [Wed, 8 Feb 2017 23:31:47 +0000 (15:31 -0800)]
Fix ZSTD_getErrorString and add tests
Nick Terrell [Thu, 9 Feb 2017 01:25:01 +0000 (17:25 -0800)]
#undef _POSIX_C_SOURCE if already defined
Sean Purcell [Wed, 8 Feb 2017 21:56:32 +0000 (13:56 -0800)]
Test multiframe legacy decoding with simple and streaming APIs
Sean Purcell [Wed, 8 Feb 2017 00:33:48 +0000 (16:33 -0800)]
Disambiguate pledgedSrcSize == 0
- Modify ZSTD CLI to only set contentSizeFlag if it _knows_ the size
- Change pzstd to stop setting contentSizeFlag without accurate pledgedSrcSize
Sean Purcell [Wed, 8 Feb 2017 00:16:55 +0000 (16:16 -0800)]
ZSTD_decompress now handles multiple frames
Sean Purcell [Tue, 7 Feb 2017 21:50:09 +0000 (13:50 -0800)]
Decompressed size functions now handle multiframes and distinguish cases
- Add ZSTD_findDecompressedSize
- Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
- Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
Przemyslaw Skibinski [Wed, 8 Feb 2017 20:11:18 +0000 (21:11 +0100)]
remove redundant "ifeq ($(HAVE_ZLIB), 1)"
Nick Terrell [Wed, 8 Feb 2017 20:00:21 +0000 (12:00 -0800)]
Remove test-longmatch from test target and only run it once
Przemyslaw Skibinski [Wed, 8 Feb 2017 17:08:09 +0000 (18:08 +0100)]
fixed gcc warnings
Przemyslaw Skibinski [Wed, 8 Feb 2017 16:37:14 +0000 (17:37 +0100)]
.gz suffix for gzip compressed files
Przemyslaw Skibinski [Wed, 8 Feb 2017 15:54:23 +0000 (16:54 +0100)]
added FIO_compressGzFrame
Przemyslaw Skibinski [Wed, 8 Feb 2017 14:17:55 +0000 (15:17 +0100)]
add "--format=gzip" option
Przemyslaw Skibinski [Wed, 8 Feb 2017 13:32:49 +0000 (14:32 +0100)]
fixed zlib detection with MinGW
Przemyslaw Skibinski [Wed, 8 Feb 2017 12:58:04 +0000 (13:58 +0100)]
turn off test-pool for qemu-ppc64-static
Przemyslaw Skibinski [Wed, 8 Feb 2017 12:49:35 +0000 (13:49 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Przemyslaw Skibinski [Wed, 8 Feb 2017 12:49:06 +0000 (13:49 +0100)]
added description of "--priority=rt"
Yann Collet [Tue, 7 Feb 2017 23:34:33 +0000 (15:34 -0800)]
Merge pull request #526 from iburinoc/educational
Add educational decoder to /contrib
Sean Purcell [Tue, 7 Feb 2017 22:44:11 +0000 (14:44 -0800)]
Minor changes to educational decoder
Yann Collet [Tue, 7 Feb 2017 20:05:28 +0000 (12:05 -0800)]
completed NEWS for v1.1.3
Yann Collet [Tue, 7 Feb 2017 19:58:59 +0000 (11:58 -0800)]
Merge pull request #534 from terrelln/dev
Attribute cover dictionary code
Nick Terrell [Tue, 7 Feb 2017 19:35:07 +0000 (11:35 -0800)]
Attribute cover dictionary code
Przemyslaw Skibinski [Tue, 7 Feb 2017 19:12:59 +0000 (20:12 +0100)]
"--rt-prio" renamed to "--priority=rt"
Przemyslaw Skibinski [Tue, 7 Feb 2017 15:48:01 +0000 (16:48 +0100)]
added the "--rt-prio" option
Przemyslaw Skibinski [Tue, 7 Feb 2017 15:36:19 +0000 (16:36 +0100)]
SET_REALTIME_PRIORITY
Yann Collet [Tue, 7 Feb 2017 01:40:38 +0000 (17:40 -0800)]
Merge branch 'dev' of github.com:facebook/zstd into dev
Yann Collet [Tue, 7 Feb 2017 01:39:54 +0000 (17:39 -0800)]
corrected contributor's name
Yann Collet [Mon, 6 Feb 2017 22:44:23 +0000 (14:44 -0800)]
Merge pull request #532 from terrelln/zlib
Change zlib include to be a system include
Nick Terrell [Mon, 6 Feb 2017 19:54:31 +0000 (11:54 -0800)]
Ensure <zlib.h> can be included in HAVE_ZLIB test
Nick Terrell [Mon, 6 Feb 2017 19:32:13 +0000 (11:32 -0800)]
Change zlib include to be a system include
Przemyslaw Skibinski [Mon, 6 Feb 2017 18:43:13 +0000 (19:43 +0100)]
platform.h added to build_package.bat
Yann Collet [Mon, 6 Feb 2017 18:01:06 +0000 (10:01 -0800)]
changed download URL for github_users sample set
Przemyslaw Skibinski [Mon, 6 Feb 2017 10:21:51 +0000 (11:21 +0100)]
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
Yann Collet [Mon, 6 Feb 2017 02:09:35 +0000 (18:09 -0800)]
updated NEWS for v1.1.3