]> git.ipfire.org Git - thirdparty/zstd.git/log
thirdparty/zstd.git
13 months agoFix typos not found by codespell 4068/head
Dimitri Papadopoulos [Wed, 19 Jun 2024 19:04:50 +0000 (21:04 +0200)] 
Fix typos not found by codespell

13 months agoFix new typos found by codespell
Dimitri Papadopoulos [Wed, 5 Jun 2024 17:41:43 +0000 (19:41 +0200)] 
Fix new typos found by codespell

13 months agoMerge pull request #4076 from facebook/fix_macos_build
Yann Collet [Wed, 19 Jun 2024 05:16:00 +0000 (22:16 -0700)] 
Merge pull request #4076 from facebook/fix_macos_build

fix macos build

13 months agofix macos build 4076/head
Yann Collet [Wed, 19 Jun 2024 03:21:25 +0000 (20:21 -0700)] 
fix macos build

weird: after replacing the UNAME line with an identical one,
it does work properly now(??).
Possibly a case of hidden special character?

13 months agoExpose size estimation helpers for Linux (#4064)
Elliot Gorokhovsky [Fri, 14 Jun 2024 18:38:48 +0000 (14:38 -0400)] 
Expose size estimation helpers for Linux (#4064)

13 months agoMerge pull request #4073 from facebook/cygwin_install
Yann Collet [Wed, 12 Jun 2024 18:58:54 +0000 (11:58 -0700)] 
Merge pull request #4073 from facebook/cygwin_install

added a cygwin install test to check #4067

13 months agoadded cygwin install test 4073/head
Yann Collet [Wed, 12 Jun 2024 16:51:01 +0000 (09:51 -0700)] 
added cygwin install test

13 months agoMerge pull request #4067 from QBos07/filterfix
Yann Collet [Wed, 12 Jun 2024 17:55:18 +0000 (10:55 -0700)] 
Merge pull request #4067 from QBos07/filterfix

Fix $filter operants and Msys/Cygwin

13 months agoMerge pull request #4065 from embg/freebsd_ci
Yann Collet [Wed, 12 Jun 2024 16:54:25 +0000 (09:54 -0700)] 
Merge pull request #4065 from embg/freebsd_ci

Drop support for FreeBSD 13.2 CI

13 months agoFix $filter and Msys/Cygwin 4067/head
Quentin Boswank [Wed, 5 Jun 2024 16:21:34 +0000 (18:21 +0200)] 
Fix $filter and Msys/Cygwin

- switched the patter and input of $filter into the right places
- added pattern wildcard to MSYS_NT & CYGWIN_NT as they change with windows versions
- correctly identify MSYS2, even in an env like MINGW64

13 months agominor:doc: specify decompression behavior in presence of multiple concatenated frames
Yann Collet [Tue, 4 Jun 2024 01:30:23 +0000 (18:30 -0700)] 
minor:doc: specify decompression behavior in presence of multiple concatenated frames

directly at ZSTD_decompress() level.

13 months agoUnit test for external sequence producer + static CCtx + streaming (#4063)
Elliot Gorokhovsky [Mon, 3 Jun 2024 16:42:27 +0000 (12:42 -0400)] 
Unit test for external sequence producer + static CCtx + streaming (#4063)

13 months agoDrop FreeBSD 13.2 CI 4065/head
Elliot Gorokhovsky [Mon, 3 Jun 2024 15:36:07 +0000 (08:36 -0700)] 
Drop FreeBSD 13.2 CI

14 months agoMerge pull request #4046 from josepho0918/iar
Yann Collet [Wed, 29 May 2024 22:33:19 +0000 (15:33 -0700)] 
Merge pull request #4046 from josepho0918/iar

Improve support for IAR compiler with attributes and intrinsics

14 months agoMerge pull request #4054 from jbajic/fix-missing-newline
Yann Collet [Wed, 29 May 2024 22:32:54 +0000 (15:32 -0700)] 
Merge pull request #4054 from jbajic/fix-missing-newline

[fix] Add newline when file exceeds 128KB

14 months agoRefactor dictionary matchfinder index safety check (#4039)
Federico Maresca [Wed, 29 May 2024 16:35:24 +0000 (18:35 +0200)] 
Refactor dictionary matchfinder index safety check (#4039)

14 months ago[fix] Add newline when file exceeds 128KB 4054/head
Jure Bajic [Sun, 26 May 2024 09:33:39 +0000 (11:33 +0200)] 
[fix] Add newline when file exceeds 128KB

14 months agoMerge pull request #4050 from Adenilson/fix_legacy_nullptr01
Yann Collet [Tue, 21 May 2024 17:30:45 +0000 (10:30 -0700)] 
Merge pull request #4050 from Adenilson/fix_legacy_nullptr01

[fix] Add check on failed allocation in legacy/zstd_v06

14 months ago[fix] Add check on failed allocation in legacy/zstd_v06 4050/head
Adenilson Cavalcanti [Fri, 17 May 2024 20:37:55 +0000 (13:37 -0700)] 
[fix] Add check on failed allocation in legacy/zstd_v06

As reported by Ben Hawkes in #4026, a failure to allocate a zstd context
would lead to a dereference of a NULL pointer due to a missing check
on the returned result of ZSTDv06_createDCtx().

This patch fix the issue by adding a check for valid returned pointer.

14 months agorevert FSE_readNCount_body attribute 4046/head
Joseph Chen [Wed, 15 May 2024 02:47:50 +0000 (10:47 +0800)] 
revert FSE_readNCount_body attribute

14 months agoImprove support for IAR compiler with attributes and intrinsics
Joseph Chen [Tue, 14 May 2024 08:51:10 +0000 (16:51 +0800)] 
Improve support for IAR compiler with attributes and intrinsics

14 months agoMerge pull request #4040 from facebook/dependabot/github_actions/msys2/setup-msys2...
Yann Collet [Mon, 13 May 2024 16:15:07 +0000 (09:15 -0700)] 
Merge pull request #4040 from facebook/dependabot/github_actions/msys2/setup-msys2-2.23.0

Bump msys2/setup-msys2 from 2.22.0 to 2.23.0

14 months agoBump msys2/setup-msys2 from 2.22.0 to 2.23.0 4040/head
dependabot[bot] [Mon, 13 May 2024 05:18:11 +0000 (05:18 +0000)] 
Bump msys2/setup-msys2 from 2.22.0 to 2.23.0

Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.22.0 to 2.23.0.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/msys2/setup-msys2/compare/cc11e9188b693c2b100158c3322424c4cc1dadea...d0e80f58dffbc64f6a3a1f43527d469b4fc7b6c8)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
15 months agoIncrease x-compatibility
Richard Barnes [Fri, 26 Apr 2024 21:44:41 +0000 (14:44 -0700)] 
Increase x-compatibility

15 months agoMake zstd.h compatible with -Wzero-as-null-pointer-constant
Richard Barnes [Fri, 26 Apr 2024 20:48:49 +0000 (13:48 -0700)] 
Make zstd.h compatible with -Wzero-as-null-pointer-constant

15 months agoMerge pull request #4029 from facebook/dependabot/github_actions/github/codeql-action...
Felix Handte [Fri, 26 Apr 2024 15:39:39 +0000 (08:39 -0700)] 
Merge pull request #4029 from facebook/dependabot/github_actions/github/codeql-action-3.25.1

Bump github/codeql-action from 3.24.10 to 3.25.1

15 months agoMerge pull request #4031 from facebook/docDStream
Yann Collet [Thu, 25 Apr 2024 21:11:32 +0000 (14:11 -0700)] 
Merge pull request #4031 from facebook/docDStream

update documentation of ZSTD_decompressStream()

15 months agoupdate documentation of ZSTD_decompressStream() 4031/head
Yann Collet [Tue, 23 Apr 2024 16:31:35 +0000 (09:31 -0700)] 
update documentation of ZSTD_decompressStream()

slightly more precise, by recommending to check the return value.
fix #4030

15 months agoBump github/codeql-action from 3.24.10 to 3.25.1 4029/head
dependabot[bot] [Mon, 22 Apr 2024 05:20:46 +0000 (05:20 +0000)] 
Bump github/codeql-action from 3.24.10 to 3.25.1

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.10 to 3.25.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/4355270be187e1b672a7a1c7c7bae5afdc1ab94a...c7f9125735019aa87cfc361530512d50ea439c71)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
15 months agoupdate test name
Yann Collet [Sun, 21 Apr 2024 17:44:50 +0000 (10:44 -0700)] 
update test name

15 months agoMerge pull request #4025 from alexlnkp/dev
Yann Collet [Sun, 21 Apr 2024 17:14:33 +0000 (10:14 -0700)] 
Merge pull request #4025 from alexlnkp/dev

Fixed all memory leaks and almost all undefined behaviour

15 months agoMerge pull request #3931 from bgilbert/pthread
Yann Collet [Sun, 21 Apr 2024 03:25:19 +0000 (20:25 -0700)] 
Merge pull request #3931 from bgilbert/pthread

meson: don't add `-pthread` to static linking flags on Windows

15 months agoMerge pull request #4019 from b7f7/fix_cmake_rc_build
Yann Collet [Tue, 16 Apr 2024 23:06:17 +0000 (16:06 -0700)] 
Merge pull request #4019 from b7f7/fix_cmake_rc_build

fix missing include folder for resource compiler

15 months agoMerge pull request #4020 from Ansuel/install-mt-pc
Yann Collet [Tue, 16 Apr 2024 23:04:23 +0000 (16:04 -0700)] 
Merge pull request #4020 from Ansuel/install-mt-pc

Provide variant pkg-config file for multi-threaded static lib

15 months agoremoved freeing of the ptr 4025/head
Alex Murkoff [Sun, 14 Apr 2024 13:44:12 +0000 (20:44 +0700)] 
removed freeing of the ptr

15 months agoinit out char array with all members as 0 x2
Alex Murkoff [Sun, 14 Apr 2024 10:57:53 +0000 (17:57 +0700)] 
init out char array with all members as 0 x2

15 months agomade initialize out char array with all elements set to 0
Alex Murkoff [Sun, 14 Apr 2024 10:45:53 +0000 (17:45 +0700)] 
made initialize out char array with all elements set to 0
fixed where i made it to init with just the first one being set to 0

15 months agoreverted the addition of free at the gz_init()
Alex Murkoff [Sat, 13 Apr 2024 05:51:20 +0000 (12:51 +0700)] 
reverted the addition of free at the gz_init()

15 months agofixed ISO C incompatibility
Alex Murkoff [Fri, 12 Apr 2024 19:37:13 +0000 (02:37 +0700)] 
fixed ISO C incompatibility
for good this time... probably

15 months agofixed ISO C incompatibility
Alex Murkoff [Fri, 12 Apr 2024 19:35:32 +0000 (02:35 +0700)] 
fixed ISO C incompatibility

15 months agofixed memory leaks and almost all undefined behaviour
Alex Murkoff [Fri, 12 Apr 2024 19:24:14 +0000 (02:24 +0700)] 
fixed memory leaks and almost all undefined behaviour

15 months agoMerge pull request #4022 from facebook/dependabot/github_actions/github/codeql-action...
Felix Handte [Fri, 12 Apr 2024 15:51:02 +0000 (08:51 -0700)] 
Merge pull request #4022 from facebook/dependabot/github_actions/github/codeql-action-3.24.10

Bump github/codeql-action from 3.24.9 to 3.24.10

15 months agomeson: don't add -pthread to static linking flags on Windows 3931/head
Benjamin Gilbert [Thu, 7 Mar 2024 12:40:27 +0000 (21:40 +0900)] 
meson: don't add -pthread to static linking flags on Windows

Meson always returns -pthread in dependency('threads') on non-MSVC
compilers.  On Windows we use Windows threading primitives, so we don't
need this.  Avoid adding -pthread to libzstd's link flags, either as a
Meson subproject or via pkg-config Libs.private, so the application
doesn't inadvertently depend on winpthreads.

Add a Meson MinGW cross-compile CI test that checks for this.  It turns
out that pzstd fails to build in that environment, so have the test
skip building contrib for now.

15 months agoMerge pull request #4021 from pstef/dev
Yann Collet [Tue, 9 Apr 2024 18:02:49 +0000 (11:02 -0700)] 
Merge pull request #4021 from pstef/dev

Fix zlibWrapper build

15 months agoProvide variant pkg-config file for multi-threaded static lib 4020/head
Christian Marangi [Sat, 6 Apr 2024 12:41:54 +0000 (14:41 +0200)] 
Provide variant pkg-config file for multi-threaded static lib

Multi-threaded static library require -pthread to correctly link and works.
The pkg-config we provide tho only works with dynamic multi-threaded library
and won't provide the correct libs and cflags values if lib-mt is used.

To handle this, introduce an env variable MT to permit advanced user to
install and generate a correct pkg-config file for lib-mt or detect if
lib-mt target is called.

With MT env set on calling make install-pc, libzstd.pc.in is a
pkg-config file for a multi-threaded static library.

On calling make lib-mt, a libzstd.pc is generated for a multi-threaded
static library as it's what asked by the user by forcing it.

libzstd.pc is changed to PHONY to force regeneration of it on calling
lib targets or install-pc to handle case where the same directory is
used for mixed compilation.

This was notice while migrating from meson to make build system where
meson generates a correct .pc file while make doesn't.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
15 months agoBump github/codeql-action from 3.24.9 to 3.24.10 4022/head
dependabot[bot] [Mon, 8 Apr 2024 05:47:58 +0000 (05:47 +0000)] 
Bump github/codeql-action from 3.24.9 to 3.24.10

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.9 to 3.24.10.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/1b1aada464948af03b950897e5eb522f92603cc2...4355270be187e1b672a7a1c7c7bae5afdc1ab94a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
15 months agoFix zlibWrapper build 4021/head
Piotr PaweÅ‚ Stefaniak [Sun, 7 Apr 2024 11:04:26 +0000 (13:04 +0200)] 
Fix zlibWrapper build

Just after a clone I'm getting this:

~/zstd/zlibWrapper$ cc -c zstd_zlibwrapper.o gz*.c -lz -lzstd -DSTDC
gzwrite.c: In function â€˜gz_write’:
gzwrite.c:226:43: error: â€˜z_uInt’ undeclared (first use in this
                         function); did you mean â€˜uInt’?
  226 |             state.state->strm.avail_in = (z_uInt)n;
      |                                           ^~~~~~
      |                                           uInt
gzwrite.c:226:43: note: each undeclared identifier is reported only
                        once for each function it appears in
gzwrite.c:226:50: error: expected â€˜;’ before â€˜n’
  226 |             state.state->strm.avail_in = (z_uInt)n;
      |                                                  ^
      |                                                  ;

z_uInt is never used directly, zconf.h redefines uInt to z_uInt under
the condition that Z_PREFIX is set. All examples use uInt, and the type
of avail_in is also uInt.

In this commit I modify the cast to refer to the same type as the type
of lvalue.

Arguably, the real fix here is to handle possible overflows, but that's
beyond the scope of this commit.

15 months agoMerge pull request #3996 from facebook/fix_nodejs_version_warning
Yann Collet [Sat, 6 Apr 2024 06:06:21 +0000 (23:06 -0700)] 
Merge pull request #3996 from facebook/fix_nodejs_version_warning

fix nodejs deprecation warning

15 months agoMerge pull request #4012 from elasota/spec-remove-fse-overflow
Yann Collet [Sat, 6 Apr 2024 06:06:06 +0000 (23:06 -0700)] 
Merge pull request #4012 from elasota/spec-remove-fse-overflow

Remove specification text stating that probability overflow is invalid

15 months agoMerge pull request #4011 from facebook/fix4005
Yann Collet [Sat, 6 Apr 2024 06:05:53 +0000 (23:05 -0700)] 
Merge pull request #4011 from facebook/fix4005

decompression errors always display the full origin filename

15 months agofix missing include folder for resource compiler 4019/head
BadWolf [Fri, 5 Apr 2024 22:05:58 +0000 (00:05 +0200)] 
fix missing include folder for resource compiler

16 months agoRemove text specifying probability overflow as invalid, the variable-size value encod... 4012/head
elasota [Tue, 2 Apr 2024 00:08:42 +0000 (20:08 -0400)] 
Remove text specifying probability overflow as invalid, the variable-size value encoding scheme makes this impossible.

16 months agodecompression errors always display the full origin filename 4011/head
Yann Collet [Mon, 1 Apr 2024 18:12:26 +0000 (11:12 -0700)] 
decompression errors always display the full origin filename

instead of the truncated size-limited version.

16 months agoFix building on windows-x86 if clang already includes
Yuriy Chernyshov [Thu, 28 Mar 2024 10:54:53 +0000 (11:54 +0100)] 
Fix building on windows-x86 if clang already includes

[D101338](https://reviews.llvm.org/D101338) landed in 2021, so clang16 should have it

16 months agoMerge pull request #3994 from sunpoet/dev
Yann Collet [Mon, 1 Apr 2024 06:09:46 +0000 (23:09 -0700)] 
Merge pull request #3994 from sunpoet/dev

Use md5sum rather than gmd5sum for FreeBSD

16 months agoMerge pull request #3997 from facebook/readme_bench_156
Yann Collet [Sun, 31 Mar 2024 20:17:01 +0000 (13:17 -0700)] 
Merge pull request #3997 from facebook/readme_bench_156

updated benchmarks for v1.5.6

16 months agoupdated benchmarks for v1.5.6 3997/head
Yann Collet [Thu, 28 Mar 2024 01:19:31 +0000 (18:19 -0700)] 
updated benchmarks for v1.5.6

16 months agoMerge pull request #3988 from facebook/dependabot/github_actions/github/codeql-action...
Yann Collet [Thu, 28 Mar 2024 01:05:42 +0000 (18:05 -0700)] 
Merge pull request #3988 from facebook/dependabot/github_actions/github/codeql-action-3.24.9

Bump github/codeql-action from 3.24.7 to 3.24.9

16 months agofix nodejs deprecation warning 3996/head
Yann Collet [Wed, 27 Mar 2024 23:10:15 +0000 (16:10 -0700)] 
fix nodejs deprecation warning

by updating msys2 action

16 months agoMerge pull request #3987 from ManuelBlanc/msbuild_vswhere
Yann Collet [Wed, 27 Mar 2024 22:48:40 +0000 (15:48 -0700)] 
Merge pull request #3987 from ManuelBlanc/msbuild_vswhere

Use vswhere to find MSBuild; add VS2022 support

16 months agoUse md5sum rather than gmd5sum for FreeBSD 3994/head
Po-Chuan Hsieh [Wed, 27 Mar 2024 12:53:52 +0000 (20:53 +0800)] 
Use md5sum rather than gmd5sum for FreeBSD

Reference: https://man.freebsd.org/cgi/man.cgi?query=md5sum

16 months ago[fuzz] Turn off -Werror by default
Nick Terrell [Tue, 26 Mar 2024 17:01:19 +0000 (10:01 -0700)] 
[fuzz] Turn off -Werror by default

This was causing OSS-Fuzz errors, due to compiler differences.
* Fix the issue
* Also turn off -Werror so we don't fail fuzzer builds for warnings
* Turn on -Werror in our CI

16 months agoBump github/codeql-action from 3.24.7 to 3.24.9 3988/head
dependabot[bot] [Mon, 25 Mar 2024 05:31:42 +0000 (05:31 +0000)] 
Bump github/codeql-action from 3.24.7 to 3.24.9

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.7 to 3.24.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/3ab4101902695724f9365a384f86c1074d94e18c...1b1aada464948af03b950897e5eb522f92603cc2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
16 months agoImprove MSBuild search; add latest option 3987/head
ManuelBlanc [Sat, 23 Mar 2024 22:34:07 +0000 (18:34 -0400)] 
Improve MSBuild search; add latest option

16 months agoUse vswhere to find MSBuild; add VS2022 support
ManuelBlanc [Sat, 23 Mar 2024 12:44:52 +0000 (08:44 -0400)] 
Use vswhere to find MSBuild; add VS2022 support

See:
https://github.com/microsoft/vswhere/wiki/Find-MSBuild

16 months agoMerge pull request #3985 from facebook/scorecard_permission 3984/head
Yann Collet [Thu, 21 Mar 2024 20:56:20 +0000 (13:56 -0700)] 
Merge pull request #3985 from facebook/scorecard_permission

try to silence some scorecard warnings

16 months agotry to silence some scorecard warnings 3985/head
Yann Collet [Thu, 21 Mar 2024 20:25:48 +0000 (13:25 -0700)] 
try to silence some scorecard warnings

16 months agoMerge pull request #3977 from facebook/doc_advanced
Yann Collet [Thu, 21 Mar 2024 19:33:15 +0000 (12:33 -0700)] 
Merge pull request #3977 from facebook/doc_advanced

Doc update

16 months agofix -Werror=pointer-arith in fuzzers (#3983)
Elliot Gorokhovsky [Thu, 21 Mar 2024 19:16:38 +0000 (15:16 -0400)] 
fix -Werror=pointer-arith in fuzzers (#3983)

16 months agoMerge pull request #3982 from embg/fuzzer_readme
Yann Collet [Thu, 21 Mar 2024 18:10:03 +0000 (11:10 -0700)] 
Merge pull request #3982 from embg/fuzzer_readme

Document the process for adding a new fuzzer

16 months agoMerge pull request #3975 from terrelln/2024-03-18-cmake-docs
Yann Collet [Thu, 21 Mar 2024 17:41:54 +0000 (10:41 -0700)] 
Merge pull request #3975 from terrelln/2024-03-18-cmake-docs

[cmake] Emit warnings for contradictory build settings

16 months agoMerge pull request #3979 from yoniko/Werror-fuzz
Yann Collet [Thu, 21 Mar 2024 17:41:34 +0000 (10:41 -0700)] 
Merge pull request #3979 from yoniko/Werror-fuzz

Fail on errors when building fuzzers

16 months agoMerge pull request #3981 from terrelln/2024-03-19-generate-sequences
Yann Collet [Thu, 21 Mar 2024 17:41:21 +0000 (10:41 -0700)] 
Merge pull request #3981 from terrelln/2024-03-19-generate-sequences

Fix & fuzz ZSTD_generateSequences

16 months agoFix & fuzz ZSTD_generateSequences 3981/head
Nick Terrell [Tue, 19 Mar 2024 19:37:55 +0000 (12:37 -0700)] 
Fix & fuzz ZSTD_generateSequences

This function was seriously flawed:
* It didn't do output bounds checks
* It produced invalid sequences when an uncompressed or RLE block was emitted
* It produced invalid sequences when the block splitter was enabled
* It produced invalid sequences when ZSTD_c_targetCBlockSize was enabled

I've attempted to fix these issues, but this function is just a bad idea,
so I've marked it as deprecated and unsafe. We should replace it with
`ZSTD_extractSequences()` which operates on a compressed frame.

16 months agoFuzzing and bugfixes for magicless-format decoding (#3976)
Elliot Gorokhovsky [Wed, 20 Mar 2024 23:22:34 +0000 (19:22 -0400)] 
Fuzzing and bugfixes for magicless-format decoding (#3976)

* fuzzing and bugfixes for magicless format

* reset dctx before each decompression

* do not memcmp empty buffers

* nit: decompressor errata

16 months agoAdd docs on how to add a new fuzzer 3982/head
Elliot Gorokhovsky [Tue, 19 Mar 2024 21:05:23 +0000 (14:05 -0700)] 
Add docs on how to add a new fuzzer

16 months agoFail on errors when building fuzzers 3979/head
Yonatan Komornik [Mon, 18 Mar 2024 22:25:22 +0000 (15:25 -0700)] 
Fail on errors when building fuzzers

Fails on errors when building fuzzers with `fuzz.py` (adds `Werror`).
Currently allows `declaration-after-statement`, `c++-compat` and
`deprecated` as they are abundant in code (some fixes to
`declaration-after-statement` are presented in this commit).

16 months agoFix bugs in simple decompression fuzzer (#3978)
Yonatan Komornik [Mon, 18 Mar 2024 22:36:40 +0000 (15:36 -0700)] 
Fix bugs in simple decompression fuzzer (#3978)

Fixes 2 issue in `simple_decompress.c`:
1. Wrong type used for storing the results of `ZSTD_findDecompressedSize` resulting in never matching to `ZSTD_CONTENTSIZE_ERROR` or `ZSTD_CONTENTSIZE_UNKNOWN`.

2. Experimental API is used (`ZSTD_findDecompressedSize`) without defining `ZSTD_STATIC_LINKING_ONLY`.

16 months agofix ZSTD_TARGETCBLOCKSIZE_MIN test 3977/head
Yann Collet [Mon, 18 Mar 2024 21:10:08 +0000 (14:10 -0700)] 
fix ZSTD_TARGETCBLOCKSIZE_MIN test

when requested CBlockSize is too low,
bound it to the minimum
instead of returning an error.

16 months agofix typo
Yann Collet [Mon, 18 Mar 2024 19:33:22 +0000 (12:33 -0700)] 
fix typo

16 months agoupdated API manual
Yann Collet [Mon, 18 Mar 2024 19:30:54 +0000 (12:30 -0700)] 
updated API manual

16 months agoadd doc on CCtx UB state
Yann Collet [Mon, 18 Mar 2024 19:30:35 +0000 (12:30 -0700)] 
add doc on CCtx UB state

16 months agoadd a paragraph on UB DCtx state after error
Yann Collet [Mon, 18 Mar 2024 19:17:41 +0000 (12:17 -0700)] 
add a paragraph on UB DCtx state after error

16 months agoupdate targetCBlockSize documentation
Yann Collet [Mon, 18 Mar 2024 19:04:02 +0000 (12:04 -0700)] 
update targetCBlockSize documentation

16 months ago[cmake] Emit warnings for contradictory build settings 3975/head
Nick Terrell [Mon, 18 Mar 2024 16:28:06 +0000 (09:28 -0700)] 
[cmake] Emit warnings for contradictory build settings

Document that the `ZSTD_BUILD_{SHARED,STATIC}` take precedence over `BUILD_SHARED_LIBS` when exactly one is ON.

Thanks to @teo-tsirpanis for pointing out the potentially confusing behavior.

16 months agopzstd: use c++14 without conditions
Alexander Kanavin [Mon, 19 Jun 2023 15:10:09 +0000 (17:10 +0200)] 
pzstd: use c++14 without conditions

Doing this check with a direct c++ snippet is prone to portability problems:

- \043 is not portable between shells: dash expands it to #,
bash does not;

- using # directly works with make 4.3 but does not with make 4.2.

Let's just use the c++ version that covers both the code and the gtest.

16 months agoImplement one-shot fallback for magicless format (#3971)
Elliot Gorokhovsky [Mon, 18 Mar 2024 14:55:53 +0000 (10:55 -0400)] 
Implement one-shot fallback for magicless format (#3971)

16 months ago[cmake] Fix up PR #3716
Nick Terrell [Thu, 14 Mar 2024 19:12:55 +0000 (12:12 -0700)] 
[cmake] Fix up PR #3716

* Make a variable `PublicHeaders` for Zstd's public headers
* Add `PublicHeaders` to `Headers`, which was missing
* Only export `${LIBRARY_DIR}` publicly, not `common/`
* Switch the `target_include_directories()` to `INTERFACE` because zstd uses relative includes internally, so doesn't need any include directories to build
* Switch installation to use the `PublicHeaders` variable, and test that the right headers are installed

16 months agoMakes it possible to use the lib through FetchContent or ExternalProject_Add
dsvi [Fri, 4 Aug 2023 22:47:46 +0000 (01:47 +0300)] 
Makes it possible to use the lib through FetchContent or ExternalProject_Add

16 months agoMerge pull request #3973 from facebook/dependabot/github_actions/github/codeql-action...
Yann Collet [Mon, 18 Mar 2024 07:44:55 +0000 (00:44 -0700)] 
Merge pull request #3973 from facebook/dependabot/github_actions/github/codeql-action-3.24.7

Bump github/codeql-action from 3.24.6 to 3.24.7

16 months agoMerge pull request #3972 from facebook/dependabot/github_actions/actions/cache-4
Yann Collet [Mon, 18 Mar 2024 07:44:12 +0000 (00:44 -0700)] 
Merge pull request #3972 from facebook/dependabot/github_actions/actions/cache-4

Bump actions/cache from 3 to 4

16 months agoBump github/codeql-action from 3.24.6 to 3.24.7 3973/head
dependabot[bot] [Mon, 18 Mar 2024 05:37:56 +0000 (05:37 +0000)] 
Bump github/codeql-action from 3.24.6 to 3.24.7

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.6 to 3.24.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/8a470fddafa5cbb6266ee11b37ef4d8aae19c571...3ab4101902695724f9365a384f86c1074d94e18c)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
16 months agoBump actions/cache from 3 to 4 3972/head
dependabot[bot] [Mon, 18 Mar 2024 05:37:52 +0000 (05:37 +0000)] 
Bump actions/cache from 3 to 4

Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
16 months agoMerge pull request #3969 from facebook/v156_prep
Yann Collet [Fri, 15 Mar 2024 17:26:49 +0000 (10:26 -0700)] 
Merge pull request #3969 from facebook/v156_prep

bump version number

16 months agoMerge pull request #3966 from facebook/debug_lineNumber
Yann Collet [Fri, 15 Mar 2024 17:26:06 +0000 (10:26 -0700)] 
Merge pull request #3966 from facebook/debug_lineNumber

add line number to debug traces

16 months agoupdate CHANGELOG for v1.5.6 3969/head
Yann Collet [Fri, 15 Mar 2024 00:01:34 +0000 (17:01 -0700)] 
update CHANGELOG for v1.5.6

16 months agoupdated version to v1.5.6
Yann Collet [Thu, 14 Mar 2024 22:38:14 +0000 (15:38 -0700)] 
updated version to v1.5.6

16 months agoRemove duplicate and incorrect docs in zstd_decompress.c (#3967)
Elliot Gorokhovsky [Thu, 14 Mar 2024 19:55:01 +0000 (15:55 -0400)] 
Remove duplicate and incorrect docs in zstd_decompress.c (#3967)

16 months agoadd line number to debug traces 3966/head
Yann Collet [Thu, 14 Mar 2024 19:11:11 +0000 (12:11 -0700)] 
add line number to debug traces

16 months ago[cmake] Always create libzstd target
Nick Terrell [Thu, 14 Mar 2024 15:47:04 +0000 (08:47 -0700)] 
[cmake] Always create libzstd target

If both `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` are set, then cmake exports the libraries `libzstd_shared` and `libzstd_static` only.
It does not export `libzstd`, which is only exported when exactly one of `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` is set.
This PR exports `libzstd` in that case, based on the value of the standard CMake variable [`BUILD_SHARED_LIBS`](https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html).
This ensures that `libzstd` can always be used to refer to the exported zstd library, since the build errors if neither `ZSTD_BUILD_SHARED` nor `ZSTD_BUILD_STATIC` are set.

I tested all the possible combinations of `ZSTD_BUILD_SHARED`, `ZSTD_BUILD_STATIC`, and `BUILD_SHARED_LIBS` and they always worked as expected:
* If only exactly one of `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` is set, that is used as `libzstd`.
* Otherwise, libzstd is set based on `BUILD_SHARED_LIBS`.

Fixes #3859.