fix msbuild action version number

Bump microsoft/setup-msbuild from 1.3.2 to 2.0.0

Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.2 to 2.0.0.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/031090342aeefe171e49f3820f3b52110c66e402...6fb02220983dee41ce7ae257b6f4d8f9bf5ed4ce)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3893 from felixhandte/gh-actions-cet-test-update-dep

Update Dependency in Intel CET Test; Re-Enable Test

Update Dependency in Intel CET Test; Re-Enable Test

Motivated by #3884.

Merge pull request #3890 from facebook/lorem

add a lorem ipsum generator

Merge pull request #3886 from facebook/sparc64

add sparc64 compilation test

blindfix meson recipe

note: absence of GLOB capability within meson makes its maintenance more painful.

fix cmake recipe

fix Visual Studio solutions

note: we probably don't want to maintain VS2008 solution anymore.
Its successor VS2010 is > 10 years old,
which is more or less the limit after which we can stop supporting old compilers.

fix paramgrill Makefile recipe

add a lorem ipsum generator

this generator replaces the statistical generator
for the general case when no statistic is requested.

Generated data features a compression level speed / ratio curve
which is more in line with expectation.

Merge pull request #3888 from facebook/dependabot/github_actions/microsoft/setup-msbuild-1.3.2

Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2

Merge pull request #3887 from facebook/dependabot/github_actions/actions/upload-artifact-4.3.0

Bump actions/upload-artifact from 4.2.0 to 4.3.0

Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2

Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/1ff57057b5cfdc39105cd07a01d78e9b0ea0c14c...031090342aeefe171e49f3820f3b52110c66e402)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Bump actions/upload-artifact from 4.2.0 to 4.3.0

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/694cdabd8bdb0f10b2cea11669e1bf5453eed0a6...26f96dfa697d77e81fd5907df203aa23a56210a8)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

add sparc64 compilation test

Merge pull request #3880 from facebook/dependabot/github_actions/actions/upload-artifact-4.2.0

Bump actions/upload-artifact from 4.1.0 to 4.2.0

Merge pull request #3883 from facebook/cmake_test

improve cmake test

fix cmakebuild test

write it in a way which is more compatible with older versions of cmake (<3.13)

Also:
fix pzstd compilation (notably on macos)

Merge pull request #3884 from facebook/disable_intelcet

disable Intel CET Compatibility tests

disable Intel CET Compatibility tests

The binary blob that must be downloaded from intel.com is no longer available

Bump actions/upload-artifact from 4.1.0 to 4.2.0

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/1eb3cb2b3e0f29609092a73eb033bb759a334595...694cdabd8bdb0f10b2cea11669e1bf5453eed0a6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3877 from facebook/oldgrep

minor: playTests.sh more compatible with older versions of grep

made playTests.sh more compatible with older versions of grep

replaced `\+` by `*`.
`\+` means `[1-N]`,
while `*` means `[0-N]`,
so it's not strictly equivalent
but `\+` happens to be badly supported on some flavors of grep,
and for the purpose of these tests, `*` is good enough.

Merge pull request #3865 from facebook/grep_e

playTests.sh does no longer needs grep -E

playTests.sh does no longer needs grep -E

it makes the test script more portable across posix systems
because `grep -E` is not guaranteed
while `grep` is fairly common.

Merge pull request #3863 from facebook/dependabot/github_actions/github/codeql-action-3.23.0

Bump github/codeql-action from 2.21.4 to 3.23.0

Merge pull request #3864 from facebook/dependabot/github_actions/actions/upload-artifact-4.1.0

Bump actions/upload-artifact from 4.0.0 to 4.1.0

Bump actions/upload-artifact from 4.0.0 to 4.1.0

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/c7d193f32edcb7bfad88892161225aeda64e9392...1eb3cb2b3e0f29609092a73eb033bb759a334595)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Bump github/codeql-action from 2.21.4 to 3.23.0

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.21.4 to 3.23.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/a09933a12a80f87b87005513f0abb1494c27a716...e5f05b81d5b6ff8cfa111c80c22c5fd02a384118)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3760 from jcelerier/patch-1

[x-compile] Fix cross-compiling for AArch64 with lld

Merge pull request #3817 from elasota/oversized-probs-clarification

Clarify that probability tables must not contain non-zero probabilities for invalid values

Merge pull request #3850 from KapJI/better-errors

cli: better errors on argument parsing

Merge pull request #3858 from eli-schwartz/msvc-dev-cmd

CI: meson: use builtin handling for MSVC

CI: meson: use builtin handling for MSVC

This avoids downloading -- and periodically bumping the checksum for --
a third-party action that isn't strictly required, and thus helps keep
down dependencies and reduce update churn.

Merge pull request #3855 from emaste/freebsd-14-ci

Cirrus-CI: Add FreeBSD 14

Merge pull request #3847 from michoecho/fix_nullptr_deref_in_createCDict

Fix a nullptr dereference in ZSTD_createCDict_advanced2()

Cirrus-CI: Add FreeBSD 14

Merge pull request #3846 from emaste/c89

zlibWrapper: convert to C89 / ANSI C

Make offload API compatible with static CCtx (#3854)

* Add ZSTD_CCtxParams_registerSequenceProducer() to public API

* add unit test

* add docs to zstd.h

* nits

* Add ZSTDLIB_STATIC_API prefix

* Add asserts

Merge pull request #3849 from facebook/dependabot/github_actions/actions/upload-artifact-4.0.0

Bump actions/upload-artifact from 3.1.3 to 4.0.0

Update windows-artifacts.yml

and fixed version number in comment

cli: better errors on arguent parsing

Bump actions/upload-artifact from 3.1.3 to 4.0.0

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3.1.3 to 4.0.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/a8a3f3ad30e3422c9c7b888a15615d19a852ae32...c7d193f32edcb7bfad88892161225aeda64e9392)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Fix a nullptr dereference in ZSTD_createCDict_advanced2()

If the relevant allocation returns NULL, ZSTD_createCDict_advanced_internal()
will return NULL. But ZSTD_createCDict_advanced2() doesn't check for
this and attempts to use the returned pointer anyway, which leads to
a segfault.

zlibWrapper: convert to C89 / ANSI C

Clang 16 (which is the system compiler in FreeBSD 14.0) no longer allows
K&R function definitions.  Formatting of the changes matches current
zlib.

Merge pull request #3845 from emaste/dev

Update FreeBSD CI: drop 12.4 (nearly EOL)

Update FreeBSD CI: drop 12.4 as it is nearly EOL

12.4 is EOL as of the end of December 2023, and pkg installation will
start failing some time after that so remove those jobs now.

Merge pull request #3839 from embg/offload_refactor

Move offload API params into ZSTD_CCtx_params

Move offload API params into ZSTD_CCtx_params

Refactor ZSTD_sequenceProducer_F typedef to ZSTD_sequenceProducer_F*

Clarify that there must be at least 2 weights, i.e. encoding all weights as 0 is invalid

Clarify that the presence of a value with weight 1 is required

Modernize macros to use `do { } while (0)`

This PR introduces no functional changes. It attempts to change all
macros currently using `{ }` or some variant of that to to
`do { } while (0)`, and introduces trailing `;` where necessary.
There were no bugs found during this migration.

The bug in Visual Studios warning on this has been fixed since VS2015.
Additionally, we have several instances of `do { } while (0)` which have
been present for several releases, so we don't have to worry about
breaking peoples builds.

Fixes Issue #3830.

Merge pull request #3820 from facebook/xxh082

update xxhash library to v0.8.2

[huf] Fix null pointer addition

`HUF_DecompressFastArgs_init()` was adding 0 to NULL. Fix it by exiting
early for empty outputs. This is no change in behavior, because the
function was already exiting 0 in this case, just slightly later.

[huf] Improve fast C & ASM performance on small data

* Rename `ilimit` to `ilowest` and set it equal to `src` instead of
  `src + 6 + 8`. This is safe because the fast decoding loops guarantee
  to never read below `ilowest` already. This allows the fast decoder to
  run for at least two more iterations, because it consumes at most 7
  bytes per iteration.
* Continue the fast loop all the way until the number of safe iterations
 is 0. Initially, I thought that when it got towards the end, the
 computation of how many iterations of safe might become expensive. But
 it ends up being slower to have to decode each of the 4 streams
 individually, which makes sense.

This drastically speeds up the Huffman decoder on the `github` dataset
for the issue raised in #3762, measured with `zstd -b1e1r github/`.

| Decoder  | Speed before | Speed after |
|----------|--------------|-------------|
| Fallback | 477 MB/s     | 477 MB/s    |
| Fast C   | 384 MB/s     | 492 MB/s    |
| Assembly | 385 MB/s     | 501 MB/s    |

We can also look at the speed delta for different block sizes of silesia
using `zstd -b1e1r silesia.tar -B#`.

| Decoder  | -B1K ∆ | -B2K ∆ | -B4K ∆ | -B8K ∆ | -B16K ∆ | -B32K ∆ | -B64K ∆ | -B128K ∆ |
|----------|--------|--------|--------|--------|---------|---------|---------|----------|
| Fast C   | +11.2% | +8.2%  | +6.1%  | +4.4%  | +2.7%   | +1.5%   | +0.6%   | +0.2%    |
| Assembly | +12.5% | +9.0%  | +6.2%  | +3.6%  | +1.5%   | +0.7%   | +0.2%   | +0.03%   |

[huf] Improve fast huffman decoding speed in linux kernel

gcc in the linux kernel was not unrolling the inner loops of the Huffman
decoder, which was destroying decoding performance. The compiler was
generating crazy code with all sorts of branches. I suspect because of
Spectre mitigations, but I'm not certain. Once the loops were manually
unrolled, performance was restored.

Additionally, when gcc couldn't prove that the variable left shift in
the 4X2 decode loop wasn't greater than 63, it inserted checks to verify
it. To fix this, mask `entry.nbBits & 0x3F`, which allows gcc to eliete
this check. This is a no op, because `entry.nbBits` is guaranteed to be
less than 64.

Lastly, introduce the `HUF_DISABLE_FAST_DECODE` macro to disable the
fast C loops for Issue #3762. So if even after this change, there is a
performance regression, users can opt-out at compile time.

[debug] Don't define g_debuglevel in the kernel

We only use this constant when `DEBUGLEVEL>=2`, but we get
-Werror=pedantic errors for empty translation units, so still define it
except in kernel environments.

Backport from the kernel:

https://lore.kernel.org/lkml/20230616144400.172683-1-ben.dooks@codethink.co.uk/

[linux] Remove usage of deprecated function

ZSTD_resetDStream() is deprecated and replaced by ZSTD_DCtx_reset().
This removes deprecation warnings from the kernel build.

This change is a no-op, see the docs suggesting this replacement.

https://github.com/facebook/zstd/blob/fcbf2fde9ac7ce1562c7b3a394350e764bcb580f/lib/zstd.h#L2655-L2663

[linux] Backport intptr_t removal

Linux started providing intptr_t in <linux/types.h> so we no longer need
to define it here.

https://lkml.kernel.org/r/ed66b9e4-1fb7-45be-9bb9-d4bc291c691f@p183

update license text

Merge pull request #3807 from gruenich/increase_minimum_cmake_version_3.5

[cmake] Require CMake version 3.5 or newer

fix the copyright linter

update xxhash to v0.8.2

List of updates : https://github.com/Cyan4973/xxHash/releases/tag/v0.8.2

This is also a preparation task before taking care of #3819

Merge pull request #3813 from elasota/overflow-clarification

Clarify that a stream containing too many Huffman weights is invalid

Merge pull request #3812 from jondo2010/bazel_build_doc

Add Bazel module instructions to README.md

Clarify that a non-zero probability for an invalid symbol is invalid

Clarify that decoding too many Huffman weights is a failure condition

Add Bazel module instructions to README.md

[cmake] Remove code for compatibility with CMake < 3.0

The required version of CMake is now 3.5.

Merge pull request #3806 from elasota/fmt-clarifications

Correct FSE probability bit consumption in specification

[cmake] Require CMake version 3.5 or newer

More recent versions of CMake emit the following warning:
CMake Deprecation Warning at cmake/CMakeLists.txt:10 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

Merge pull request #3800 from facebook/dependabot/github_actions/actions/checkout-4.1.1

Bump actions/checkout from 4.1.0 to 4.1.1

Merge pull request #3795 from Saverio976/doc/cmake-fetch-content

Add doc on how to use it with cmake FetchContent

Add definition of "log2sup" function

Clarify that the log2 of the largest possible symbol is the maximum number of bits consumed

Add target_include_directories because windows and macos need it for me

Bump actions/checkout from 4.1.0 to 4.1.1

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/8ade135a41bc03ea155e62e844d188df1ea18608...b4ffde65f46336ab88eb53be808477a3936bae11)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3789 from facebook/fix_flexArray_fse

solving flexArray issue #3785 in fse

revert to manually defining DTable

thus avoiding the analyzer and ubsan to associate DTable to a size of 1.

baby-step towards solving flexArray issue #3785

the flexArray in structure FSE_DecompressWksp
is just a way to derive a pointer easily,
without risk/complexity of calculating it manually.

Not sure if this change is good enough to avoid ubsan warnings though.

Add doc on how to use it with cmake FetchContent

Merge pull request #3786 from facebook/fix_flexarray_cctx

Remove FlexArray pattern from ZSTDMT

extended the fix to ZSTDMT's Buffer Pool

removed unused macro constant

fixes suggested by @ebiggers

removed FlexArray pattern from CCtxPool

within ZSTDMT_.
This pattern is flagged by less forgiving variants of ubsan
notably used during compilation of the Linux Kernel.

There are 2 other places in the code where this pattern is used.
This fixes just one of them.

Merge pull request #3752 from paulmenzel/fix-pzstd-makefile

Fix pzstd Makefile to allow setting `DESTDIR` and `BINDIR` separately

Merge pull request #3763 from dloidolt/fix_lib/README.md

Fix a very small formatting typo in the lib/README.md file

Merge pull request #3772 from DimitriPapadopoulos/WIN32

Do not test WIN32, instead test _WIN32

Merge pull request #3771 from DimitriPapadopoulos/codespell

Fix new typos found by codespell

Merge pull request #3777 from facebook/fix_x32

fix x32 tests on Github CI

Merge pull request #3777 from facebook/fix_x32

fix x32 tests on Github CI

Stop suppressing pointer-overflow UBSAN errors

* Remove all pointer-overflow suppressions from our UBSAN builds/tests.
* Add `ZSTD_ALLOW_POINTER_OVERFLOW_ATTR` macro to suppress
  pointer-overflow at a per-function level. This is a superior approach
  because it also applies to users who build zstd with UBSAN.
* Add `ZSTD_wrappedPtr{Diff,Add,Sub}()` that use these suppressions.
  The end goal is to only tag these functions with
  `ZSTD_ALLOW_POINTER_OVERFLOW`. But we can start by annoting functions
  that rely on pointer overflow, and gradually transition to using
  these.
* Add `ZSTD_maybeNullPtrAdd()` to simplify pointer addition when the
  pointer may be `NULL`.
* Fix all the fuzzer issues that came up. I'm sure there will be a lot
  more, but these are the ones that came up within a few minutes of
  running the fuzzers, and while running GitHub CI.

Revert "Work around nullptr-with-nonzero-offset warning"

This reverts commit c27fa399042f466080e79bb4fd8a4871bc0bcf28.

fix x32 tests on Github CI

ubuntu-22.04 seems to have problems with x32 recently
switching to ubuntu-20.04 which seems to work fine so far

https://github.com/actions/runner-images/issues/8397

Merge pull request #3774 from facebook/dependabot/github_actions/actions/checkout-4.1.0

Bump actions/checkout from 4.0.0 to 4.1.0

Bump actions/checkout from 4.0.0 to 4.1.0

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/3df4ab11eba7bda6032a0b82a6bb43b11571feac...8ade135a41bc03ea155e62e844d188df1ea18608)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Do not test WIN32, instead test _WIN32

To the best of my knowledge:
* `_WIN32` and `_WIN64` are defined by the compiler,
* `WIN32` and `WIN64` are defined by the user, to indicate whatever
  the user chooses them to indicate. They mean 32-bit and 64-bit Windows
  compilation by convention only.

See:
https://accu.org/journals/overload/24/132/wilson_2223/

Windows compilers in general, and MSVC in particular, have been defining
`_WIN32` and `_WIN64` for a long time, provably at least since Visual Studio
2015, and in practice as early as in the days of 16-bit Windows.

See:
https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-140
https://learn.microsoft.com/en-us/windows/win32/winprog64/the-tools

Tests used to be inconsistent, sometimes testing `_WIN32`, sometimes
`_WIN32` and `WIN32`. This brings consistency to Windows detection.