Fixed zstd cmake shared build on windows

Merge pull request #3720 from QBos07/cygwin-msys2-support

Updated Makefiles for full MSYS2 and Cygwin installation and testing …

Merge pull request #3728 from 0o001/dev

fix: ZSTD_BUILD_DECOMPRESSION message

No longer reject dictionaries with literals maxSymbolValue < 255

We already have logic in our Huffman encoder to validate Huffman tables with missing symbols.
We use this for higher compression levels to re-use the previous blocks statistics, or when the dictionaries table has zero-weighted symbols.
This check was leftover as an oversight from before we added validation for Huffman tables.

I validated that the `dictionary_loader` fuzzer has coverage of every line in the `ZSTD_loadCEntropy()` function to validate that it is correctly testing this function.

Merge pull request #3726 from facebook/fullbench_dctx

added ZSTD_decompressDCtx() benchmark option to fullbench

Merge pull request #3730 from facebook/dependabot/github_actions/github/codeql-action-2.21.4

Bump github/codeql-action from 2.20.3 to 2.21.4

Bump github/codeql-action from 2.20.3 to 2.21.4

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.20.3 to 2.21.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/46ed16ded91731b2df79a2893d3aea8e9f03b5c4...a09933a12a80f87b87005513f0abb1494c27a716)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

fix: ZSTD_BUILD_DECOMPRESSION message

added ZSTD_decompressDCtx() benchmark option to fullbench

useful to compare the difference between ZSTD_decompress
and ZSTD_decompressDCtx().

Merge pull request #3725 from felixhandte/msan-unpoison-cwksp-on-free

Unpoison Workspace Memory Before Custom-Free

Unpoison Workspace Memory Before Freeing to Custom Free

MSAN is hooked into the system malloc, but when the user provides a custom
allocator, it may not provide the same cleansing behavior. So if we leave
memory poisoned and return it to the user's allocator, where it is re-used
elsewhere, our poisoning can blow up in some other context.

Easy: Move Helper Functions Up

Improve dual license wording in README

We are licensed under BSD or GPLv2. It is clear in our headers, but not in the README.

Fixes #3717

Updated Makefiles for full MSYS2 and Cygwin installation and testing support.

They are Linux-like environments under Windows and have all the tools needed to support staged installation and testing.

Beware: this only affects the make build system.

Merge pull request #3712 from alexsifivetw/fix_typo

Fixed typo

Fixed typo

Merge pull request #3701 from nikohoffren/grammar-fix

Fix typographical error in README.md

Merge pull request #3704 from void0red/dev

fileio_asyncio: handle malloc fails in AIO_ReadPool_create

fileio_asyncio: handle malloc fails in AIO_ReadPool_create

Update fileio.c: fix build failure with enabled LTO

For some reasons when LTO is enabled, the compiler complains about statbuf variable not being correctly initialized, even though the variable has an assert != NULL just few lines below (FIO_getDictFileStat)

This is the fixed build failure:
x86_64-linux-gnu-gcc -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fdebug-prefix-map=/<<PKGBUILDDIR>>=/usr/src/libzstd-1.5.5+dfsg2-1 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wmissing-prototypes -Wc++-compat -g -Werror -Wa,--noexecstack -Wdate-time -D_FORTIFY_SOURCE=2 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=1 -DZSTD_LEGACY_SUPPORT=5 -DZSTD_MULTITHREAD -DZSTD_GZCOMPRESS -DZSTD_GZDECOMPRESS -DZSTD_LZMACOMPRESS -DZSTD_LZMADECOMPRESS -DZSTD_LZ4COMPRESS -DZSTD_LZ4DECOMPRESS -DZSTD_LEGACY_SUPPORT=5  -c -MT obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.o -MMD -MP -MF obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.d -o obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.o zstdcli_trace.c
In function ‘UTIL_isRegularFileStat’,
    inlined from ‘UTIL_getFileSizeStat’ at util.c:524:10,
    inlined from ‘FIO_createDResources’ at fileio.c:2230:30:
util.c:209:12: error: ‘statbuf.st_mode’ may be used uninitialized [-Werror=maybe-uninitialized]
  209 |     return S_ISREG(statbuf->st_mode) != 0;
      |            ^
fileio.c: In function ‘FIO_createDResources’:
fileio.c:2223:12: note: ‘statbuf’ declared here
 2223 |     stat_t statbuf;
      |            ^
lto1: all warnings being treated as errors

Save one byte on the frame epilogue

Merge pull request #3665 from gjasny/fix-asm-for-xcode

Fix Intel Xcode builds with assembly

Fix typographical error in README.md

Merge pull request #3697 from facebook/dependabot/github_actions/github/codeql-action-2.20.3

Bump github/codeql-action from 2.20.1 to 2.20.3

Bump github/codeql-action from 2.20.1 to 2.20.3

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.20.1 to 2.20.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/f6e388ebf0efc915c6c5b165b019ee61a6746a38...46ed16ded91731b2df79a2893d3aea8e9f03b5c4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3677 from facebook/detectOverflow

Changed the decoding loop to detect more invalid cases of corruption sooner

Merge pull request #3688 from nidhijaju/hide-asm-apple

Hide ASM symbols on Apple platforms

Merge pull request #3689 from facebook/dependabot/github_actions/github/codeql-action-2.20.1

Bump github/codeql-action from 2.3.2 to 2.20.1

Merge pull request #3690 from facebook/dependabot/github_actions/ossf/scorecard-action-2.2.0

Bump ossf/scorecard-action from 2.1.3 to 2.2.0

Merge pull request #3686 from embg/ldm_error

Clean up a false error message in the LDM debug log

Bump ossf/scorecard-action from 2.1.3 to 2.2.0

Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.1.3 to 2.2.0.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)
- [Commits](https://github.com/ossf/scorecard-action/compare/80e868c13c90f172d68d1f4501dee99e2479f7af...08b4669551908b1024bb425080c797723083c031)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Bump github/codeql-action from 2.3.2 to 2.20.1

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.2 to 2.20.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/f3feb00acb00f31a6f60280e6ace9ca31d91c76a...f6e388ebf0efc915c6c5b165b019ee61a6746a38)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

hide asm functions on apple platforms

Merge pull request #3684 from emaste/dev

Update FreeBSD CI images to latest supported releases

suppress false error message in LDM mode

Update FreeBSD CI images to latest supported releases

fixed incorrect test in Win32 pthread wrapper

reported by @Banzai24-yht in #3683

fixed static analyzer false positive regarding @sequence initialization

make a mock initialization to please the tool

adapted long decoder to new decodeSequences

removed older decodeSequences

changed ZSTD_decompressSequences_bodySplitLitBuffer() decoding loop

to behave more like the regular decoding loop.

removed _old variant from splitLit

changed (partially) the decodeSequences flow logic

this allows detecting overflow events without a checksum.

Merge pull request #3676 from facebook/overflow_zeroes

Bitstream produces only zeroes after an overflow event

fixed MEM_STATIC already defined in Linux Kernel mode

fix : unused attribute for FORCE_INLINE functions

fix2 : reloadDStreamFast is used by decompress4x2,
modified the entry point, so that it works fine in this case too.

make the bitstream generate only 0-value bits after an overflow

Merge pull request #3674 from facebook/zeroSeq_noExtra

detect extraneous bytes in the Sequences section

detect extraneous bytes in the Sequences section

when nbSeq == 0.

Reported by @ip7z

Merge pull request #3671 from facebook/dependabot/github_actions/actions/checkout-3.5.3

Bump actions/checkout from 3.5.2 to 3.5.3

Bump actions/checkout from 3.5.2 to 3.5.3

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/8e5e7e5ab8b370d6c329ec480221332ada57f0ab...c85c95e3d7251135ab7dc9ce3241c5835cc595a9)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3669 from facebook/zeroSeq_2B

fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes

fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes

The sequence section starts with a number, which tells how sequences are present in the section.
If this number if 0, the section automatically ends.

The number 0 can be represented using the 1 byte or the 2 bytes formats.
That's because the 2-bytes formats fully overlaps the 1 byte format.

However, when 0 is represented using the 2-bytes format,
the decoder was expecting the sequence section to continue,
and was looking for FSE tables, which is incorrect.

Fixed this behavior, in both the reference decoder and the educational behavior.

In practice, this behavior never happens,
because the encoder will always select the 1-byte format to represent 0,
since this is more efficient.

Completed the fix with a new golden sample for tests,
a clarification of the specification,
and a decoder errata paragraph.

Merge pull request #3664 from facebook/llu

changed LLU suffix into ULL for Visual 2012 and lower

Merge pull request #3659 from facebook/fixHarness

Fixed a bug in the educational decoder

Merge pull request #3668 from facebook/fix3667

fix a minor inefficiency in compress_superblock

fix a minor inefficiency in compress_superblock

and in `decodecorpus`:
the specific case `nbSeq=127` can be represented using the 1-byte format.
Note that both the 1-byte and the 2-bytes formats are valid to represent this case,
so there was no "error", produced data remains valid,
it's just that the 1-byte format is more efficient.

fix #3667

Credit to @ip7z for finding this issue.

Fix Intel Xcode builds with assembly

When forcing the source file language to `C`, Xcode enforces
the file to be compiled as `C` by appending `-x c` to the
compiler command line.

For now try to limit the damage and only enforce the language
if the ASM and C compilers differ.

Reproducer (CMake `3.26.4`, Xcode `14.3`):
```
cmake -S build/cmake -B _b -GXcode -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build _b
```

Fix: #3622

changed LLU suffix into ULL for Visual 2012 and lower

both suffixes are supposed to be valid,
but for some reason, Visual 2012 and lower only support ULL.

Fixed a bug in the educational decoder

Credit to Igor Pavlov

Add options in Makefile to cmake

Signed-off-by: Tao He <sighingnow@gmail.com>

Fix UBSAN issue (zero addition to NULL)

Fix UBSAN issue that came up internally.

Merge pull request #3631 from facebook/example_streaming_mt

minor : update streaming_compression example

Get zstd working with ARM64EC on Windows

Merge pull request #3623 from felixhandte/compile-out-compressors

Allow Build-Time Exclusion of Individual Compression Strategies

Reorder Definitions in zstd_opt.c to Group Under Macro Guards (Slightly)

Reorder Definitions in zstd_lazy.c to Group Under Macro Guards

Also Allow/Document/Test Excluding dfast and Up

Add Documentation to lib/README.md

Also Reorganize Zstd Opt Declarations

Remove blockCompressor NULL Checks

Abort if Unsupported Parameters Used

Re-Order Lazy Declarations; Minimize ifndefs

Add CI Test for Excluding Matchfinders

Add ZSTD_LIB_EXCLUDE_COMPRESSORS_DFAST_AND_UP Build Variable

Define Macros for Possibly-Present Functions; Use Them Rather than Ifdef Guards

Merge Ultra and Ultra2 Exclusion

Ultra2 does not exist for dict compression, and so uses ultra. So ultra must
be present if ultra2 is.

Tweak Ultra/Opt Guards

Avoid Ratio Regression Tests When Compressors are Excluded

Adjust Strategy in CParams to Avoid Using Excluded Block Compressors

Macro-Exclude Block Compressors from Declaration/Definition

NULL Out Block Compressor Table Entries When Excluded

Don't check about excluding `ZSTD_fast`. It's always included so that we know
we can resolve downwards and hit a strategy that's present.

Allow `ZSTD_selectBlockCompressor()` to Return NULL

Return an error rather than segfaulting.

Merge pull request #3634 from facebook/dependabot/github_actions/github/codeql-action-2.3.2

Bump github/codeql-action from 2.3.0 to 2.3.2

Bump github/codeql-action from 2.3.0 to 2.3.2

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.0 to 2.3.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b2c19fb9a2a485599ccf4ed5d65527d94bc57226...f3feb00acb00f31a6f60280e6ace9ca31d91c76a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

minor : update streaming_compression example

display a warning when requesting multi-threading
while linking to a library that doesn't support multi-threading.

Merge pull request #3629 from facebook/dependabot/github_actions/github/codeql-action-2.3.0

Bump github/codeql-action from 2.2.11 to 2.3.0

Bump github/codeql-action from 2.2.11 to 2.3.0

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.11 to 2.3.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/d186a2a36cc67bfa1b860e6170d37fb9634742c7...b2c19fb9a2a485599ccf4ed5d65527d94bc57226)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #3624 from danlark1/patch-4

Disable unused variable warning in msan configurations

Disable unused variable warning in msan configurations

Merge pull request #3614 from mredig/darwin-fat

add makefile entry to build fat binary on macos

Merge pull request #3620 from facebook/errata_128k

[doc] add decoder errata paragraph

added golden file for the new decompressor erratum

Add ZSTD_d_maxBlockSize parameter

Reduces memory when blocks are guaranteed to be smaller than allowed by
the format. This is useful for streaming compression in conjunction with
ZSTD_c_maxBlockSize.

This PR saves 2 * (formatMaxBlockSize - paramMaxBlockSize) when streaming.
Once it is rebased on top of PR #3616 it will save
3 * (formatMaxBlockSize - paramMaxBlockSize).

Merge pull request #3621 from facebook/remove_CIs

removed travis & appveyor scripts

Reduce streaming decompression memory by 128KB

The split literals buffer patch increased streaming decompression memory
by 64KB (shrunk lit buffer from 128KB to 64KB, and added 128KB). This
patch removes the added 128KB buffer, because it isn't necessary.

The buffer was there because the literals compression code didn't know
the true `blockSizeMax` of the frame, and always put split literals so
they ended 128KB - 32 from the beginning of the block. Instead, we can
pass down the true `blockSizeMax` and ensure that the split literals
end up at `blockSizeMax - 32` from the beginning of the block. We
already reserve a full `blockSizeMax` bytes in streaming mode, so we
won't be overwriting the extDict window.

removed travis & appveyor scripts

we don't employ these CI systems anymore

added decoder errata paragraph

for compressed blocks of size exactly 128 KB
which used to be disallowed by the spec
but have become allowed in more recent version of the spec.

While this limitation is fixed in decoders v1.5.4+,
implementers should refrain from generating such block with their custom encoder
as they could be misclassified as corrupted by older decoder versions.

Merge pull request #3619 from facebook/dependabot/github_actions/actions/checkout-3.5.2

Bump actions/checkout from 3.5.0 to 3.5.2

Bump actions/checkout from 3.5.0 to 3.5.2

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/8f4b7f84864484a7bf31766abe9204da3cbe65b3...8e5e7e5ab8b370d6c329ec480221332ada57f0ab)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

add makefile entry to build fat binary on macos