Nick Terrell [Thu, 15 Dec 2022 00:07:22 +0000 (16:07 -0800)]
[legacy] Remove FORCE_MEMORY_ACCESS and only use memcpy
Delete unaligned memory access code from the legacy codebase by removing all the
non-memcpy functions. We don't care about speed at all for this codebase, only
simplicity.
Fix an instance of `NULL + 0` in `ZSTD_decompressStream()`. Also, improve our
`stream_decompress` fuzzer to pass `NULL` in/out buffers to
`ZSTD_decompressStream()`, and fix 2 issues that were immediately surfaced.
Nick Terrell [Wed, 14 Dec 2022 23:17:05 +0000 (15:17 -0800)]
[fileio] Separate parameter adaption from display update rate
Split the logic for parameter adaption from the logic to update the display rate.
This decouples the two updates, so changes to display updates don't affect
parameter adaption.
Also add a test case that checks that parameter adaption actually happens.
This fixes Issue #3353, where --adapt is broken when --no-progress is passed.
Instead of using packed attribute hack, just use aligned attribute. It
improves code generation on armv6 and armv7, and slightly improves code
generation on aarch64. GCC generates identical code to regular aligned
access on ARMv6 for all versions between 4.5 and trunk, except GCC 5
which is buggy and generates the same (bad) code as packed access:
https://gcc.godbolt.org/z/hq37rz7sb
Nick Terrell [Fri, 7 Jan 2022 23:07:28 +0000 (15:07 -0800)]
Refactor progress bar & summary line logic
* Centralize the logic about whether to print the progress bar or not in
the `*_PROGRESS()` macros.
* Centralize the logc about whether to print the summary line or not in
`FIO_shouldDisplayFileSummary()` and
`FIO_shouldDisplayMultipleFileSummary()`.
* Make `--progress` work for non-zstd (de)compressors.
* Clean up several edge cases in compression and decompression progress
printing along the way. E.g. wrong log level, or missing summary line.
One thing I don't like about stdout mode, which sets the display level
to 1, is that warnings aren't displayed. After this PR, we could change
stdout mode from lowering the display level, to defaulting to implied
`--no-progress`. But, I think that deserves a separate PR.
Nick Terrell [Fri, 14 Jan 2022 20:37:32 +0000 (12:37 -0800)]
Allow tests to fake stdin/stdout/stderr is a console
We've been unable to effectively test cases where stdin/stdout/stderr
are consoles, because in our test cases they generally aren't. Allow the
command line flags `--fake-std{in,out,err}-is-console` to tell the CLI
to pretend that std{in,out,err} is a console.
Nick Terrell [Tue, 13 Dec 2022 23:03:23 +0000 (15:03 -0800)]
[contrib][linux-kernel] Fix stack detection for newer gcc
Newer gcc versions were getting smart and omitting the `memset()`.
Get around this issue by outlining the `memset()` into a different
function. This test is still hacky, but it works...
dependabot[bot] [Thu, 8 Dec 2022 18:39:55 +0000 (18:39 +0000)]
Bump skx/github-action-publish-binaries from release-1.3 to 2.0
Bumps [skx/github-action-publish-binaries](https://github.com/skx/github-action-publish-binaries) from release-1.3 to 2.0. This release includes the previously tagged commit.
- [Release notes](https://github.com/skx/github-action-publish-binaries/releases)
- [Commits](https://github.com/skx/github-action-publish-binaries/compare/release-1.3...release-2.0)
Eli Schwartz [Sun, 6 Nov 2022 01:52:51 +0000 (21:52 -0400)]
meson: get version up front
Run the scraper command to establish the project version immediately,
rather than wait for the build to be configured. This simplifies the
code and ensures that project introspection works correctly.
Nick Terrell [Tue, 18 Oct 2022 01:40:06 +0000 (18:40 -0700)]
[contrib][linux] Disable ASM in the kernel
Disable ASM in the kernel for now. It requires a few changes & setup to
get working. Instead of doing it in a zstd version update, I'd prefer to
package that change as a single patch, and propose it separately from
the version update. This makes the version update easier, and reduces
some risk.
Nick Terrell [Sat, 22 Oct 2022 00:14:02 +0000 (17:14 -0700)]
[lazy] Use switch instead of indirect function calls. (#3295)
Use a switch statement to select the search function instead of an
indirect function call. This results in a sizable performance win.
This PR is a modification of the approach taken in PR #2828.
When I measured performance for that commit, it was neutral.
However, I now see a performance regression on gcc, but still
neutral on clang. I'm measuring on the same platform, but with
newer compilers. The new approach beats both the current dev
branch and the baseline before PR #2828 was merged.
This PR is necessary for Issue #3275, to update zstd in the kernel.
Without this PR there is a large regression in greedy - btlazy2
compression speed. With this PR it is about neutral.
Add a `--spdx` option to the freestanding script to prefix
files with a line like (for `.c` files):
// SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause
or (for `.h` and `.S` files):
/* SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause */
Given the style of the line to be used depends on the extension,
a simple `sed` insert command would not work.
It also skips the file if an existing SPDX line is there,
as well as raising an error if an unexpected SPDX line appears
anywhere else in the file, as well as for unexpected
file extensions.
I double-checked that all currently generated files appear
to be license as expected with:
grep -LRF 'This source code is licensed under both the BSD-style license (found in the' linux/lib/zstd
grep -LRF 'LICENSE file in the root directory of this source tree) and the GPLv2 (found' linux/lib/zstd
but somebody knowledgable on the licensing of the project should
double-check this is the intended case.
Fixes: https://github.com/facebook/zstd/issues/3293 Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
W. Felix Handte [Sat, 15 Oct 2022 02:53:03 +0000 (22:53 -0400)]
Make ZSTD_getDictID_fromDDict() Read DictID from DDict
Currently this function actually reads the dict ID from the dictionary's
header, via `ZSTD_getDictID_fromDict()`. But during decompression the decomp-
ressor actually compares the dict ID in the frame header with the dict ID in
the DDict. Now of course the dict ID in the dictionary contents and the dict
ID in the DDict struct *should* be the same. But in cases of memory corrupt-
ion, where they can drift out of sync, it's misleading for this function to
read it again from the dict buffer rather then return the dict ID that will
actually be used.
Also doing it this way avoids rechecking the magic and so on and so it is a
tiny bit more efficient.
Ilya Kurdyukov [Wed, 5 Oct 2022 10:32:16 +0000 (17:32 +0700)]
fixed zstd-pgo target for GCC
Since your Makefile uses obj/$(HASH_DIR) for object files, this code does not work correctly for GCC. Because profiles are saved in one directory, and are expected in another when reading.
`$(RM) zstd *.o` - this line doesn't delete object files.
Clang stores profiles in the current directory, so the problem doesn't appear when compiling with Clang.
Fangrui Song [Thu, 22 Sep 2022 19:30:21 +0000 (12:30 -0700)]
Move ZSTD_DEPRECATED before ZSTDLIB_API/ZSTDLIB_STATIC_API
Clang doesn't allow [[deprecated(...)]] attribute after __attribute__.
Move [[deprecated(...)]] before __attribute__ to fix C++14/C++17 uses
with Clang.
Andrea Pappacoda [Tue, 20 Sep 2022 20:13:23 +0000 (22:13 +0200)]
ci: test pkg-config file
As mentioned in
https://github.com/facebook/zstd/pull/3252#issuecomment-1251733791 ,
this patch adds a CI job that builds and installs libzstd on the job
runner, and then compiles a sample binary linking against the installed
library; the needed build flags are passed by invoking pkg-config.
Jun He [Fri, 24 Jun 2022 09:07:06 +0000 (17:07 +0800)]
compress:check more bytes to reduce ZSTD_count call
Comparing 4B instead of comparing 1B in ZSTD_noDict
mode, thus it can avoid cases like match in match[ml]
but mismatch in match[ml-3]..match[ml-1]. So the call
count of ZSTD_count can be reduced.
Signed-off-by: Jun He <jun.he@arm.com>
Change-Id: I3449ea423d5c8e8344f75341f19a2d1643c703f6
projects / thirdparty / zstd.git / log
