Rewrite inflate memory allocation.
Inflate used to allocate state during init, but window would be allocated
when/if needed and could be resized and that required a new free/alloc round.
- Now, we allocate state and a 32K window during init, allowing the latency cost
of allocs to be done during init instead of at one or more times later.
- Total memory allocation is about the same when requesting a 32K window, but
if now window or a smaller window was requested, then it is an increase.
- While doing alloc(), we now store pointer to corresponding free(), avoiding crashes
with applications that incorrectly set alloc/free pointers after running init function.
- After init has succeeded, inflate will no longer possibly fail due to a failing malloc.
Rewrite deflate memory allocation.
Deflate used to call allocate 5 times during init.
- 5 calls to external alloc function now becomes 1
- Handling alignment of allocated buffers is simplified
- Efforts to align the allocated buffer now needs to happen only once.
- Individual buffers are ordered so that they have natural sequential alignment.
- Due to reduced losses to alignment, we allocate less memory in total.
- While doing alloc(), we now store pointer to corresponding free(), avoiding crashes
with applications that incorrectly set alloc/free pointers after running init function.
- Removed need for extra padding after window, chunked reads can now go beyond the window
buffer without causing a segfault.
Fix illegal instruction usage in Xeon Phi x200 processors
The Xeon Phi x200 family of processors (Knights Landing) supports
AVX512 (F, CD, ER, PF) but does not support AVX512 (VL, DQ, BW).
Because of processors like this, the Intel Software Developer's Manual
suggests the bits AVX512 (DQ,BW,VL) are also tested in EBX together with
AVX512F before deciding to run AVX512 (DQ,BW,VL) instructions.
This also adds a new x86 feature called avx512_common that indicates
that AVX512 (F,DQ,BW,VL) are all available and start using this for both
adler32_avx512 and crc32_vpclmulqdq implementations because they are
both built with -mavx512dq -mavx512bw -mavx512vl.
This has been reported downstream as
https://bugzilla.redhat.com/show_bug.cgi?id=2280347 .
Update s390x CI setup.
- New dockerfile
- Using native actions-runner instead of relying on qemu.
- To support s390x, we include patches to actions-runner.
- Using Almalinux 9 instead of Ubuntu, with functional .Net.
- Update CI workflow.
- Update readme guide.
Currently the DFLTCC sanitizer instrumentation is limited to
MSAN-unpoisoning the parameter block. Add ASAN and MSAN checks;
also MSAN-unpoison the window.
Introduce the generic instrument_read(), instrument_write() and
instrument_read_write() macros, that are modeled after the repsective
functions in the Linux kernel.
Deniz Bahadir [Wed, 24 Apr 2024 14:37:34 +0000 (16:37 +0200)]
.gitattributes: Enforce LF line-endings on all non-binary files
Although Git is able to automatically modify line-endings during checkin
and checkout, this brings a lot of trouble, especially when trying to
use a repository from different platforms (as Windows and Linux). This
is due to the fact that Git consults different local gitconfig settings
and uses problematic defaults if not set.
Therefore, Git should enfoce one type of line-ending (LF) and not
consult the local config, which is what the change from this commit
does.
Signed-off-by: Deniz Bahadir <deniz@code.bahadir.email>
Matt McCormick [Tue, 9 Apr 2024 14:44:07 +0000 (10:44 -0400)]
Bump max CMake policy version to 3.29.0
Addresses:
3.29.0/share/cmake/Modules/CMakeDependentOption.cmake:89 (message):
Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
CMakeLists.txt:107 (cmake_dependent_option)
This warning is for project developers. Use -Wno-dev to suppress it.
/home/autobuild/autobuild/instance-2/output-1/build/zlib-ng-2.1.6/arch/riscv/riscv_features.c:4:10: fatal error: sys/auxv.h: No such file or directory
4 | #include <sys/auxv.h>
| ^~~~~~~~~~~~
Deniz Bahadir [Fri, 5 Apr 2024 20:37:11 +0000 (22:37 +0200)]
CMake: Replace ';' by '$<SEMICOLON>' in generator-expression
Note: CMake generator-expressions should not contain semicolons,
especially if they might end up in a CMake list, because a semicolon
would be interpreted as list-item separator and therefore render the
generator-expression invalid. The generator-expression `$<SEMICOLON>`
should be used instead.
Signed-off-by: Deniz Bahadir <deniz@code.bahadir.email>
Mika Lindqvist [Sun, 25 Feb 2024 14:42:43 +0000 (16:42 +0200)]
[ARM] Override Clang x4 NEON intrinsics for Android
* Clang for Android requires 256-bit alignment for x4 loads and stores, which can't be guaranteed and is unnecessary
Move update_hash(), insert_string() and quick_insert_string() out of functable
and remove SSE4.2 and ACLE optimizations. The functable overhead is higher
than the benefit from using optimized functions.
Pavel P [Sun, 11 Feb 2024 15:33:03 +0000 (17:33 +0200)]
Increase alignment from 8 to 16 to avoid warnings with ms compiler
Fixing align attribute, makes ms compiler warn: 'internal_state': Alignment specifier is less than actual alignment (16), and will be ignored.
Increasing alignemnt fixes the warning
Pavel P [Sun, 11 Feb 2024 15:02:37 +0000 (17:02 +0200)]
Fix `deflate_state` alignment with MS or clang-cl compilers
When building with clang-cl, compiler produces the following warning:
zlib-ng/deflate.h(287,3): warning : attribute 'align' is ignored, place it after "struct" to apply attribute to type declaration [-Wignored-attributes]
zlib-ng/zbuild.h(196,34): note: expanded from macro 'ALIGNED_'
Repositioning align attribute after "struct" fixes the warning and aligns `deflate_state` correctly.
Don Olmstead [Thu, 1 Feb 2024 20:49:11 +0000 (12:49 -0800)]
Relocate CMake target export definitions
When exporting a CMake target the `install(TARGETS ${ZLIB_INSTALL_LIBRARIES})` is repeated with an `EXPORT` parameter. On Windows this would install the `.dll` files into the `lib` directory.
Set the `EXPORT_NAME` earlier and unconditionally on the `TARGET`. Then move the `EXPORT` to a single `install(TARGETS)`.
Use zng_alloc_aligned in unit tests to prevent having to use C++17.
alloc_aligned when using in C++ requires C++17 standard. zutil_p.h
include removed from test_crc32 since it was causing the same issue and was
not really needed.
Mark Adler [Tue, 30 Jan 2024 00:38:32 +0000 (16:38 -0800)]
Add LIT_MEM define to use more memory for a small deflate speedup.
A bug fix in zlib 1.2.12 resulted in a slight slowdown (1-2%) of
deflate. This commit provides the option to #define LIT_MEM, which
uses more memory to reverse most of that slowdown. The memory for
the pending buffer and symbol buffers is increased by 25%, which
increases the total memory usage with the default parameters by
about 6%.
projects / thirdparty / zlib-ng.git / log
