]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
20 months agoMove VERM16 enums to the end of the list 191/head
Konstantinos Margaritis [Thu, 16 Nov 2023 19:50:30 +0000 (03:50 +0800)] 
Move VERM16 enums to the end of the list

This was causing a hard-to-track segfault with Fat Runtime on SVE2 hw,
because of the macro-based hard-coded way to calculate offsets for each
implementation. This needs a rewrite.

21 months agoMerge pull request #186 from VectorCamp/bugfix/fix-compilation-arm-ubuntu-20.04
Konstantinos Margaritis [Wed, 25 Oct 2023 10:53:44 +0000 (13:53 +0300)] 
Merge pull request #186 from VectorCamp/bugfix/fix-compilation-arm-ubuntu-20.04

Ubuntu 20.04 gcc does not define HWCAP2_SVE2 #180

21 months agoMerge pull request #185 from VectorCamp/bugfix/fix-inconsistent-version-header
Konstantinos Margaritis [Wed, 11 Oct 2023 16:52:22 +0000 (19:52 +0300)] 
Merge pull request #185 from VectorCamp/bugfix/fix-inconsistent-version-header

Fix version getting out of sync #175

21 months agoUbuntu 20.04 gcc does not define HWCAP2_SVE2 #180 186/head
Konstantinos Margaritis [Tue, 10 Oct 2023 10:30:12 +0000 (18:30 +0800)] 
Ubuntu 20.04 gcc does not define HWCAP2_SVE2 #180

21 months agoMerge pull request #181 from VectorCamp/bugfix/fix-clang15-compilation-errors
Konstantinos Margaritis [Tue, 10 Oct 2023 10:14:10 +0000 (13:14 +0300)] 
Merge pull request #181 from VectorCamp/bugfix/fix-clang15-compilation-errors

Fix clang 15,16 compilation errors on all platforms, refactor CMake build system

21 months agofix default arch definition for non fat builds on arm 181/head
Konstantinos Margaritis [Mon, 9 Oct 2023 16:55:02 +0000 (00:55 +0800)] 
fix default arch definition for non fat builds on arm

21 months agoset default value
Konstantinos Margaritis [Mon, 9 Oct 2023 12:38:19 +0000 (20:38 +0800)] 
set default value

21 months agoSVE2 is armv9-a but gcc 11 does not recognize that
Konstantinos Margaritis [Mon, 9 Oct 2023 12:02:37 +0000 (20:02 +0800)] 
SVE2 is armv9-a but gcc 11 does not recognize that

21 months agoremove vermicelli_simd.cpp to fix redefinition build failure on SVE2 builds
Konstantinos Margaritis [Mon, 9 Oct 2023 12:01:26 +0000 (20:01 +0800)] 
remove vermicelli_simd.cpp to fix redefinition build failure on SVE2 builds

21 months agoremove unneeded option
Konstantinos Margaritis [Mon, 9 Oct 2023 10:26:08 +0000 (10:26 +0000)] 
remove unneeded option

21 months ago is not known at that stage
Konstantinos Margaritis [Mon, 9 Oct 2023 10:16:40 +0000 (10:16 +0000)] 
 is not known at that stage

21 months agofix cmake refactor for arm builds
Konstantinos Margaritis [Mon, 9 Oct 2023 10:03:53 +0000 (10:03 +0000)] 
fix cmake refactor for arm builds

21 months agofix default arch/tune flags for ppc64le
Konstantinos Margaritis [Sun, 8 Oct 2023 21:44:12 +0000 (00:44 +0300)] 
fix default arch/tune flags for ppc64le

21 months agofix x86 fat binary build
Konstantinos Margaritis [Sun, 8 Oct 2023 21:42:39 +0000 (00:42 +0300)] 
fix x86 fat binary build

21 months agofix sqlite3 version detection
Konstantinos Margaritis [Sun, 8 Oct 2023 21:23:29 +0000 (00:23 +0300)] 
fix sqlite3 version detection

21 months agofix cflags detection for x86
Konstantinos Margaritis [Sun, 8 Oct 2023 21:23:08 +0000 (00:23 +0300)] 
fix cflags detection for x86

21 months agofix defaults for -march for x86
Konstantinos Margaritis [Sun, 8 Oct 2023 21:22:52 +0000 (00:22 +0300)] 
fix defaults for -march for x86

21 months agoincrease cmake_minimum_version, basically the one in Debian 11
Konstantinos Margaritis [Sun, 8 Oct 2023 21:22:31 +0000 (00:22 +0300)] 
increase cmake_minimum_version, basically the one in Debian 11

21 months agoadd missing file
Konstantinos Margaritis [Sun, 8 Oct 2023 20:54:06 +0000 (23:54 +0300)] 
add missing file

21 months agoWIP: Refactor CMake build system to more modular
Konstantinos Margaritis [Sun, 8 Oct 2023 20:26:07 +0000 (23:26 +0300)] 
WIP: Refactor CMake build system to more modular

21 months agoSVE2 needs armv9-a, fix build
Konstantinos Margaritis [Sat, 7 Oct 2023 16:00:42 +0000 (00:00 +0800)] 
SVE2 needs armv9-a, fix build

21 months agoadd src/nfa/vermicelli_simd.cpp to ppc64le
Konstantinos Margaritis [Sat, 7 Oct 2023 15:02:00 +0000 (18:02 +0300)] 
add src/nfa/vermicelli_simd.cpp to ppc64le

21 months agoinclude extra sources for Arm on non-fat builds
Konstantinos Margaritis [Sat, 7 Oct 2023 14:27:26 +0000 (22:27 +0800)] 
include extra sources for Arm on non-fat builds

21 months agoadd missing file
Konstantinos Margaritis [Sat, 7 Oct 2023 09:10:06 +0000 (12:10 +0300)] 
add missing file

21 months agoadd cpuid_flags to ppc64le as well
Konstantinos Margaritis [Sat, 7 Oct 2023 09:06:47 +0000 (12:06 +0300)] 
add cpuid_flags to ppc64le as well

21 months agore-add missing file to x86 builds
Konstantinos Margaritis [Sat, 7 Oct 2023 08:44:57 +0000 (11:44 +0300)] 
re-add missing file to x86 builds

21 months agofix fat & normal build errors on arm
Konstantinos Margaritis [Fri, 6 Oct 2023 22:17:18 +0000 (06:17 +0800)] 
fix fat & normal build errors on arm

21 months agodetect arm_sve.h when using clang on fat runtime builds
Konstantinos Margaritis [Fri, 6 Oct 2023 12:46:24 +0000 (20:46 +0800)] 
detect arm_sve.h when using clang on fat runtime builds

21 months agoremove extra print
Konstantinos Margaritis [Fri, 6 Oct 2023 09:08:36 +0000 (12:08 +0300)] 
remove extra print

21 months ago__builtin_constant_p is true in the wrong case on gcc 13.2. Exclude for now
Konstantinos Margaritis [Fri, 6 Oct 2023 08:44:41 +0000 (11:44 +0300)] 
__builtin_constant_p is true in the wrong case on gcc 13.2. Exclude for now

21 months agoReduce debug unit tests runtime even more
Konstantinos Margaritis [Thu, 5 Oct 2023 16:12:38 +0000 (19:12 +0300)] 
Reduce debug unit tests runtime even more
In single.cpp featuremask with AVX512 features is not relevant to non-x86 platforms,
and just extends the runtime for no reason.

22 months agoDon't run regression UE_2595 on debug, it times out CI
Konstantinos Margaritis [Thu, 5 Oct 2023 07:40:30 +0000 (10:40 +0300)] 
Don't run regression UE_2595 on debug, it times out CI

22 months agouse the right type of cast
Konstantinos Margaritis [Wed, 4 Oct 2023 20:35:10 +0000 (23:35 +0300)] 
use the right type of cast

22 months agoclang 16 as well
Konstantinos Margaritis [Wed, 4 Oct 2023 19:07:34 +0000 (22:07 +0300)] 
clang 16 as well

22 months agomissed one pragma
Konstantinos Margaritis [Wed, 4 Oct 2023 17:54:57 +0000 (20:54 +0300)] 
missed one pragma

22 months agouse the conditional in the right way
Konstantinos Margaritis [Wed, 4 Oct 2023 17:35:58 +0000 (20:35 +0300)] 
use the conditional in the right way

22 months agoadd conditional for __clang__
Konstantinos Margaritis [Wed, 4 Oct 2023 17:28:35 +0000 (20:28 +0300)] 
add conditional for __clang__

22 months agoclang 15 (but not 16) fails on ppc64le with -Wdeprecate-lax-vec-conv-all
Konstantinos Margaritis [Wed, 4 Oct 2023 17:09:45 +0000 (20:09 +0300)] 
clang 15 (but not 16) fails on ppc64le with -Wdeprecate-lax-vec-conv-all

22 months agoReduce unit test runtimes dramatically for debug builds
Konstantinos Margaritis [Wed, 4 Oct 2023 16:20:45 +0000 (19:20 +0300)] 
Reduce unit test runtimes dramatically for debug builds

22 months agofix -Wunused warnings on debug
Konstantinos Margaritis [Wed, 4 Oct 2023 07:16:12 +0000 (07:16 +0000)] 
fix -Wunused warnings on debug

22 months agomore std::move fixes 179/head
Konstantinos Margaritis [Tue, 3 Oct 2023 18:01:51 +0000 (21:01 +0300)] 
more std::move fixes

22 months agomove definition of RAGEL_C_FLAGS earlier to catch tools/hscollider
Konstantinos Margaritis [Tue, 3 Oct 2023 18:01:35 +0000 (21:01 +0300)] 
move definition of RAGEL_C_FLAGS earlier to catch tools/hscollider

22 months agoFix 'unqualified call to std::move' errors in clang 15+
Konstantinos Margaritis [Tue, 3 Oct 2023 17:24:39 +0000 (20:24 +0300)] 
Fix 'unqualified call to std::move' errors in clang 15+

22 months agoFix version getting out of sync #175 178/head 185/head
Konstantinos Margaritis [Tue, 3 Oct 2023 06:57:10 +0000 (09:57 +0300)] 
Fix version getting out of sync #175

22 months agoMerge pull request #174 from VectorCamp/develop vectorscan/5.4.10.1
Konstantinos Margaritis [Fri, 8 Sep 2023 10:42:33 +0000 (13:42 +0300)] 
Merge pull request #174 from VectorCamp/develop

Minor bugfix release 5.4.10.1

22 months agoMerge pull request #173 from VectorCamp/bugfix/disable-fat-macos-arm 174/head
Konstantinos Margaritis [Fri, 8 Sep 2023 09:18:11 +0000 (12:18 +0300)] 
Merge pull request #173 from VectorCamp/bugfix/disable-fat-macos-arm

Bugfix/disable fat macos arm

22 months agoHWCAP is only available on Linux 173/head
Konstantinos Margaritis [Fri, 8 Sep 2023 07:08:44 +0000 (10:08 +0300)] 
HWCAP is only available on Linux

22 months agoforce disable FAT_RUNTIME on MacOS on Arm
Konstantinos Margaritis [Fri, 8 Sep 2023 07:08:18 +0000 (10:08 +0300)] 
force disable FAT_RUNTIME on MacOS on Arm

22 months agoforgot to update changelog for latest entry
Konstantinos Margaritis [Thu, 7 Sep 2023 17:10:20 +0000 (20:10 +0300)] 
forgot to update changelog for latest entry

22 months agoMerge pull request #167 from VectorCamp/develop
Konstantinos Margaritis [Thu, 7 Sep 2023 17:03:49 +0000 (20:03 +0300)] 
Merge pull request #167 from VectorCamp/develop

Prepare for 5.4.10

22 months agominor fix 167/head
Konstantinos Margaritis [Thu, 7 Sep 2023 14:53:25 +0000 (17:53 +0300)] 
minor fix

22 months agobump version, add Vectorscan Changelog
Konstantinos Margaritis [Thu, 7 Sep 2023 14:51:07 +0000 (17:51 +0300)] 
bump version, add Vectorscan Changelog

22 months agoMerge pull request #169 from VectorCamp/feature/backport-hyperscan-2023Q3
Konstantinos Margaritis [Tue, 5 Sep 2023 17:11:30 +0000 (20:11 +0300)] 
Merge pull request #169 from VectorCamp/feature/backport-hyperscan-2023Q3

Feature/backport hyperscan 2023 q3

22 months agochangelog: updates for 5.4.2 release 169/head
Hong, Yang A [Wed, 19 Apr 2023 09:18:45 +0000 (09:18 +0000)] 
changelog: updates for 5.4.2 release

22 months agoremove invalid nfa dump info
Hong, Yang A [Thu, 23 Mar 2023 06:43:46 +0000 (06:43 +0000)] 
remove invalid nfa dump info

22 months agoscratch: remove quick validity check
Hong, Yang A [Wed, 1 Mar 2023 14:42:27 +0000 (14:42 +0000)] 
scratch: remove quick validity check
Roll back fix for github issue #350

About Scratch Usage:
For compile time, scratch space is strongly recommended to be
allocated immediately after database generation.
For runtime, besides using scratch for corresponding database,
Hyperscan also allows user to use larger scratch space allocated
for another database.
When multiple concurrent threads need to use the same databases
and a new scratch space is required, cloning the largest one is
always safe. This is realized based on API hs_scratch_size() and
hs_clone_scratch().
Behaviors beyond above are discouraged and results are undefined.

22 months agochangelog: updates for 5.4.1 release
Chang, Harry [Tue, 21 Feb 2023 22:52:57 +0000 (22:52 +0000)] 
changelog: updates for 5.4.1 release

22 months agosanitiser bugfix
Hong, Yang A [Wed, 29 Dec 2021 22:30:18 +0000 (22:30 +0000)] 
sanitiser bugfix

22 months agobugfix: add vbmi case for test in database.cpp
Hong, Yang A [Mon, 7 Jun 2021 08:24:51 +0000 (16:24 +0800)] 
bugfix: add vbmi case for test in database.cpp

22 months agobugfix: add vbmi platform parameter for tests in single.cpp
Hong, Yang A [Mon, 7 Jun 2021 07:35:57 +0000 (15:35 +0800)] 
bugfix: add vbmi platform parameter for tests in single.cpp

22 months agofix nfa dump error
Hong, Yang A [Thu, 20 Oct 2022 08:47:03 +0000 (08:47 +0000)] 
fix nfa dump error

22 months agoscratch: add quick validity check
Hong, Yang A [Thu, 20 Oct 2022 08:48:46 +0000 (08:48 +0000)] 
scratch: add quick validity check

fix github issue #350

22 months agostream close: free stream to avoid memory leak
Hong, Yang A [Wed, 19 Oct 2022 16:50:02 +0000 (16:50 +0000)] 
stream close: free stream to avoid memory leak

fix github issue #303

22 months agoSilence clang-14 warnings
Hong, Yang A [Mon, 15 Aug 2022 03:00:22 +0000 (03:00 +0000)] 
Silence clang-14 warnings

22 months agoFix cmake CMP0115 warning for CMake 3.20 and above
Hong, Yang A [Tue, 2 Aug 2022 19:25:27 +0000 (19:25 +0000)] 
Fix cmake CMP0115 warning for CMake 3.20 and above

22 months agoupdate year 2022
Hong, Yang A [Mon, 1 Aug 2022 17:13:25 +0000 (17:13 +0000)] 
update year 2022

22 months agoUTF-8 validation: fix one cotec check corner issue
Hong, Yang A [Thu, 28 Jul 2022 21:24:31 +0000 (21:24 +0000)] 
UTF-8 validation: fix one cotec check corner issue

fix github issue #362

22 months agostringop-overflow compatible fix
Hong, Yang A [Tue, 12 Jul 2022 08:42:05 +0000 (08:42 +0000)] 
stringop-overflow compatible fix

22 months agogcc-10(and above): fix compile issue caused by stringop-overflow
Hong, Yang A [Tue, 5 Jul 2022 17:11:18 +0000 (17:11 +0000)] 
gcc-10(and above): fix compile issue caused by stringop-overflow

22 months agoklocwork: fix risk issues
Hong, Yang A [Tue, 31 May 2022 06:30:18 +0000 (06:30 +0000)] 
klocwork: fix risk issues

23 months agoMerge pull request #164 from jeffplaisance/develop
Konstantinos Margaritis [Mon, 4 Sep 2023 20:11:34 +0000 (23:11 +0300)] 
Merge pull request #164 from jeffplaisance/develop

adding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically linked into another shared library without exporting symbols

23 months agoMerge pull request #165 from VectorCamp/feature/enable-fat-runtime-arm
Konstantinos Margaritis [Mon, 4 Sep 2023 19:50:11 +0000 (22:50 +0300)] 
Merge pull request #165 from VectorCamp/feature/enable-fat-runtime-arm

Feature/enable fat runtime arm

23 months agoadding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically... 164/head
jplaisance [Tue, 27 Jun 2023 14:47:14 +0000 (09:47 -0500)] 
adding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically linked into another shared library without exporting symbols

23 months agofix SVE flags detection order #145 165/head
Konstantinos Margaritis [Wed, 23 Aug 2023 10:21:02 +0000 (10:21 +0000)] 
fix SVE flags detection order #145

23 months agoinitial attempt for fat binary on Aarch64
Konstantinos Margaritis [Wed, 23 Aug 2023 09:42:00 +0000 (09:42 +0000)] 
initial attempt for fat binary on Aarch64

2 years agoMerge pull request #149 from azat-ch/small-vector-msan
Konstantinos Margaritis [Tue, 23 May 2023 15:45:10 +0000 (18:45 +0300)] 
Merge pull request #149 from azat-ch/small-vector-msan

Use std::vector instead of boost::container::small_vector under MSan

2 years agoMerge pull request #148 from azat-ch/getData128-msan
Konstantinos Margaritis [Tue, 23 May 2023 15:44:47 +0000 (18:44 +0300)] 
Merge pull request #148 from azat-ch/getData128-msan

Fix use-of-uninitialized-value due to getData128()

2 years agoFix use-of-uninitialized-value due to getData128() 148/head
Azat Khuzhin [Wed, 10 May 2023 13:29:23 +0000 (15:29 +0200)] 
Fix use-of-uninitialized-value due to getData128()

When temporary buffer is used in getData128(), then it may return
uninitialized data.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2 years agoUse std::vector instead of boost::container::small_vector under MSan 149/head
Azat Khuzhin [Wed, 10 May 2023 13:38:27 +0000 (15:38 +0200)] 
Use std::vector instead of boost::container::small_vector under MSan

There are some issues with dtors in boost::container::small_vector
and/or vector, which is reported by MSan as an error.

The suppression __attribute__((no_sanitize_memory)) works until
clang-15, but since clang-16 it does not. It looks like before clang-16
this no_sanitize_memory works for all child functions, while since
clang-16 only for this function. I've tried to add few others, but a) it
looks icky b) I haven't managed to finish this process.

Also I've measured the performance and it hadn't been changed. Though
boost::small_vector should be faster then std::vector, but apparently my
particular case hadn't affected too much.

And one more thing, MSan reports this only with -O0, with -O3 - it is
not reproduced.

<details>

<summary>MSan report:</summary>

_Note: it was slightly trimmed_

```
==11364==WARNING: MemorySanitizer: use-of-uninitialized-value
2023.05.10 15:40:53.000233 [ 11620 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 1012.32 MiB, peak 1012.32 MiB, free memory in arenas 0.00 B, will set to 1015.82 MiB (RSS), difference: 3.50 MiB
    0 0x55558d13289f in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::deallocate(std::__1::pair<unsigned char, unsigned char>* const&, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:455:7
    1 0x55558d139e8e in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::~vector_alloc_holder() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:420:16
    2 0x55558d139e0b in boost::container::vector<std::__1::pair<unsigned char, unsigned char>, boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, void>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1141:4
    3 0x55558d12a4fa in boost::container::small_vector_base<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    4 0x55558d12a4fa in boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    5 0x55558d13a21b in std::__1::__tuple_leaf<0ul, boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>, false>::~__tuple_leaf() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:265:7
    6 0x55558d13a13a in std::__1::__tuple_impl<>::~__tuple_impl() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:451:37
    7 0x55558d13a05b in std::__1::tuple<>::~tuple() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:538:28
    8 0x55558d139f7b in ue2::flat_detail::flat_base<>::~flat_base() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:89:7
    9 0x55558d1299da in ue2::flat_set<>::~flat_set() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:152:7
    10 0x55558d4e4dda in ue2::(anonymous namespace)::DAccelScheme::~DAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:301:8
    11 0x55558d4ff6cf in void boost::container::allocator_traits<>::priv_destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::move_detail::integral_constant<bool, false>, boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:403:11
    12 0x55558d4fefde in void boost::container::allocator_traits<>::destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:331:7
    13 0x55558d4fc364 in boost::container::dtl::disable_if_trivially_destructible<>::type boost::container::destroy_alloc_n<>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/detail/copy_move_algo.hpp:988:7
    14 0x55558d517962 in boost::container::vector<>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1138:7
    15 0x55558d4f724d in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    16 0x55558d4f724d in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    17 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    18 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    19 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    20 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    21 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    22 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    23 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    24 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    25 0x55558d4e4af5 in ue2::findBestDoubleAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:556:5
    26 0x55558d4e2659 in ue2::findBestAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:569:27
    27 0x55558d3aa8ff in ue2::look_for_offset_accel(ue2::raw_dfa const&, unsigned short, unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:197:22
    28 0x55558d3a9727 in ue2::accel_dfa_build_strat::find_escape_strings(unsigned short) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:414:13
    29 0x55558d3b2119 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&)::$_0::operator()(unsigned long) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:606:26
    30 0x55558d3aefd4 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:627:13
    31 0x55558d2fc61f in ue2::mcclellanCompile8() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:935:22
    32 0x55558d2e89ec in ue2::mcclellanCompile_i() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1510:15
    33 0x55558d2ff502 in ue2::mcclellanCompile() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1527:12
    34 0x55558fb13b52 in ue2::getDfa() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:646:15
    35 0x55558fb7e8c8 in ue2::makeLeftNfa() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:854:22
    36 0x55558fb6bd36 in ue2::buildLeftfix() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:1123:15
    37 0x55558fb21020 in ue2::buildLeftfixes() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:1579:9
    38 0x55558fad972c in ue2::buildNfas() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:2063:10
    39 0x55558fac9843 in ue2::RoseBuildImpl::buildFinalEngine(unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:3660:10
    40 0x55558f2b2d86 in ue2::RoseBuildImpl::buildRose(unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_compile.cpp:1796:12

  Uninitialized value was stored to memory at
    0 0x55558d132898 in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::deallocate(std::__1::pair<unsigned char, unsigned char>* const&, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:455:56
    1 0x55558d139e8e in boost::container::vector_alloc_holder<>::~vector_alloc_holder() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:420:16
    2 0x55558d139e0b in boost::container::vector<>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1141:4
    3 0x55558d12a4fa in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    4 0x55558d12a4fa in boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    5 0x55558d13a21b in std::__1::__tuple_leaf<>::~__tuple_leaf() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:265:7
    6 0x55558d13a13a in std::__1::__tuple_impl<>::~__tuple_impl .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:451:37
    7 0x55558d13a05b in std::__1::tuple<>::~tuple() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:538:28
    8 0x55558d139f7b in ue2::flat_detail::flat_base<>::~flat_base() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:89:7
    9 0x55558d1299da in ue2::flat_set<>::~flat_set() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:1
52:7
    10 0x55558d4e4dda in ue2::(anonymous namespace)::DAccelScheme::~DAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:301:8
    11 0x55558d4ff6cf in void boost::container::allocator_traits<>::priv_destroy<ue2::(anonymous namespace)::DAccelScheme>() .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:403:11
    12 0x55558d4fefde in void boost::container::allocator_traits<>::destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::container::small_vector_allocator<>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:331:7
    13 0x55558d4fc364 in boost::container::dtl::disable_if_trivially_destructible<>::type boost::container::destroy_alloc_n<boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>, ue2::(anonymous namespace)::DAccelScheme*, unsigned long>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/detail/copy_move_algo.hpp:988:7
    14 0x55558d517962 in boost::container::vector<ue2::(anonymous namespace)::DAccelScheme, boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>, void>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1138:7
    15 0x55558d4f724d in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    16 0x55558d4f724d in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    17 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    18 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    19 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    20 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    21 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9

  Member fields were destroyed
    0 0x5555652e08dd in __sanitizer_dtor_callback_fields /src/llvm/worktrees/llvm-16/compiler-rt/lib/msan/msan_interceptors.cpp:961:5
    1 0x55558d4f71a6 in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:528:8
    2 0x55558d4f71a6 in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    3 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    4 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    5 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    6 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    7 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    8 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    9 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    10 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    11 0x55558d4e4af5 in ue2::findBestDoubleAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:556:5
    12 0x55558d4e2659 in ue2::findBestAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:569:27
    13 0x55558d3aa8ff in ue2::look_for_offset_accel(ue2::raw_dfa const&, unsigned short, unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:197:22
    14 0x55558d3a9727 in ue2::accel_dfa_build_strat::find_escape_strings(unsigned short) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:414:13
    15 0x55558d3b2119 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&)::$_0::operator()(unsigned long) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:606:26
    16 0x55558d3aefd4 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:627:13
    17 0x55558d2fc61f in ue2::mcclellanCompile8(ue2::(anonymous namespace)::dfa_info&, ue2::CompileContext const&, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:935:22
    18 0x55558d2e89ec in ue2::mcclellanCompile_i(ue2::raw_dfa&, ue2::accel_dfa_build_strat&, ue2::CompileContext const&, bool, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1510:15
    19 0x55558d2ff502 in ue2::mcclellanCompile(ue2::raw_dfa&, ue2::CompileContext const&, ue2::ReportManager const&, bool, bool, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1527:12
    20 0x55558fb13b52 in ue2::getDfa(ue2::raw_dfa&, bool, ue2::CompileContext const&, ue2::ReportManager const&) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:646:15
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2 years agoMerge pull request #144 from rschu1ze/rs/fix-libcxx16
Konstantinos Margaritis [Wed, 29 Mar 2023 09:55:48 +0000 (12:55 +0300)] 
Merge pull request #144 from rschu1ze/rs/fix-libcxx16

Fix compilation with libcxx 16

2 years agoFix compilation with libcxx 16 144/head
Robert Schulze [Tue, 28 Mar 2023 21:34:35 +0000 (21:34 +0000)] 
Fix compilation with libcxx 16

After upgrading our (ClickHouse's) libcxx from 15 to 16, the compiler
started to complain about usage of an incomplete type "RoseInstruction"
in this (header) function:

  void RoseProgram::replace(Iter it, std::unique_ptr<RoseInstruction> ri) {
    ...

The reason is that libcxx 16 is the first version which implements C++23
constexpr std::unique_ptr (P2273R3, see (*)). RoseProgram::replace()
happens to be be const-evaluatable and the compiler tries to run
std::unique_ptr's ctor + dtor. This fails because at this point
RoseInstruction isn't defined yet.

There are two ways of fixing this:
1. Include rose_build_instruction.h (which contains RoseInstruction)
   into rose_build_program.h. Disadvantage: The new include will
   propagate transitively into all callers.
2. Move the function implementation into the source file which sees
   RoseInstruction's definition already. Disadvantage: Template
   instantiation is no longer automatic, instead there must be either a)
   explicit template instantiation (e.g. in rose_build_program.cpp) or
   b) all callers which instantiate the function must live in the same
   source file and do the instantiations by themselves. Fortunately, the
   latter is the case here, but potential future code outside
   rose_build_program.cpp will require ugly explicit instantiation.

(*) https://en.cppreference.com/w/cpp/23

2 years agoMerge pull request #143 from VectorCamp/develop vectorscan/5.4.9
Konstantinos Margaritis [Thu, 23 Mar 2023 14:11:37 +0000 (16:11 +0200)] 
Merge pull request #143 from VectorCamp/develop

Prepare for new release 5.4.9

2 years agoMerge pull request #142 from VectorCamp/feature/bump-version 143/head
Konstantinos Margaritis [Thu, 23 Mar 2023 08:31:06 +0000 (10:31 +0200)] 
Merge pull request #142 from VectorCamp/feature/bump-version

Bump version

2 years agoBump version 142/head
Konstantinos Margaritis [Thu, 23 Mar 2023 08:29:20 +0000 (08:29 +0000)] 
Bump version

2 years agoMerge pull request #141 from VectorCamp/bugfix/hs-flag-utf8-signed-char-on-arm
Konstantinos Margaritis [Thu, 23 Mar 2023 08:14:59 +0000 (10:14 +0200)] 
Merge pull request #141 from VectorCamp/bugfix/hs-flag-utf8-signed-char-on-arm

Set Ragel.rl char type to unsigned, #135

2 years agoclang 14 makes some test failed because val is uninitialized 141/head
Konstantinos Margaritis [Wed, 22 Mar 2023 19:39:03 +0000 (21:39 +0200)] 
clang 14 makes some test failed because val is uninitialized

2 years agofix ExpressionParser.cpp path
Konstantinos Margaritis [Wed, 22 Mar 2023 09:29:28 +0000 (11:29 +0200)] 
fix ExpressionParser.cpp path

2 years agoadd -funsigned-char to RAGEL_C_FLAGS, move util build after that
Konstantinos Margaritis [Wed, 22 Mar 2023 09:29:02 +0000 (11:29 +0200)] 
add -funsigned-char to RAGEL_C_FLAGS, move util build after that

2 years agoForce -funsigned-char to RAGEL_C_FLAGS
Konstantinos Margaritis [Wed, 22 Mar 2023 08:49:19 +0000 (08:49 +0000)] 
Force -funsigned-char to RAGEL_C_FLAGS

2 years agofix typo
Konstantinos Margaritis [Tue, 21 Mar 2023 18:11:17 +0000 (18:11 +0000)] 
fix typo

2 years agoSet Ragel.rl char type to unsigned, #135
Konstantinos Margaritis [Tue, 21 Mar 2023 18:07:06 +0000 (18:07 +0000)] 
Set Ragel.rl char type to unsigned, #135

2 years agoMerge pull request #136 from VectorCamp/feature/prefix-assume-aligned
Konstantinos Margaritis [Tue, 1 Nov 2022 15:41:39 +0000 (17:41 +0200)] 
Merge pull request #136 from VectorCamp/feature/prefix-assume-aligned

prefix assume_aligned to avoid clash with std::assume_aligned in c++20

2 years agoprefix assume_aligned to avoid clash with std::assume_aligned in c++20 136/head
Konstantinos Margaritis [Tue, 1 Nov 2022 10:29:22 +0000 (10:29 +0000)] 
prefix assume_aligned to avoid clash with std::assume_aligned in c++20

2 years agoMerge pull request #131 from VectorCamp/develop
Konstantinos Margaritis [Mon, 19 Sep 2022 14:59:40 +0000 (17:59 +0300)] 
Merge pull request #131 from VectorCamp/develop

Prepare for new release 5.4.9

2 years agoMerge pull request #125 from abondarev84/master 131/head
Konstantinos Margaritis [Mon, 19 Sep 2022 09:44:14 +0000 (12:44 +0300)] 
Merge pull request #125 from abondarev84/master

cmake change for correct placement of autodetected tune & arch flags of GCC and SVE enablement on AARCH64

2 years agoclang SVE build fix 125/head
Alex Bondarev [Sun, 18 Sep 2022 16:42:45 +0000 (19:42 +0300)] 
clang SVE build fix

2 years agoSVE enabled on user input. updated README
Alex Bondarev [Sun, 18 Sep 2022 09:04:05 +0000 (12:04 +0300)] 
SVE enabled on user input. updated README

tune and arch flags will be applied from autodetect only if they have been created by the process, otherwise the old logical flow remains wrt the flags

2 years agoMerge pull request #129 from VectorCamp/bugfix/fix-clang-on-power
Konstantinos Margaritis [Fri, 16 Sep 2022 16:04:06 +0000 (19:04 +0300)] 
Merge pull request #129 from VectorCamp/bugfix/fix-clang-on-power

Fix compile errors on clang and Power

2 years agomove variable 129/head
Konstantinos Margaritis [Fri, 16 Sep 2022 11:05:31 +0000 (14:05 +0300)] 
move variable

2 years agoclang 13+ gives wrong -Wunused-but-set-variable error on nfa/mcclellancompile.cpp...
Konstantinos Margaritis [Fri, 16 Sep 2022 11:04:59 +0000 (14:04 +0300)] 
clang 13+ gives wrong -Wunused-but-set-variable error on nfa/mcclellancompile.cpp about total_daddy variable, disabling