]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
22 months agoFix version getting out of sync #175 178/head 185/head
Konstantinos Margaritis [Tue, 3 Oct 2023 06:57:10 +0000 (09:57 +0300)] 
Fix version getting out of sync #175

22 months agoMerge pull request #174 from VectorCamp/develop vectorscan/5.4.10.1
Konstantinos Margaritis [Fri, 8 Sep 2023 10:42:33 +0000 (13:42 +0300)] 
Merge pull request #174 from VectorCamp/develop

Minor bugfix release 5.4.10.1

22 months agoMerge pull request #173 from VectorCamp/bugfix/disable-fat-macos-arm 174/head
Konstantinos Margaritis [Fri, 8 Sep 2023 09:18:11 +0000 (12:18 +0300)] 
Merge pull request #173 from VectorCamp/bugfix/disable-fat-macos-arm

Bugfix/disable fat macos arm

22 months agoHWCAP is only available on Linux 173/head
Konstantinos Margaritis [Fri, 8 Sep 2023 07:08:44 +0000 (10:08 +0300)] 
HWCAP is only available on Linux

22 months agoforce disable FAT_RUNTIME on MacOS on Arm
Konstantinos Margaritis [Fri, 8 Sep 2023 07:08:18 +0000 (10:08 +0300)] 
force disable FAT_RUNTIME on MacOS on Arm

22 months agoforgot to update changelog for latest entry
Konstantinos Margaritis [Thu, 7 Sep 2023 17:10:20 +0000 (20:10 +0300)] 
forgot to update changelog for latest entry

22 months agoMerge pull request #167 from VectorCamp/develop
Konstantinos Margaritis [Thu, 7 Sep 2023 17:03:49 +0000 (20:03 +0300)] 
Merge pull request #167 from VectorCamp/develop

Prepare for 5.4.10

22 months agominor fix 167/head
Konstantinos Margaritis [Thu, 7 Sep 2023 14:53:25 +0000 (17:53 +0300)] 
minor fix

22 months agobump version, add Vectorscan Changelog
Konstantinos Margaritis [Thu, 7 Sep 2023 14:51:07 +0000 (17:51 +0300)] 
bump version, add Vectorscan Changelog

22 months agoMerge pull request #169 from VectorCamp/feature/backport-hyperscan-2023Q3
Konstantinos Margaritis [Tue, 5 Sep 2023 17:11:30 +0000 (20:11 +0300)] 
Merge pull request #169 from VectorCamp/feature/backport-hyperscan-2023Q3

Feature/backport hyperscan 2023 q3

22 months agochangelog: updates for 5.4.2 release 169/head
Hong, Yang A [Wed, 19 Apr 2023 09:18:45 +0000 (09:18 +0000)] 
changelog: updates for 5.4.2 release

22 months agoremove invalid nfa dump info
Hong, Yang A [Thu, 23 Mar 2023 06:43:46 +0000 (06:43 +0000)] 
remove invalid nfa dump info

22 months agoscratch: remove quick validity check
Hong, Yang A [Wed, 1 Mar 2023 14:42:27 +0000 (14:42 +0000)] 
scratch: remove quick validity check
Roll back fix for github issue #350

About Scratch Usage:
For compile time, scratch space is strongly recommended to be
allocated immediately after database generation.
For runtime, besides using scratch for corresponding database,
Hyperscan also allows user to use larger scratch space allocated
for another database.
When multiple concurrent threads need to use the same databases
and a new scratch space is required, cloning the largest one is
always safe. This is realized based on API hs_scratch_size() and
hs_clone_scratch().
Behaviors beyond above are discouraged and results are undefined.

22 months agochangelog: updates for 5.4.1 release
Chang, Harry [Tue, 21 Feb 2023 22:52:57 +0000 (22:52 +0000)] 
changelog: updates for 5.4.1 release

22 months agosanitiser bugfix
Hong, Yang A [Wed, 29 Dec 2021 22:30:18 +0000 (22:30 +0000)] 
sanitiser bugfix

22 months agobugfix: add vbmi case for test in database.cpp
Hong, Yang A [Mon, 7 Jun 2021 08:24:51 +0000 (16:24 +0800)] 
bugfix: add vbmi case for test in database.cpp

22 months agobugfix: add vbmi platform parameter for tests in single.cpp
Hong, Yang A [Mon, 7 Jun 2021 07:35:57 +0000 (15:35 +0800)] 
bugfix: add vbmi platform parameter for tests in single.cpp

22 months agofix nfa dump error
Hong, Yang A [Thu, 20 Oct 2022 08:47:03 +0000 (08:47 +0000)] 
fix nfa dump error

22 months agoscratch: add quick validity check
Hong, Yang A [Thu, 20 Oct 2022 08:48:46 +0000 (08:48 +0000)] 
scratch: add quick validity check

fix github issue #350

22 months agostream close: free stream to avoid memory leak
Hong, Yang A [Wed, 19 Oct 2022 16:50:02 +0000 (16:50 +0000)] 
stream close: free stream to avoid memory leak

fix github issue #303

22 months agoSilence clang-14 warnings
Hong, Yang A [Mon, 15 Aug 2022 03:00:22 +0000 (03:00 +0000)] 
Silence clang-14 warnings

22 months agoFix cmake CMP0115 warning for CMake 3.20 and above
Hong, Yang A [Tue, 2 Aug 2022 19:25:27 +0000 (19:25 +0000)] 
Fix cmake CMP0115 warning for CMake 3.20 and above

22 months agoupdate year 2022
Hong, Yang A [Mon, 1 Aug 2022 17:13:25 +0000 (17:13 +0000)] 
update year 2022

22 months agoUTF-8 validation: fix one cotec check corner issue
Hong, Yang A [Thu, 28 Jul 2022 21:24:31 +0000 (21:24 +0000)] 
UTF-8 validation: fix one cotec check corner issue

fix github issue #362

22 months agostringop-overflow compatible fix
Hong, Yang A [Tue, 12 Jul 2022 08:42:05 +0000 (08:42 +0000)] 
stringop-overflow compatible fix

22 months agogcc-10(and above): fix compile issue caused by stringop-overflow
Hong, Yang A [Tue, 5 Jul 2022 17:11:18 +0000 (17:11 +0000)] 
gcc-10(and above): fix compile issue caused by stringop-overflow

22 months agoklocwork: fix risk issues
Hong, Yang A [Tue, 31 May 2022 06:30:18 +0000 (06:30 +0000)] 
klocwork: fix risk issues

23 months agoMerge pull request #164 from jeffplaisance/develop
Konstantinos Margaritis [Mon, 4 Sep 2023 20:11:34 +0000 (23:11 +0300)] 
Merge pull request #164 from jeffplaisance/develop

adding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically linked into another shared library without exporting symbols

23 months agoMerge pull request #165 from VectorCamp/feature/enable-fat-runtime-arm
Konstantinos Margaritis [Mon, 4 Sep 2023 19:50:11 +0000 (22:50 +0300)] 
Merge pull request #165 from VectorCamp/feature/enable-fat-runtime-arm

Feature/enable fat runtime arm

23 months agoadding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically... 164/head
jplaisance [Tue, 27 Jun 2023 14:47:14 +0000 (09:47 -0500)] 
adding ifndef around HS_PUBLIC_API definition so that vectorscan can be statically linked into another shared library without exporting symbols

23 months agofix SVE flags detection order #145 165/head
Konstantinos Margaritis [Wed, 23 Aug 2023 10:21:02 +0000 (10:21 +0000)] 
fix SVE flags detection order #145

23 months agoinitial attempt for fat binary on Aarch64
Konstantinos Margaritis [Wed, 23 Aug 2023 09:42:00 +0000 (09:42 +0000)] 
initial attempt for fat binary on Aarch64

2 years agoMerge pull request #149 from azat-ch/small-vector-msan
Konstantinos Margaritis [Tue, 23 May 2023 15:45:10 +0000 (18:45 +0300)] 
Merge pull request #149 from azat-ch/small-vector-msan

Use std::vector instead of boost::container::small_vector under MSan

2 years agoMerge pull request #148 from azat-ch/getData128-msan
Konstantinos Margaritis [Tue, 23 May 2023 15:44:47 +0000 (18:44 +0300)] 
Merge pull request #148 from azat-ch/getData128-msan

Fix use-of-uninitialized-value due to getData128()

2 years agoFix use-of-uninitialized-value due to getData128() 148/head
Azat Khuzhin [Wed, 10 May 2023 13:29:23 +0000 (15:29 +0200)] 
Fix use-of-uninitialized-value due to getData128()

When temporary buffer is used in getData128(), then it may return
uninitialized data.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2 years agoUse std::vector instead of boost::container::small_vector under MSan 149/head
Azat Khuzhin [Wed, 10 May 2023 13:38:27 +0000 (15:38 +0200)] 
Use std::vector instead of boost::container::small_vector under MSan

There are some issues with dtors in boost::container::small_vector
and/or vector, which is reported by MSan as an error.

The suppression __attribute__((no_sanitize_memory)) works until
clang-15, but since clang-16 it does not. It looks like before clang-16
this no_sanitize_memory works for all child functions, while since
clang-16 only for this function. I've tried to add few others, but a) it
looks icky b) I haven't managed to finish this process.

Also I've measured the performance and it hadn't been changed. Though
boost::small_vector should be faster then std::vector, but apparently my
particular case hadn't affected too much.

And one more thing, MSan reports this only with -O0, with -O3 - it is
not reproduced.

<details>

<summary>MSan report:</summary>

_Note: it was slightly trimmed_

```
==11364==WARNING: MemorySanitizer: use-of-uninitialized-value
2023.05.10 15:40:53.000233 [ 11620 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 1012.32 MiB, peak 1012.32 MiB, free memory in arenas 0.00 B, will set to 1015.82 MiB (RSS), difference: 3.50 MiB
    0 0x55558d13289f in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::deallocate(std::__1::pair<unsigned char, unsigned char>* const&, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:455:7
    1 0x55558d139e8e in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::~vector_alloc_holder() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:420:16
    2 0x55558d139e0b in boost::container::vector<std::__1::pair<unsigned char, unsigned char>, boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, void>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1141:4
    3 0x55558d12a4fa in boost::container::small_vector_base<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    4 0x55558d12a4fa in boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    5 0x55558d13a21b in std::__1::__tuple_leaf<0ul, boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>, false>::~__tuple_leaf() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:265:7
    6 0x55558d13a13a in std::__1::__tuple_impl<>::~__tuple_impl() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:451:37
    7 0x55558d13a05b in std::__1::tuple<>::~tuple() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:538:28
    8 0x55558d139f7b in ue2::flat_detail::flat_base<>::~flat_base() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:89:7
    9 0x55558d1299da in ue2::flat_set<>::~flat_set() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:152:7
    10 0x55558d4e4dda in ue2::(anonymous namespace)::DAccelScheme::~DAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:301:8
    11 0x55558d4ff6cf in void boost::container::allocator_traits<>::priv_destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::move_detail::integral_constant<bool, false>, boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:403:11
    12 0x55558d4fefde in void boost::container::allocator_traits<>::destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:331:7
    13 0x55558d4fc364 in boost::container::dtl::disable_if_trivially_destructible<>::type boost::container::destroy_alloc_n<>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/detail/copy_move_algo.hpp:988:7
    14 0x55558d517962 in boost::container::vector<>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1138:7
    15 0x55558d4f724d in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    16 0x55558d4f724d in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    17 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    18 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    19 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    20 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    21 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    22 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    23 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    24 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    25 0x55558d4e4af5 in ue2::findBestDoubleAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:556:5
    26 0x55558d4e2659 in ue2::findBestAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:569:27
    27 0x55558d3aa8ff in ue2::look_for_offset_accel(ue2::raw_dfa const&, unsigned short, unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:197:22
    28 0x55558d3a9727 in ue2::accel_dfa_build_strat::find_escape_strings(unsigned short) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:414:13
    29 0x55558d3b2119 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&)::$_0::operator()(unsigned long) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:606:26
    30 0x55558d3aefd4 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:627:13
    31 0x55558d2fc61f in ue2::mcclellanCompile8() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:935:22
    32 0x55558d2e89ec in ue2::mcclellanCompile_i() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1510:15
    33 0x55558d2ff502 in ue2::mcclellanCompile() .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1527:12
    34 0x55558fb13b52 in ue2::getDfa() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:646:15
    35 0x55558fb7e8c8 in ue2::makeLeftNfa() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:854:22
    36 0x55558fb6bd36 in ue2::buildLeftfix() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:1123:15
    37 0x55558fb21020 in ue2::buildLeftfixes() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:1579:9
    38 0x55558fad972c in ue2::buildNfas() .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:2063:10
    39 0x55558fac9843 in ue2::RoseBuildImpl::buildFinalEngine(unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:3660:10
    40 0x55558f2b2d86 in ue2::RoseBuildImpl::buildRose(unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_compile.cpp:1796:12

  Uninitialized value was stored to memory at
    0 0x55558d132898 in boost::container::vector_alloc_holder<boost::container::small_vector_allocator<std::__1::pair<unsigned char, unsigned char>, std::__1::allocator<void>, void>, unsigned long, boost::move_detail::integral_constant<unsigned int, 1u>>::deallocate(std::__1::pair<unsigned char, unsigned char>* const&, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:455:56
    1 0x55558d139e8e in boost::container::vector_alloc_holder<>::~vector_alloc_holder() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:420:16
    2 0x55558d139e0b in boost::container::vector<>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1141:4
    3 0x55558d12a4fa in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    4 0x55558d12a4fa in boost::container::small_vector<std::__1::pair<unsigned char, unsigned char>, 1ul, std::__1::allocator<std::__1::pair<unsigned char, unsigned char>>, void>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    5 0x55558d13a21b in std::__1::__tuple_leaf<>::~__tuple_leaf() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:265:7
    6 0x55558d13a13a in std::__1::__tuple_impl<>::~__tuple_impl .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:451:37
    7 0x55558d13a05b in std::__1::tuple<>::~tuple() .cmake-llvm16-msan/./contrib/llvm-project/libcxx/include/tuple:538:28
    8 0x55558d139f7b in ue2::flat_detail::flat_base<>::~flat_base() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:89:7
    9 0x55558d1299da in ue2::flat_set<>::~flat_set() .cmake-llvm16-msan/./contrib/vectorscan/src/util/flat_containers.h:1
52:7
    10 0x55558d4e4dda in ue2::(anonymous namespace)::DAccelScheme::~DAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:301:8
    11 0x55558d4ff6cf in void boost::container::allocator_traits<>::priv_destroy<ue2::(anonymous namespace)::DAccelScheme>() .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:403:11
    12 0x55558d4fefde in void boost::container::allocator_traits<>::destroy<ue2::(anonymous namespace)::DAccelScheme>(boost::container::small_vector_allocator<>&, ue2::(anonymous namespace)::DAccelScheme*) .cmake-llvm16-msan/./contrib/boost/boost/container/allocator_traits.hpp:331:7
    13 0x55558d4fc364 in boost::container::dtl::disable_if_trivially_destructible<>::type boost::container::destroy_alloc_n<boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>, ue2::(anonymous namespace)::DAccelScheme*, unsigned long>(boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>&, ue2::(anonymous namespace)::DAccelScheme*, unsigned long) .cmake-llvm16-msan/./contrib/boost/boost/container/detail/copy_move_algo.hpp:988:7
    14 0x55558d517962 in boost::container::vector<ue2::(anonymous namespace)::DAccelScheme, boost::container::small_vector_allocator<ue2::(anonymous namespace)::DAccelScheme, boost::container::new_allocator<void>, void>, void>::~vector() .cmake-llvm16-msan/./contrib/boost/boost/container/vector.hpp:1138:7
    15 0x55558d4f724d in boost::container::small_vector_base<>::~small_vector_base() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:445:80
    16 0x55558d4f724d in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    17 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    18 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    19 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    20 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    21 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9

  Member fields were destroyed
    0 0x5555652e08dd in __sanitizer_dtor_callback_fields /src/llvm/worktrees/llvm-16/compiler-rt/lib/msan/msan_interceptors.cpp:961:5
    1 0x55558d4f71a6 in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:528:8
    2 0x55558d4f71a6 in boost::container::small_vector<>::~small_vector() .cmake-llvm16-msan/./contrib/boost/boost/container/small_vector.hpp:564:7
    3 0x55558d4f2ff3 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:444:1
    4 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    5 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    6 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    7 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    8 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    9 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    10 0x55558d4f2f41 in ue2::findDoubleBest() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:442:9
    11 0x55558d4e4af5 in ue2::findBestDoubleAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:556:5
    12 0x55558d4e2659 in ue2::findBestAccelScheme() .cmake-llvm16-msan/./contrib/vectorscan/src/nfagraph/ng_limex_accel.cpp:569:27
    13 0x55558d3aa8ff in ue2::look_for_offset_accel(ue2::raw_dfa const&, unsigned short, unsigned int) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:197:22
    14 0x55558d3a9727 in ue2::accel_dfa_build_strat::find_escape_strings(unsigned short) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:414:13
    15 0x55558d3b2119 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&)::$_0::operator()(unsigned long) const .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:606:26
    16 0x55558d3aefd4 in ue2::accel_dfa_build_strat::getAccelInfo(ue2::Grey const&) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/accel_dfa_build_strat.cpp:627:13
    17 0x55558d2fc61f in ue2::mcclellanCompile8(ue2::(anonymous namespace)::dfa_info&, ue2::CompileContext const&, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:935:22
    18 0x55558d2e89ec in ue2::mcclellanCompile_i(ue2::raw_dfa&, ue2::accel_dfa_build_strat&, ue2::CompileContext const&, bool, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1510:15
    19 0x55558d2ff502 in ue2::mcclellanCompile(ue2::raw_dfa&, ue2::CompileContext const&, ue2::ReportManager const&, bool, bool, std::__1::set<unsigned short, std::__1::less<unsigned short>, std::__1::allocator<unsigned short>>*) .cmake-llvm16-msan/./contrib/vectorscan/src/nfa/mcclellancompile.cpp:1527:12
    20 0x55558fb13b52 in ue2::getDfa(ue2::raw_dfa&, bool, ue2::CompileContext const&, ue2::ReportManager const&) .cmake-llvm16-msan/./contrib/vectorscan/src/rose/rose_build_bytecode.cpp:646:15
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2 years agoMerge pull request #144 from rschu1ze/rs/fix-libcxx16
Konstantinos Margaritis [Wed, 29 Mar 2023 09:55:48 +0000 (12:55 +0300)] 
Merge pull request #144 from rschu1ze/rs/fix-libcxx16

Fix compilation with libcxx 16

2 years agoFix compilation with libcxx 16 144/head
Robert Schulze [Tue, 28 Mar 2023 21:34:35 +0000 (21:34 +0000)] 
Fix compilation with libcxx 16

After upgrading our (ClickHouse's) libcxx from 15 to 16, the compiler
started to complain about usage of an incomplete type "RoseInstruction"
in this (header) function:

  void RoseProgram::replace(Iter it, std::unique_ptr<RoseInstruction> ri) {
    ...

The reason is that libcxx 16 is the first version which implements C++23
constexpr std::unique_ptr (P2273R3, see (*)). RoseProgram::replace()
happens to be be const-evaluatable and the compiler tries to run
std::unique_ptr's ctor + dtor. This fails because at this point
RoseInstruction isn't defined yet.

There are two ways of fixing this:
1. Include rose_build_instruction.h (which contains RoseInstruction)
   into rose_build_program.h. Disadvantage: The new include will
   propagate transitively into all callers.
2. Move the function implementation into the source file which sees
   RoseInstruction's definition already. Disadvantage: Template
   instantiation is no longer automatic, instead there must be either a)
   explicit template instantiation (e.g. in rose_build_program.cpp) or
   b) all callers which instantiate the function must live in the same
   source file and do the instantiations by themselves. Fortunately, the
   latter is the case here, but potential future code outside
   rose_build_program.cpp will require ugly explicit instantiation.

(*) https://en.cppreference.com/w/cpp/23

2 years agoMerge pull request #143 from VectorCamp/develop vectorscan/5.4.9
Konstantinos Margaritis [Thu, 23 Mar 2023 14:11:37 +0000 (16:11 +0200)] 
Merge pull request #143 from VectorCamp/develop

Prepare for new release 5.4.9

2 years agoMerge pull request #142 from VectorCamp/feature/bump-version 143/head
Konstantinos Margaritis [Thu, 23 Mar 2023 08:31:06 +0000 (10:31 +0200)] 
Merge pull request #142 from VectorCamp/feature/bump-version

Bump version

2 years agoBump version 142/head
Konstantinos Margaritis [Thu, 23 Mar 2023 08:29:20 +0000 (08:29 +0000)] 
Bump version

2 years agoMerge pull request #141 from VectorCamp/bugfix/hs-flag-utf8-signed-char-on-arm
Konstantinos Margaritis [Thu, 23 Mar 2023 08:14:59 +0000 (10:14 +0200)] 
Merge pull request #141 from VectorCamp/bugfix/hs-flag-utf8-signed-char-on-arm

Set Ragel.rl char type to unsigned, #135

2 years agoclang 14 makes some test failed because val is uninitialized 141/head
Konstantinos Margaritis [Wed, 22 Mar 2023 19:39:03 +0000 (21:39 +0200)] 
clang 14 makes some test failed because val is uninitialized

2 years agofix ExpressionParser.cpp path
Konstantinos Margaritis [Wed, 22 Mar 2023 09:29:28 +0000 (11:29 +0200)] 
fix ExpressionParser.cpp path

2 years agoadd -funsigned-char to RAGEL_C_FLAGS, move util build after that
Konstantinos Margaritis [Wed, 22 Mar 2023 09:29:02 +0000 (11:29 +0200)] 
add -funsigned-char to RAGEL_C_FLAGS, move util build after that

2 years agoForce -funsigned-char to RAGEL_C_FLAGS
Konstantinos Margaritis [Wed, 22 Mar 2023 08:49:19 +0000 (08:49 +0000)] 
Force -funsigned-char to RAGEL_C_FLAGS

2 years agofix typo
Konstantinos Margaritis [Tue, 21 Mar 2023 18:11:17 +0000 (18:11 +0000)] 
fix typo

2 years agoSet Ragel.rl char type to unsigned, #135
Konstantinos Margaritis [Tue, 21 Mar 2023 18:07:06 +0000 (18:07 +0000)] 
Set Ragel.rl char type to unsigned, #135

2 years agoMerge pull request #136 from VectorCamp/feature/prefix-assume-aligned
Konstantinos Margaritis [Tue, 1 Nov 2022 15:41:39 +0000 (17:41 +0200)] 
Merge pull request #136 from VectorCamp/feature/prefix-assume-aligned

prefix assume_aligned to avoid clash with std::assume_aligned in c++20

2 years agoprefix assume_aligned to avoid clash with std::assume_aligned in c++20 136/head
Konstantinos Margaritis [Tue, 1 Nov 2022 10:29:22 +0000 (10:29 +0000)] 
prefix assume_aligned to avoid clash with std::assume_aligned in c++20

2 years agoMerge pull request #131 from VectorCamp/develop
Konstantinos Margaritis [Mon, 19 Sep 2022 14:59:40 +0000 (17:59 +0300)] 
Merge pull request #131 from VectorCamp/develop

Prepare for new release 5.4.9

2 years agoMerge pull request #125 from abondarev84/master 131/head
Konstantinos Margaritis [Mon, 19 Sep 2022 09:44:14 +0000 (12:44 +0300)] 
Merge pull request #125 from abondarev84/master

cmake change for correct placement of autodetected tune & arch flags of GCC and SVE enablement on AARCH64

2 years agoclang SVE build fix 125/head
Alex Bondarev [Sun, 18 Sep 2022 16:42:45 +0000 (19:42 +0300)] 
clang SVE build fix

2 years agoSVE enabled on user input. updated README
Alex Bondarev [Sun, 18 Sep 2022 09:04:05 +0000 (12:04 +0300)] 
SVE enabled on user input. updated README

tune and arch flags will be applied from autodetect only if they have been created by the process, otherwise the old logical flow remains wrt the flags

2 years agoMerge pull request #129 from VectorCamp/bugfix/fix-clang-on-power
Konstantinos Margaritis [Fri, 16 Sep 2022 16:04:06 +0000 (19:04 +0300)] 
Merge pull request #129 from VectorCamp/bugfix/fix-clang-on-power

Fix compile errors on clang and Power

2 years agomove variable 129/head
Konstantinos Margaritis [Fri, 16 Sep 2022 11:05:31 +0000 (14:05 +0300)] 
move variable

2 years agoclang 13+ gives wrong -Wunused-but-set-variable error on nfa/mcclellancompile.cpp...
Konstantinos Margaritis [Fri, 16 Sep 2022 11:04:59 +0000 (14:04 +0300)] 
clang 13+ gives wrong -Wunused-but-set-variable error on nfa/mcclellancompile.cpp about total_daddy variable, disabling

2 years agoremove leftover debug print
Konstantinos Margaritis [Fri, 16 Sep 2022 11:03:17 +0000 (14:03 +0300)] 
remove leftover debug print

2 years agoclang 14 does not allow bitwise OR for bools
Konstantinos Margaritis [Fri, 16 Sep 2022 11:02:53 +0000 (14:02 +0300)] 
clang 14 does not allow bitwise OR for bools

2 years agoclang 14 complains about this, needs investigation
Konstantinos Margaritis [Fri, 16 Sep 2022 11:02:26 +0000 (14:02 +0300)] 
clang 14 complains about this, needs investigation

2 years ago[VSX] movemask needs to be explicitly aligned on clang for vec_ste
Konstantinos Margaritis [Fri, 16 Sep 2022 09:50:33 +0000 (12:50 +0300)] 
[VSX] movemask needs to be explicitly aligned on clang for vec_ste

2 years agoenv vars have to be in quotes
Konstantinos Margaritis [Fri, 16 Sep 2022 09:46:35 +0000 (12:46 +0300)] 
env vars have to be in quotes

2 years ago[VSX] clang complains about the order of __vector 128/head
Konstantinos Margaritis [Fri, 16 Sep 2022 09:41:08 +0000 (12:41 +0300)] 
[VSX] clang complains about the order of __vector

2 years agoDeclarative Pipeline Jenkins environment
Konstantinos Margaritis [Fri, 16 Sep 2022 08:59:36 +0000 (11:59 +0300)] 
Declarative Pipeline Jenkins environment

2 years agoDeclarative Pipeline Jenkins environment attempt
Konstantinos Margaritis [Fri, 16 Sep 2022 08:54:23 +0000 (11:54 +0300)] 
Declarative Pipeline Jenkins environment attempt

2 years agoJenkins change envVars -> withEnv
Konstantinos Margaritis [Fri, 16 Sep 2022 08:46:09 +0000 (11:46 +0300)] 
Jenkins change envVars -> withEnv

2 years agoadditional mcpu flag cleanup
Alex Bondarev [Thu, 15 Sep 2022 21:03:08 +0000 (00:03 +0300)] 
additional mcpu flag cleanup

2 years agoremoved cpu reference flags and fixed tune flag
Alex Bondarev [Thu, 15 Sep 2022 15:38:01 +0000 (18:38 +0300)] 
removed cpu reference flags and fixed tune flag

2 years agoupdated README to reflect CMake changes
Alex Bondarev [Tue, 13 Sep 2022 15:29:06 +0000 (18:29 +0300)] 
updated README to reflect CMake changes

2 years agofix to correctly place the autodetected flags and to activate SVE options
Alex Bondarev [Tue, 13 Sep 2022 15:21:10 +0000 (18:21 +0300)] 
fix to correctly place the autodetected flags and to activate SVE options

2 years agobump version vectorscan/5.4.8
Konstantinos Margaritis [Tue, 13 Sep 2022 12:57:08 +0000 (12:57 +0000)] 
bump version

2 years agoMerge pull request #124 from VectorCamp/develop
Konstantinos Margaritis [Tue, 13 Sep 2022 12:52:20 +0000 (15:52 +0300)] 
Merge pull request #124 from VectorCamp/develop

Merge develop to master

2 years agoMerge pull request #123 from VectorCamp/feature/neon-shift-optimizations 124/head
Konstantinos Margaritis [Tue, 13 Sep 2022 06:13:05 +0000 (09:13 +0300)] 
Merge pull request #123 from VectorCamp/feature/neon-shift-optimizations

[NEON] simplify/optimize shift/align primitives

2 years ago[NEON] simplify/optimize shift/align primitives 123/head
Konstantinos Margaritis [Mon, 12 Sep 2022 13:09:51 +0000 (13:09 +0000)] 
[NEON] simplify/optimize shift/align primitives

2 years agoMerge pull request #121 from liquidaty/mingw64-develop
Konstantinos Margaritis [Fri, 9 Sep 2022 10:42:49 +0000 (13:42 +0300)] 
Merge pull request #121 from liquidaty/mingw64-develop

fix to enable successful build with mingw64

2 years agofix to enable successful build with mingw64 121/head
liquidaty [Thu, 8 Sep 2022 16:59:37 +0000 (09:59 -0700)] 
fix to enable successful build with mingw64

2 years agoMerge pull request #119 from VectorCamp/feature/vsx-optimizations
Konstantinos Margaritis [Thu, 8 Sep 2022 10:41:13 +0000 (13:41 +0300)] 
Merge pull request #119 from VectorCamp/feature/vsx-optimizations

VSX optimizations

2 years agouse correct intrinsic for lshiftbyte_m128 119/head
Konstantinos Margaritis [Wed, 7 Sep 2022 13:00:10 +0000 (16:00 +0300)] 
use correct intrinsic for lshiftbyte_m128

2 years agoprovide non-immediate versions of lshiftbyte/rshiftbyte on x86
Konstantinos Margaritis [Wed, 7 Sep 2022 12:07:20 +0000 (15:07 +0300)] 
provide non-immediate versions of lshiftbyte/rshiftbyte on x86

2 years agoreadd simd_onebit_masks for x86, needs more work
Konstantinos Margaritis [Wed, 7 Sep 2022 10:42:25 +0000 (13:42 +0300)] 
readd simd_onebit_masks for x86, needs more work

2 years ago[NEON] optimize mask1bit128, get rid of simd_onebit_masks
Konstantinos Margaritis [Wed, 7 Sep 2022 10:20:01 +0000 (10:20 +0000)] 
[NEON] optimize mask1bit128, get rid of simd_onebit_masks

2 years agoremove simd_onebit_masks from arm/x86 headers, as they moved to common
Konstantinos Margaritis [Wed, 7 Sep 2022 09:41:32 +0000 (12:41 +0300)] 
remove simd_onebit_masks from arm/x86 headers, as they moved to common

2 years ago[VSX] optimize alignr method
Konstantinos Margaritis [Wed, 7 Sep 2022 09:35:28 +0000 (12:35 +0300)] 
[VSX] optimize alignr method

2 years ago[VSX] optimize shift operators
Konstantinos Margaritis [Wed, 7 Sep 2022 09:16:14 +0000 (12:16 +0300)] 
[VSX] optimize shift operators

2 years ago[VSX] optimize shifting methods, replace template Unroller
Konstantinos Margaritis [Wed, 7 Sep 2022 08:52:08 +0000 (11:52 +0300)] 
[VSX] optimize shifting methods, replace template Unroller

2 years agooptimize comparemask implementation, clean up code, use union types instead of casts
Konstantinos Margaritis [Tue, 6 Sep 2022 23:02:11 +0000 (02:02 +0300)] 
optimize comparemask implementation, clean up code, use union types instead of casts

2 years ago[VSX] add algorithm for alignr w/o use of immediates
Konstantinos Margaritis [Tue, 6 Sep 2022 21:01:54 +0000 (00:01 +0300)] 
[VSX] add algorithm for alignr w/o use of immediates

2 years ago[VSX] correct lshiftbyte_m128/rshiftbyte_m128, variable_byte_shift
Konstantinos Margaritis [Tue, 6 Sep 2022 20:39:44 +0000 (23:39 +0300)] 
[VSX] correct lshiftbyte_m128/rshiftbyte_m128, variable_byte_shift

2 years ago[VSX] huge optimization of movemask128
Konstantinos Margaritis [Tue, 6 Sep 2022 17:08:44 +0000 (20:08 +0300)] 
[VSX] huge optimization of movemask128

2 years ago[VSX] optimize and correct lshift_m128/rshift_m128
Konstantinos Margaritis [Tue, 6 Sep 2022 15:46:39 +0000 (18:46 +0300)] 
[VSX] optimize and correct lshift_m128/rshift_m128

2 years ago[VSX] optimized mask1bit128(), moved simd_onebit_masks to common
Konstantinos Margaritis [Tue, 6 Sep 2022 15:10:55 +0000 (18:10 +0300)] 
[VSX] optimized mask1bit128(), moved simd_onebit_masks to common

2 years agoadd popcount32x4, popcount64x4 helper functions
Konstantinos Margaritis [Fri, 2 Sep 2022 12:12:56 +0000 (15:12 +0300)] 
add popcount32x4, popcount64x4 helper functions

2 years agoMerge pull request #118 from VectorCamp/bugfix/hyperscan-backport-202208
Konstantinos Margaritis [Sat, 3 Sep 2022 06:32:43 +0000 (09:32 +0300)] 
Merge pull request #118 from VectorCamp/bugfix/hyperscan-backport-202208

Bugfix/hyperscan backport 202208

2 years agoAdd missing <memory> header 118/head
Konstantinos Margaritis [Tue, 30 Aug 2022 17:40:23 +0000 (20:40 +0300)] 
Add missing <memory> header

2 years agofix build with glibc-2.34
Liu Zixian [Mon, 27 Jun 2022 08:07:16 +0000 (16:07 +0800)] 
fix build with glibc-2.34

SIGTSKSZ is no long a constant after glibc 2.34
https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html

2 years agoliteral API: add empty string check.
Hong, Yang A [Thu, 4 Mar 2021 16:13:46 +0000 (16:13 +0000)] 
literal API: add empty string check.

fixes github issue #302, #304

2 years agochimera: fix SKIP flag issue
Hong, Yang A [Thu, 28 Jul 2022 04:59:34 +0000 (04:59 +0000)] 
chimera: fix SKIP flag issue

fix github issue #360

2 years agoCorpus editor: fix random char value of UTF-8.
Chang, Harry [Thu, 12 May 2022 08:20:29 +0000 (08:20 +0000)] 
Corpus editor: fix random char value of UTF-8.

2 years agoCorpus generator: fix random char value of UTF-8.
Chang, Harry [Thu, 12 May 2022 02:15:07 +0000 (02:15 +0000)] 
Corpus generator: fix random char value of UTF-8.

fixes github issue #184

2 years agobugfix: fix overflow risk of strlen function
Hong, Yang A [Thu, 28 Apr 2022 10:11:32 +0000 (10:11 +0000)] 
bugfix: fix overflow risk of strlen function