]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
8 years agoDon't ignore unused attrib warnings in test
Matthew Barr [Mon, 12 Dec 2016 03:52:06 +0000 (14:52 +1100)] 
Don't ignore unused attrib warnings in test

8 years agoDon't set flags for common files
Matthew Barr [Mon, 12 Dec 2016 00:55:56 +0000 (11:55 +1100)] 
Don't set flags for common files

8 years agocmake: default to fat runtime for release builds
Matthew Barr [Mon, 12 Dec 2016 00:34:15 +0000 (11:34 +1100)] 
cmake: default to fat runtime for release builds

8 years agocmake: unneeded header check
Matthew Barr [Tue, 29 Nov 2016 05:19:04 +0000 (16:19 +1100)] 
cmake: unneeded header check

8 years agoapi: hs_valid_platform
Matthew Barr [Tue, 29 Nov 2016 05:08:14 +0000 (16:08 +1100)] 
api: hs_valid_platform

8 years agoFat runtime
Matthew Barr [Wed, 2 Nov 2016 00:01:28 +0000 (11:01 +1100)] 
Fat runtime

8 years agoConvert compile-time code to not require SIMD
Matthew Barr [Thu, 6 Oct 2016 04:33:24 +0000 (15:33 +1100)] 
Convert compile-time code to not require SIMD

8 years agoRemove erroneous semicolon
Matthew Barr [Thu, 6 Oct 2016 02:51:14 +0000 (13:51 +1100)] 
Remove erroneous semicolon

8 years agomultibit, fatbit: make _size build-time only
Justin Viiret [Mon, 5 Dec 2016 05:20:52 +0000 (16:20 +1100)] 
multibit, fatbit: make _size build-time only

This commit makes mmbit_size() and fatbit_size compile-time only, and
adds a resource limit for very large multibits.

8 years agocalcDepthFromSource: only take one copy of the graph
Alex Coyte [Fri, 2 Dec 2016 05:02:09 +0000 (16:02 +1100)] 
calcDepthFromSource: only take one copy of the graph

8 years agomcsheng: remove dead code
Alex Coyte [Thu, 1 Dec 2016 23:42:26 +0000 (10:42 +1100)] 
mcsheng: remove dead code

8 years agointroduce Sheng-McClellan hybrid
Alex Coyte [Thu, 1 Dec 2016 03:32:47 +0000 (14:32 +1100)] 
introduce Sheng-McClellan hybrid

8 years agohsbench: add Hyperscan benchmarker
Matthew Barr [Wed, 14 Dec 2016 04:26:01 +0000 (15:26 +1100)] 
hsbench: add Hyperscan benchmarker

The hsbench tool provides an easy way to measure Hyperscan's
performance for a particular set of patterns and corpus of data
to be scanned.

8 years agong_literal_analysis: use ue2_graph
Justin Viiret [Tue, 29 Nov 2016 03:49:01 +0000 (14:49 +1100)] 
ng_literal_analysis: use ue2_graph

This reduces compile time ~10% on a number of large cases.

8 years agomcclellan: fix printf format specifiers
Justin Viiret [Mon, 28 Nov 2016 05:19:42 +0000 (16:19 +1100)] 
mcclellan: fix printf format specifiers

8 years agorefactor mcclellan runtime
Alex Coyte [Fri, 25 Nov 2016 04:33:27 +0000 (15:33 +1100)] 
refactor mcclellan runtime

1. use u32 to hold the state during runtime to prevent repeated zero extension
2. have a tight small loop for processing characters which breaks when
   something interesting happens

8 years agorose_build_merge: correctly merge NFA outfixes
Justin Viiret [Wed, 23 Nov 2016 04:38:26 +0000 (15:38 +1100)] 
rose_build_merge: correctly merge NFA outfixes

We were not doing our bookkeeping properly for merges where the number
of NFAs was greater than the batch size of 200.

8 years agoRework the creation of startDs edges in fillHolderForLockCheck().
Alex Coyte [Tue, 22 Nov 2016 02:48:00 +0000 (13:48 +1100)] 
Rework the creation of startDs edges in fillHolderForLockCheck().

This prevents clearing of edges created for virtual starts in late regions.

8 years agoprecise dverm fix for 5a32993: handle sign extension correctly
Alex Coyte [Thu, 17 Nov 2016 05:47:20 +0000 (16:47 +1100)] 
precise dverm fix for 5a32993: handle sign extension correctly

8 years agocheck for partial matches at end of dverm
Alex Coyte [Wed, 16 Nov 2016 22:41:23 +0000 (09:41 +1100)] 
check for partial matches at end of dverm

8 years agogough: don't dump int data unless it's requested
Justin Viiret [Tue, 15 Nov 2016 23:56:37 +0000 (10:56 +1100)] 
gough: don't dump int data unless it's requested

8 years agoshufti: slightly faster short shufti operation
Matthew Barr [Tue, 13 Sep 2016 05:07:31 +0000 (15:07 +1000)] 
shufti: slightly faster short shufti operation

It is better to shift the high lane values in an XMM
before then using insert to combine the high and low lanes.

8 years agoFix combine2x128
Matthew Barr [Tue, 13 Sep 2016 05:07:16 +0000 (15:07 +1000)] 
Fix combine2x128

8 years agocorrect dump filenames of som rev engines
Alex Coyte [Thu, 3 Nov 2016 23:59:33 +0000 (10:59 +1100)] 
correct dump filenames of som rev engines

8 years agohave single dump function per engine
Alex Coyte [Tue, 1 Nov 2016 23:36:24 +0000 (10:36 +1100)] 
have single dump function per engine

8 years agonfa_api: remove subtype from dispatch
Alex Coyte [Thu, 27 Oct 2016 22:50:09 +0000 (09:50 +1100)] 
nfa_api: remove subtype from dispatch

8 years agoImplement some ue2_graph functions using TMP rather than friends
Alex Coyte [Thu, 15 Sep 2016 05:44:35 +0000 (15:44 +1000)] 
Implement some ue2_graph functions using TMP rather than friends

This helps work around issues with some compilers

8 years agoallow edge_descriptors to be created from pair<edge_descriptor, bool>
Alex Coyte [Tue, 30 Aug 2016 06:08:49 +0000 (16:08 +1000)] 
allow edge_descriptors to be created from pair<edge_descriptor, bool>

8 years agoIntroduce custom adjacency-list based graph
Alex Coyte [Wed, 24 Aug 2016 06:12:51 +0000 (16:12 +1000)] 
Introduce custom adjacency-list based graph

8 years agoremove unused define and old inlining controls
Alex Coyte [Thu, 27 Oct 2016 03:29:34 +0000 (14:29 +1100)] 
remove unused define and old inlining controls

8 years agouse stable_sort in analysis from 47f53f6; missed review comment
Alex Coyte [Thu, 27 Oct 2016 03:26:00 +0000 (14:26 +1100)] 
use stable_sort in analysis from 47f53f6; missed review comment

8 years agorose_dump: remove stray newline
Justin Viiret [Thu, 13 Oct 2016 22:14:28 +0000 (09:14 +1100)] 
rose_dump: remove stray newline

8 years agogetData256(): data needs to be 32-byte aligned
Justin Viiret [Wed, 12 Oct 2016 03:59:20 +0000 (14:59 +1100)] 
getData256(): data needs to be 32-byte aligned

8 years agosmallwrite: minimize DFAs if they have been pruned
Justin Viiret [Tue, 11 Oct 2016 04:36:16 +0000 (15:36 +1100)] 
smallwrite: minimize DFAs if they have been pruned

8 years agosmallwrite: prune overlong nfa graphs early
Justin Viiret [Tue, 11 Oct 2016 00:39:26 +0000 (11:39 +1100)] 
smallwrite: prune overlong nfa graphs early

8 years agong_limex: add edges in deterministic ordering
Justin Viiret [Tue, 11 Oct 2016 22:53:12 +0000 (09:53 +1100)] 
ng_limex: add edges in deterministic ordering

8 years agong_restructuring: wire start to tops in idx order
Justin Viiret [Tue, 11 Oct 2016 22:49:09 +0000 (09:49 +1100)] 
ng_restructuring: wire start to tops in idx order

8 years agong_equivalence: don't use ptr_vector
Justin Viiret [Tue, 11 Oct 2016 03:03:28 +0000 (14:03 +1100)] 
ng_equivalence: don't use ptr_vector

Switch over ptr_vector<T> to vector<unique_ptr<T>>. This works around
some issues we were seeing with MSVC builds, where the contents of the
ptr_vector were being destroyed when it was returned.

8 years agofix for analysis in previous commit
Alex Coyte [Thu, 6 Oct 2016 23:06:46 +0000 (10:06 +1100)] 
fix for analysis in previous commit

Properly distinguish between vertices that will get set after the cyclic
from vertices that may get set alongside the cyclic

8 years agosimple pass to pick up paths redundant with those from cyclic's succs
Alex Coyte [Thu, 6 Oct 2016 04:54:48 +0000 (15:54 +1100)] 
simple pass to pick up paths redundant with those from cyclic's succs

8 years agorose: call loadLongLiteralState() earlier
Justin Viiret [Thu, 29 Sep 2016 00:29:42 +0000 (10:29 +1000)] 
rose: call loadLongLiteralState() earlier

The ll_buf, ll_buf_nocase buffers must be initialised before anyh path
that could lead to storeLongLiteralState().

8 years agoproperly consider report behaviour when finding start verts
Alex Coyte [Thu, 29 Sep 2016 00:42:43 +0000 (10:42 +1000)] 
properly consider report behaviour when finding start verts

8 years agorose: don't unconditionally init ll_buf etc
Justin Viiret [Wed, 28 Sep 2016 07:17:58 +0000 (17:17 +1000)] 
rose: don't unconditionally init ll_buf etc

This is only necessary (and already always done) if there is a long
literal table.

8 years agoCreate combo tops for trigger limexes
Alex Coyte [Wed, 28 Sep 2016 03:57:24 +0000 (13:57 +1000)] 
Create combo tops for trigger limexes

8 years agoMerge master into develop
Matthew Barr [Fri, 2 Dec 2016 00:21:29 +0000 (11:21 +1100)] 
Merge master into develop

8 years agoBump version number for release v4.3.2
Matthew Barr [Mon, 14 Nov 2016 22:54:28 +0000 (09:54 +1100)] 
Bump version number for release

8 years agoUpdate changelog for 4.3.2
Matthew Barr [Mon, 14 Nov 2016 22:54:04 +0000 (09:54 +1100)] 
Update changelog for 4.3.2

8 years agoAdd the missing degree() in Boost 1.62
Matthew Barr [Wed, 19 Oct 2016 22:49:10 +0000 (09:49 +1100)] 
Add the missing degree() in Boost 1.62

This is a workaround that fixes issue #39

8 years agoremove unused includes
Alex Coyte [Wed, 28 Sep 2016 03:56:15 +0000 (13:56 +1000)] 
remove unused includes

8 years agofdr: reduce confirm size to a u8
Justin Viiret [Tue, 13 Sep 2016 05:52:39 +0000 (15:52 +1000)] 
fdr: reduce confirm size to a u8

Also removes the flexible array member from the LitInfo structure.

8 years agonoodle: correct history req calculation
Justin Viiret [Mon, 19 Sep 2016 01:23:37 +0000 (11:23 +1000)] 
noodle: correct history req calculation

8 years agorose: simplify long lit table, add bloom filter
Justin Viiret [Thu, 22 Sep 2016 03:58:42 +0000 (13:58 +1000)] 
rose: simplify long lit table, add bloom filter

Replaces the original long lit hash table (used in streaming mode) with a
smaller, simpler linear probing approach. Adds a bloom filter in front
of it to reduce time spent on false positives.

Sizing of both the hash table and bloom filter are done based on max
load.

8 years agofdr: move long literal handling into Rose
Justin Viiret [Wed, 7 Sep 2016 05:59:23 +0000 (15:59 +1000)] 
fdr: move long literal handling into Rose

Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.

This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.

8 years agoDisable asserts everywhere when we say so
Matthew Barr [Thu, 15 Sep 2016 06:45:00 +0000 (16:45 +1000)] 
Disable asserts everywhere when we say so

8 years agoUE-3025: There is no need to prune tops from non-triggered graphs
Alex Coyte [Thu, 15 Sep 2016 01:22:29 +0000 (11:22 +1000)] 
UE-3025: There is no need to prune tops from non-triggered graphs

8 years agoextern "C" for mask1bit table
Matthew Barr [Wed, 14 Sep 2016 04:00:43 +0000 (14:00 +1000)] 
extern "C" for mask1bit table

8 years agoallow sets of tops on edges
Alex Coyte [Thu, 1 Sep 2016 04:58:55 +0000 (14:58 +1000)] 
allow sets of tops on edges

8 years agohinted insert operations for flat_set
Alex Coyte [Thu, 1 Sep 2016 04:40:17 +0000 (14:40 +1000)] 
hinted insert operations for flat_set

8 years agoOperator precedence matters
Matthew Barr [Wed, 7 Sep 2016 05:47:26 +0000 (15:47 +1000)] 
Operator precedence matters

8 years agoMSVC prefers the attrib at the beginning
Matthew Barr [Wed, 7 Sep 2016 01:50:00 +0000 (11:50 +1000)] 
MSVC prefers the attrib at the beginning

8 years agocmake: don't be so heavyhanded with flags
Matthew Barr [Wed, 7 Sep 2016 04:09:49 +0000 (14:09 +1000)] 
cmake: don't be so heavyhanded with flags

8 years agorose: move END instruction to start of enum
Justin Viiret [Tue, 13 Sep 2016 00:55:26 +0000 (10:55 +1000)] 
rose: move END instruction to start of enum

Stop overloading END as the last Rose interpreter instruction, use new
sentinel LAST_ROSE_INSTRUCTION for that.

This change will also make it easier to add new instructions without
renumbering END and thus changing all generated bytecodes.

8 years agorose: add CHECK_SINGLE_LOOKAROUND instruction
Xu, Chi [Mon, 5 Sep 2016 04:19:15 +0000 (12:19 +0800)] 
rose: add CHECK_SINGLE_LOOKAROUND instruction

This specialisation is cheaper than the shufti-based variants, so we
prefer it for single character class tests.

8 years agorose: enable generation of shufti32x16 case
Justin Viiret [Wed, 7 Sep 2016 00:33:44 +0000 (10:33 +1000)] 
rose: enable generation of shufti32x16 case

8 years agoremove code preventing firing callbacks in the history buffer
Alex Coyte [Wed, 31 Aug 2016 23:55:53 +0000 (09:55 +1000)] 
remove code preventing firing callbacks in the history buffer

8 years agoAdd short avx2 shufti form
Matthew Barr [Mon, 22 Aug 2016 05:59:32 +0000 (15:59 +1000)] 
Add short avx2 shufti form

8 years agorose: add shufti-based lookaround instructions
Xu, Chi [Wed, 31 Aug 2016 23:48:04 +0000 (07:48 +0800)] 
rose: add shufti-based lookaround instructions

More lookaround specialisations that use the shufti approach.

8 years agolimex: make NFAAccept::squash rel to LimEx base
Justin Viiret [Sun, 4 Sep 2016 23:56:58 +0000 (09:56 +1000)] 
limex: make NFAAccept::squash rel to LimEx base

8 years agolimex: rework accept handling
Justin Viiret [Tue, 30 Aug 2016 04:24:23 +0000 (14:24 +1000)] 
limex: rework accept handling

Rather that iterating over NFAAccept structures and testing individual
bits in the state structure, iterate over the state vector and index
into accept structures.

Adds report list support to this path, unified with the report lists
used for exception handling.

8 years agolimex: fold TESTEOD_REV_FN use into TESTEOD_FN
Justin Viiret [Thu, 1 Sep 2016 06:42:00 +0000 (16:42 +1000)] 
limex: fold TESTEOD_REV_FN use into TESTEOD_FN

8 years agolimex: remove constant do_br arg from TESTEOD_FN
Justin Viiret [Thu, 1 Sep 2016 06:30:08 +0000 (16:30 +1000)] 
limex: remove constant do_br arg from TESTEOD_FN

8 years agorose: move sparse iter cache to RoseEngineBlob
Justin Viiret [Wed, 24 Aug 2016 00:16:12 +0000 (10:16 +1000)] 
rose: move sparse iter cache to RoseEngineBlob

This enables its use for iterators written by instructions.

8 years agohash: add hash_all variadic tpl func, use in rose
Justin Viiret [Tue, 23 Aug 2016 06:12:34 +0000 (16:12 +1000)] 
hash: add hash_all variadic tpl func, use in rose

8 years agorose: decouple build-time program representation
Justin Viiret [Thu, 4 Aug 2016 04:21:51 +0000 (14:21 +1000)] 
rose: decouple build-time program representation

This commit replaces the build-time representation of the Rose
interpreter programs, from a class containing a discriminated union of
the bytecode structures to a class hierarchy of build-time prototypes.

This makes it easier to reason about and manipulate Rose programs during
compilation.

8 years agocontainer.h: include <vector>
Justin Viiret [Mon, 22 Aug 2016 02:37:46 +0000 (12:37 +1000)] 
container.h: include <vector>

8 years agoUE-3019: limex_compile: correctly access the dominator map
Alex Coyte [Wed, 31 Aug 2016 00:50:46 +0000 (10:50 +1000)] 
UE-3019: limex_compile: correctly access the dominator map

8 years agorose: tighten up requirements for catch up
Justin Viiret [Thu, 25 Aug 2016 05:12:28 +0000 (15:12 +1000)] 
rose: tighten up requirements for catch up

We only need to catch up when there is an actual anchored table, not
merely when there are successors of anchored_root in the Rose graph.

8 years agorework load_m128_from_u64a()
Alex Coyte [Mon, 29 Aug 2016 05:07:22 +0000 (15:07 +1000)] 
rework load_m128_from_u64a()

8 years agoLimex: don't not build accel schemes for impossible state sets
Alex Coyte [Fri, 26 Aug 2016 05:19:10 +0000 (15:19 +1000)] 
Limex: don't not build accel schemes for impossible state sets

8 years agoIntroduce a 64-bit LimEx model.
Alex Coyte [Fri, 26 Aug 2016 04:17:41 +0000 (14:17 +1000)] 
Introduce a 64-bit LimEx model.

On 64-bit platforms, the Limex 64 model is implemented in normal GPRs.
On 32-bit platforms, however, 128-bit SSE registers are used for the
runtime implementation.

8 years agodebug: always use %zu in format string for size_t
Justin Viiret [Thu, 25 Aug 2016 03:26:27 +0000 (13:26 +1000)] 
debug: always use %zu in format string for size_t

8 years agolimex_dump: silence gcc large alignment warning
Justin Viiret [Thu, 25 Aug 2016 03:25:01 +0000 (13:25 +1000)] 
limex_dump: silence gcc large alignment warning

Passing mask by const ref silences the warning: "The ABI for passing
parameters with 32-byte alignment has changed in GCC 4.6".

8 years agorose: remove CHECK_LIT_MASK instruction
Justin Viiret [Mon, 8 Aug 2016 06:12:44 +0000 (16:12 +1000)] 
rose: remove CHECK_LIT_MASK instruction

8 years agorose: use lookarounds to implement benefits masks
Justin Viiret [Mon, 8 Aug 2016 05:39:43 +0000 (15:39 +1000)] 
rose: use lookarounds to implement benefits masks

This replaces the CHECK_LIT_MASK instruction.

8 years agorose: add new instruction CHECK_MASK_32
Xu, Chi [Thu, 21 Jul 2016 19:35:53 +0000 (03:35 +0800)] 
rose: add new instruction CHECK_MASK_32

This is a specialisation of the "lookaround" code.

8 years agorose: don't merge large acyclic suffixes
Justin Viiret [Fri, 5 Aug 2016 00:52:24 +0000 (10:52 +1000)] 
rose: don't merge large acyclic suffixes

Check earlier on in mergeSuffixes that we're not proposing to merge
suffixes above our limit from the acyclic merge path.

8 years agoUpdated nfa source file permissions
jason taylor [Sun, 4 Sep 2016 23:25:59 +0000 (19:25 -0400)] 
Updated nfa source file permissions

Resolves issue github issue #37

8 years agoUpdate changelog for 4.3.1 v4.3.1
Matthew Barr [Mon, 29 Aug 2016 03:51:17 +0000 (13:51 +1000)] 
Update changelog for 4.3.1

8 years agoMerge branch develop into master
Matthew Barr [Mon, 29 Aug 2016 01:30:51 +0000 (11:30 +1000)] 
Merge branch develop into master

8 years agoBump version number for release
Matthew Barr [Mon, 29 Aug 2016 01:26:16 +0000 (11:26 +1000)] 
Bump version number for release

8 years agoOnly initialise stream state history when required.
Matthew Barr [Fri, 26 Aug 2016 01:08:29 +0000 (11:08 +1000)] 
Only initialise stream state history when required.

This fixes 01org/hyperscan#30.

8 years agounit test for github issue #30
Matthew Barr [Thu, 25 Aug 2016 22:30:12 +0000 (08:30 +1000)] 
unit test for github issue #30

9 years agoMerge branch develop into master v4.3.0
Matthew Barr [Wed, 24 Aug 2016 04:29:28 +0000 (14:29 +1000)] 
Merge branch develop into master

9 years agoBump version number for release
Matthew Barr [Wed, 24 Aug 2016 04:27:59 +0000 (14:27 +1000)] 
Bump version number for release

9 years agochangelog: updates for 4.3 release
Justin Viiret [Wed, 24 Aug 2016 04:21:57 +0000 (14:21 +1000)] 
changelog: updates for 4.3 release

9 years agoChange SONAME to only use the major version number
Matthew Barr [Mon, 22 Aug 2016 05:54:18 +0000 (15:54 +1000)] 
Change SONAME to only use the major version number

Hyperscan will only break ABI on major version changes, and the SONAME
used for shared library versions should reflect this.

9 years agoRemove problematic debug output
Matthew Barr [Mon, 15 Aug 2016 01:55:08 +0000 (11:55 +1000)] 
Remove problematic debug output

9 years agotamarama: check for match halt in _Q2 as well
Justin Viiret [Sun, 14 Aug 2016 23:58:06 +0000 (09:58 +1000)] 
tamarama: check for match halt in _Q2 as well

9 years agoset an appropriate default value for RoleInfo::score
Alex Coyte [Thu, 11 Aug 2016 03:10:34 +0000 (13:10 +1000)] 
set an appropriate default value for RoleInfo::score

Coverity CID 131843

9 years agoAdd assertion indicating valid range of rank_in_mask's bit param
Alex Coyte [Thu, 11 Aug 2016 03:04:26 +0000 (13:04 +1000)] 
Add assertion indicating valid range of rank_in_mask's bit param

Coverity CID 141632