]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
8 years agong_undirected: modernize code
Justin Viiret [Tue, 21 Mar 2017 01:31:08 +0000 (12:31 +1100)] 
ng_undirected: modernize code

8 years agong_calc_components: add Grey box control
Justin Viiret [Mon, 20 Mar 2017 23:58:26 +0000 (10:58 +1100)] 
ng_calc_components: add Grey box control

8 years agong_calc_components: rework to move graphs
Justin Viiret [Mon, 20 Mar 2017 23:27:14 +0000 (10:27 +1100)] 
ng_calc_components: rework to move graphs

Rather than cloning graphs for output, rework calc components so that it
moves them.

8 years agong: split NGWrapper into NGHolder, ExpressionInfo
Justin Viiret [Thu, 16 Mar 2017 07:18:34 +0000 (18:18 +1100)] 
ng: split NGWrapper into NGHolder, ExpressionInfo

We now use NGHolder for all graph information, while other expression
properties (report, flag information, etc) go in new class
ExpressionInfo.

8 years agoInstall example source in docdir/examples
Matthew Barr [Sun, 19 Mar 2017 23:00:51 +0000 (10:00 +1100)] 
Install example source in docdir/examples

8 years agoUse GNUInstallDirs for install paths
Matthew Barr [Thu, 2 Mar 2017 04:26:03 +0000 (15:26 +1100)] 
Use GNUInstallDirs for install paths

8 years agocmake: simplify g++ version check
Matthew Barr [Thu, 16 Mar 2017 22:11:53 +0000 (09:11 +1100)] 
cmake: simplify g++ version check

8 years agocmake: override gcc's tuning guess
Matthew Barr [Wed, 15 Mar 2017 05:49:05 +0000 (16:49 +1100)] 
cmake: override gcc's tuning guess

8 years agocmake: move boost tests
Matthew Barr [Thu, 16 Mar 2017 22:25:03 +0000 (09:25 +1100)] 
cmake: move boost tests

8 years agocmake: look for Boost in-tree first
Matthew Barr [Thu, 16 Mar 2017 05:58:42 +0000 (16:58 +1100)] 
cmake: look for Boost in-tree first

During development we tend to use a symlink to the most recent version of
Boost from the Hyperscan source tree, so look there first.

Also, unset some cached results so we recheck when the Boost version changes.

8 years agoUse the default allocator for std::vector
Matthew Barr [Thu, 16 Mar 2017 05:30:01 +0000 (16:30 +1100)] 
Use the default allocator for std::vector

8 years agomake rose responsible for dumping its bytecode
Alex Coyte [Thu, 16 Mar 2017 04:30:33 +0000 (15:30 +1100)] 
make rose responsible for dumping its bytecode

8 years agorose_build_bytecode: make build_context, RoseBuildImpl params const
Alex Coyte [Thu, 16 Mar 2017 01:51:24 +0000 (12:51 +1100)] 
rose_build_bytecode: make build_context, RoseBuildImpl params const

8 years agobuild: More robust temp file handling
Matthew Barr [Wed, 15 Mar 2017 02:27:18 +0000 (13:27 +1100)] 
build: More robust temp file handling

8 years agong_dominators: use a vector for doms internally
Justin Viiret [Mon, 13 Mar 2017 04:18:12 +0000 (15:18 +1100)] 
ng_dominators: use a vector for doms internally

8 years agolimex_compile: don't repeatedly calc dominators
Justin Viiret [Mon, 13 Mar 2017 00:49:19 +0000 (11:49 +1100)] 
limex_compile: don't repeatedly calc dominators

8 years agong_corpus_generator: small fix for MSVC compat
Justin Viiret [Fri, 10 Mar 2017 04:58:11 +0000 (15:58 +1100)] 
ng_corpus_generator: small fix for MSVC compat

8 years agosmall_vector: add header to handle older Boost
Justin Viiret [Fri, 10 Mar 2017 02:43:31 +0000 (13:43 +1100)] 
small_vector: add header to handle older Boost

This provides ue2::small_vector<T, N, Alloc> which will fall back to
std::vector<T, Alloc> where the Boost version is too old or not present.

8 years agong_corpus_generator: restore use of boost::next()
Justin Viiret [Thu, 9 Mar 2017 22:42:57 +0000 (09:42 +1100)] 
ng_corpus_generator: restore use of boost::next()

libc++ checks for forward_iterator_tag in std::next(), which isn't
provided by our (Boost-derived) ue2_graph iterators.

8 years agoUE-3098: add unaligned load for andn without BMI
Wang, Xiang W [Thu, 9 Mar 2017 14:33:29 +0000 (09:33 -0500)] 
UE-3098: add unaligned load for andn without BMI

8 years agong_corpus_generator: use std::next, not boost::next
Justin Viiret [Tue, 7 Mar 2017 23:42:28 +0000 (10:42 +1100)] 
ng_corpus_generator: use std::next, not boost::next

8 years agong_corpus_generator: stop using ptr_vector
Justin Viiret [Tue, 7 Mar 2017 23:04:20 +0000 (10:04 +1100)] 
ng_corpus_generator: stop using ptr_vector

Nowadays we can use vector<unique_ptr<T>>.

8 years agoserialize: tidy
Justin Viiret [Wed, 1 Mar 2017 06:12:45 +0000 (17:12 +1100)] 
serialize: tidy

8 years agounit: modernise test_util
Justin Viiret [Wed, 1 Mar 2017 05:55:24 +0000 (16:55 +1100)] 
unit: modernise test_util

8 years agoserialize: parameterize on pattern as well
Justin Viiret [Wed, 1 Mar 2017 05:07:28 +0000 (16:07 +1100)] 
serialize: parameterize on pattern as well

8 years agoserialize: add vectored mode
Justin Viiret [Wed, 1 Mar 2017 02:12:39 +0000 (13:12 +1100)] 
serialize: add vectored mode

8 years agong_find_matches: use wd.active for accepts too
Justin Viiret [Thu, 9 Mar 2017 00:12:16 +0000 (11:12 +1100)] 
ng_find_matches: use wd.active for accepts too

8 years agong_find_matches: speed up gather...ByDepth
Justin Viiret [Wed, 8 Mar 2017 23:55:22 +0000 (10:55 +1100)] 
ng_find_matches: speed up gather...ByDepth

8 years agogetMatches: simplify
Justin Viiret [Wed, 8 Mar 2017 05:27:41 +0000 (16:27 +1100)] 
getMatches: simplify

8 years agofindMatches: persist working data
Justin Viiret [Wed, 8 Mar 2017 04:57:29 +0000 (15:57 +1100)] 
findMatches: persist working data

8 years agogetSuccessors: reuse a vector<State> for output
Justin Viiret [Wed, 8 Mar 2017 04:46:25 +0000 (15:46 +1100)] 
getSuccessors: reuse a vector<State> for output

8 years agogetActiveStates: return a sorted, uniqued vector
Justin Viiret [Wed, 8 Mar 2017 04:41:06 +0000 (15:41 +1100)] 
getActiveStates: return a sorted, uniqued vector

8 years agocmake: don't build static libs if we don't have to
Matthew Barr [Tue, 7 Mar 2017 05:08:02 +0000 (16:08 +1100)] 
cmake: don't build static libs if we don't have to

8 years agohsbench: don't create the greybox in release build
Matthew Barr [Tue, 7 Mar 2017 05:06:46 +0000 (16:06 +1100)] 
hsbench: don't create the greybox in release build

8 years agong_fuzzy: apply a resource limit to vertex count
Justin Viiret [Wed, 8 Mar 2017 00:36:24 +0000 (11:36 +1100)] 
ng_fuzzy: apply a resource limit to vertex count

8 years agoFDR: delete dead confirm code
Wang, Xiang W [Wed, 8 Mar 2017 11:35:23 +0000 (06:35 -0500)] 
FDR: delete dead confirm code

8 years agong_find_matches: limit how big we're willing to go
Justin Viiret [Tue, 7 Mar 2017 00:47:10 +0000 (11:47 +1100)] 
ng_find_matches: limit how big we're willing to go

Tests which require tracking more than 15K states (including edit
distance states) are very, very slow.

8 years agoUse intrinsic to get correct movq everywhere
Matthew Barr [Mon, 6 Mar 2017 22:58:24 +0000 (09:58 +1100)] 
Use intrinsic to get correct movq everywhere

The real trick here is that _mm_set_epi64x() (note the 'x') takes a 64-bit
value - not a ptr to a 128-bit value like the non-x - so compilers don't
twist themselves in knots with alignment or whatever confuses them.

8 years agorose: guard writeNfaInfo from zero NFAs
Justin Viiret [Mon, 6 Mar 2017 22:41:51 +0000 (09:41 +1100)] 
rose: guard writeNfaInfo from zero NFAs

8 years agong: check can_never_match before validate_fuzzy
Justin Viiret [Mon, 6 Mar 2017 03:15:56 +0000 (14:15 +1100)] 
ng: check can_never_match before validate_fuzzy

8 years agoUse std::move explicitly to avoid ambiguity with boost
Matthew Barr [Fri, 3 Mar 2017 03:53:41 +0000 (14:53 +1100)] 
Use std::move explicitly to avoid ambiguity with boost

8 years agoAdd approximate matching documentation
Anatoly Burakov [Fri, 10 Feb 2017 15:46:29 +0000 (15:46 +0000)] 
Add approximate matching documentation

8 years agoAdd support for approximate matching in other tools
Anatoly Burakov [Fri, 10 Feb 2017 15:45:09 +0000 (15:45 +0000)] 
Add support for approximate matching in other tools

8 years agoAdd support for approximate matching in NFA matcher unit tests
Anatoly Burakov [Fri, 10 Feb 2017 15:44:16 +0000 (15:44 +0000)] 
Add support for approximate matching in NFA matcher unit tests

8 years agoAdd support for approximate matching in ue2collider
Anatoly Burakov [Fri, 10 Feb 2017 15:42:36 +0000 (15:42 +0000)] 
Add support for approximate matching in ue2collider

8 years agoAdding support for compiling approximate matching patterns
Anatoly Burakov [Fri, 10 Feb 2017 15:37:35 +0000 (15:37 +0000)] 
Adding support for compiling approximate matching patterns

Adds new "edit_distance" extparam

8 years agorose: use ROUNDUP_N for alignment
Justin Viiret [Wed, 1 Mar 2017 23:06:29 +0000 (10:06 +1100)] 
rose: use ROUNDUP_N for alignment

8 years agorose: remove unnecessary engine blob size check
Justin Viiret [Wed, 1 Mar 2017 22:51:19 +0000 (09:51 +1100)] 
rose: remove unnecessary engine blob size check

8 years agorose: do state work before writing NfaInfo structs
Justin Viiret [Wed, 1 Mar 2017 22:46:59 +0000 (09:46 +1100)] 
rose: do state work before writing NfaInfo structs

8 years agorose: move matcher bytecode to engine blob
Justin Viiret [Wed, 1 Mar 2017 22:39:23 +0000 (09:39 +1100)] 
rose: move matcher bytecode to engine blob

8 years agorose: consistent naming in updateNfaState
Justin Viiret [Wed, 1 Mar 2017 22:18:28 +0000 (09:18 +1100)] 
rose: consistent naming in updateNfaState

8 years agoengine_blob: add_range() member function
Justin Viiret [Wed, 1 Mar 2017 00:28:14 +0000 (11:28 +1100)] 
engine_blob: add_range() member function

8 years agorose: clean up nfa state alloc
Justin Viiret [Tue, 28 Feb 2017 06:13:47 +0000 (17:13 +1100)] 
rose: clean up nfa state alloc

8 years agorose: update nfa info earlier, in engine blob
Justin Viiret [Tue, 28 Feb 2017 06:03:59 +0000 (17:03 +1100)] 
rose: update nfa info earlier, in engine blob

8 years agorose: move NfaInfo to engine blob
Justin Viiret [Tue, 28 Feb 2017 03:39:37 +0000 (14:39 +1100)] 
rose: move NfaInfo to engine blob

8 years agorose: move leftfix info into engine blob, refactor
Justin Viiret [Tue, 28 Feb 2017 02:55:46 +0000 (13:55 +1100)] 
rose: move leftfix info into engine blob, refactor

8 years agorose: move dkey info to engine blob
Justin Viiret [Tue, 28 Feb 2017 02:46:07 +0000 (13:46 +1100)] 
rose: move dkey info to engine blob

8 years agorose: move lookaround tables to engine blob
Justin Viiret [Tue, 28 Feb 2017 00:18:23 +0000 (11:18 +1100)] 
rose: move lookaround tables to engine blob

8 years agorose: move active leftfix iter to engine blob
Justin Viiret [Mon, 27 Feb 2017 05:55:02 +0000 (16:55 +1100)] 
rose: move active leftfix iter to engine blob

8 years agorose: move SOM reverse NFAs to engine_blob
Justin Viiret [Mon, 27 Feb 2017 05:33:43 +0000 (16:33 +1100)] 
rose: move SOM reverse NFAs to engine_blob

8 years agorose: build RoseEngine prototype for bytecode
Justin Viiret [Thu, 23 Feb 2017 05:36:28 +0000 (16:36 +1100)] 
rose: build RoseEngine prototype for bytecode

Rather than tracking a great many offsets in u32 variables, build them
in a RoseEngine prototype.

8 years agoFDR: front end loop improvement
Wang, Xiang W [Mon, 23 Jan 2017 22:15:40 +0000 (17:15 -0500)] 
FDR: front end loop improvement

8 years agorose: create new ProgramBuild struct
Justin Viiret [Thu, 23 Feb 2017 00:43:49 +0000 (11:43 +1100)] 
rose: create new ProgramBuild struct

8 years agorose: make buildLiteralProgram take one lit_id
Justin Viiret [Wed, 22 Feb 2017 22:52:52 +0000 (09:52 +1100)] 
rose: make buildLiteralProgram take one lit_id

8 years agorose: don't assign fragments for event lits
Justin Viiret [Wed, 22 Feb 2017 22:29:11 +0000 (09:29 +1100)] 
rose: don't assign fragments for event lits

8 years agorose: remove final_id
Justin Viiret [Wed, 22 Feb 2017 05:01:40 +0000 (16:01 +1100)] 
rose: remove final_id

8 years agorose: new program construction code
Justin Viiret [Wed, 22 Feb 2017 03:48:23 +0000 (14:48 +1100)] 
rose: new program construction code

8 years agorose_build_bytecode: clean up findEdgesByLiteral()
Justin Viiret [Wed, 22 Feb 2017 01:46:52 +0000 (12:46 +1100)] 
rose_build_bytecode: clean up findEdgesByLiteral()

8 years agorose_build_bytecode: less final_id
Justin Viiret [Tue, 21 Feb 2017 23:56:26 +0000 (10:56 +1100)] 
rose_build_bytecode: less final_id

8 years agorose: eliminate global final to fragment map
Justin Viiret [Tue, 21 Feb 2017 23:28:40 +0000 (10:28 +1100)] 
rose: eliminate global final to fragment map

8 years agorose dump: simplify
Justin Viiret [Tue, 21 Feb 2017 23:19:56 +0000 (10:19 +1100)] 
rose dump: simplify

8 years agorose: tidy
Justin Viiret [Tue, 21 Feb 2017 23:12:56 +0000 (10:12 +1100)] 
rose: tidy

8 years agorose_build_matchers: simplify literal build
Justin Viiret [Tue, 21 Feb 2017 22:53:09 +0000 (09:53 +1100)] 
rose_build_matchers: simplify literal build

8 years agorose: dump support for delay programs
Justin Viiret [Thu, 16 Feb 2017 04:39:11 +0000 (15:39 +1100)] 
rose: dump support for delay programs

8 years agocontainer: allow sort_and_unique to have a comparator
Justin Viiret [Thu, 16 Feb 2017 00:03:05 +0000 (11:03 +1100)] 
container: allow sort_and_unique to have a comparator

8 years agorose: reorganise delay program generation
Justin Viiret [Wed, 15 Feb 2017 23:47:24 +0000 (10:47 +1100)] 
rose: reorganise delay program generation

8 years agorose: dedupe anch programs and RECORD_ANCHOREDs
Justin Viiret [Wed, 15 Feb 2017 22:19:44 +0000 (09:19 +1100)] 
rose: dedupe anch programs and RECORD_ANCHOREDs

8 years agorose: don't emit RECORD_ANCHORED in anchored progs
Justin Viiret [Wed, 15 Feb 2017 03:03:30 +0000 (14:03 +1100)] 
rose: don't emit RECORD_ANCHORED in anchored progs

8 years agorose: rearrange anchored program generation
Justin Viiret [Wed, 15 Feb 2017 02:54:09 +0000 (13:54 +1100)] 
rose: rearrange anchored program generation

8 years agorose: reduce delay program dep on final_id
Justin Viiret [Tue, 14 Feb 2017 00:11:10 +0000 (11:11 +1100)] 
rose: reduce delay program dep on final_id

8 years agorose: remove now-unused anchored_base_id
Justin Viiret [Mon, 13 Feb 2017 23:23:53 +0000 (10:23 +1100)] 
rose: remove now-unused anchored_base_id

8 years agorose: reduce anchored program dep on final_id
Justin Viiret [Mon, 13 Feb 2017 23:19:25 +0000 (10:19 +1100)] 
rose: reduce anchored program dep on final_id

We only need to build anchored programs for cases where a
RECORD_ANCHORED instruction has been generated, and we can key those
directly rather than using final_id.

8 years agorose: build fragments directly
Justin Viiret [Mon, 13 Feb 2017 22:14:58 +0000 (09:14 +1100)] 
rose: build fragments directly

8 years agorose: use fragment ids earlier for anchored dfas
Justin Viiret [Mon, 13 Feb 2017 05:41:08 +0000 (16:41 +1100)] 
rose: use fragment ids earlier for anchored dfas

8 years agorose: write fragment ids into literal_info
Justin Viiret [Mon, 13 Feb 2017 04:54:16 +0000 (15:54 +1100)] 
rose: write fragment ids into literal_info

8 years agorose_build_impl: fix header guard
Justin Viiret [Mon, 13 Feb 2017 04:38:05 +0000 (15:38 +1100)] 
rose_build_impl: fix header guard

8 years agorose_build_anchored: tbi->build
Justin Viiret [Mon, 13 Feb 2017 04:35:38 +0000 (15:35 +1100)] 
rose_build_anchored: tbi->build

8 years agodeterminisation: use unordered_map to hold state set -> id mapping
Alex Coyte [Mon, 13 Feb 2017 22:56:43 +0000 (09:56 +1100)] 
determinisation: use unordered_map to hold state set -> id mapping

8 years agofdr: add grey box control for flood detection
Justin Viiret [Fri, 10 Feb 2017 02:04:22 +0000 (13:04 +1100)] 
fdr: add grey box control for flood detection

8 years agoClear upper half of AVX register before libm call
Matthew Barr [Fri, 10 Feb 2017 00:29:42 +0000 (11:29 +1100)] 
Clear upper half of AVX register before libm call

Clearing the upper half of the AVX register is required before calling SSE
code to avoid AVX-to-SSE transition penalties.

8 years agoparser: handle "control verbs" without close paren
Justin Viiret [Fri, 3 Feb 2017 00:39:06 +0000 (11:39 +1100)] 
parser: handle "control verbs" without close paren

8 years agoparser: check for std::out_of_range from stoul
Justin Viiret [Fri, 3 Feb 2017 00:17:47 +0000 (11:17 +1100)] 
parser: check for std::out_of_range from stoul

8 years agorose: only dump delay rebuild table if streaming
Justin Viiret [Thu, 2 Feb 2017 22:40:00 +0000 (09:40 +1100)] 
rose: only dump delay rebuild table if streaming

8 years agoparser: use control_verb parser inline
Justin Viiret [Thu, 2 Feb 2017 04:49:26 +0000 (15:49 +1100)] 
parser: use control_verb parser inline

8 years agoparser: use stoul(), not strtol()
Justin Viiret [Thu, 2 Feb 2017 04:01:30 +0000 (15:01 +1100)] 
parser: use stoul(), not strtol()

8 years agoparser: switch to using char* pointers
Justin Viiret [Thu, 2 Feb 2017 02:56:30 +0000 (13:56 +1100)] 
parser: switch to using char* pointers

8 years agoparser: add initial parser for control verbs
Justin Viiret [Thu, 12 Jan 2017 01:35:54 +0000 (12:35 +1100)] 
parser: add initial parser for control verbs

This more reliably handles control verbs like (*UTF8) that can only
happen at the start of the pattern, and allows them in any ordering.

8 years agorose: delete dead code for cloneVertex
Justin Viiret [Wed, 1 Feb 2017 23:35:40 +0000 (10:35 +1100)] 
rose: delete dead code for cloneVertex

8 years agong_literal_analysis: refactor splitOffLeadingLiteral
Justin Viiret [Wed, 1 Feb 2017 23:33:20 +0000 (10:33 +1100)] 
ng_literal_analysis: refactor splitOffLeadingLiteral

8 years agodead code: remove splitOffAnchoredLeadingLiteral
Justin Viiret [Wed, 1 Feb 2017 23:26:11 +0000 (10:26 +1100)] 
dead code: remove splitOffAnchoredLeadingLiteral