]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
9 years agolimex: add CANNOT_DIE flag and loop without test
Justin Viiret [Mon, 3 Apr 2017 01:40:42 +0000 (11:40 +1000)] 
limex: add CANNOT_DIE flag and loop without test

9 years agodefault the move special functions
Alex Coyte [Tue, 4 Apr 2017 05:43:57 +0000 (15:43 +1000)] 
default the move special functions

9 years agoensure that engines added to the leftQueueMap are real rather than lookarounds
Alex Coyte [Tue, 4 Apr 2017 01:38:27 +0000 (11:38 +1000)] 
ensure that engines added to the leftQueueMap are real rather than lookarounds

9 years agodepth: use hash_all()
Justin Viiret [Thu, 30 Mar 2017 06:00:52 +0000 (17:00 +1100)] 
depth: use hash_all()

9 years agodepth: use totally_ordered
Justin Viiret [Thu, 30 Mar 2017 05:57:43 +0000 (16:57 +1100)] 
depth: use totally_ordered

9 years agodepth: make constructor explicit
Justin Viiret [Thu, 30 Mar 2017 05:33:11 +0000 (16:33 +1100)] 
depth: make constructor explicit

9 years agorose_build: reduce size/scope of context objects
Alex Coyte [Thu, 30 Mar 2017 03:34:33 +0000 (14:34 +1100)] 
rose_build: reduce size/scope of context objects

9 years agolimex: move load from input closer to use
Justin Viiret [Thu, 30 Mar 2017 00:18:24 +0000 (11:18 +1100)] 
limex: move load from input closer to use

9 years agorose: add multi-path shufti 16x8, 32x8, 32x16, 64x8 and multi-path lookaround instruc...
Xu, Chi [Thu, 30 Mar 2017 20:37:33 +0000 (04:37 +0800)] 
rose: add multi-path shufti 16x8, 32x8, 32x16, 64x8 and multi-path lookaround instructions.

9 years agoDon't fail if sqlite is not present.
Matthew Barr [Thu, 23 Mar 2017 01:19:35 +0000 (12:19 +1100)] 
Don't fail if sqlite is not present.

9 years agononcopyable: switch over from boost
Justin Viiret [Thu, 30 Mar 2017 03:37:35 +0000 (14:37 +1100)] 
noncopyable: switch over from boost

9 years agononcopyable: add simple ue2::noncopyable class
Justin Viiret [Thu, 30 Mar 2017 03:30:36 +0000 (14:30 +1100)] 
noncopyable: add simple ue2::noncopyable class

9 years agooperators: add simple ue2::totally_ordered class
Justin Viiret [Thu, 30 Mar 2017 02:55:36 +0000 (13:55 +1100)] 
operators: add simple ue2::totally_ordered class

9 years agoWrapper for system intrin header
Matthew Barr [Thu, 30 Mar 2017 02:40:52 +0000 (13:40 +1100)] 
Wrapper for system intrin header

9 years agoCheck compiler architecture flags in one place
Matthew Barr [Wed, 29 Mar 2017 05:39:16 +0000 (16:39 +1100)] 
Check compiler architecture flags in one place

9 years agorose: allow lits that squash to share fragments
Justin Viiret [Wed, 29 Mar 2017 02:10:07 +0000 (13:10 +1100)] 
rose: allow lits that squash to share fragments

9 years agorose: add CLEAR_WORK_DONE instruction
Justin Viiret [Wed, 29 Mar 2017 02:06:26 +0000 (13:06 +1100)] 
rose: add CLEAR_WORK_DONE instruction

Preparatory work for allowing fragments to be shared between literals
that squash groups and those that don't.

9 years agoICC doesn't have a __POPCNT__ macro
Matthew Barr [Wed, 29 Mar 2017 02:04:27 +0000 (13:04 +1100)] 
ICC doesn't have a __POPCNT__ macro

9 years agodump_util: move into namespace ue2
Justin Viiret [Wed, 29 Mar 2017 04:08:16 +0000 (15:08 +1100)] 
dump_util: move into namespace ue2

9 years agosom: move som_type into namespace ue2
Justin Viiret [Wed, 29 Mar 2017 03:33:37 +0000 (14:33 +1100)] 
som: move som_type into namespace ue2

9 years agounit: better output from expr_info unit tests
Justin Viiret [Thu, 23 Mar 2017 23:41:16 +0000 (10:41 +1100)] 
unit: better output from expr_info unit tests

9 years agong_extparam: split up work and do per-comp reduce
Justin Viiret [Thu, 23 Mar 2017 03:10:14 +0000 (14:10 +1100)] 
ng_extparam: split up work and do per-comp reduce

This change breaks extparam processing up into:

 - propagateExtendedParams: propagates min_length, min_offset and
   max_offset into the reports on the graph
 - reduceExtendedParams: runs graph reductions based on extparams

Then, we apply the reduce pass to the whole graph, and later as well to
each component after calc_components.

9 years agorose: only use live reports for dedupe assignment
Justin Viiret [Tue, 28 Mar 2017 00:26:40 +0000 (11:26 +1100)] 
rose: only use live reports for dedupe assignment

9 years agorose_build_groups: allow simple-exh lits to squash
Justin Viiret [Wed, 22 Mar 2017 23:12:12 +0000 (10:12 +1100)] 
rose_build_groups: allow simple-exh lits to squash

9 years agoICC doesn't have a __BMI2__ macro
Matthew Barr [Mon, 27 Mar 2017 23:51:41 +0000 (10:51 +1100)] 
ICC doesn't have a __BMI2__ macro

9 years agommbBuildInitRangePlan: correct offset if initial block is not block 0
Alex Coyte [Wed, 22 Mar 2017 01:43:42 +0000 (12:43 +1100)] 
mmbBuildInitRangePlan: correct offset if initial block is not block 0

9 years agong_region: no need to copy enters
Justin Viiret [Tue, 21 Mar 2017 23:56:59 +0000 (10:56 +1100)] 
ng_region: no need to copy enters

9 years agong_region: clean up and modernise
Justin Viiret [Tue, 21 Mar 2017 22:43:50 +0000 (09:43 +1100)] 
ng_region: clean up and modernise

9 years agong_region: don't realloc exits
Justin Viiret [Tue, 21 Mar 2017 06:19:56 +0000 (17:19 +1100)] 
ng_region: don't realloc exits

9 years agong_region: simplify checkAndAddExitCandidate
Justin Viiret [Tue, 14 Mar 2017 22:37:21 +0000 (09:37 +1100)] 
ng_region: simplify checkAndAddExitCandidate

9 years agong_region: clean up refineExits
Justin Viiret [Tue, 14 Mar 2017 22:29:44 +0000 (09:29 +1100)] 
ng_region: clean up refineExits

9 years agong_region: use flat_sets in exit_info
Justin Viiret [Tue, 14 Mar 2017 07:43:42 +0000 (18:43 +1100)] 
ng_region: use flat_sets in exit_info

9 years agong_calc_components: filter vertices from ug
Justin Viiret [Tue, 21 Mar 2017 02:09:53 +0000 (13:09 +1100)] 
ng_calc_components: filter vertices from ug

9 years agong_undirected: avoid parallel edge at construction
Justin Viiret [Tue, 21 Mar 2017 01:37:52 +0000 (12:37 +1100)] 
ng_undirected: avoid parallel edge at construction

(Rather than using setS for out edges)

9 years agong_undirected: modernize code
Justin Viiret [Tue, 21 Mar 2017 01:31:08 +0000 (12:31 +1100)] 
ng_undirected: modernize code

9 years agong_calc_components: add Grey box control
Justin Viiret [Mon, 20 Mar 2017 23:58:26 +0000 (10:58 +1100)] 
ng_calc_components: add Grey box control

9 years agong_calc_components: rework to move graphs
Justin Viiret [Mon, 20 Mar 2017 23:27:14 +0000 (10:27 +1100)] 
ng_calc_components: rework to move graphs

Rather than cloning graphs for output, rework calc components so that it
moves them.

9 years agong: split NGWrapper into NGHolder, ExpressionInfo
Justin Viiret [Thu, 16 Mar 2017 07:18:34 +0000 (18:18 +1100)] 
ng: split NGWrapper into NGHolder, ExpressionInfo

We now use NGHolder for all graph information, while other expression
properties (report, flag information, etc) go in new class
ExpressionInfo.

9 years agoInstall example source in docdir/examples
Matthew Barr [Sun, 19 Mar 2017 23:00:51 +0000 (10:00 +1100)] 
Install example source in docdir/examples

9 years agoUse GNUInstallDirs for install paths
Matthew Barr [Thu, 2 Mar 2017 04:26:03 +0000 (15:26 +1100)] 
Use GNUInstallDirs for install paths

9 years agocmake: simplify g++ version check
Matthew Barr [Thu, 16 Mar 2017 22:11:53 +0000 (09:11 +1100)] 
cmake: simplify g++ version check

9 years agocmake: override gcc's tuning guess
Matthew Barr [Wed, 15 Mar 2017 05:49:05 +0000 (16:49 +1100)] 
cmake: override gcc's tuning guess

9 years agocmake: move boost tests
Matthew Barr [Thu, 16 Mar 2017 22:25:03 +0000 (09:25 +1100)] 
cmake: move boost tests

9 years agocmake: look for Boost in-tree first
Matthew Barr [Thu, 16 Mar 2017 05:58:42 +0000 (16:58 +1100)] 
cmake: look for Boost in-tree first

During development we tend to use a symlink to the most recent version of
Boost from the Hyperscan source tree, so look there first.

Also, unset some cached results so we recheck when the Boost version changes.

9 years agoUse the default allocator for std::vector
Matthew Barr [Thu, 16 Mar 2017 05:30:01 +0000 (16:30 +1100)] 
Use the default allocator for std::vector

9 years agomake rose responsible for dumping its bytecode
Alex Coyte [Thu, 16 Mar 2017 04:30:33 +0000 (15:30 +1100)] 
make rose responsible for dumping its bytecode

9 years agorose_build_bytecode: make build_context, RoseBuildImpl params const
Alex Coyte [Thu, 16 Mar 2017 01:51:24 +0000 (12:51 +1100)] 
rose_build_bytecode: make build_context, RoseBuildImpl params const

9 years agobuild: More robust temp file handling
Matthew Barr [Wed, 15 Mar 2017 02:27:18 +0000 (13:27 +1100)] 
build: More robust temp file handling

9 years agong_dominators: use a vector for doms internally
Justin Viiret [Mon, 13 Mar 2017 04:18:12 +0000 (15:18 +1100)] 
ng_dominators: use a vector for doms internally

9 years agolimex_compile: don't repeatedly calc dominators
Justin Viiret [Mon, 13 Mar 2017 00:49:19 +0000 (11:49 +1100)] 
limex_compile: don't repeatedly calc dominators

9 years agong_corpus_generator: small fix for MSVC compat
Justin Viiret [Fri, 10 Mar 2017 04:58:11 +0000 (15:58 +1100)] 
ng_corpus_generator: small fix for MSVC compat

9 years agosmall_vector: add header to handle older Boost
Justin Viiret [Fri, 10 Mar 2017 02:43:31 +0000 (13:43 +1100)] 
small_vector: add header to handle older Boost

This provides ue2::small_vector<T, N, Alloc> which will fall back to
std::vector<T, Alloc> where the Boost version is too old or not present.

9 years agong_corpus_generator: restore use of boost::next()
Justin Viiret [Thu, 9 Mar 2017 22:42:57 +0000 (09:42 +1100)] 
ng_corpus_generator: restore use of boost::next()

libc++ checks for forward_iterator_tag in std::next(), which isn't
provided by our (Boost-derived) ue2_graph iterators.

9 years agoUE-3098: add unaligned load for andn without BMI
Wang, Xiang W [Thu, 9 Mar 2017 14:33:29 +0000 (09:33 -0500)] 
UE-3098: add unaligned load for andn without BMI

9 years agong_corpus_generator: use std::next, not boost::next
Justin Viiret [Tue, 7 Mar 2017 23:42:28 +0000 (10:42 +1100)] 
ng_corpus_generator: use std::next, not boost::next

9 years agong_corpus_generator: stop using ptr_vector
Justin Viiret [Tue, 7 Mar 2017 23:04:20 +0000 (10:04 +1100)] 
ng_corpus_generator: stop using ptr_vector

Nowadays we can use vector<unique_ptr<T>>.

9 years agoserialize: tidy
Justin Viiret [Wed, 1 Mar 2017 06:12:45 +0000 (17:12 +1100)] 
serialize: tidy

9 years agounit: modernise test_util
Justin Viiret [Wed, 1 Mar 2017 05:55:24 +0000 (16:55 +1100)] 
unit: modernise test_util

9 years agoserialize: parameterize on pattern as well
Justin Viiret [Wed, 1 Mar 2017 05:07:28 +0000 (16:07 +1100)] 
serialize: parameterize on pattern as well

9 years agoserialize: add vectored mode
Justin Viiret [Wed, 1 Mar 2017 02:12:39 +0000 (13:12 +1100)] 
serialize: add vectored mode

9 years agong_find_matches: use wd.active for accepts too
Justin Viiret [Thu, 9 Mar 2017 00:12:16 +0000 (11:12 +1100)] 
ng_find_matches: use wd.active for accepts too

9 years agong_find_matches: speed up gather...ByDepth
Justin Viiret [Wed, 8 Mar 2017 23:55:22 +0000 (10:55 +1100)] 
ng_find_matches: speed up gather...ByDepth

9 years agogetMatches: simplify
Justin Viiret [Wed, 8 Mar 2017 05:27:41 +0000 (16:27 +1100)] 
getMatches: simplify

9 years agofindMatches: persist working data
Justin Viiret [Wed, 8 Mar 2017 04:57:29 +0000 (15:57 +1100)] 
findMatches: persist working data

9 years agogetSuccessors: reuse a vector<State> for output
Justin Viiret [Wed, 8 Mar 2017 04:46:25 +0000 (15:46 +1100)] 
getSuccessors: reuse a vector<State> for output

9 years agogetActiveStates: return a sorted, uniqued vector
Justin Viiret [Wed, 8 Mar 2017 04:41:06 +0000 (15:41 +1100)] 
getActiveStates: return a sorted, uniqued vector

9 years agocmake: don't build static libs if we don't have to
Matthew Barr [Tue, 7 Mar 2017 05:08:02 +0000 (16:08 +1100)] 
cmake: don't build static libs if we don't have to

9 years agohsbench: don't create the greybox in release build
Matthew Barr [Tue, 7 Mar 2017 05:06:46 +0000 (16:06 +1100)] 
hsbench: don't create the greybox in release build

9 years agong_fuzzy: apply a resource limit to vertex count
Justin Viiret [Wed, 8 Mar 2017 00:36:24 +0000 (11:36 +1100)] 
ng_fuzzy: apply a resource limit to vertex count

9 years agoFDR: delete dead confirm code
Wang, Xiang W [Wed, 8 Mar 2017 11:35:23 +0000 (06:35 -0500)] 
FDR: delete dead confirm code

9 years agong_find_matches: limit how big we're willing to go
Justin Viiret [Tue, 7 Mar 2017 00:47:10 +0000 (11:47 +1100)] 
ng_find_matches: limit how big we're willing to go

Tests which require tracking more than 15K states (including edit
distance states) are very, very slow.

9 years agoUse intrinsic to get correct movq everywhere
Matthew Barr [Mon, 6 Mar 2017 22:58:24 +0000 (09:58 +1100)] 
Use intrinsic to get correct movq everywhere

The real trick here is that _mm_set_epi64x() (note the 'x') takes a 64-bit
value - not a ptr to a 128-bit value like the non-x - so compilers don't
twist themselves in knots with alignment or whatever confuses them.

9 years agorose: guard writeNfaInfo from zero NFAs
Justin Viiret [Mon, 6 Mar 2017 22:41:51 +0000 (09:41 +1100)] 
rose: guard writeNfaInfo from zero NFAs

9 years agong: check can_never_match before validate_fuzzy
Justin Viiret [Mon, 6 Mar 2017 03:15:56 +0000 (14:15 +1100)] 
ng: check can_never_match before validate_fuzzy

9 years agoUse std::move explicitly to avoid ambiguity with boost
Matthew Barr [Fri, 3 Mar 2017 03:53:41 +0000 (14:53 +1100)] 
Use std::move explicitly to avoid ambiguity with boost

9 years agoAdd approximate matching documentation
Anatoly Burakov [Fri, 10 Feb 2017 15:46:29 +0000 (15:46 +0000)] 
Add approximate matching documentation

9 years agoAdd support for approximate matching in other tools
Anatoly Burakov [Fri, 10 Feb 2017 15:45:09 +0000 (15:45 +0000)] 
Add support for approximate matching in other tools

9 years agoAdd support for approximate matching in NFA matcher unit tests
Anatoly Burakov [Fri, 10 Feb 2017 15:44:16 +0000 (15:44 +0000)] 
Add support for approximate matching in NFA matcher unit tests

9 years agoAdd support for approximate matching in ue2collider
Anatoly Burakov [Fri, 10 Feb 2017 15:42:36 +0000 (15:42 +0000)] 
Add support for approximate matching in ue2collider

9 years agoAdding support for compiling approximate matching patterns
Anatoly Burakov [Fri, 10 Feb 2017 15:37:35 +0000 (15:37 +0000)] 
Adding support for compiling approximate matching patterns

Adds new "edit_distance" extparam

9 years agorose: use ROUNDUP_N for alignment
Justin Viiret [Wed, 1 Mar 2017 23:06:29 +0000 (10:06 +1100)] 
rose: use ROUNDUP_N for alignment

9 years agorose: remove unnecessary engine blob size check
Justin Viiret [Wed, 1 Mar 2017 22:51:19 +0000 (09:51 +1100)] 
rose: remove unnecessary engine blob size check

9 years agorose: do state work before writing NfaInfo structs
Justin Viiret [Wed, 1 Mar 2017 22:46:59 +0000 (09:46 +1100)] 
rose: do state work before writing NfaInfo structs

9 years agorose: move matcher bytecode to engine blob
Justin Viiret [Wed, 1 Mar 2017 22:39:23 +0000 (09:39 +1100)] 
rose: move matcher bytecode to engine blob

9 years agorose: consistent naming in updateNfaState
Justin Viiret [Wed, 1 Mar 2017 22:18:28 +0000 (09:18 +1100)] 
rose: consistent naming in updateNfaState

9 years agoengine_blob: add_range() member function
Justin Viiret [Wed, 1 Mar 2017 00:28:14 +0000 (11:28 +1100)] 
engine_blob: add_range() member function

9 years agorose: clean up nfa state alloc
Justin Viiret [Tue, 28 Feb 2017 06:13:47 +0000 (17:13 +1100)] 
rose: clean up nfa state alloc

9 years agorose: update nfa info earlier, in engine blob
Justin Viiret [Tue, 28 Feb 2017 06:03:59 +0000 (17:03 +1100)] 
rose: update nfa info earlier, in engine blob

9 years agorose: move NfaInfo to engine blob
Justin Viiret [Tue, 28 Feb 2017 03:39:37 +0000 (14:39 +1100)] 
rose: move NfaInfo to engine blob

9 years agorose: move leftfix info into engine blob, refactor
Justin Viiret [Tue, 28 Feb 2017 02:55:46 +0000 (13:55 +1100)] 
rose: move leftfix info into engine blob, refactor

9 years agorose: move dkey info to engine blob
Justin Viiret [Tue, 28 Feb 2017 02:46:07 +0000 (13:46 +1100)] 
rose: move dkey info to engine blob

9 years agorose: move lookaround tables to engine blob
Justin Viiret [Tue, 28 Feb 2017 00:18:23 +0000 (11:18 +1100)] 
rose: move lookaround tables to engine blob

9 years agorose: move active leftfix iter to engine blob
Justin Viiret [Mon, 27 Feb 2017 05:55:02 +0000 (16:55 +1100)] 
rose: move active leftfix iter to engine blob

9 years agorose: move SOM reverse NFAs to engine_blob
Justin Viiret [Mon, 27 Feb 2017 05:33:43 +0000 (16:33 +1100)] 
rose: move SOM reverse NFAs to engine_blob

9 years agorose: build RoseEngine prototype for bytecode
Justin Viiret [Thu, 23 Feb 2017 05:36:28 +0000 (16:36 +1100)] 
rose: build RoseEngine prototype for bytecode

Rather than tracking a great many offsets in u32 variables, build them
in a RoseEngine prototype.

9 years agoFDR: front end loop improvement
Wang, Xiang W [Mon, 23 Jan 2017 22:15:40 +0000 (17:15 -0500)] 
FDR: front end loop improvement

9 years agorose: create new ProgramBuild struct
Justin Viiret [Thu, 23 Feb 2017 00:43:49 +0000 (11:43 +1100)] 
rose: create new ProgramBuild struct

9 years agorose: make buildLiteralProgram take one lit_id
Justin Viiret [Wed, 22 Feb 2017 22:52:52 +0000 (09:52 +1100)] 
rose: make buildLiteralProgram take one lit_id

9 years agorose: don't assign fragments for event lits
Justin Viiret [Wed, 22 Feb 2017 22:29:11 +0000 (09:29 +1100)] 
rose: don't assign fragments for event lits

9 years agorose: remove final_id
Justin Viiret [Wed, 22 Feb 2017 05:01:40 +0000 (16:01 +1100)] 
rose: remove final_id