]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
8 years agorose: minor improvements to avoid unneeded program instructions
Alex Coyte [Mon, 24 Apr 2017 00:22:44 +0000 (10:22 +1000)] 
rose: minor improvements to avoid unneeded program instructions

- strip out lonely check handled instructions
- avoid producing programs for empty ghost roles

8 years agorose: minor clean up of catchup
Alex Coyte [Sun, 23 Apr 2017 23:51:58 +0000 (09:51 +1000)] 
rose: minor clean up of catchup

- anchored dfa do not mean that catchup is required
- remove needsCatchup from rose bytecode as catchup is based on interpreter

8 years agobe more selective about generating CLEAR_WORK_DONE instructions
Alex Coyte [Tue, 11 Apr 2017 00:50:16 +0000 (10:50 +1000)] 
be more selective about generating CLEAR_WORK_DONE instructions

8 years agominor clean up of rose_build_bytecode.cpp
Alex Coyte [Fri, 21 Apr 2017 05:46:13 +0000 (15:46 +1000)] 
minor clean up of rose_build_bytecode.cpp

- consistently name functions creating programs as makeFoo()
- replace writeLiteralProgram() with makeFragmentProgram()
- make applyFinalSpecialisation() part of writeProgram(bc, prog)
- seperate users who want to make a program for a single literal and fragments

8 years agoMake the build wrapper less fragile
Matthew Barr [Tue, 18 Apr 2017 01:48:17 +0000 (11:48 +1000)] 
Make the build wrapper less fragile

This script was failing for certain paths (dots, whitespace), and using
sed to replace parts of the command line was overkill. Do less mangling,
and quote command line args.

Fixes 01org/hyperscan#51

8 years agosmallwrite: cope when everything has been pruned
Justin Viiret [Wed, 19 Apr 2017 05:45:35 +0000 (15:45 +1000)] 
smallwrite: cope when everything has been pruned

8 years agomcclellancompile: docs for main compile function
Justin Viiret [Wed, 12 Apr 2017 01:24:30 +0000 (11:24 +1000)] 
mcclellancompile: docs for main compile function

8 years agosmallwrite: prune trie when rose is high quality
Justin Viiret [Tue, 11 Apr 2017 06:13:43 +0000 (16:13 +1000)] 
smallwrite: prune trie when rose is high quality

8 years agosmallwrite: construct DFA states in BFS order
Justin Viiret [Tue, 11 Apr 2017 04:21:02 +0000 (14:21 +1000)] 
smallwrite: construct DFA states in BFS order

8 years agosmallwrite: bfs ordering, refine daddy selection
Justin Viiret [Tue, 11 Apr 2017 03:56:51 +0000 (13:56 +1000)] 
smallwrite: bfs ordering, refine daddy selection

8 years agosmallwrite: use failure map to set dfa daddy state
Justin Viiret [Tue, 11 Apr 2017 03:37:56 +0000 (13:37 +1000)] 
smallwrite: use failure map to set dfa daddy state

8 years agodfa: allow smwr to avoid lengthy daddy recalc
Justin Viiret [Tue, 11 Apr 2017 02:44:20 +0000 (12:44 +1000)] 
dfa: allow smwr to avoid lengthy daddy recalc

8 years agomcsheng_compile: factor out find_daddy_candidates
Justin Viiret [Tue, 18 Apr 2017 06:35:06 +0000 (16:35 +1000)] 
mcsheng_compile: factor out find_daddy_candidates

8 years agomcclellancompile: factor out find_daddy_candidates
Justin Viiret [Tue, 11 Apr 2017 01:36:20 +0000 (11:36 +1000)] 
mcclellancompile: factor out find_daddy_candidates

8 years agodfa: only accel init states from smwr path
Justin Viiret [Tue, 11 Apr 2017 00:56:22 +0000 (10:56 +1000)] 
dfa: only accel init states from smwr path

If the small-write DFA has been built from literals, then we only need
to look for accel states at init.

8 years agodfa: prune_overlong -> clear_deeper_reports
Justin Viiret [Thu, 13 Apr 2017 03:18:33 +0000 (13:18 +1000)] 
dfa: prune_overlong -> clear_deeper_reports

Rather than pruning states, we simply clear reports on states that are
too deep and allow Hopcroft minimisation to reduce the size of the DFA
afterwards.

8 years agodfa_min: clean up and improve minimize code
Justin Viiret [Thu, 13 Apr 2017 06:18:22 +0000 (16:18 +1000)] 
dfa_min: clean up and improve minimize code

8 years agoaccel_dfa_build_strat: use flat_map for small maps
Justin Viiret [Mon, 10 Apr 2017 04:23:00 +0000 (14:23 +1000)] 
accel_dfa_build_strat: use flat_map for small maps

8 years agoaccel_dfa_build_strat: use small_vector for paths
Justin Viiret [Mon, 10 Apr 2017 01:05:06 +0000 (11:05 +1000)] 
accel_dfa_build_strat: use small_vector for paths

8 years agomcclellancompile_util: reduce malloc traffic, tidy
Justin Viiret [Mon, 10 Apr 2017 00:32:37 +0000 (10:32 +1000)] 
mcclellancompile_util: reduce malloc traffic, tidy

8 years agong_limex_accel: fix broken termination condition
Justin Viiret [Mon, 10 Apr 2017 06:39:31 +0000 (16:39 +1000)] 
ng_limex_accel: fix broken termination condition

8 years agoexpressions: add much faster limitToSignatures()
Justin Viiret [Fri, 7 Apr 2017 07:02:00 +0000 (17:02 +1000)] 
expressions: add much faster limitToSignatures()

8 years agoEnsure simd types are aligned to what we assume
Matthew Barr [Fri, 7 Apr 2017 06:40:11 +0000 (16:40 +1000)] 
Ensure simd types are aligned to what we assume

Turns out Clang is sensitive to the location of the alignment attribute.

8 years agong_limex_accel: compile time speedups, tidy up
Justin Viiret [Thu, 6 Apr 2017 03:47:30 +0000 (13:47 +1000)] 
ng_limex_accel: compile time speedups, tidy up

8 years agofind_better_daddy: use flat_set for hinted states
Justin Viiret [Thu, 6 Apr 2017 01:43:56 +0000 (11:43 +1000)] 
find_better_daddy: use flat_set for hinted states

8 years agofind_escape_strings: find outs2_broken faster
Justin Viiret [Thu, 6 Apr 2017 06:14:43 +0000 (16:14 +1000)] 
find_escape_strings: find outs2_broken faster

8 years agodoc: update copyright year
Justin Viiret [Fri, 20 Jan 2017 03:25:50 +0000 (14:25 +1100)] 
doc: update copyright year

8 years agohsbench: documentation
Justin Viiret [Sun, 30 Oct 2016 22:46:41 +0000 (09:46 +1100)] 
hsbench: documentation

8 years agocmake: check for libc++ directly
Matthew Barr [Thu, 6 Apr 2017 00:53:28 +0000 (10:53 +1000)] 
cmake: check for libc++ directly

Rather than trying to infer libc++ use from platform and compiler, just
look for a defined symbol.

8 years agoUse ue2::noncopyable instead of deleting copy ctor
Matthew Barr [Fri, 7 Apr 2017 01:11:53 +0000 (11:11 +1000)] 
Use ue2::noncopyable instead of deleting copy ctor

8 years agowindows: tools are unix only
Matthew Barr [Wed, 22 Mar 2017 21:13:00 +0000 (08:13 +1100)] 
windows: tools are unix only

8 years agosmallwrite: aho-corasick construction for literals
Justin Viiret [Fri, 31 Mar 2017 03:04:44 +0000 (14:04 +1100)] 
smallwrite: aho-corasick construction for literals

8 years agosmallwrite: simple trie experiment
Justin Viiret [Thu, 2 Mar 2017 00:09:27 +0000 (11:09 +1100)] 
smallwrite: simple trie experiment

8 years agordfa_merge: sort esets before alphabet calc
Justin Viiret [Wed, 5 Apr 2017 06:40:07 +0000 (16:40 +1000)] 
rdfa_merge: sort esets before alphabet calc

8 years agorose: fix CHECK_MULTIPATH_LOOKAROUND match difference bug
Xu, Chi [Wed, 5 Apr 2017 20:20:17 +0000 (04:20 +0800)] 
rose: fix CHECK_MULTIPATH_LOOKAROUND match difference bug

8 years agobytecode_ptr: needs <stdexcept> for logic_error
Justin Viiret [Thu, 6 Apr 2017 00:09:34 +0000 (10:09 +1000)] 
bytecode_ptr: needs <stdexcept> for logic_error

8 years agobytecode_ptr: avoid shadowing size()
Justin Viiret [Wed, 5 Apr 2017 22:37:05 +0000 (08:37 +1000)] 
bytecode_ptr: avoid shadowing size()

8 years agobytecode_ptr: add shrink() member function
Justin Viiret [Wed, 5 Apr 2017 06:58:53 +0000 (16:58 +1000)] 
bytecode_ptr: add shrink() member function

8 years agobytecode_ptr: fix shadow/conversion issues (gcc48)
Justin Viiret [Wed, 5 Apr 2017 05:54:52 +0000 (15:54 +1000)] 
bytecode_ptr: fix shadow/conversion issues (gcc48)

8 years agorose: eliminate roseSize, use bytecode_ptr size
Justin Viiret [Wed, 5 Apr 2017 00:29:22 +0000 (10:29 +1000)] 
rose: eliminate roseSize, use bytecode_ptr size

8 years agoalloc: remove aligned_unique_ptr
Justin Viiret [Tue, 4 Apr 2017 01:52:25 +0000 (11:52 +1000)] 
alloc: remove aligned_unique_ptr

8 years agorose_build_long_lit: use bytecode_ptr
Justin Viiret [Tue, 4 Apr 2017 01:50:10 +0000 (11:50 +1000)] 
rose_build_long_lit: use bytecode_ptr

8 years agounit: use bytecode_ptr instead of aligned_unique_ptr
Justin Viiret [Tue, 4 Apr 2017 01:41:50 +0000 (11:41 +1000)] 
unit: use bytecode_ptr instead of aligned_unique_ptr

8 years agorose: use bytecode_ptr for interpreter programs
Justin Viiret [Tue, 4 Apr 2017 01:19:07 +0000 (11:19 +1000)] 
rose: use bytecode_ptr for interpreter programs

8 years agorose_build_bytecode: fix up indentation
Justin Viiret [Tue, 4 Apr 2017 01:06:02 +0000 (11:06 +1000)] 
rose_build_bytecode: fix up indentation

8 years agobytecode_ptr: add make_zeroed_bytecode_ptr
Justin Viiret [Tue, 4 Apr 2017 01:02:33 +0000 (11:02 +1000)] 
bytecode_ptr: add make_zeroed_bytecode_ptr

Rather than always zeroing memory.

8 years agobytecode_ptr updates
Justin Viiret [Tue, 4 Apr 2017 00:51:09 +0000 (10:51 +1000)] 
bytecode_ptr updates

8 years agoteddy: fix forward decl (namespace ue2)
Justin Viiret [Mon, 3 Apr 2017 23:29:41 +0000 (09:29 +1000)] 
teddy: fix forward decl (namespace ue2)

8 years agobytecode_ptr: some small improvements
Justin Viiret [Mon, 3 Apr 2017 07:38:26 +0000 (17:38 +1000)] 
bytecode_ptr: some small improvements

8 years agonfa: switch to using bytecode_ptr<NFA>
Justin Viiret [Mon, 3 Apr 2017 07:21:37 +0000 (17:21 +1000)] 
nfa: switch to using bytecode_ptr<NFA>

8 years agorose_build_anchored: use bytecode_ptr
Justin Viiret [Mon, 3 Apr 2017 06:47:21 +0000 (16:47 +1000)] 
rose_build_anchored: use bytecode_ptr

8 years agohwlm: use bytecode_ptr for HWLM structures
Justin Viiret [Mon, 3 Apr 2017 03:56:49 +0000 (13:56 +1000)] 
hwlm: use bytecode_ptr for HWLM structures

8 years agorose: use bytecode_ptr<RoseEngine>
Justin Viiret [Fri, 31 Mar 2017 02:22:04 +0000 (13:22 +1100)] 
rose: use bytecode_ptr<RoseEngine>

8 years agompvcompile: use bytecode_ptr
Justin Viiret [Fri, 31 Mar 2017 00:25:04 +0000 (11:25 +1100)] 
mpvcompile: use bytecode_ptr

8 years agofdr: use bytecode_ptr in fdr/teddy compilers
Justin Viiret [Mon, 20 Mar 2017 06:26:49 +0000 (17:26 +1100)] 
fdr: use bytecode_ptr in fdr/teddy compilers

8 years agonoodle_build: use bytecode_ptr
Justin Viiret [Mon, 20 Mar 2017 06:18:41 +0000 (17:18 +1100)] 
noodle_build: use bytecode_ptr

8 years agosmallwrite_build: delete smwrSize
Justin Viiret [Mon, 20 Mar 2017 05:31:01 +0000 (16:31 +1100)] 
smallwrite_build: delete smwrSize

8 years agosmallwrite_build: use bytecode_ptr
Justin Viiret [Mon, 20 Mar 2017 05:30:08 +0000 (16:30 +1100)] 
smallwrite_build: use bytecode_ptr

8 years agolimex_compile: use bytecode_ptr
Justin Viiret [Mon, 20 Mar 2017 04:35:50 +0000 (15:35 +1100)] 
limex_compile: use bytecode_ptr

8 years agorose_build_engine_blob: add func for bytecode_ptr
Justin Viiret [Mon, 20 Mar 2017 04:11:44 +0000 (15:11 +1100)] 
rose_build_engine_blob: add func for bytecode_ptr

8 years agofdr_confirm_compile: use bytecode_ptr
Justin Viiret [Mon, 20 Mar 2017 03:51:36 +0000 (14:51 +1100)] 
fdr_confirm_compile: use bytecode_ptr

8 years agofdr: use bytecode_ptr internally
Justin Viiret [Mon, 20 Mar 2017 02:53:40 +0000 (13:53 +1100)] 
fdr: use bytecode_ptr internally

8 years agobytecode_ptr: new smart pointer type
Justin Viiret [Mon, 20 Mar 2017 00:51:15 +0000 (11:51 +1100)] 
bytecode_ptr: new smart pointer type

bytecode_ptr<T> is an aligned_unique_ptr<T> that can be queried for its
size and alignment.

8 years agolimex: more general CANNOT_DIE analysis
Justin Viiret [Tue, 4 Apr 2017 04:36:52 +0000 (14:36 +1000)] 
limex: more general CANNOT_DIE analysis

8 years agolimex: refactor non-accel loops into function
Justin Viiret [Mon, 3 Apr 2017 03:05:17 +0000 (13:05 +1000)] 
limex: refactor non-accel loops into function

8 years agolimex_dump: dump LimEx type and flags in text
Justin Viiret [Mon, 3 Apr 2017 02:30:30 +0000 (12:30 +1000)] 
limex_dump: dump LimEx type and flags in text

8 years agolimex: add CANNOT_DIE flag and loop without test
Justin Viiret [Mon, 3 Apr 2017 01:40:42 +0000 (11:40 +1000)] 
limex: add CANNOT_DIE flag and loop without test

8 years agodefault the move special functions
Alex Coyte [Tue, 4 Apr 2017 05:43:57 +0000 (15:43 +1000)] 
default the move special functions

8 years agoensure that engines added to the leftQueueMap are real rather than lookarounds
Alex Coyte [Tue, 4 Apr 2017 01:38:27 +0000 (11:38 +1000)] 
ensure that engines added to the leftQueueMap are real rather than lookarounds

8 years agodepth: use hash_all()
Justin Viiret [Thu, 30 Mar 2017 06:00:52 +0000 (17:00 +1100)] 
depth: use hash_all()

8 years agodepth: use totally_ordered
Justin Viiret [Thu, 30 Mar 2017 05:57:43 +0000 (16:57 +1100)] 
depth: use totally_ordered

8 years agodepth: make constructor explicit
Justin Viiret [Thu, 30 Mar 2017 05:33:11 +0000 (16:33 +1100)] 
depth: make constructor explicit

8 years agorose_build: reduce size/scope of context objects
Alex Coyte [Thu, 30 Mar 2017 03:34:33 +0000 (14:34 +1100)] 
rose_build: reduce size/scope of context objects

8 years agolimex: move load from input closer to use
Justin Viiret [Thu, 30 Mar 2017 00:18:24 +0000 (11:18 +1100)] 
limex: move load from input closer to use

8 years agorose: add multi-path shufti 16x8, 32x8, 32x16, 64x8 and multi-path lookaround instruc...
Xu, Chi [Thu, 30 Mar 2017 20:37:33 +0000 (04:37 +0800)] 
rose: add multi-path shufti 16x8, 32x8, 32x16, 64x8 and multi-path lookaround instructions.

8 years agoDon't fail if sqlite is not present.
Matthew Barr [Thu, 23 Mar 2017 01:19:35 +0000 (12:19 +1100)] 
Don't fail if sqlite is not present.

8 years agononcopyable: switch over from boost
Justin Viiret [Thu, 30 Mar 2017 03:37:35 +0000 (14:37 +1100)] 
noncopyable: switch over from boost

8 years agononcopyable: add simple ue2::noncopyable class
Justin Viiret [Thu, 30 Mar 2017 03:30:36 +0000 (14:30 +1100)] 
noncopyable: add simple ue2::noncopyable class

8 years agooperators: add simple ue2::totally_ordered class
Justin Viiret [Thu, 30 Mar 2017 02:55:36 +0000 (13:55 +1100)] 
operators: add simple ue2::totally_ordered class

8 years agoWrapper for system intrin header
Matthew Barr [Thu, 30 Mar 2017 02:40:52 +0000 (13:40 +1100)] 
Wrapper for system intrin header

8 years agoCheck compiler architecture flags in one place
Matthew Barr [Wed, 29 Mar 2017 05:39:16 +0000 (16:39 +1100)] 
Check compiler architecture flags in one place

8 years agorose: allow lits that squash to share fragments
Justin Viiret [Wed, 29 Mar 2017 02:10:07 +0000 (13:10 +1100)] 
rose: allow lits that squash to share fragments

8 years agorose: add CLEAR_WORK_DONE instruction
Justin Viiret [Wed, 29 Mar 2017 02:06:26 +0000 (13:06 +1100)] 
rose: add CLEAR_WORK_DONE instruction

Preparatory work for allowing fragments to be shared between literals
that squash groups and those that don't.

8 years agoICC doesn't have a __POPCNT__ macro
Matthew Barr [Wed, 29 Mar 2017 02:04:27 +0000 (13:04 +1100)] 
ICC doesn't have a __POPCNT__ macro

8 years agodump_util: move into namespace ue2
Justin Viiret [Wed, 29 Mar 2017 04:08:16 +0000 (15:08 +1100)] 
dump_util: move into namespace ue2

8 years agosom: move som_type into namespace ue2
Justin Viiret [Wed, 29 Mar 2017 03:33:37 +0000 (14:33 +1100)] 
som: move som_type into namespace ue2

8 years agounit: better output from expr_info unit tests
Justin Viiret [Thu, 23 Mar 2017 23:41:16 +0000 (10:41 +1100)] 
unit: better output from expr_info unit tests

8 years agong_extparam: split up work and do per-comp reduce
Justin Viiret [Thu, 23 Mar 2017 03:10:14 +0000 (14:10 +1100)] 
ng_extparam: split up work and do per-comp reduce

This change breaks extparam processing up into:

 - propagateExtendedParams: propagates min_length, min_offset and
   max_offset into the reports on the graph
 - reduceExtendedParams: runs graph reductions based on extparams

Then, we apply the reduce pass to the whole graph, and later as well to
each component after calc_components.

8 years agorose: only use live reports for dedupe assignment
Justin Viiret [Tue, 28 Mar 2017 00:26:40 +0000 (11:26 +1100)] 
rose: only use live reports for dedupe assignment

8 years agorose_build_groups: allow simple-exh lits to squash
Justin Viiret [Wed, 22 Mar 2017 23:12:12 +0000 (10:12 +1100)] 
rose_build_groups: allow simple-exh lits to squash

8 years agoICC doesn't have a __BMI2__ macro
Matthew Barr [Mon, 27 Mar 2017 23:51:41 +0000 (10:51 +1100)] 
ICC doesn't have a __BMI2__ macro

8 years agommbBuildInitRangePlan: correct offset if initial block is not block 0
Alex Coyte [Wed, 22 Mar 2017 01:43:42 +0000 (12:43 +1100)] 
mmbBuildInitRangePlan: correct offset if initial block is not block 0

8 years agong_region: no need to copy enters
Justin Viiret [Tue, 21 Mar 2017 23:56:59 +0000 (10:56 +1100)] 
ng_region: no need to copy enters

8 years agong_region: clean up and modernise
Justin Viiret [Tue, 21 Mar 2017 22:43:50 +0000 (09:43 +1100)] 
ng_region: clean up and modernise

8 years agong_region: don't realloc exits
Justin Viiret [Tue, 21 Mar 2017 06:19:56 +0000 (17:19 +1100)] 
ng_region: don't realloc exits

8 years agong_region: simplify checkAndAddExitCandidate
Justin Viiret [Tue, 14 Mar 2017 22:37:21 +0000 (09:37 +1100)] 
ng_region: simplify checkAndAddExitCandidate

8 years agong_region: clean up refineExits
Justin Viiret [Tue, 14 Mar 2017 22:29:44 +0000 (09:29 +1100)] 
ng_region: clean up refineExits

8 years agong_region: use flat_sets in exit_info
Justin Viiret [Tue, 14 Mar 2017 07:43:42 +0000 (18:43 +1100)] 
ng_region: use flat_sets in exit_info

8 years agong_calc_components: filter vertices from ug
Justin Viiret [Tue, 21 Mar 2017 02:09:53 +0000 (13:09 +1100)] 
ng_calc_components: filter vertices from ug

8 years agong_undirected: avoid parallel edge at construction
Justin Viiret [Tue, 21 Mar 2017 01:37:52 +0000 (12:37 +1100)] 
ng_undirected: avoid parallel edge at construction

(Rather than using setS for out edges)