]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
9 years agosimple pass to pick up paths redundant with those from cyclic's succs
Alex Coyte [Thu, 6 Oct 2016 04:54:48 +0000 (15:54 +1100)] 
simple pass to pick up paths redundant with those from cyclic's succs

9 years agorose: call loadLongLiteralState() earlier
Justin Viiret [Thu, 29 Sep 2016 00:29:42 +0000 (10:29 +1000)] 
rose: call loadLongLiteralState() earlier

The ll_buf, ll_buf_nocase buffers must be initialised before anyh path
that could lead to storeLongLiteralState().

9 years agoproperly consider report behaviour when finding start verts
Alex Coyte [Thu, 29 Sep 2016 00:42:43 +0000 (10:42 +1000)] 
properly consider report behaviour when finding start verts

9 years agorose: don't unconditionally init ll_buf etc
Justin Viiret [Wed, 28 Sep 2016 07:17:58 +0000 (17:17 +1000)] 
rose: don't unconditionally init ll_buf etc

This is only necessary (and already always done) if there is a long
literal table.

9 years agoCreate combo tops for trigger limexes
Alex Coyte [Wed, 28 Sep 2016 03:57:24 +0000 (13:57 +1000)] 
Create combo tops for trigger limexes

9 years agoMerge master into develop
Matthew Barr [Fri, 2 Dec 2016 00:21:29 +0000 (11:21 +1100)] 
Merge master into develop

9 years agoBump version number for release v4.3.2
Matthew Barr [Mon, 14 Nov 2016 22:54:28 +0000 (09:54 +1100)] 
Bump version number for release

9 years agoUpdate changelog for 4.3.2
Matthew Barr [Mon, 14 Nov 2016 22:54:04 +0000 (09:54 +1100)] 
Update changelog for 4.3.2

9 years agoAdd the missing degree() in Boost 1.62
Matthew Barr [Wed, 19 Oct 2016 22:49:10 +0000 (09:49 +1100)] 
Add the missing degree() in Boost 1.62

This is a workaround that fixes issue #39

9 years agoremove unused includes
Alex Coyte [Wed, 28 Sep 2016 03:56:15 +0000 (13:56 +1000)] 
remove unused includes

9 years agofdr: reduce confirm size to a u8
Justin Viiret [Tue, 13 Sep 2016 05:52:39 +0000 (15:52 +1000)] 
fdr: reduce confirm size to a u8

Also removes the flexible array member from the LitInfo structure.

9 years agonoodle: correct history req calculation
Justin Viiret [Mon, 19 Sep 2016 01:23:37 +0000 (11:23 +1000)] 
noodle: correct history req calculation

9 years agorose: simplify long lit table, add bloom filter
Justin Viiret [Thu, 22 Sep 2016 03:58:42 +0000 (13:58 +1000)] 
rose: simplify long lit table, add bloom filter

Replaces the original long lit hash table (used in streaming mode) with a
smaller, simpler linear probing approach. Adds a bloom filter in front
of it to reduce time spent on false positives.

Sizing of both the hash table and bloom filter are done based on max
load.

9 years agofdr: move long literal handling into Rose
Justin Viiret [Wed, 7 Sep 2016 05:59:23 +0000 (15:59 +1000)] 
fdr: move long literal handling into Rose

Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.

This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.

9 years agoDisable asserts everywhere when we say so
Matthew Barr [Thu, 15 Sep 2016 06:45:00 +0000 (16:45 +1000)] 
Disable asserts everywhere when we say so

9 years agoUE-3025: There is no need to prune tops from non-triggered graphs
Alex Coyte [Thu, 15 Sep 2016 01:22:29 +0000 (11:22 +1000)] 
UE-3025: There is no need to prune tops from non-triggered graphs

9 years agoextern "C" for mask1bit table
Matthew Barr [Wed, 14 Sep 2016 04:00:43 +0000 (14:00 +1000)] 
extern "C" for mask1bit table

9 years agoallow sets of tops on edges
Alex Coyte [Thu, 1 Sep 2016 04:58:55 +0000 (14:58 +1000)] 
allow sets of tops on edges

9 years agohinted insert operations for flat_set
Alex Coyte [Thu, 1 Sep 2016 04:40:17 +0000 (14:40 +1000)] 
hinted insert operations for flat_set

9 years agoOperator precedence matters
Matthew Barr [Wed, 7 Sep 2016 05:47:26 +0000 (15:47 +1000)] 
Operator precedence matters

9 years agoMSVC prefers the attrib at the beginning
Matthew Barr [Wed, 7 Sep 2016 01:50:00 +0000 (11:50 +1000)] 
MSVC prefers the attrib at the beginning

9 years agocmake: don't be so heavyhanded with flags
Matthew Barr [Wed, 7 Sep 2016 04:09:49 +0000 (14:09 +1000)] 
cmake: don't be so heavyhanded with flags

9 years agorose: move END instruction to start of enum
Justin Viiret [Tue, 13 Sep 2016 00:55:26 +0000 (10:55 +1000)] 
rose: move END instruction to start of enum

Stop overloading END as the last Rose interpreter instruction, use new
sentinel LAST_ROSE_INSTRUCTION for that.

This change will also make it easier to add new instructions without
renumbering END and thus changing all generated bytecodes.

9 years agorose: add CHECK_SINGLE_LOOKAROUND instruction
Xu, Chi [Mon, 5 Sep 2016 04:19:15 +0000 (12:19 +0800)] 
rose: add CHECK_SINGLE_LOOKAROUND instruction

This specialisation is cheaper than the shufti-based variants, so we
prefer it for single character class tests.

9 years agorose: enable generation of shufti32x16 case
Justin Viiret [Wed, 7 Sep 2016 00:33:44 +0000 (10:33 +1000)] 
rose: enable generation of shufti32x16 case

9 years agoremove code preventing firing callbacks in the history buffer
Alex Coyte [Wed, 31 Aug 2016 23:55:53 +0000 (09:55 +1000)] 
remove code preventing firing callbacks in the history buffer

9 years agoAdd short avx2 shufti form
Matthew Barr [Mon, 22 Aug 2016 05:59:32 +0000 (15:59 +1000)] 
Add short avx2 shufti form

9 years agorose: add shufti-based lookaround instructions
Xu, Chi [Wed, 31 Aug 2016 23:48:04 +0000 (07:48 +0800)] 
rose: add shufti-based lookaround instructions

More lookaround specialisations that use the shufti approach.

9 years agolimex: make NFAAccept::squash rel to LimEx base
Justin Viiret [Sun, 4 Sep 2016 23:56:58 +0000 (09:56 +1000)] 
limex: make NFAAccept::squash rel to LimEx base

9 years agolimex: rework accept handling
Justin Viiret [Tue, 30 Aug 2016 04:24:23 +0000 (14:24 +1000)] 
limex: rework accept handling

Rather that iterating over NFAAccept structures and testing individual
bits in the state structure, iterate over the state vector and index
into accept structures.

Adds report list support to this path, unified with the report lists
used for exception handling.

9 years agolimex: fold TESTEOD_REV_FN use into TESTEOD_FN
Justin Viiret [Thu, 1 Sep 2016 06:42:00 +0000 (16:42 +1000)] 
limex: fold TESTEOD_REV_FN use into TESTEOD_FN

9 years agolimex: remove constant do_br arg from TESTEOD_FN
Justin Viiret [Thu, 1 Sep 2016 06:30:08 +0000 (16:30 +1000)] 
limex: remove constant do_br arg from TESTEOD_FN

9 years agorose: move sparse iter cache to RoseEngineBlob
Justin Viiret [Wed, 24 Aug 2016 00:16:12 +0000 (10:16 +1000)] 
rose: move sparse iter cache to RoseEngineBlob

This enables its use for iterators written by instructions.

9 years agohash: add hash_all variadic tpl func, use in rose
Justin Viiret [Tue, 23 Aug 2016 06:12:34 +0000 (16:12 +1000)] 
hash: add hash_all variadic tpl func, use in rose

9 years agorose: decouple build-time program representation
Justin Viiret [Thu, 4 Aug 2016 04:21:51 +0000 (14:21 +1000)] 
rose: decouple build-time program representation

This commit replaces the build-time representation of the Rose
interpreter programs, from a class containing a discriminated union of
the bytecode structures to a class hierarchy of build-time prototypes.

This makes it easier to reason about and manipulate Rose programs during
compilation.

9 years agocontainer.h: include <vector>
Justin Viiret [Mon, 22 Aug 2016 02:37:46 +0000 (12:37 +1000)] 
container.h: include <vector>

9 years agoUE-3019: limex_compile: correctly access the dominator map
Alex Coyte [Wed, 31 Aug 2016 00:50:46 +0000 (10:50 +1000)] 
UE-3019: limex_compile: correctly access the dominator map

9 years agorose: tighten up requirements for catch up
Justin Viiret [Thu, 25 Aug 2016 05:12:28 +0000 (15:12 +1000)] 
rose: tighten up requirements for catch up

We only need to catch up when there is an actual anchored table, not
merely when there are successors of anchored_root in the Rose graph.

9 years agorework load_m128_from_u64a()
Alex Coyte [Mon, 29 Aug 2016 05:07:22 +0000 (15:07 +1000)] 
rework load_m128_from_u64a()

9 years agoLimex: don't not build accel schemes for impossible state sets
Alex Coyte [Fri, 26 Aug 2016 05:19:10 +0000 (15:19 +1000)] 
Limex: don't not build accel schemes for impossible state sets

9 years agoIntroduce a 64-bit LimEx model.
Alex Coyte [Fri, 26 Aug 2016 04:17:41 +0000 (14:17 +1000)] 
Introduce a 64-bit LimEx model.

On 64-bit platforms, the Limex 64 model is implemented in normal GPRs.
On 32-bit platforms, however, 128-bit SSE registers are used for the
runtime implementation.

9 years agodebug: always use %zu in format string for size_t
Justin Viiret [Thu, 25 Aug 2016 03:26:27 +0000 (13:26 +1000)] 
debug: always use %zu in format string for size_t

9 years agolimex_dump: silence gcc large alignment warning
Justin Viiret [Thu, 25 Aug 2016 03:25:01 +0000 (13:25 +1000)] 
limex_dump: silence gcc large alignment warning

Passing mask by const ref silences the warning: "The ABI for passing
parameters with 32-byte alignment has changed in GCC 4.6".

9 years agorose: remove CHECK_LIT_MASK instruction
Justin Viiret [Mon, 8 Aug 2016 06:12:44 +0000 (16:12 +1000)] 
rose: remove CHECK_LIT_MASK instruction

9 years agorose: use lookarounds to implement benefits masks
Justin Viiret [Mon, 8 Aug 2016 05:39:43 +0000 (15:39 +1000)] 
rose: use lookarounds to implement benefits masks

This replaces the CHECK_LIT_MASK instruction.

9 years agorose: add new instruction CHECK_MASK_32
Xu, Chi [Thu, 21 Jul 2016 19:35:53 +0000 (03:35 +0800)] 
rose: add new instruction CHECK_MASK_32

This is a specialisation of the "lookaround" code.

9 years agorose: don't merge large acyclic suffixes
Justin Viiret [Fri, 5 Aug 2016 00:52:24 +0000 (10:52 +1000)] 
rose: don't merge large acyclic suffixes

Check earlier on in mergeSuffixes that we're not proposing to merge
suffixes above our limit from the acyclic merge path.

9 years agoUpdated nfa source file permissions
jason taylor [Sun, 4 Sep 2016 23:25:59 +0000 (19:25 -0400)] 
Updated nfa source file permissions

Resolves issue github issue #37

9 years agoUpdate changelog for 4.3.1 v4.3.1
Matthew Barr [Mon, 29 Aug 2016 03:51:17 +0000 (13:51 +1000)] 
Update changelog for 4.3.1

9 years agoMerge branch develop into master
Matthew Barr [Mon, 29 Aug 2016 01:30:51 +0000 (11:30 +1000)] 
Merge branch develop into master

9 years agoBump version number for release
Matthew Barr [Mon, 29 Aug 2016 01:26:16 +0000 (11:26 +1000)] 
Bump version number for release

9 years agoOnly initialise stream state history when required.
Matthew Barr [Fri, 26 Aug 2016 01:08:29 +0000 (11:08 +1000)] 
Only initialise stream state history when required.

This fixes 01org/hyperscan#30.

9 years agounit test for github issue #30
Matthew Barr [Thu, 25 Aug 2016 22:30:12 +0000 (08:30 +1000)] 
unit test for github issue #30

9 years agoMerge branch develop into master v4.3.0
Matthew Barr [Wed, 24 Aug 2016 04:29:28 +0000 (14:29 +1000)] 
Merge branch develop into master

9 years agoBump version number for release
Matthew Barr [Wed, 24 Aug 2016 04:27:59 +0000 (14:27 +1000)] 
Bump version number for release

9 years agochangelog: updates for 4.3 release
Justin Viiret [Wed, 24 Aug 2016 04:21:57 +0000 (14:21 +1000)] 
changelog: updates for 4.3 release

9 years agoChange SONAME to only use the major version number
Matthew Barr [Mon, 22 Aug 2016 05:54:18 +0000 (15:54 +1000)] 
Change SONAME to only use the major version number

Hyperscan will only break ABI on major version changes, and the SONAME
used for shared library versions should reflect this.

9 years agoRemove problematic debug output
Matthew Barr [Mon, 15 Aug 2016 01:55:08 +0000 (11:55 +1000)] 
Remove problematic debug output

9 years agotamarama: check for match halt in _Q2 as well
Justin Viiret [Sun, 14 Aug 2016 23:58:06 +0000 (09:58 +1000)] 
tamarama: check for match halt in _Q2 as well

9 years agoset an appropriate default value for RoleInfo::score
Alex Coyte [Thu, 11 Aug 2016 03:10:34 +0000 (13:10 +1000)] 
set an appropriate default value for RoleInfo::score

Coverity CID 131843

9 years agoAdd assertion indicating valid range of rank_in_mask's bit param
Alex Coyte [Thu, 11 Aug 2016 03:04:26 +0000 (13:04 +1000)] 
Add assertion indicating valid range of rank_in_mask's bit param

Coverity CID 141632

9 years agoUse SOURCE_DATE_EPOCH for timestamp if present
Matthew Barr [Wed, 10 Aug 2016 01:14:54 +0000 (11:14 +1000)] 
Use SOURCE_DATE_EPOCH for timestamp if present

The Debian reproducible builds effort suggests using
this environment variable for timestamps.

9 years agounit: Disable warnings
Matthew Barr [Mon, 18 Jul 2016 23:28:46 +0000 (09:28 +1000)] 
unit: Disable warnings

9 years agoGCC 6 warns about unused attributes - disable the warning
Matthew Barr [Mon, 18 Jul 2016 23:23:57 +0000 (09:23 +1000)] 
GCC 6 warns about unused attributes - disable the warning

9 years agocmake: take control of our compiler flags
Matthew Barr [Tue, 28 Jun 2016 01:59:14 +0000 (11:59 +1000)] 
cmake: take control of our compiler flags

9 years agorose: use min of max_offset in left merges
Justin Viiret [Tue, 9 Aug 2016 04:38:58 +0000 (14:38 +1000)] 
rose: use min of max_offset in left merges

Be more careful with max_offset, since we rely on it ofr ANCH history
cases. Also adds tighter assertions.

9 years agorose: ensure anch small block literals have bounds
Justin Viiret [Tue, 9 Aug 2016 05:53:21 +0000 (15:53 +1000)] 
rose: ensure anch small block literals have bounds

9 years agoprogram_runtime: remove commented-out code
Justin Viiret [Thu, 4 Aug 2016 00:40:35 +0000 (10:40 +1000)] 
program_runtime: remove commented-out code

9 years agodfa_build_strat: remove dead pure virtual impl
Justin Viiret [Thu, 4 Aug 2016 00:36:09 +0000 (10:36 +1000)] 
dfa_build_strat: remove dead pure virtual impl

9 years agoremove exceptionMap from LimExNFA
Alex Coyte [Wed, 3 Aug 2016 04:41:34 +0000 (14:41 +1000)] 
remove exceptionMap from LimExNFA

9 years agodfa: adding new Sheng engine
Anatoly Burakov [Thu, 10 Mar 2016 09:57:41 +0000 (09:57 +0000)] 
dfa: adding new Sheng engine

A new shuffle-based DFA engine, complete with acceleration and smallwrite.

9 years agoMcClellan refactor
Anatoly Burakov [Thu, 19 May 2016 13:16:35 +0000 (14:16 +0100)] 
McClellan refactor

Taking dfa strat out of McClellan, to be reused by other DFAs

9 years agonfa: delete largely-unused struct LimExNFABase
Justin Viiret [Tue, 2 Aug 2016 04:49:38 +0000 (14:49 +1000)] 
nfa: delete largely-unused struct LimExNFABase

9 years agoadd a csv version of rose_components
Alex Coyte [Mon, 1 Aug 2016 02:43:13 +0000 (12:43 +1000)] 
add a csv version of rose_components

9 years agoCheck for misaligned memory in compile error code
Matthew Barr [Wed, 27 Jul 2016 06:55:28 +0000 (16:55 +1000)] 
Check for misaligned memory in compile error code

We now check that mem alloc for error message is aligned, and
fail with an appropriate message in the compile error.

9 years agorose: don't leave stray reports when copying the subgraph leading to a report
Alex Coyte [Fri, 29 Jul 2016 05:47:55 +0000 (15:47 +1000)] 
rose: don't leave stray reports when copying the subgraph leading to a report

9 years agodump: offsets for SOM_REL, SOM_ABS reports
Justin Viiret [Mon, 25 Jul 2016 23:45:57 +0000 (09:45 +1000)] 
dump: offsets for SOM_REL, SOM_ABS reports

9 years agoparser: ignore \E that is not preceded by \Q
Justin Viiret [Thu, 28 Jul 2016 02:38:26 +0000 (12:38 +1000)] 
parser: ignore \E that is not preceded by \Q

This conforms to PCRE's behaviour, where an isolated \E that is not
preceded by \Q is ignored.

9 years agong_violet: fix non-det in lookForCleanEarlySplits
Justin Viiret [Wed, 27 Jul 2016 01:19:58 +0000 (11:19 +1000)] 
ng_violet: fix non-det in lookForCleanEarlySplits

9 years agong_violet: trivial typo fixes
Justin Viiret [Wed, 27 Jul 2016 00:42:46 +0000 (10:42 +1000)] 
ng_violet: trivial typo fixes

9 years agong_violet: fix non-determinism in deanchorIfNeeded
Justin Viiret [Wed, 27 Jul 2016 00:23:42 +0000 (10:23 +1000)] 
ng_violet: fix non-determinism in deanchorIfNeeded

9 years agong_violet: fix non-determinism in splitEdgesByCut
Justin Viiret [Tue, 26 Jul 2016 23:29:39 +0000 (09:29 +1000)] 
ng_violet: fix non-determinism in splitEdgesByCut

9 years agoOlder gcc doesn't like shadowing the function
Matthew Barr [Mon, 25 Jul 2016 06:24:30 +0000 (16:24 +1000)] 
Older gcc doesn't like shadowing the function

9 years agoRemove enum typedef
Matthew Barr [Mon, 25 Jul 2016 06:06:37 +0000 (16:06 +1000)] 
Remove enum typedef

9 years agoDon't shadow names
Matthew Barr [Mon, 25 Jul 2016 05:33:40 +0000 (15:33 +1000)] 
Don't shadow names

9 years agong_violet: det. vertex ordering in splitRoseEdge
Justin Viiret [Thu, 21 Jul 2016 04:22:05 +0000 (14:22 +1000)] 
ng_violet: det. vertex ordering in splitRoseEdge

9 years agong_violet: iterate in edge order
Justin Viiret [Thu, 21 Jul 2016 02:57:12 +0000 (12:57 +1000)] 
ng_violet: iterate in edge order

9 years agorose: use dumpString for debug output
Justin Viiret [Thu, 21 Jul 2016 03:32:55 +0000 (13:32 +1000)] 
rose: use dumpString for debug output

9 years agoflood_compile: escape chars in debugging
Justin Viiret [Thu, 21 Jul 2016 03:25:26 +0000 (13:25 +1000)] 
flood_compile: escape chars in debugging

9 years agong_violet: use dumpString for debug output
Justin Viiret [Thu, 21 Jul 2016 02:46:07 +0000 (12:46 +1000)] 
ng_violet: use dumpString for debug output

9 years agorose: clear reports in removeFalsePaths
Justin Viiret [Thu, 21 Jul 2016 00:33:21 +0000 (10:33 +1000)] 
rose: clear reports in removeFalsePaths

9 years agorose: only write out report programs if in use
Justin Viiret [Thu, 14 Jul 2016 06:17:06 +0000 (16:17 +1000)] 
rose: only write out report programs if in use

These programs are only used by output-exposed engines.

9 years agong: ensure that only match states have reports
Justin Viiret [Mon, 18 Jul 2016 02:41:31 +0000 (12:41 +1000)] 
ng: ensure that only match states have reports

Ensure (and assert) that vertices without an edge to {accept, acceptEod}
do not have reports set.

9 years agoteddy: apply poison mask after prep_conf_ work
Justin Viiret [Mon, 18 Jul 2016 01:32:18 +0000 (11:32 +1000)] 
teddy: apply poison mask after prep_conf_ work

This simplifies the code, and removes all the all-ones p_mask uses,
which we were otherwise trusting the optimizer to remove.

9 years agotake mask overhang into account for hwlm accel, float min dist
Alex Coyte [Mon, 18 Jul 2016 01:33:13 +0000 (11:33 +1000)] 
take mask overhang into account for hwlm accel, float min dist

9 years agoviolet: 32bit fix
Alex Coyte [Fri, 15 Jul 2016 04:32:21 +0000 (14:32 +1000)] 
violet: 32bit fix

9 years agoviolet: initial implementation
Alex Coyte [Fri, 15 Jul 2016 03:07:00 +0000 (13:07 +1000)] 
violet: initial implementation

9 years agocompiledump: allow disabling of early graphs for large compiles
Alex Coyte [Wed, 29 Jun 2016 23:57:18 +0000 (09:57 +1000)] 
compiledump: allow disabling of early graphs for large compiles

9 years agorose: expose smwr builder, tidy up engine build
Justin Viiret [Thu, 14 Jul 2016 23:50:08 +0000 (09:50 +1000)] 
rose: expose smwr builder, tidy up engine build

9 years agorose_build_anchored: clean up remapping
Justin Viiret [Thu, 14 Jul 2016 03:34:56 +0000 (13:34 +1000)] 
rose_build_anchored: clean up remapping

Note that there are no EOD reports in the anchored matcher raw_dfas.