]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
9 years agofdr: move fdrSize into namespace ue2 (build only)
Justin Viiret [Wed, 27 Apr 2016 05:50:16 +0000 (15:50 +1000)] 
fdr: move fdrSize into namespace ue2 (build only)

9 years agomcclellan compile: more efficient discovery of accel
Alex Coyte [Thu, 14 Apr 2016 02:30:46 +0000 (12:30 +1000)] 
mcclellan compile: more efficient discovery of accel

9 years agonoodle_build: update interface to use hwlmLiteral
Justin Viiret [Fri, 22 Apr 2016 06:09:39 +0000 (16:09 +1000)] 
noodle_build: update interface to use hwlmLiteral

9 years agocmpForward: assume (and assert) p2 is upper-case
Justin Viiret [Thu, 21 Apr 2016 03:39:16 +0000 (13:39 +1000)] 
cmpForward: assume (and assert) p2 is upper-case

9 years agounit: fix up noodle unit tests
Justin Viiret [Thu, 21 Apr 2016 05:07:21 +0000 (15:07 +1000)] 
unit: fix up noodle unit tests

9 years agohwlm_literal: coerce nocase lits to upper-case
Justin Viiret [Thu, 21 Apr 2016 04:17:45 +0000 (14:17 +1000)] 
hwlm_literal: coerce nocase lits to upper-case

9 years agoue2string: bring caseless cmp inline
Justin Viiret [Thu, 21 Apr 2016 03:57:57 +0000 (13:57 +1000)] 
ue2string: bring caseless cmp inline

9 years agonoodle: remove redundant bitmask
Matthew Barr [Thu, 21 Apr 2016 05:50:01 +0000 (15:50 +1000)] 
noodle: remove redundant bitmask

9 years agonoodle: use SSE palignr
Matthew Barr [Thu, 21 Apr 2016 05:39:47 +0000 (15:39 +1000)] 
noodle: use SSE palignr

9 years agong_mcclellan: move logic into base class
Justin Viiret [Wed, 20 Apr 2016 07:11:25 +0000 (17:11 +1000)] 
ng_mcclellan: move logic into base class

Move all of the Automaton logic into Automaton_Base, which is templated
on its StateSet/StateMap types.

9 years agong_haig: move logic into base class
Justin Viiret [Wed, 20 Apr 2016 05:37:17 +0000 (15:37 +1000)] 
ng_haig: move logic into base class

Move all of the Automaton logic into Automaton_Base, which is templated
on its StateSet/StateMap types.

9 years agorose: assert that program offset is sane
Justin Viiret [Wed, 20 Apr 2016 02:41:02 +0000 (12:41 +1000)] 
rose: assert that program offset is sane

9 years agonfa_kind: documentation
Justin Viiret [Tue, 19 Apr 2016 23:47:12 +0000 (09:47 +1000)] 
nfa_kind: documentation

9 years agompv: native report remapping
Justin Viiret [Tue, 19 Apr 2016 05:22:20 +0000 (15:22 +1000)] 
mpv: native report remapping

9 years agocastle, lbr: native report remap
Justin Viiret [Tue, 19 Apr 2016 04:42:45 +0000 (14:42 +1000)] 
castle, lbr: native report remap

9 years agocastle: add nfa kind to CastleProto
Justin Viiret [Tue, 19 Apr 2016 03:51:41 +0000 (13:51 +1000)] 
castle: add nfa kind to CastleProto

9 years agomcclellan, gough: native report remapping
Justin Viiret [Tue, 19 Apr 2016 01:53:10 +0000 (11:53 +1000)] 
mcclellan, gough: native report remapping

9 years agorose: remap reports to program offsets
Justin Viiret [Mon, 18 Apr 2016 04:40:58 +0000 (14:40 +1000)] 
rose: remap reports to program offsets

9 years agoreport_manager: store report -> program mapping
Justin Viiret [Mon, 18 Apr 2016 04:20:45 +0000 (14:20 +1000)] 
report_manager: store report -> program mapping

9 years agoMerge functions no longer need transfer function
Justin Viiret [Mon, 18 Apr 2016 03:21:34 +0000 (13:21 +1000)] 
Merge functions no longer need transfer function

9 years agoRemove OutfixInfo::chained (which meant "is MPV")
Justin Viiret [Mon, 18 Apr 2016 03:12:15 +0000 (13:12 +1000)] 
Remove OutfixInfo::chained (which meant "is MPV")

9 years agoUse a visitor for outfix engine construction
Justin Viiret [Sun, 17 Apr 2016 23:49:13 +0000 (09:49 +1000)] 
Use a visitor for outfix engine construction

9 years agoOutfixInfo: use boost::variant for engines
Justin Viiret [Fri, 15 Apr 2016 06:41:35 +0000 (16:41 +1000)] 
OutfixInfo: use boost::variant for engines

9 years agoWrap MPV puffettes in a struct
Justin Viiret [Fri, 15 Apr 2016 05:28:23 +0000 (15:28 +1000)] 
Wrap MPV puffettes in a struct

9 years agoscratch: delete unused hs_scratch::qNfaState
Justin Viiret [Mon, 18 Apr 2016 01:05:01 +0000 (11:05 +1000)] 
scratch: delete unused hs_scratch::qNfaState

9 years agong_limex_accel: delete unused DAccelScheme ctor
Justin Viiret [Fri, 15 Apr 2016 00:56:43 +0000 (10:56 +1000)] 
ng_limex_accel: delete unused DAccelScheme ctor

9 years agoreport: remove dead function isInternalSomReport
Justin Viiret [Fri, 15 Apr 2016 00:48:09 +0000 (10:48 +1000)] 
report: remove dead function isInternalSomReport

9 years agoreport: remove dead function satisfiesMinLength
Justin Viiret [Fri, 15 Apr 2016 00:45:52 +0000 (10:45 +1000)] 
report: remove dead function satisfiesMinLength

9 years agoFINAL_REPORT: Add specialised instruction
Justin Viiret [Thu, 14 Apr 2016 00:08:36 +0000 (10:08 +1000)] 
FINAL_REPORT: Add specialised instruction

Specialisation of the REPORT instruction that also terminates execution
of the program. Improves performance on programs that generate many
reports.

9 years agoRose: replace internal_report with program
Justin Viiret [Tue, 22 Mar 2016 05:45:09 +0000 (16:45 +1100)] 
Rose: replace internal_report with program

Replace the use of the internal_report structure (for reports from
engines, MPV etc) with the Rose program interpreter.

SOM processing was reworked to use a new som_operation structure that is
embedded in the appropriate instructions.

9 years agoHelp ICC perform a not m128
Matthew Barr [Wed, 13 Apr 2016 02:00:36 +0000 (12:00 +1000)] 
Help ICC perform a not m128

9 years agoshufti: small C++11 fix to silence clang warning
Justin Viiret [Wed, 13 Apr 2016 00:19:39 +0000 (10:19 +1000)] 
shufti: small C++11 fix to silence clang warning

9 years agobitfield: unroll main operators
Alex Coyte [Mon, 11 Apr 2016 23:52:53 +0000 (09:52 +1000)] 
bitfield: unroll main operators

9 years agoCharReach operators inline
Alex Coyte [Mon, 11 Apr 2016 03:47:10 +0000 (13:47 +1000)] 
CharReach operators inline

9 years agounify some accel code/structures between limex and mcclellan
Alex Coyte [Wed, 6 Apr 2016 06:43:32 +0000 (16:43 +1000)] 
unify some accel code/structures between limex and mcclellan

9 years agomore efficent accel path discovery
Alex Coyte [Tue, 5 Apr 2016 04:31:13 +0000 (14:31 +1000)] 
more efficent accel path discovery

9 years agosmallwrite: don't recompile the dfa if prune fails
Alex Coyte [Tue, 5 Apr 2016 01:07:26 +0000 (11:07 +1000)] 
smallwrite: don't recompile the dfa if prune fails

9 years agoraise the limit of strings in double shufti
Alex Coyte [Mon, 4 Apr 2016 05:54:09 +0000 (15:54 +1000)] 
raise the limit of strings in double shufti

9 years agoallow double shufti to share buckets
Alex Coyte [Mon, 4 Apr 2016 01:02:59 +0000 (11:02 +1000)] 
allow double shufti to share buckets

9 years agomake dverm more precise
Alex Coyte [Wed, 30 Mar 2016 00:14:59 +0000 (11:14 +1100)] 
make dverm more precise

9 years agomasked version of dverm
Alex Coyte [Mon, 21 Mar 2016 05:19:46 +0000 (16:19 +1100)] 
masked version of dverm

9 years agorefactoring of double byte offset accel to use paths and add to mcclellan
Alex Coyte [Wed, 9 Mar 2016 22:58:28 +0000 (09:58 +1100)] 
refactoring of double byte offset accel to use paths and add to mcclellan

9 years agolook for normal accel schemes using compressed alpha
Alex Coyte [Fri, 4 Mar 2016 03:20:53 +0000 (14:20 +1100)] 
look for normal accel schemes using compressed alpha

9 years agosimple offset accel for mcclellan start state
Alex Coyte [Tue, 16 Feb 2016 02:37:39 +0000 (13:37 +1100)] 
simple offset accel for mcclellan start state

9 years agodump contents of double shufti masks
Alex Coyte [Sun, 20 Mar 2016 22:38:30 +0000 (09:38 +1100)] 
dump contents of double shufti masks

9 years agomove MultibyteAccelInfo to accelcompile.h
Alex Coyte [Thu, 7 Apr 2016 05:25:05 +0000 (15:25 +1000)] 
move MultibyteAccelInfo to accelcompile.h

9 years agoexhaust: Update interface
Justin Viiret [Thu, 7 Apr 2016 06:33:11 +0000 (16:33 +1000)] 
exhaust: Update interface

 - Only use functions in exhaust.h for valid ekeys
 - Use INVALID_EKEY everywhere (remove dupe END_EXHAUST sentinel)

9 years agoRuntime: be more careful with status bits
Justin Viiret [Tue, 5 Apr 2016 03:25:39 +0000 (13:25 +1000)] 
Runtime: be more careful with status bits

9 years agoRose: use a multibit for the exhaustion vector
Justin Viiret [Mon, 4 Apr 2016 06:04:28 +0000 (16:04 +1000)] 
Rose: use a multibit for the exhaustion vector

Previously, the exhaustion vector was a standard bitvector, which
required an expensive memset() call at init for databases with a large
number of exhaustion keys.

9 years agommbit_all: True if all keys are switched on
Justin Viiret [Mon, 4 Apr 2016 06:19:55 +0000 (16:19 +1000)] 
mmbit_all: True if all keys are switched on

9 years agompv: fire only one report when simple-exhaustible
Justin Viiret [Wed, 6 Apr 2016 05:06:48 +0000 (15:06 +1000)] 
mpv: fire only one report when simple-exhaustible

9 years agompv: use size_t for count, not u32
Justin Viiret [Wed, 6 Apr 2016 04:44:22 +0000 (14:44 +1000)] 
mpv: use size_t for count, not u32

Small cleanup in processReportsForRange.

9 years agoBuild convenience libraries as static libs
Matthew Barr [Wed, 6 Apr 2016 01:09:08 +0000 (11:09 +1000)] 
Build convenience libraries as static libs

9 years agoRemove unused storeu macros
Matthew Barr [Tue, 5 Apr 2016 00:36:50 +0000 (10:36 +1000)] 
Remove unused storeu macros

9 years agoRemove unused loadu2x128
Matthew Barr [Tue, 5 Apr 2016 00:13:34 +0000 (10:13 +1000)] 
Remove unused loadu2x128

9 years agommbit_iterate_bounded: make block_width u64a
Justin Viiret [Tue, 5 Apr 2016 05:18:33 +0000 (15:18 +1000)] 
mmbit_iterate_bounded: make block_width u64a

9 years agoremove HAVE_PTHREAD_H
Matthew Barr [Tue, 29 Mar 2016 05:41:27 +0000 (16:41 +1100)] 
remove HAVE_PTHREAD_H

9 years agoremove ENABLE_TOOLS_THREADS
Matthew Barr [Tue, 29 Mar 2016 05:39:58 +0000 (16:39 +1100)] 
remove ENABLE_TOOLS_THREADS

9 years agomcclellan: Remove the use of state numbering
Justin Viiret [Wed, 23 Mar 2016 00:47:54 +0000 (11:47 +1100)] 
mcclellan: Remove the use of state numbering

McClellan and Haig only need to know which states are not participants
(i.e. unused start, startDs)

9 years agong_find_matches: Simplify and improve performance
Justin Viiret [Thu, 5 Nov 2015 05:32:10 +0000 (16:32 +1100)] 
ng_find_matches: Simplify and improve performance

Improve performance by using bitsets rather than sets of vertex
indices.

9 years agoshufticompile: Remove unused mergeShuftiMask
Justin Viiret [Tue, 15 Mar 2016 05:36:39 +0000 (16:36 +1100)] 
shufticompile: Remove unused mergeShuftiMask

9 years agoscratch: Remove (now unused) tctxtToScratch
Justin Viiret [Tue, 15 Mar 2016 05:31:13 +0000 (16:31 +1100)] 
scratch: Remove (now unused) tctxtToScratch

9 years agocastle/repeat: fix a number of bugs
Justin Viiret [Sun, 13 Mar 2016 22:28:17 +0000 (09:28 +1100)] 
castle/repeat: fix a number of bugs

- Add fits_in_len_bytes assertions for packed stores.
  Corrects the assertion formerly on line 888.

- In exclusive mode, don't overwrite packedCtrlSize with the max of the
  group; each repeat should know how many bytes it is using, even if
  they share the same stream state.

- Ensure that exclusive mode stream state is sized correctly.

9 years agoPrintTo functions for google test and valgrind
Matthew Barr [Tue, 8 Mar 2016 03:53:03 +0000 (14:53 +1100)] 
PrintTo functions for google test and valgrind

9 years agoUE-2919: add termination checks around handleReportInternally
Alex Coyte [Mon, 7 Mar 2016 03:48:01 +0000 (14:48 +1100)] 
UE-2919: add termination checks around handleReportInternally

9 years agomakeReport: remove unused param
Justin Viiret [Fri, 4 Mar 2016 03:46:42 +0000 (14:46 +1100)] 
makeReport: remove unused param

9 years agoRose: be more precise with CATCHUP instruction
Justin Viiret [Fri, 4 Mar 2016 02:54:30 +0000 (13:54 +1100)] 
Rose: be more precise with CATCHUP instruction

- Fix bugs introduced by recent addition of the boundary program. It's
  not safe to do catchup there.
- Only do catchup once per report set, when necessary.

9 years agoAlignedAllocator: simplify and fix for MSVC, C++11
Justin Viiret [Thu, 3 Mar 2016 23:10:26 +0000 (10:10 +1100)] 
AlignedAllocator: simplify and fix for MSVC, C++11

- Simplify thanks to minimal allocator requirements in C++11.
- Add required copy constructor.

9 years agoRose: use program for boundary reports
Justin Viiret [Thu, 3 Mar 2016 04:36:10 +0000 (15:36 +1100)] 
Rose: use program for boundary reports

Use the program to handle report lists at boundaries, rather than the
previous list-of-reports approach.

9 years agoRose: make all NFA callbacks take scratch as ctx
Justin Viiret [Thu, 3 Mar 2016 03:23:08 +0000 (14:23 +1100)] 
Rose: make all NFA callbacks take scratch as ctx

9 years agoRose: make HWLM callbacks take scratch as ctx
Justin Viiret [Thu, 3 Mar 2016 03:14:30 +0000 (14:14 +1100)] 
Rose: make HWLM callbacks take scratch as ctx

9 years agoRose: remove dead function roseHandleMatch
Justin Viiret [Thu, 3 Mar 2016 02:44:54 +0000 (13:44 +1100)] 
Rose: remove dead function roseHandleMatch

9 years agoRose build: move HWLM build code to own file
Justin Viiret [Wed, 2 Mar 2016 02:05:33 +0000 (13:05 +1100)] 
Rose build: move HWLM build code to own file

To reduce the size of rose_build_bytecode.cpp a little, move the code
that deals with HWLM literal tables into its own new file.

9 years agoAdd hs_expression_ext_info() API function
Justin Viiret [Fri, 23 Oct 2015 06:15:24 +0000 (17:15 +1100)] 
Add hs_expression_ext_info() API function

This is a variant of hs_expression_info() that can also accept extended
parameters.

9 years agoRose: use program for all literal matches
Justin Viiret [Wed, 17 Feb 2016 22:45:37 +0000 (09:45 +1100)] 
Rose: use program for all literal matches

Unifies all literal match paths so that the Rose program is used for all
of them. This removes the previous specialised "direct report" and
"multi direct report" paths. Some additional REPORT instruction work was
necessary for this.

Reworked literal construction path at compile time in prep for using
program offsets as literal IDs.

Completely removed the anchored log runtime, which is no longer worth
the extra complexity.

9 years agoFixed some DEBUG_OUTPUT format string.
Mohammad Abdul Awal [Tue, 1 Mar 2016 16:06:00 +0000 (16:06 +0000)] 
Fixed some DEBUG_OUTPUT format string.

9 years agoComponentCondReference: mark ctors explicit
Justin Viiret [Wed, 17 Feb 2016 06:00:30 +0000 (17:00 +1100)] 
ComponentCondReference: mark ctors explicit

9 years agoUse using directives to silence hiding warning
Justin Viiret [Wed, 17 Feb 2016 05:54:45 +0000 (16:54 +1100)] 
Use using directives to silence hiding warning

9 years agoRemove CMake tests for pthread barriers
Justin Viiret [Wed, 11 Nov 2015 23:52:48 +0000 (10:52 +1100)] 
Remove CMake tests for pthread barriers

9 years agoDon't enable -Werror for release builds
Matthew Barr [Tue, 1 Mar 2016 00:04:09 +0000 (11:04 +1100)] 
Don't enable -Werror for release builds

Using -Werror is a very good thing during development, but it has the
potential to break the user's build, for example, if they are using a slightly
different compiler combination to one that has been tested previously.

9 years agoFix typo
Matthew Barr [Mon, 29 Feb 2016 23:56:58 +0000 (10:56 +1100)] 
Fix typo

9 years agoInstall pkgconfig file everywhere but Windows
Matthew Barr [Mon, 29 Feb 2016 23:56:22 +0000 (10:56 +1100)] 
Install pkgconfig file everywhere but Windows

9 years agoAlways init NFAContext::cached_br
Justin Viiret [Mon, 29 Feb 2016 06:06:31 +0000 (17:06 +1100)] 
Always init NFAContext::cached_br

Now that it's on stack, this should be initialised. Silences a warning
from valgrind.

9 years agoCorrect asserts
Matthew Barr [Sun, 28 Feb 2016 21:25:02 +0000 (08:25 +1100)] 
Correct asserts

9 years agoCastle: exclusive analysis for multiple subcastle chunks
Xiang Wang [Fri, 11 Dec 2015 18:27:53 +0000 (13:27 -0500)] 
Castle: exclusive analysis for multiple subcastle chunks

Apply clique analysis to subcastle chunks if the number of
subcastles is large and check the status of each chunk
separately at runtime.

9 years agoRose: further generalise literal dedupe work
Justin Viiret [Sun, 21 Feb 2016 23:58:23 +0000 (10:58 +1100)] 
Rose: further generalise literal dedupe work

9 years agoRose: don't assume roles with >1 lit need dedupe
Justin Viiret [Thu, 18 Feb 2016 05:37:15 +0000 (16:37 +1100)] 
Rose: don't assume roles with >1 lit need dedupe

We only require dedupe for such roles when they have literals that can
arrive simultaneously (i.e. one literal overlaps with the suffix of
another).

9 years agoRose: make DR runtime path external report only
Justin Viiret [Tue, 16 Feb 2016 23:42:41 +0000 (10:42 +1100)] 
Rose: make DR runtime path external report only

9 years agoRose: only use direct reports for external reports
Justin Viiret [Tue, 16 Feb 2016 23:36:22 +0000 (10:36 +1100)] 
Rose: only use direct reports for external reports

9 years agoRose: allow DR literals to share vertices
Justin Viiret [Tue, 16 Feb 2016 06:14:13 +0000 (17:14 +1100)] 
Rose: allow DR literals to share vertices

9 years agoRose: allow direct reports to be aliased
Justin Viiret [Mon, 15 Feb 2016 22:29:07 +0000 (09:29 +1100)] 
Rose: allow direct reports to be aliased

9 years agoRose: assign DRs in allocateFinalLiteralId
Justin Viiret [Mon, 15 Feb 2016 22:25:18 +0000 (09:25 +1100)] 
Rose: assign DRs in allocateFinalLiteralId

Previously, direct reports were allocated earlier; now all final IDs are
assigned in the same place.

9 years agoImproved test for AVX2 support
Matthew Barr [Mon, 15 Feb 2016 04:16:15 +0000 (15:16 +1100)] 
Improved test for AVX2 support

Test for xsave and XMM/YMM registers. Use ICC dynamic feature detection
where available.

9 years agoMake comparison signed (fix warning)
Justin Viiret [Mon, 15 Feb 2016 05:17:58 +0000 (16:17 +1100)] 
Make comparison signed (fix warning)

9 years agoDump: give abs jump targets in output
Justin Viiret [Mon, 15 Feb 2016 02:50:05 +0000 (13:50 +1100)] 
Dump: give abs jump targets in output

9 years agoRose program: Improvements to debug/assertions
Justin Viiret [Mon, 15 Feb 2016 02:41:21 +0000 (13:41 +1100)] 
Rose program: Improvements to debug/assertions

- Add current pc to debug printf.
- Assert that pc doesn't escape the RoseEngine structure.

9 years agoUpdate Rose state layout description
Justin Viiret [Thu, 11 Feb 2016 04:57:56 +0000 (15:57 +1100)] 
Update Rose state layout description

Many of the things in this list no longer existed.

9 years agoSplit CHECK_LEFTFIX into CHECK_{INFIX,PREFIX}
Justin Viiret [Thu, 11 Feb 2016 04:21:18 +0000 (15:21 +1100)] 
Split CHECK_LEFTFIX into CHECK_{INFIX,PREFIX}

9 years agoreduceInfixQueue: factor out any_work check
Justin Viiret [Thu, 11 Feb 2016 03:54:10 +0000 (14:54 +1100)] 
reduceInfixQueue: factor out any_work check

9 years agoRename rosePrefixCheckMiracles to roseLeftfix...
Justin Viiret [Thu, 11 Feb 2016 03:40:04 +0000 (14:40 +1100)] 
Rename rosePrefixCheckMiracles to roseLeftfix...