]>
git.ipfire.org Git - thirdparty/vectorscan.git/log
Justin Viiret [Fri, 4 Mar 2016 02:54:30 +0000 (13:54 +1100)]
Rose: be more precise with CATCHUP instruction
- Fix bugs introduced by recent addition of the boundary program. It's
not safe to do catchup there.
- Only do catchup once per report set, when necessary.
Justin Viiret [Thu, 3 Mar 2016 23:10:26 +0000 (10:10 +1100)]
AlignedAllocator: simplify and fix for MSVC, C++11
- Simplify thanks to minimal allocator requirements in C++11.
- Add required copy constructor.
Justin Viiret [Thu, 3 Mar 2016 04:36:10 +0000 (15:36 +1100)]
Rose: use program for boundary reports
Use the program to handle report lists at boundaries, rather than the
previous list-of-reports approach.
Justin Viiret [Thu, 3 Mar 2016 03:23:08 +0000 (14:23 +1100)]
Rose: make all NFA callbacks take scratch as ctx
Justin Viiret [Thu, 3 Mar 2016 03:14:30 +0000 (14:14 +1100)]
Rose: make HWLM callbacks take scratch as ctx
Justin Viiret [Thu, 3 Mar 2016 02:44:54 +0000 (13:44 +1100)]
Rose: remove dead function roseHandleMatch
Justin Viiret [Wed, 2 Mar 2016 02:05:33 +0000 (13:05 +1100)]
Rose build: move HWLM build code to own file
To reduce the size of rose_build_bytecode.cpp a little, move the code
that deals with HWLM literal tables into its own new file.
Justin Viiret [Fri, 23 Oct 2015 06:15:24 +0000 (17:15 +1100)]
Add hs_expression_ext_info() API function
This is a variant of hs_expression_info() that can also accept extended
parameters.
Justin Viiret [Wed, 17 Feb 2016 22:45:37 +0000 (09:45 +1100)]
Rose: use program for all literal matches
Unifies all literal match paths so that the Rose program is used for all
of them. This removes the previous specialised "direct report" and
"multi direct report" paths. Some additional REPORT instruction work was
necessary for this.
Reworked literal construction path at compile time in prep for using
program offsets as literal IDs.
Completely removed the anchored log runtime, which is no longer worth
the extra complexity.
Mohammad Abdul Awal [Tue, 1 Mar 2016 16:06:00 +0000 (16:06 +0000)]
Fixed some DEBUG_OUTPUT format string.
Justin Viiret [Wed, 17 Feb 2016 06:00:30 +0000 (17:00 +1100)]
ComponentCondReference: mark ctors explicit
Justin Viiret [Wed, 17 Feb 2016 05:54:45 +0000 (16:54 +1100)]
Use using directives to silence hiding warning
Justin Viiret [Wed, 11 Nov 2015 23:52:48 +0000 (10:52 +1100)]
Remove CMake tests for pthread barriers
Matthew Barr [Tue, 1 Mar 2016 00:04:09 +0000 (11:04 +1100)]
Don't enable -Werror for release builds
Using -Werror is a very good thing during development, but it has the
potential to break the user's build, for example, if they are using a slightly
different compiler combination to one that has been tested previously.
Matthew Barr [Mon, 29 Feb 2016 23:56:58 +0000 (10:56 +1100)]
Fix typo
Matthew Barr [Mon, 29 Feb 2016 23:56:22 +0000 (10:56 +1100)]
Install pkgconfig file everywhere but Windows
Justin Viiret [Mon, 29 Feb 2016 06:06:31 +0000 (17:06 +1100)]
Always init NFAContext::cached_br
Now that it's on stack, this should be initialised. Silences a warning
from valgrind.
Matthew Barr [Sun, 28 Feb 2016 21:25:02 +0000 (08:25 +1100)]
Correct asserts
Xiang Wang [Fri, 11 Dec 2015 18:27:53 +0000 (13:27 -0500)]
Castle: exclusive analysis for multiple subcastle chunks
Apply clique analysis to subcastle chunks if the number of
subcastles is large and check the status of each chunk
separately at runtime.
Justin Viiret [Sun, 21 Feb 2016 23:58:23 +0000 (10:58 +1100)]
Rose: further generalise literal dedupe work
Justin Viiret [Thu, 18 Feb 2016 05:37:15 +0000 (16:37 +1100)]
Rose: don't assume roles with >1 lit need dedupe
We only require dedupe for such roles when they have literals that can
arrive simultaneously (i.e. one literal overlaps with the suffix of
another).
Justin Viiret [Tue, 16 Feb 2016 23:42:41 +0000 (10:42 +1100)]
Rose: make DR runtime path external report only
Justin Viiret [Tue, 16 Feb 2016 23:36:22 +0000 (10:36 +1100)]
Rose: only use direct reports for external reports
Justin Viiret [Tue, 16 Feb 2016 06:14:13 +0000 (17:14 +1100)]
Rose: allow DR literals to share vertices
Justin Viiret [Mon, 15 Feb 2016 22:29:07 +0000 (09:29 +1100)]
Rose: allow direct reports to be aliased
Justin Viiret [Mon, 15 Feb 2016 22:25:18 +0000 (09:25 +1100)]
Rose: assign DRs in allocateFinalLiteralId
Previously, direct reports were allocated earlier; now all final IDs are
assigned in the same place.
Matthew Barr [Mon, 15 Feb 2016 04:16:15 +0000 (15:16 +1100)]
Improved test for AVX2 support
Test for xsave and XMM/YMM registers. Use ICC dynamic feature detection
where available.
Justin Viiret [Mon, 15 Feb 2016 05:17:58 +0000 (16:17 +1100)]
Make comparison signed (fix warning)
Justin Viiret [Mon, 15 Feb 2016 02:50:05 +0000 (13:50 +1100)]
Dump: give abs jump targets in output
Justin Viiret [Mon, 15 Feb 2016 02:41:21 +0000 (13:41 +1100)]
Rose program: Improvements to debug/assertions
- Add current pc to debug printf.
- Assert that pc doesn't escape the RoseEngine structure.
Justin Viiret [Thu, 11 Feb 2016 04:57:56 +0000 (15:57 +1100)]
Update Rose state layout description
Many of the things in this list no longer existed.
Justin Viiret [Thu, 11 Feb 2016 04:21:18 +0000 (15:21 +1100)]
Split CHECK_LEFTFIX into CHECK_{INFIX,PREFIX}
Justin Viiret [Thu, 11 Feb 2016 03:54:10 +0000 (14:54 +1100)]
reduceInfixQueue: factor out any_work check
Justin Viiret [Thu, 11 Feb 2016 03:40:04 +0000 (14:40 +1100)]
Rename rosePrefixCheckMiracles to roseLeftfix...
Justin Viiret [Thu, 11 Feb 2016 03:38:12 +0000 (14:38 +1100)]
Rename reduceQueue to reduceInfixQueue
Justin Viiret [Thu, 11 Feb 2016 02:55:48 +0000 (13:55 +1100)]
Use rose_inline rather than really_inline
Justin Viiret [Thu, 11 Feb 2016 02:54:51 +0000 (13:54 +1100)]
roseTestLeftfix: unify common "nfa is dead" code
Justin Viiret [Mon, 15 Feb 2016 01:35:03 +0000 (12:35 +1100)]
Add CATCH_UP to report_block, not "parent" program
Also ensure that exhaustion check happens after catch up, as catch up
may fire reports (which could exhaust).
Justin Viiret [Fri, 12 Feb 2016 02:58:17 +0000 (13:58 +1100)]
limex_runtime.h: scratch header no longer needed
Justin Viiret [Fri, 12 Feb 2016 02:52:39 +0000 (13:52 +1100)]
Remove more unused structures from unit tests
The NFA, LBR no longer need scratch or the NFAContext structure stored
outside the NFA stack.
Justin Viiret [Thu, 11 Feb 2016 05:51:59 +0000 (16:51 +1100)]
NFA API: Remove unused scratch ptr from struct mq
Justin Viiret [Thu, 11 Feb 2016 05:46:15 +0000 (16:46 +1100)]
NFA API: Remove nfaBlockExecReverse scratch arg
Scratch is no longer used by this function's implementations.
Justin Viiret [Thu, 11 Feb 2016 05:40:16 +0000 (16:40 +1100)]
NFA: Move NFAContext to stack (from scratch)
Justin Viiret [Mon, 8 Feb 2016 23:01:53 +0000 (10:01 +1100)]
Rose: allow block-mode merge of small prefixes
Previously, we disallowed the merging of all Rose prefixes in block mode
where the literal sets are not identical.
This change allows merging if the prefix graphs to be merged are very
small, as a small performance improvement for cases with lots of tiny
prefixes.
This check is deliberately conservative: graphs must have some common
vertices, and the result of the merge must not give up any
accelerability.
Justin Viiret [Wed, 10 Feb 2016 04:14:49 +0000 (15:14 +1100)]
NFA merging: permit different reports
For cases where the edges from start are not to a mix of accept and
acceptEod, report sets can be combined.
Justin Viiret [Wed, 10 Feb 2016 04:33:48 +0000 (15:33 +1100)]
Dump: don't call dumpNfaNotes for SOM reverse NFAs
These NFAs have no queue index.
Justin Viiret [Mon, 8 Feb 2016 02:32:21 +0000 (13:32 +1100)]
SET_GROUPS instr: don't generate more than one
Justin Viiret [Mon, 8 Feb 2016 05:04:41 +0000 (16:04 +1100)]
dedupeCatchup: only call when necessary at runtime
Justin Viiret [Sun, 7 Feb 2016 23:21:17 +0000 (10:21 +1100)]
DEDUPE instr: generate only when necessary
Justin Viiret [Thu, 4 Feb 2016 01:46:53 +0000 (12:46 +1100)]
Rose: clean up use of scratch, RoseContext
Justin Viiret [Mon, 1 Feb 2016 22:42:00 +0000 (09:42 +1100)]
Rose: pack global state bits into one u8
Eliminate the RoseRuntimeState structure in favour of a single status
byte that is stored in scratch and copied to/from stream state.
Justin Viiret [Mon, 1 Feb 2016 00:07:07 +0000 (11:07 +1100)]
Rose: remove alignment req for anchored DFA state
Justin Viiret [Mon, 18 Jan 2016 00:56:01 +0000 (11:56 +1100)]
Rose: move more report handling work into program
Move report preconditions (bounds, exhaustion, etc) into program
instructions and use a more direct path to the user match callback than
the adaptor functions.
Report handling has been moved to new file src/report.h. Reporting from
EOD now uses the same instructions as normal report handling, rather
than its own.
Jump target tracking in rose_build_bytecode.cpp has been cleaned up.
Justin Viiret [Thu, 28 Jan 2016 06:16:53 +0000 (17:16 +1100)]
ng_filter: Fix bug introduced in
98eff64
If the max width is modified for a region, use the modified version when
checking to see if a self-loop must be added on the last vertex.
Justin Viiret [Thu, 14 Jan 2016 05:55:39 +0000 (16:55 +1100)]
RoseRuntimeState no longer needs to be packed
This structure only contains u8 values now. In the future we may wish to
eliminate it entirely and store the few bits we need more directly.
Justin Viiret [Thu, 14 Jan 2016 02:45:44 +0000 (13:45 +1100)]
Docs for Rose callback types
Justin Viiret [Wed, 13 Jan 2016 02:20:38 +0000 (13:20 +1100)]
Make Rose callback types explicitly take scratch
Justin Viiret [Wed, 13 Jan 2016 01:39:28 +0000 (12:39 +1100)]
Remove RoseContext::userCtx
All Rose callbacks receive scratch as their context.
Justin Viiret [Sun, 10 Jan 2016 22:25:32 +0000 (09:25 +1100)]
Move cyclic path redundancy into reduce loop
Sometimes cyclic path redundancy can uncover further reduction work that
can be done by the other passes in the reduce loop.
Justin Viiret [Sun, 17 Jan 2016 22:18:19 +0000 (09:18 +1100)]
roseEodRunMatcher: correct early return value
Justin Viiret [Mon, 11 Jan 2016 04:19:09 +0000 (15:19 +1100)]
nfaCheckFinalState: define return value
Make nfaCheckFinalState return MO_HALT_MATCHING when the user instructs
us (via the callback return value) to halt matching. In the caller,
check this value and stop matching if told.
Anatoly Burakov [Tue, 12 Jan 2016 16:21:20 +0000 (16:21 +0000)]
Don't look for accel friends for multibyte acceleration
Justin Viiret [Thu, 14 Jan 2016 03:48:22 +0000 (14:48 +1100)]
scratch: correctly align fatbit arrays
This fixes an assertion failure on 32-bit targets.
Justin Viiret [Thu, 7 Jan 2016 00:56:57 +0000 (11:56 +1100)]
Use fatbit for anch log, delay slots in scratch
Since these structures are in scratch, they do not have to be as small
as possible and we can use fatbit instead of multibit to improve
performance.
Justin Viiret [Wed, 13 Jan 2016 23:38:24 +0000 (10:38 +1100)]
rose_build_anchored: take ref, not pointer
Justin Viiret [Wed, 13 Jan 2016 23:24:19 +0000 (10:24 +1100)]
Account for multi-dfa case with ANCHORED_DELAY
Specifically, we must set build_context::floatingMinLiteralMatchOffset
to 1 when thew anchored table contains multiple DFAs, as they can
produce unordered matches.
This check was already been done, but too late to affect the generation
of ANCHORED_DELAY instructions.
Justin Viiret [Wed, 13 Jan 2016 22:53:15 +0000 (09:53 +1100)]
Use correct type for anchored matcher build
Justin Viiret [Wed, 13 Jan 2016 21:52:18 +0000 (08:52 +1100)]
Fix release build (unused var)
Justin Viiret [Tue, 12 Jan 2016 03:10:23 +0000 (14:10 +1100)]
Remove dupe engine, state ptrs from RoseContext
Remove the RoseEngine and stream state pointers frose RoseContext, as
they are also present in core_info.
Unify stream state handing in Rose to always use a char * (we were often
a u8 * for no particularly good reason) and tidy up.
Matthew Barr [Tue, 12 Jan 2016 03:48:35 +0000 (14:48 +1100)]
Coverity: Restore output stream format
Justin Viiret [Fri, 18 Dec 2015 04:24:52 +0000 (15:24 +1100)]
Rose: Move all literal operations into program
Replace the RoseLiteral structure with more program instructions; now,
instead of each literal ID leading to a RoseLiteral, it simply has a
program to run (and a delay rebuild program).
This commit also makes some other improvements:
* CHECK_STATE instruction, for use instead of a sparse iterator over a
single element.
* Elide some checks (CHECK_LIT_EARLY, ANCHORED_DELAY, etc) when not
needed.
* Flatten PUSH_DELAYED behaviour to one instruction per delayed
literal, rather than the mask/index-list approach used before.
* Simple program cache at compile time for deduplication.
Alex Coyte [Mon, 11 Jan 2016 02:14:58 +0000 (13:14 +1100)]
squashing: prevent generation of pairs of squash states
Matthew Barr [Mon, 11 Jan 2016 03:28:27 +0000 (14:28 +1100)]
alignof() should operate on a type-id
Justin Viiret [Sun, 10 Jan 2016 21:58:08 +0000 (08:58 +1100)]
Delete unused build_context::depths
Justin Viiret [Thu, 7 Jan 2016 23:10:10 +0000 (10:10 +1100)]
Remove use of depth from Rose entirely
Justin Viiret [Thu, 7 Jan 2016 22:58:20 +0000 (09:58 +1100)]
Don't use depth for in-flight check
Justin Viiret [Thu, 7 Jan 2016 22:11:09 +0000 (09:11 +1100)]
Remove CHECK_DEPTH instruction
Justin Viiret [Wed, 6 Jan 2016 00:45:31 +0000 (11:45 +1100)]
Remove "dot" entries from leftfix lookarounds
Note that we have to be careful to leave the first lookaround entry in
place, if it's a dot. This should eventually be done with a program
instruction.
Justin Viiret [Mon, 4 Jan 2016 04:33:05 +0000 (15:33 +1100)]
ComponentRepeat: remove firsts_cache, precalc code
Firsts are easy to compute in ComponentRepeat::first() now.
Justin Viiret [Mon, 4 Jan 2016 04:23:28 +0000 (15:23 +1100)]
ComponentRepeat: wire X{0,N} and (X?){N} the same
Justin Viiret [Tue, 15 Dec 2015 03:34:25 +0000 (14:34 +1100)]
ComponentRepeat: wire R{0,N} as (R{1,N})?
Change the way that we wire up the edges in a bounded repeat to avoid
large fan-out from predecessors.
Justin Viiret [Sun, 13 Dec 2015 23:08:57 +0000 (10:08 +1100)]
ng_prefilter: turn large max bound into inf
During prefilter region replacement, turn regions with very large max
bounds into repeats with inf max bound. This improves compile time and
the likelihood that we will actually be able to build an implementation
for such patterns.
Anatoly Burakov [Wed, 9 Dec 2015 13:39:16 +0000 (13:39 +0000)]
Multibyte matcher unit-tests
Anatoly Burakov [Wed, 9 Dec 2015 11:24:16 +0000 (11:24 +0000)]
Bitmatcher unit-tests
Anatoly Burakov [Wed, 9 Dec 2015 13:38:58 +0000 (13:38 +0000)]
Multibyte acceleration compile side
Anatoly Burakov [Wed, 9 Dec 2015 13:10:57 +0000 (13:10 +0000)]
Multibyte truffle runtime
Anatoly Burakov [Wed, 9 Dec 2015 12:20:34 +0000 (12:20 +0000)]
Multibyte shufti runtime
Anatoly Burakov [Wed, 9 Dec 2015 11:46:19 +0000 (11:46 +0000)]
Multibyte vermicelli runtime
Anatoly Burakov [Wed, 9 Dec 2015 11:11:49 +0000 (11:11 +0000)]
Adding bitmatchers
Anatoly Burakov [Wed, 9 Dec 2015 12:36:12 +0000 (12:36 +0000)]
Adding AVX2 version of truffle
Justin Viiret [Mon, 4 Jan 2016 05:04:19 +0000 (16:04 +1100)]
scratch: remove sparse iter state (now unused)
Justin Viiret [Mon, 4 Jan 2016 05:02:20 +0000 (16:02 +1100)]
roseRunProgram: iter state on stack
Justin Viiret [Mon, 4 Jan 2016 02:46:51 +0000 (13:46 +1100)]
roseCatchUpLeftfixes: iter state on stack
Justin Viiret [Mon, 4 Jan 2016 02:44:26 +0000 (13:44 +1100)]
roseBlockHasEodWork: iter state on stack
Justin Viiret [Mon, 4 Jan 2016 02:42:21 +0000 (13:42 +1100)]
roseFlushLastByteHistory: iter state on stack
Justin Viiret [Wed, 23 Dec 2015 04:12:28 +0000 (15:12 +1100)]
roseCheckNfaEod: use sparse iterator for EOD
Rather than checking all active outfix/suffix engines, use a sparse
iterator to check only those engines that accept at EOD.
Justin Viiret [Fri, 18 Dec 2015 04:50:56 +0000 (15:50 +1100)]
runtime: hoist broken check in streaming mode
Matthew Barr [Tue, 8 Dec 2015 03:40:20 +0000 (14:40 +1100)]
Build the tools dir only if the cmake file exists
Justin Viiret [Fri, 18 Dec 2015 00:48:33 +0000 (11:48 +1100)]
writeEodProgram: avoid make_move_iterator warning
Avoid an ambiguity between std:: and boost::make_move_iterator on builds
against libc++.
Justin Viiret [Thu, 10 Dec 2015 00:41:47 +0000 (11:41 +1100)]
rose: Extend program to handle literals, iterators
- cleanups
- add sparse iter instructions
- merge "root" and "sparse iter" programs together
- move program execution to new file program_runtime.h
- simplify EOD execution