]> git.ipfire.org Git - thirdparty/vectorscan.git/log
thirdparty/vectorscan.git
8 years agoBump version number for release
Matthew Barr [Thu, 15 Jun 2017 23:51:23 +0000 (09:51 +1000)] 
Bump version number for release

8 years agochangelog: updates for 4.5.1 release
Matthew Barr [Thu, 15 Jun 2017 23:27:03 +0000 (09:27 +1000)] 
changelog: updates for 4.5.1 release

8 years agofdr_confirm_compile: fix bug in table sizing
Justin Viiret [Wed, 14 Jun 2017 06:04:26 +0000 (16:04 +1000)] 
fdr_confirm_compile: fix bug in table sizing

8 years agohsbench: cpuset portability
Matthew Barr [Wed, 14 Jun 2017 02:58:51 +0000 (12:58 +1000)] 
hsbench: cpuset portability

8 years agohsbench: make output more clear
Matthew Barr [Tue, 13 Jun 2017 23:04:43 +0000 (09:04 +1000)] 
hsbench: make output more clear

8 years agohsbench: fix flag handling
Matthew Barr [Tue, 13 Jun 2017 23:04:21 +0000 (09:04 +1000)] 
hsbench: fix flag handling

8 years agocmake: remove duplicate else()
Matthew Barr [Tue, 13 Jun 2017 03:45:52 +0000 (13:45 +1000)] 
cmake: remove duplicate else()

8 years agoflat_set/map: workaround for gcc-4.8 C++11 defect
Justin Viiret [Tue, 13 Jun 2017 00:36:11 +0000 (10:36 +1000)] 
flat_set/map: workaround for gcc-4.8 C++11 defect

The STL shipped with gcc-4.8 does not provide
vector::erase(const_iterator) for C++11, instead only taking a mutable
iterator.  This causes problems with flat_set/map if we don't have
Boost small_vector available and we fall back to std::vector.

We work around this by providing a function to construct a mutable
iterator given a const_iterator for internal use.

8 years agoBump version number for release
Matthew Barr [Thu, 8 Jun 2017 23:50:23 +0000 (09:50 +1000)] 
Bump version number for release

8 years agochangelog: updates for 4.5 release
Justin Viiret [Thu, 8 Jun 2017 23:47:03 +0000 (09:47 +1000)] 
changelog: updates for 4.5 release

8 years agochangelog: quote function name
Justin Viiret [Mon, 29 May 2017 23:47:53 +0000 (09:47 +1000)] 
changelog: quote function name

8 years agong_expr_info: more complete analysis passes
Justin Viiret [Thu, 8 Jun 2017 00:33:23 +0000 (10:33 +1000)] 
ng_expr_info: more complete analysis passes

8 years agohs_compile: add note to hs_expression_info() docs
Justin Viiret [Mon, 5 Jun 2017 04:33:09 +0000 (14:33 +1000)] 
hs_compile: add note to hs_expression_info() docs

Successful analysis of a pattern with hs_expression_info() does not
imply that the pattern will successfully compile with hs_compile(), etc.
It is merely a utility function for pattern analysis.

Addresses Github issue #54.

8 years agohs_expression_info: check unsupported constructs
Justin Viiret [Mon, 5 Jun 2017 04:38:57 +0000 (14:38 +1000)] 
hs_expression_info: check unsupported constructs

8 years agoUse env to get python bin, don't hard code path
Matthew Barr [Wed, 7 Jun 2017 04:45:57 +0000 (14:45 +1000)] 
Use env to get python bin, don't hard code path

8 years agoDocs: avx512 support
Matthew Barr [Wed, 7 Jun 2017 04:10:19 +0000 (14:10 +1000)] 
Docs: avx512 support

8 years agomsvc: use the vectorcall calling convention
Matthew Barr [Fri, 10 Mar 2017 04:48:38 +0000 (15:48 +1100)] 
msvc: use the vectorcall calling convention

This requires declaring external interfaces with the cdecl
calling convention.

8 years agocmake: Improve the microarch checks
Matthew Barr [Wed, 7 Jun 2017 01:16:30 +0000 (11:16 +1000)] 
cmake: Improve the microarch checks

8 years agocmake: scope fun
Matthew Barr [Wed, 23 Nov 2016 05:43:34 +0000 (16:43 +1100)] 
cmake: scope fun

8 years agoaccel: limit how far we will search in findBest
Justin Viiret [Wed, 17 May 2017 04:43:15 +0000 (14:43 +1000)] 
accel: limit how far we will search in findBest

8 years agoDisable part of unit test on FreeBSD 10
Matthew Barr [Wed, 17 May 2017 03:48:13 +0000 (13:48 +1000)] 
Disable part of unit test on FreeBSD 10

8 years agoIf we can shift by an immediate, do it. Otherwise, don't.
Matthew Barr [Tue, 16 May 2017 01:05:53 +0000 (11:05 +1000)] 
If we can shift by an immediate, do it. Otherwise, don't.

8 years agoUse our intrin header
Matthew Barr [Tue, 16 May 2017 00:37:19 +0000 (10:37 +1000)] 
Use our intrin header

8 years agocmake: use the correct command
Matthew Barr [Mon, 15 May 2017 22:52:11 +0000 (08:52 +1000)] 
cmake: use the correct command

8 years agotest for pthread_setaffinity_np
Matthew Barr [Fri, 12 May 2017 01:29:58 +0000 (11:29 +1000)] 
test for pthread_setaffinity_np

Only enable setting threads per core when available

8 years agorose_build_engine_blob: rose unused data() func
Justin Viiret [Mon, 15 May 2017 04:40:11 +0000 (14:40 +1000)] 
rose_build_engine_blob: rose unused data() func

We should no longer be reading out of the engine blob.

8 years agogcc7: comments for falling through a switch case
Matthew Barr [Mon, 10 Apr 2017 03:25:07 +0000 (13:25 +1000)] 
gcc7: comments for falling through a switch case

GCC 7 adds a warning -Wimplicit-fallthrough to catch when falling
through a switch statement without a break. Since we actually want that
behaviour sometimes, we can add a comment so the compiler knows we
intended the fallthrough.

8 years agocmake: build shared libs with the PIC objs
Matthew Barr [Wed, 10 May 2017 04:24:43 +0000 (14:24 +1000)] 
cmake: build shared libs with the PIC objs

8 years agorename pshufb to pshufb_m128
Matthew Barr [Fri, 5 May 2017 00:43:37 +0000 (10:43 +1000)] 
rename pshufb to pshufb_m128

8 years agorename vpshufb to pshufb_m256
Matthew Barr [Thu, 4 May 2017 23:59:29 +0000 (09:59 +1000)] 
rename vpshufb to pshufb_m256

8 years agoavx512: shufti
Matthew Barr [Tue, 27 Sep 2016 06:01:08 +0000 (16:01 +1000)] 
avx512: shufti

8 years agoavx512: truffle
Matthew Barr [Wed, 14 Sep 2016 06:09:47 +0000 (16:09 +1000)] 
avx512: truffle

8 years agoavx512: noodle
Matthew Barr [Mon, 8 Aug 2016 00:55:52 +0000 (10:55 +1000)] 
avx512: noodle

8 years agoavx512 fat runtime support: experimental
Matthew Barr [Wed, 14 Dec 2016 00:47:28 +0000 (11:47 +1100)] 
avx512 fat runtime support: experimental

8 years agoavx512: CPU detection and platform hints
Matthew Barr [Tue, 4 Oct 2016 00:18:10 +0000 (11:18 +1100)] 
avx512: CPU detection and platform hints

8 years agoavx512: add basic functions to simd_utils
Matthew Barr [Wed, 20 Jul 2016 01:31:34 +0000 (11:31 +1000)] 
avx512: add basic functions to simd_utils

Extends the m512 type to use avx512 and also changes required
for limex.

8 years agoAllow the full cpuid flags for fat runtimes
Matthew Barr [Tue, 9 May 2017 01:20:47 +0000 (11:20 +1000)] 
Allow the full cpuid flags for fat runtimes

8 years agohsbench: output max throughput result
Matthew Barr [Tue, 11 Apr 2017 04:27:59 +0000 (14:27 +1000)] 
hsbench: output max throughput result

8 years agodebug: use before set
Matthew Barr [Tue, 28 Mar 2017 00:15:55 +0000 (11:15 +1100)] 
debug: use before set

8 years agoHeader guards
Matthew Barr [Tue, 27 Sep 2016 05:56:40 +0000 (15:56 +1000)] 
Header guards

8 years agoDe-multiaccel
Matthew Barr [Thu, 30 Mar 2017 23:38:03 +0000 (10:38 +1100)] 
De-multiaccel

8 years agodeterministic assembleProgramBlocks()
Alex Coyte [Mon, 8 May 2017 00:51:19 +0000 (10:51 +1000)] 
deterministic assembleProgramBlocks()

8 years agohsbench: use boost:crc for db filename
Matthew Barr [Thu, 4 May 2017 00:37:45 +0000 (10:37 +1000)] 
hsbench: use boost:crc for db filename

8 years agowhitespace changes
Matthew Barr [Wed, 3 May 2017 04:47:01 +0000 (14:47 +1000)] 
whitespace changes

8 years agong_equivalence: use flat_set for VertexInfoSet
Justin Viiret [Tue, 2 May 2017 01:54:22 +0000 (11:54 +1000)] 
ng_equivalence: use flat_set for VertexInfoSet

8 years agosmallwrite: remove unnecessary assertion
Justin Viiret [Tue, 2 May 2017 00:20:59 +0000 (10:20 +1000)] 
smallwrite: remove unnecessary assertion

The alpha remap array is always big enough to remap characters. Silences
Coverity issue CID 167663.

8 years agosmwr: more aggressive pruning of overlong paths in NFA
Alex Coyte [Mon, 1 May 2017 03:31:09 +0000 (13:31 +1000)] 
smwr: more aggressive pruning of overlong paths in NFA

8 years agodeterminisation: cleanups, remove shrinkStateSet
Alex Coyte [Mon, 1 May 2017 01:33:37 +0000 (11:33 +1000)] 
determinisation: cleanups, remove shrinkStateSet

8 years agong_depth: modernize findLoopReachable
Justin Viiret [Mon, 1 May 2017 05:08:06 +0000 (15:08 +1000)] 
ng_depth: modernize findLoopReachable

8 years agong_depth: rename calcDepth functions, return vec
Justin Viiret [Mon, 1 May 2017 04:57:05 +0000 (14:57 +1000)] 
ng_depth: rename calcDepth functions, return vec

8 years agorose: rework storage of extra lookaround information
Alex Coyte [Mon, 1 May 2017 06:09:10 +0000 (16:09 +1000)] 
rose: rework storage of extra lookaround information

- remove explicit lookaround table from bytecode
- make the RoseInstr responsible for adding required info to blob

8 years agoverify_types: add type static assertions
Justin Viiret [Mon, 1 May 2017 04:21:51 +0000 (14:21 +1000)] 
verify_types: add type static assertions

8 years agoverify_types: throw on failure
Justin Viiret [Wed, 26 Apr 2017 07:22:22 +0000 (17:22 +1000)] 
verify_types: throw on failure

In release builds, we would like a verify_u32 (etc) failure to be more
than just an assertion.

8 years agopass roleStateIndices by reference
Alex Coyte [Thu, 27 Apr 2017 05:42:02 +0000 (15:42 +1000)] 
pass roleStateIndices by reference

8 years agoStrengthen assert to indicate that q->end is less than q->cur
Alex Coyte [Thu, 27 Apr 2017 03:58:55 +0000 (13:58 +1000)] 
Strengthen assert to indicate that q->end is less than q->cur

Coverity CID-167665

8 years agorose: replace RoseLiteralMap use of bimap
Justin Viiret [Wed, 26 Apr 2017 05:12:27 +0000 (15:12 +1000)] 
rose: replace RoseLiteralMap use of bimap

This apoproach is simpler and more efficient for cases with large
numbers of literals.

8 years agorose: remove hasLiteral()
Justin Viiret [Sun, 23 Apr 2017 23:27:16 +0000 (09:27 +1000)] 
rose: remove hasLiteral()

8 years agoue2_literal: add hash_value()
Justin Viiret [Sun, 23 Apr 2017 23:26:35 +0000 (09:26 +1000)] 
ue2_literal: add hash_value()

8 years agorose: shift program construction functions to rose_build_program
Alex Coyte [Wed, 26 Apr 2017 03:45:31 +0000 (13:45 +1000)] 
rose: shift program construction functions to rose_build_program

8 years agorose_build: move dedupe analysis into own file
Justin Viiret [Wed, 26 Apr 2017 01:21:19 +0000 (11:21 +1000)] 
rose_build: move dedupe analysis into own file

8 years agoreport_manager: more use of unordered_map
Justin Viiret [Mon, 24 Apr 2017 04:58:20 +0000 (14:58 +1000)] 
report_manager: more use of unordered_map

8 years agoreport_manager: use unordered externalIdMap
Justin Viiret [Mon, 24 Apr 2017 04:40:47 +0000 (14:40 +1000)] 
report_manager: use unordered externalIdMap

8 years agosplit out instruction details to own files
Alex Coyte [Wed, 26 Apr 2017 00:38:55 +0000 (10:38 +1000)] 
split out instruction details to own files

8 years agorose_program: merge RECORD_ANCHORED instruction into ANCHORED_DELAY
Alex Coyte [Tue, 25 Apr 2017 23:31:04 +0000 (09:31 +1000)] 
rose_program: merge RECORD_ANCHORED instruction into ANCHORED_DELAY

8 years agorose: minor improvements to avoid unneeded program instructions
Alex Coyte [Mon, 24 Apr 2017 00:22:44 +0000 (10:22 +1000)] 
rose: minor improvements to avoid unneeded program instructions

- strip out lonely check handled instructions
- avoid producing programs for empty ghost roles

8 years agorose: minor clean up of catchup
Alex Coyte [Sun, 23 Apr 2017 23:51:58 +0000 (09:51 +1000)] 
rose: minor clean up of catchup

- anchored dfa do not mean that catchup is required
- remove needsCatchup from rose bytecode as catchup is based on interpreter

8 years agobe more selective about generating CLEAR_WORK_DONE instructions
Alex Coyte [Tue, 11 Apr 2017 00:50:16 +0000 (10:50 +1000)] 
be more selective about generating CLEAR_WORK_DONE instructions

8 years agominor clean up of rose_build_bytecode.cpp
Alex Coyte [Fri, 21 Apr 2017 05:46:13 +0000 (15:46 +1000)] 
minor clean up of rose_build_bytecode.cpp

- consistently name functions creating programs as makeFoo()
- replace writeLiteralProgram() with makeFragmentProgram()
- make applyFinalSpecialisation() part of writeProgram(bc, prog)
- seperate users who want to make a program for a single literal and fragments

8 years agoMake the build wrapper less fragile
Matthew Barr [Tue, 18 Apr 2017 01:48:17 +0000 (11:48 +1000)] 
Make the build wrapper less fragile

This script was failing for certain paths (dots, whitespace), and using
sed to replace parts of the command line was overkill. Do less mangling,
and quote command line args.

Fixes 01org/hyperscan#51

8 years agosmallwrite: cope when everything has been pruned
Justin Viiret [Wed, 19 Apr 2017 05:45:35 +0000 (15:45 +1000)] 
smallwrite: cope when everything has been pruned

8 years agomcclellancompile: docs for main compile function
Justin Viiret [Wed, 12 Apr 2017 01:24:30 +0000 (11:24 +1000)] 
mcclellancompile: docs for main compile function

8 years agosmallwrite: prune trie when rose is high quality
Justin Viiret [Tue, 11 Apr 2017 06:13:43 +0000 (16:13 +1000)] 
smallwrite: prune trie when rose is high quality

8 years agosmallwrite: construct DFA states in BFS order
Justin Viiret [Tue, 11 Apr 2017 04:21:02 +0000 (14:21 +1000)] 
smallwrite: construct DFA states in BFS order

8 years agosmallwrite: bfs ordering, refine daddy selection
Justin Viiret [Tue, 11 Apr 2017 03:56:51 +0000 (13:56 +1000)] 
smallwrite: bfs ordering, refine daddy selection

8 years agosmallwrite: use failure map to set dfa daddy state
Justin Viiret [Tue, 11 Apr 2017 03:37:56 +0000 (13:37 +1000)] 
smallwrite: use failure map to set dfa daddy state

8 years agodfa: allow smwr to avoid lengthy daddy recalc
Justin Viiret [Tue, 11 Apr 2017 02:44:20 +0000 (12:44 +1000)] 
dfa: allow smwr to avoid lengthy daddy recalc

8 years agomcsheng_compile: factor out find_daddy_candidates
Justin Viiret [Tue, 18 Apr 2017 06:35:06 +0000 (16:35 +1000)] 
mcsheng_compile: factor out find_daddy_candidates

8 years agomcclellancompile: factor out find_daddy_candidates
Justin Viiret [Tue, 11 Apr 2017 01:36:20 +0000 (11:36 +1000)] 
mcclellancompile: factor out find_daddy_candidates

8 years agodfa: only accel init states from smwr path
Justin Viiret [Tue, 11 Apr 2017 00:56:22 +0000 (10:56 +1000)] 
dfa: only accel init states from smwr path

If the small-write DFA has been built from literals, then we only need
to look for accel states at init.

8 years agodfa: prune_overlong -> clear_deeper_reports
Justin Viiret [Thu, 13 Apr 2017 03:18:33 +0000 (13:18 +1000)] 
dfa: prune_overlong -> clear_deeper_reports

Rather than pruning states, we simply clear reports on states that are
too deep and allow Hopcroft minimisation to reduce the size of the DFA
afterwards.

8 years agodfa_min: clean up and improve minimize code
Justin Viiret [Thu, 13 Apr 2017 06:18:22 +0000 (16:18 +1000)] 
dfa_min: clean up and improve minimize code

8 years agoaccel_dfa_build_strat: use flat_map for small maps
Justin Viiret [Mon, 10 Apr 2017 04:23:00 +0000 (14:23 +1000)] 
accel_dfa_build_strat: use flat_map for small maps

8 years agoaccel_dfa_build_strat: use small_vector for paths
Justin Viiret [Mon, 10 Apr 2017 01:05:06 +0000 (11:05 +1000)] 
accel_dfa_build_strat: use small_vector for paths

8 years agomcclellancompile_util: reduce malloc traffic, tidy
Justin Viiret [Mon, 10 Apr 2017 00:32:37 +0000 (10:32 +1000)] 
mcclellancompile_util: reduce malloc traffic, tidy

8 years agong_limex_accel: fix broken termination condition
Justin Viiret [Mon, 10 Apr 2017 06:39:31 +0000 (16:39 +1000)] 
ng_limex_accel: fix broken termination condition

8 years agoexpressions: add much faster limitToSignatures()
Justin Viiret [Fri, 7 Apr 2017 07:02:00 +0000 (17:02 +1000)] 
expressions: add much faster limitToSignatures()

8 years agoEnsure simd types are aligned to what we assume
Matthew Barr [Fri, 7 Apr 2017 06:40:11 +0000 (16:40 +1000)] 
Ensure simd types are aligned to what we assume

Turns out Clang is sensitive to the location of the alignment attribute.

8 years agong_limex_accel: compile time speedups, tidy up
Justin Viiret [Thu, 6 Apr 2017 03:47:30 +0000 (13:47 +1000)] 
ng_limex_accel: compile time speedups, tidy up

8 years agofind_better_daddy: use flat_set for hinted states
Justin Viiret [Thu, 6 Apr 2017 01:43:56 +0000 (11:43 +1000)] 
find_better_daddy: use flat_set for hinted states

8 years agofind_escape_strings: find outs2_broken faster
Justin Viiret [Thu, 6 Apr 2017 06:14:43 +0000 (16:14 +1000)] 
find_escape_strings: find outs2_broken faster

8 years agodoc: update copyright year
Justin Viiret [Fri, 20 Jan 2017 03:25:50 +0000 (14:25 +1100)] 
doc: update copyright year

8 years agohsbench: documentation
Justin Viiret [Sun, 30 Oct 2016 22:46:41 +0000 (09:46 +1100)] 
hsbench: documentation

8 years agocmake: check for libc++ directly
Matthew Barr [Thu, 6 Apr 2017 00:53:28 +0000 (10:53 +1000)] 
cmake: check for libc++ directly

Rather than trying to infer libc++ use from platform and compiler, just
look for a defined symbol.

8 years agoUse ue2::noncopyable instead of deleting copy ctor
Matthew Barr [Fri, 7 Apr 2017 01:11:53 +0000 (11:11 +1000)] 
Use ue2::noncopyable instead of deleting copy ctor

8 years agowindows: tools are unix only
Matthew Barr [Wed, 22 Mar 2017 21:13:00 +0000 (08:13 +1100)] 
windows: tools are unix only

8 years agosmallwrite: aho-corasick construction for literals
Justin Viiret [Fri, 31 Mar 2017 03:04:44 +0000 (14:04 +1100)] 
smallwrite: aho-corasick construction for literals

8 years agosmallwrite: simple trie experiment
Justin Viiret [Thu, 2 Mar 2017 00:09:27 +0000 (11:09 +1100)] 
smallwrite: simple trie experiment

8 years agordfa_merge: sort esets before alphabet calc
Justin Viiret [Wed, 5 Apr 2017 06:40:07 +0000 (16:40 +1000)] 
rdfa_merge: sort esets before alphabet calc

8 years agorose: fix CHECK_MULTIPATH_LOOKAROUND match difference bug
Xu, Chi [Wed, 5 Apr 2017 20:20:17 +0000 (04:20 +0800)] 
rose: fix CHECK_MULTIPATH_LOOKAROUND match difference bug

8 years agobytecode_ptr: needs <stdexcept> for logic_error
Justin Viiret [Thu, 6 Apr 2017 00:09:34 +0000 (10:09 +1000)] 
bytecode_ptr: needs <stdexcept> for logic_error