Merge pull request #1224 in SNORT/snort3 from doc_trace to master

Squashed commit of the following:

commit ebf90a7526091f3aae85bea6a62b757fa325160f
Author: mdagon <mdagon@cisco.com>
Date:   Fri May 11 14:25:20 2018 -0400

    doc: add doc for module trace

diff --git a/doc/CMakeLists.txt b/doc/CMakeLists.txt
index 41842be80a88ff6c353d8b0c35a20c3cf4385086..a8ed0905c18cc8427534554fcb4f72a1e69e6e3e 100644 (file)
--- a/doc/CMakeLists.txt
+++ b/doc/CMakeLists.txt
@@ -56,6 +56,7 @@ if ( MAKE_DOC )
        high_availability.txt
        http_inspect.txt
        http2_inspect.txt
+       module_trace.txt
        overview.txt
        params.txt
        perf_monitor.txt

diff --git a/doc/features.txt b/doc/features.txt
index 528e8cef76fc5ff35a1f805f8946c806660033a0..1aed8106c097e47e41559ae1c6dbb543c75eeb05 100644 (file)
--- a/doc/features.txt
+++ b/doc/features.txt
@@ -60,6 +60,10 @@ include::http_inspect.txt[]
 
 include::http2_inspect.txt[]
 
+=== Module Trace
+
+include::module_trace.txt[]
+
 === Performance Monitor
 
 include::perf_monitor.txt[]

diff --git a/doc/module_trace.txt b/doc/module_trace.txt
new file mode 100644 (file)
index 0000000..7421bf5
--- /dev/null
+++ b/doc/module_trace.txt
@@ -0,0 +1,159 @@
+Snort 3 retired the different flavors of debug macros that used to be set
+through environment variable SNORT_DEBUG. It was replaced by a module specific
+trace. Trace is turned on by setting the module-specific trace bitmask in
+snort.lua. As before, in order to enable it, snort has to be configured and
+built with  --enable-debug-msgs.
+
+==== Debugging rules using detection trace
+
+Detection engine is responsible for rule evaluation. Turning on the 
+trace for it can help with debugging new rules.
+
+The relevant options for detection are as follow (represented as hex):
+
+   0x2 - follow rule evaluation
+   0x4 - print evaluated buffer if it changed 
+   0x8 - print evaluated buffer at every step
+   0x10 - print value of ips rule options vars
+   0x20 - print information on fast pattern search
+
+Buffer print is useful, but in case the buffer is very big can be too verbose.
+Choose between 0x4, 0x8 or no buffer trace accordingly.
+
+0x10 is useful when the rule is using ips rule options vars.
+
+==== Example - rule evaluation traces:
+
+In snort.lua, the following line was added:
+
+detection = {trace = 0x20 + 0x10 + 0x2 + 0x4}
+
+The pcap has a single packet with payload: 
+10.AAAAAAAfoobar
+
+Evaluated on rules:
+
+    # byte_math + oper with byte extract and content
+    # VAL = 1, byte_math = 0 + 10 
+    alert tcp ( byte_extract: 1, 0, VAL, string, dec; 
+    byte_math:bytes 1,offset VAL,oper +, rvalue 10, result var1, string dec;
+    content:"foo", offset var1; sid:3)
+
+    #This rule should not trigger
+    alert tcp (content:"AAAAA"; byte_jump:2,0,relative; 
+    content:"foo", within 3; sid:2)
+
+
+The output:
+
+    detection: packet 1 C2S 127.0.0.1:1234 127.0.0.1:5678
+    detection: Fast pattern search
+    detection: 1 fp packet[16]
+
+    snort.raw[16]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    31 30 00 41 41 41 41 41 41 41  66 6F 6F 62 61 72              10.AAAAAAAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Processing pattern match #1
+    detection: Fast pattern packet[5] = 'AAAAA' |41 41 41 41 41 | ( )
+    detection: Starting tree eval
+    detection: Evaluating option content, cursor name pkt_data, cursor position 0
+
+    snort.raw[16]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    31 30 00 41 41 41 41 41 41 41  66 6F 6F 62 61 72              10.AAAAAAAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Rule options variables: 
+    var[0]=0 var[1]=0 var[2]=0 
+    detection: Evaluating option byte_jump, cursor name pkt_data, cursor position 8
+
+    snort.raw[8]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    41 41 66 6F 6F 62 61 72                                       AAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: no match
+    detection: Rule options variables: 
+    var[0]=0 var[1]=0 var[2]=0 
+    detection: Evaluating option byte_jump, cursor name pkt_data, cursor position 9
+
+    snort.raw[7]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    41 66 6F 6F 62 61 72                                          Afoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: no match
+    detection: Rule options variables: 
+    var[0]=0 var[1]=0 var[2]=0 
+    detection: Evaluating option byte_jump, cursor name pkt_data, cursor position 10
+
+    snort.raw[6]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    66 6F 6F 62 61 72                                             foobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: no match
+    detection: no match
+    detection: Processing pattern match #2
+    detection: Fast pattern packet[3] = 'foo' |66 6F 6F | ( )
+    detection: Starting tree eval
+    detection: Evaluating option byte_extract, cursor name pkt_data, cursor position 0
+
+    snort.raw[16]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    31 30 00 41 41 41 41 41 41 41  66 6F 6F 62 61 72              10.AAAAAAAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Rule options variables: 
+    var[0]=1 var[1]=0 var[2]=0 
+    detection: Evaluating option byte_math, cursor name pkt_data, cursor position 1
+
+    snort.raw[15]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    30 00 41 41 41 41 41 41 41 66  6F 6F 62 61 72                 0.AAAAAAAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Rule options variables: 
+    var[0]=1 var[1]=10 var[2]=0 
+    detection: Evaluating option content, cursor name pkt_data, cursor position 2
+
+    snort.raw[14]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    00 41 41 41 41 41 41 41 66 6F  6F 62 61 72                    .AAAAAAAfoobar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Rule options variables: 
+    var[0]=1 var[1]=10 var[2]=0 
+    detection: Reached leaf, cursor name pkt_data, cursor position 13
+
+    snort.raw[3]:
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    62 61 72                                                      bar
+    - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - -  - - - - -  - - - - -
+    detection: Matched rule gid:sid:rev 1:3:0
+    detection: Rule options variables: 
+    var[0]=1 var[1]=10 var[2]=0 
+    04/22-20:21:40.905630, 1, TCP, raw, 56, C2S, 127.0.0.1:1234, 127.0.0.1:5678, 1:3:0, allow
+
+
+==== Other available traces
+
+There are more trace options supported by detection:
+
+    0x1 - prints statistics about the engine 
+    0x40 - prints a message when disabling content detect for packet
+    0x80 - prints option tree data structure
+    0x100 - prints a message when a new tag is added
+
+Detection is the only module that support multiple options for trace.
+
+The rest support only 1 option, and can be turned on by adding trace = 1 to
+their lua config.
+
+* stream module trace:
+
+When turned on prints a message in case inspection is stopped on a flow.
+Example for output:
+
+stream: stop inspection on flow, dir BOTH
+
+* stream_ip, stream_user: trace will output general processing messages
+
+Other modules that support trace have messages as seemed fit to the developer.
+Some are for corner cases, other for complex data structures prints.  Current
+list of additional modules supporting trace: appid, dce_smb, gtp_inspect and
+dce_udp.