From: Victor Julien Date: Wed, 10 May 2017 07:04:17 +0000 (+0200) Subject: doc: move parts out of snort difference doc X-Git-Tag: suricata-4.0.0-beta1~72 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=0af562d4c8c3903a1688e505ebbb37bce8ecc5b6;p=thirdparty%2Fsuricata.git doc: move parts out of snort difference doc Move generic keyword descriptions to the keyword documentation. --- diff --git a/doc/userguide/rules/differences-from-snort.rst b/doc/userguide/rules/differences-from-snort.rst index 8a58324522..2dcb45ec9e 100644 --- a/doc/userguide/rules/differences-from-snort.rst +++ b/doc/userguide/rules/differences-from-snort.rst @@ -1,6 +1,6 @@ -==================================== -Suricata Rule Differences From Snort -==================================== +====================== +Differences From Snort +====================== Overview -------- @@ -22,6 +22,7 @@ Automatic Protocol Detection application layer protocols: - dcerpc + - dnp3 - dns - http - imap (detection only by default; no parsing) @@ -32,7 +33,7 @@ Automatic Protocol Detection - smb2 (disabled internally inside the engine) - smtp - ssh - - tls (SSLv2, SSLv3 & TLSv1) + - tls (SSLv2, SSLv3, TLSv1, TLSv1.1 and TLSv1.2) - In Suricata, protocol detection is port agnostic (in most cases). In Snort, in order for the ``http_inspect`` and other preprocessors to be @@ -245,114 +246,15 @@ Automatic Protocol Detection - Corresponding PCRE modifier: ``C`` (same as Snort) -``http_user_agent`` Buffer --------------------------- - -- Suricata has a ``http_user_agent`` buffer, Snort does not. - -- The ``http_user_agent`` buffer will NOT include the header name, - colon, or leading whitespace. i.e. it will not include - "User-Agent: ". - -- The ``http_user_agent`` buffer does not include a CRLF (0x0D - 0x0A) at the end. If you want to match the end of the buffer, use a - relative ``isdataat`` or a PCRE (although PCRE will be worse on - performance). - -- If a request contains multiple "User-Agent" headers, the values will - be concatenated in the ``http_user_agent`` buffer, in the order - seen from top to bottom, with a comma and space (", ") between each - of them. - - Example request:: - - GET /test.html HTTP/1.1 - User-Agent: SuriTester/0.8 - User-Agent: GGGG - - ``http_user_agent`` buffer contents:: - - SuriTester/0.8, GGGG - -- Corresponding PCRE modifier: ``V`` - -- Using the ``http_user_agent`` buffer is more efficient when it - comes to performance than using the ``http_header`` buffer (~10% - better). - -- `http://blog.inliniac.net/2012/07/09/suricata-http\_user\_agent-vs-http\_header/ `_ - -``http_host`` and ``http_raw_host`` Buffers -------------------------------------------- - -- Suricata has ``http_host`` and ``http_raw_host`` buffers, - Snort does not. - -- The ``http_host`` and ``http_raw_host`` buffers are populated - from either the URI (if the full URI is present in the request like - in a proxy request) or the HTTP Host header. If both are present, the - URI is used. - -- The ``http_host`` and ``http_raw_host`` buffers will NOT - include the header name, colon, or leading whitespace if populated - from the Host header. i.e. they will not include "Host: ". - -- The ``http_host`` and ``http_raw_host`` buffers do not - include a CRLF (0x0D 0x0A) at the end. If you want to match the end - of the buffer, use a relative 'isdataat' or a PCRE (although PCRE - will be worse on performance). - -- The ``http_host`` buffer is normalized to be all lower case. - -- The content match that ``http_host`` applies to must be all lower - case or have the ``nocase`` flag set. - -- ``http_raw_host`` matches the unnormalized buffer so matching - will be case-sensitive (unless ``nocase`` is set). - -- If a request contains multiple "Host" headers, the values will be - concatenated in the ``http_host`` and ``http_raw_host`` - buffers, in the order seen from top to bottom, with a comma and space - (", ") between each of them. - - Example request:: - - GET /test.html HTTP/1.1 - Host: ABC.com - Accept: */* - Host: efg.net - - ``http_host`` buffer contents:: - - abc.com, efg.net - - ``http_raw_host`` buffer contents:: - - ABC.com, efg.net - -- Corresponding PCRE modifier (``http_host``): ``W`` -- Corresponding PCRE modifier (``http_raw_host``): ``Z`` - -``http_server_body`` Buffer ---------------------------- - -- Suricata has the ``http_server_body`` buffer, Snort does not. - -- This tells Suricata to match in the HTTP server response body. +New HTTP keywords +----------------- -- Using ``http_server_body`` is similar to having content matches - that come after ``file_data`` except that it doesn't permanently - (unless reset) set the detection pointer to the beginning of the - server response body. i.e. it is not a sticky buffer. +Suricata supports several HTTP keywords that Snort doesn't have. -- ``http_server_body`` will match on gzip decoded data just like - ``file_data`` does. +Examples are ``http_user_agent``, ``http_host`` and ``http_content_type``. -- Since ``http_server_body`` matches on a server response, it - can't be used with the ``to_server`` or ``from_client`` flow - directives. +See :doc:`http-keywords` for all HTTP keywords. -- Corresponding PCRE modifier: ``Q`` ``byte_extract`` Keyword ------------------------ @@ -465,41 +367,7 @@ For details see :doc:`tls-keywords`. - Use ``pkt_data`` to reset the detection pointer to the beginning of the packet payload. -- **Buffer is normalized!** - - - Contains literal domain name - - - values (as seen in a raw DNS request) - are literal '.' characters - - no leading value - - No terminating NULL (0x00) byte (use a negated relative ``isdataat`` - to match the end) - - Example DNS request for "mail.google.com" (for readability, hex - values are encoded between pipes): - - DNS query on the wire (snippet):: - - |04|mail|06|google|03|com|00| - - ``dns_query`` buffer:: - - mail.google.com - -- :doc:`dns-keywords` - -``geoip`` Keyword ------------------ - -- Suricata has the ``geoip`` keyword, Snort does not. - -- Only supports IPv4 -- Uses GeoIP API of Maxmind - - - libgeoip - - Must be compiled in. - -- See :doc:`header-keywords`, "Geoip" section +- See :doc:`dns-keywords` for details. IP Reputation and ``iprep`` Keyword ----------------------------------- @@ -528,31 +396,6 @@ IP Reputation and ``iprep`` Keyword - :doc:`../reputation/ipreputation/ip-reputation-format` - `http://blog.inliniac.net/2012/11/21/ip-reputation-in-suricata/ `_ -``xbits`` ---------- - -- Suricata supports ``xbits`` which are like ``flowbits`` but - can apply to disparate streams – "global flowbits" -- Can track by ``ip_src``, ``ip_dst``, or ``ip_pair`` - - - No difference between using ``hostbits`` and ``xbits`` - with ``track ip_`` - - - If you ``set`` on a client request and use - ``track ip_dst``, if you want to match on the server response, - you check it (``isset``) with ``track ip_src``. - - - ``track ip_pair`` has to have the same src and dst IPs on the - setter and checker - -- To not alert, use ``flowbits:noalert;`` -- there - is no such thing as ``xbits:noalert;`` - -- See also: - - - `https://blog.inliniac.net/2014/12/21/crossing-the-streams-in-suricata/ `_ - - `http://www.cipherdyne.org/blog/2013/07/crossing-the-streams-in-ids-signature-languages.html `_ - Flowbits -------- @@ -574,6 +417,15 @@ Flowbits - :doc:`flow-keywords` +flowbits:noalert; +----------------- + +A common pattern in existing rules is to use ``flowbits:noalert;`` to make +sure a rule doesn't generate an alert if it matches. + +Suricata allows using just ``noalert;`` as well. Both have an identical meaning +in Suricata. + Negated Content Match Special Case ---------------------------------- @@ -643,8 +495,9 @@ File Extraction Lua Scripting ------------- -- Suricata has the ``luajit`` keyword which allows for a rule to reference - a Lua script that can access the packet, payload, HTTP buffers, etc. +- Suricata has the ``lua`` (or ``luajit``) keyword which allows for a + rule to reference a Lua script that can access the packet, payload, + HTTP buffers, etc. - Provides powerful flexibility and capabilities that Snort does not have. - :doc:`rule-lua-scripting` diff --git a/doc/userguide/rules/dns-keywords.rst b/doc/userguide/rules/dns-keywords.rst index e7e4fc1d61..8149cebda5 100644 --- a/doc/userguide/rules/dns-keywords.rst +++ b/doc/userguide/rules/dns-keywords.rst @@ -1,8 +1,6 @@ DNS Keywords ============ -(from v2.0) - There are some more content modifiers (If you are unfamiliar with content modifiers, please visit the page :doc:`payload-keywords` These ones make sure the signature checks a specific part of the @@ -23,3 +21,25 @@ used in a rule all contents following it are affected by it. Example: The dns_query keyword affects all following contents, until pkt_data is used or it reaches the end of the rule. + +Normalized Buffer +~~~~~~~~~~~~~~~~~ + +Buffer contains literal domain name + +- values (as seen in a raw DNS request) + are literal '.' characters +- no leading value +- No terminating NULL (0x00) byte (use a negated relative ``isdataat`` + to match the end) + +Example DNS request for "mail.google.com" (for readability, hex +values are encoded between pipes): + +DNS query on the wire (snippet):: + + |04|mail|06|google|03|com|00| + +``dns_query`` buffer:: + + mail.google.com diff --git a/doc/userguide/rules/header-keywords.rst b/doc/userguide/rules/header-keywords.rst index b0bfa79a60..de386e276e 100644 --- a/doc/userguide/rules/header-keywords.rst +++ b/doc/userguide/rules/header-keywords.rst @@ -136,6 +136,10 @@ direction you would like to match:: dest: if the destination matches with the given geoip. src: the source matches with the given geoip. +The keyword only supports IPv4. As it uses the GeoIP API of Maxmind, +libgeoip must be compiled in. + + Fragments --------- diff --git a/doc/userguide/rules/http-keywords.rst b/doc/userguide/rules/http-keywords.rst index b23c7acbf1..5e0eaeab4d 100644 --- a/doc/userguide/rules/http-keywords.rst +++ b/doc/userguide/rules/http-keywords.rst @@ -142,8 +142,8 @@ specifically and only on the HTTP method buffer. The keyword can be used in combination with all previously mentioned content modifiers such as: ``depth``, ``distance``, ``offset``, ``nocase`` and ``within``. -Methods are: **GET**, **POST**, **PUT**, **HEAD**, **DELETE**, **TRACE**, -**OPTIONS**, **CONNECT** and **PATCH**. +Examples of methods are: **GET**, **POST**, **PUT**, **HEAD**, +**DELETE**, **TRACE**, **OPTIONS**, **CONNECT** and **PATCH**. Example of a method in a HTTP request: @@ -312,10 +312,6 @@ modifiers like ``depth``, ``distance``, ``offset``, ``nocase`` and ``within``. Note that the ``pcre`` keyword can also inspect this buffer when using the ``/V`` modifier. -An analysis into the performance of ``http_user_agent`` -vs. ``http_header`` is found at: -http://blog.inliniac.net/2012/07/09/suricata-http_user_agent-vs-http_header/ - Normalization: leading spaces **are not** part of this buffer. So "User-Agent: \r\n" will result in an empty ``http_user_agent`` buffer. @@ -327,6 +323,41 @@ Example of the purpose of ``http_user_agent``: .. image:: http-keywords/user_agent_match.png +Notes +~~~~~ + +- The ``http_user_agent`` buffer will NOT include the header name, + colon, or leading whitespace. i.e. it will not include + "User-Agent: ". + +- The ``http_user_agent`` buffer does not include a CRLF (0x0D + 0x0A) at the end. If you want to match the end of the buffer, use a + relative ``isdataat`` or a PCRE (although PCRE will be worse on + performance). + +- If a request contains multiple "User-Agent" headers, the values will + be concatenated in the ``http_user_agent`` buffer, in the order + seen from top to bottom, with a comma and space (", ") between each + of them. + + Example request:: + + GET /test.html HTTP/1.1 + User-Agent: SuriTester/0.8 + User-Agent: GGGG + + ``http_user_agent`` buffer contents:: + + SuriTester/0.8, GGGG + +- Corresponding PCRE modifier: ``V`` + +- Using the ``http_user_agent`` buffer is more efficient when it + comes to performance than using the ``http_header`` buffer (~10% + better). + +- `http://blog.inliniac.net/2012/07/09/suricata-http\_user\_agent-vs-http\_header/ `_ + http_accept ----------- @@ -543,6 +574,23 @@ in your :ref:`libhtp configuration section ` via the ``response-body-limit`` setting. +Notes +~~~~~ + +- Using ``http_server_body`` is similar to having content matches + that come after ``file_data`` except that it doesn't permanently + (unless reset) set the detection pointer to the beginning of the + server response body. i.e. it is not a sticky buffer. + +- ``http_server_body`` will match on gzip decoded data just like + ``file_data`` does. + +- Since ``http_server_body`` matches on a server response, it + can't be used with the ``to_server`` or ``from_client`` flow + directives. + +- Corresponding PCRE modifier: ``Q`` + http_host and http_raw_host --------------------------- @@ -553,9 +601,57 @@ The ``http_raw_host`` inspects the raw hostname. The keyword can be used in combination with most of the content modifiers like ``distance``, ``offset``, ``within``, etc. -The ``nocase`` keyword ist not allowed anymore. Keep in mind that you need +The ``nocase`` keyword is not allowed anymore. Keep in mind that you need to specify a lowercase pattern. +Notes +~~~~~ + +- The ``http_host`` and ``http_raw_host`` buffers are populated + from either the URI (if the full URI is present in the request like + in a proxy request) or the HTTP Host header. If both are present, the + URI is used. + +- The ``http_host`` and ``http_raw_host`` buffers will NOT + include the header name, colon, or leading whitespace if populated + from the Host header. i.e. they will not include "Host: ". + +- The ``http_host`` and ``http_raw_host`` buffers do not + include a CRLF (0x0D 0x0A) at the end. If you want to match the end + of the buffer, use a relative 'isdataat' or a PCRE (although PCRE + will be worse on performance). + +- The ``http_host`` buffer is normalized to be all lower case. + +- The content match that ``http_host`` applies to must be all lower + case or have the ``nocase`` flag set. + +- ``http_raw_host`` matches the unnormalized buffer so matching + will be case-sensitive (unless ``nocase`` is set). + +- If a request contains multiple "Host" headers, the values will be + concatenated in the ``http_host`` and ``http_raw_host`` + buffers, in the order seen from top to bottom, with a comma and space + (", ") between each of them. + + Example request:: + + GET /test.html HTTP/1.1 + Host: ABC.com + Accept: */* + Host: efg.net + + ``http_host`` buffer contents:: + + abc.com, efg.net + + ``http_raw_host`` buffer contents:: + + ABC.com, efg.net + +- Corresponding PCRE modifier (``http_host``): ``W`` +- Corresponding PCRE modifier (``http_raw_host``): ``Z`` + file_data --------- diff --git a/doc/userguide/rules/xbits.rst b/doc/userguide/rules/xbits.rst index 3af99cff94..e574e3fcf3 100644 --- a/doc/userguide/rules/xbits.rst +++ b/doc/userguide/rules/xbits.rst @@ -12,12 +12,31 @@ Syntax:: xbits:,,track \ [,expire ]; +Notes +~~~~~ + +- No difference between using ``hostbits`` and ``xbits`` + with ``track ip_`` + +- If you ``set`` on a client request and use + ``track ip_dst``, if you want to match on the server response, + you check it (``isset``) with ``track ip_src``. + +- To not alert, use ``noalert;`` + +- See also: + + - `https://blog.inliniac.net/2014/12/21/crossing-the-streams-in-suricata/ `_ + - `http://www.cipherdyne.org/blog/2013/07/crossing-the-streams-in-ids-signature-languages.html `_ + YAML settings ------------- -Bits that are stored per host are stored in the Host table. +Bits that are stored per host are stored in the Host table. This means that +host table settings affect hostsbits and xbits per host. -Bits that are stored per IP pair are stored in the IPPair table. +Bits that are stored per IP pair are stored in the IPPair table. This means +that ippair table settings, especially memcap, affect xbits per ip_pair. Threading ---------