From: jason taylor Date: Fri, 19 Apr 2024 21:50:27 +0000 (+0000) Subject: doc: update normalization notes X-Git-Tag: suricata-8.0.0-beta1~1339 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=abb74245cc71fb9e8e05d128eb093d754f961bfd;p=thirdparty%2Fsuricata.git doc: update normalization notes Ticket: #6781 Signed-off-by: jason taylor --- diff --git a/doc/userguide/rules/http-keywords.rst b/doc/userguide/rules/http-keywords.rst index 34794c8bd9..a453846bcb 100644 --- a/doc/userguide/rules/http-keywords.rst +++ b/doc/userguide/rules/http-keywords.rst @@ -92,6 +92,35 @@ Request or Response Keywords: * :ref:`http.protocol` * :ref:`http.start` +.. _http.normalization: + +Normalization +------------- + +There are times when Suricata performs formatting/normalization changes to +traffic that is seen. + +Duplicate Header Names +~~~~~~~~~~~~~~~~~~~~~~ + +If there are multiple values for the same header name, they are concatenated +with a comma and space (", ") between each value. More information can be +found in RFC 2616 ``_ + +Example Duplicate HTTP Header:: + + GET / HTTP/1.1 + Host: suricata.io + User-Agent: Mozilla/5.0 + User-Agent: Chrome/121.0.0 + +.. container:: example-rule + + alert http $HOME_NET -> $EXTERNAL_NET (msg:"Example Duplicate Header"; \ + flow:established,to_server; :example-rule-options:`http.user_agent; \ + content:"Mozilla/5.0, Chrome/121.0.0";` classtype:bad-unknown; sid:103; \ + rev:1;) + .. _file.name: file.name @@ -142,6 +171,9 @@ Example HTTP Request:: .. note:: ``http.accept`` does not include the leading space or trailing \\r\\n +.. note:: ``http.accept`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. + .. _http.accept_enc: http.accept_enc @@ -169,6 +201,9 @@ Example HTTP Request:: .. note:: ``http.accept_enc`` does not include the leading space or trailing \\r\\n +.. note:: ``http.accept_enc`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. + .. _http.accept_lang: http.accept_lang @@ -196,6 +231,9 @@ Example HTTP Request:: .. note:: ``http.accept_lang`` does not include the leading space or trailing \\r\\n +.. note:: ``http.accept_lang`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. + .. _http.host: http.host @@ -238,24 +276,8 @@ In the above example the host buffer would contain `suricata.io`. in a proxy request) or the HTTP Host header. If both are present, the URI is used. -.. note:: If a request contains multiple "Host" headers, the values will be - concatenated in the ``http.host`` and ``http.host.raw`` - buffers, in the order seen from top to bottom, with a comma and space - (", ") between each of them. - -Example Duplicate Host Header Request:: - - GET /index.html HTTP/1.1 - User-Agent: Chrome/2.0 - Cookie: PHPSESSION=123 - Host: suricata.io - Host: oisf.net - -.. container:: example-rule - - alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"HTTP Two Host Example"; \ - flow:established,to_server; :example-rule-options:`http.host; \ - content:"suricata.io, oisf.net";` classtype:bad-unknown; sid:125; rev:1;) +.. note:: ``http.host`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. .. _http.host.raw: @@ -284,24 +306,8 @@ Example HTTP Request:: in a proxy request) or the HTTP Host header. If both are present, the URI is used. -.. note:: If a request contains multiple "Host" headers, the values will be - concatenated in the ``http.host`` and ``http.host.raw`` buffers, in the - order seen from top to bottom, with a comma and space (", ") between each - of them. - -Example Duplicate Host Header Request:: - - GET /index.html HTTP/1.1 - User-Agent: Chrome/2.0 - Cookie: PHPSESSION=123 - Host: suricata.io - Host: oisf.net - -.. container:: example-rule - - alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"HTTP Two Host Example"; \ - flow:established,to_server; :example-rule-options:`http.host.raw; \ - content:"suricata.io, oisf.net";` classtype:bad-unknown; sid:125; rev:1;) +.. note:: ``http.host.raw`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. .. _http.method: @@ -362,6 +368,9 @@ Example HTTP Request:: .. note:: ``http.referer`` does not include the leading space or trailing \\r\\n +.. note:: ``http.referer`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. + .. _http.request_body: http.request_body @@ -555,24 +564,9 @@ Example HTTP Request:: .. note:: Using the ``http.user_agent`` generally provides better performance than using :ref:`http.header`. -.. note:: If a request contains multiple "User-Agent" headers, the values will - be concatenated in the ``http.user_agent`` buffer, in the order seen from - top to bottom, with a comma and space (", ") between each of them. - -Example Duplicate User-Agent Header Request:: - - GET /index.html HTTP/1.1 - User-Agent: Mozilla/5.0 - User-Agent: Chrome/2.0 - Cookie: PHPSESSION=123 - Host: suricata.io - -.. container:: example-rule - - alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"HTTP User-Agent Example"; \ - flow:established,to_server; :example-rule-options:`http.user_agent; \ - content:"Mozilla/5.0, Chrome/2.0";` bsize:23; classtype:bad-unknown; sid:90; \ - rev:1;) +.. note:: ``http.user_agent`` can have additional formatting/normalization + applied to buffer contents, see :ref:`http.normalization` for additional + details. .. _urilen: @@ -646,6 +640,9 @@ Example HTTP Response:: .. note:: ``http.location`` does not include the leading space or trailing \\r\\n +.. note:: ``http.location`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. + .. _http.response_body: http.response_body @@ -762,6 +759,10 @@ Example HTTP Response:: .. note:: ``http.server`` does not include the leading space or trailing \\r\\n +.. note:: ``http.server`` can have additional formatting/normalization + applied to buffer contents, see :ref:`http.normalization` for additional + details. + .. _http.stat_code: http.stat_code @@ -916,6 +917,10 @@ Example HTTP Request:: .. note:: ``http.connection`` does not include the leading space or trailing \\r\\n +.. note:: ``http.connection`` can have additional formatting/normalization + applied to buffer contents, see :ref:`http.normalization` for additional + details. + .. _http.content_len: http.content_len @@ -1010,6 +1015,10 @@ Example HTTP Response:: .. note:: ``http.content_type`` does not include the leading space or trailing \\r\\n +.. note:: ``http.content_type`` can have additional formatting/normalization + applied to buffer contents, see :ref:`http.normalization` for additional + details. + .. _http.cookie: http.cookie @@ -1021,12 +1030,6 @@ present in HTTP request (Cookie) or HTTP response (Set-Cookie) headers. It is possible to use any of the :doc:`payload-keywords` with both ``http.header`` keywords. -.. note:: Cookies are passed in HTTP headers but Suricata extracts the cookie - data to ``http.cookie`` and will not match cookie content put in the - :ref:`http.header` sticky buffer. - -.. note:: ``http.cookie`` does not include the leading space or trailing \\r\\n - Example HTTP Request:: GET /index.html HTTP/1.1 @@ -1040,6 +1043,16 @@ Example HTTP Request:: flow:established,to_server; :example-rule-emphasis:`http.cookie; \ content:"PHPSESSIONID=123";` bsize:14; classtype:bad-unknown; sid:80; rev:1;) +.. note:: Cookies are passed in HTTP headers but Suricata extracts the cookie + data to ``http.cookie`` and will not match cookie content put in the + :ref:`http.header` sticky buffer. + +.. note:: ``http.cookie`` does not include the leading space or trailing \\r\\n + +.. note:: ``http.cookie`` can have additional formatting/normalization + applied to buffer contents, see :ref:`http.normalization` for additional + details. + .. _http.header: http.header @@ -1092,10 +1105,8 @@ Example HTTP Request:: .. note:: There are headers that will not be included in the ``http.header`` buffer, specifically the :ref:`http.cookie` buffer. -.. note:: If there are multiple values for the same header name, they are - concatenated with a comma and space (", ") between each value. - More information can be found in RFC 2616 - ``_ +.. note:: ``http.header`` can have additional formatting/normalization applied + to buffer contents, see :ref:`http.normalization` for additional details. .. _http.header.raw: @@ -1119,6 +1130,9 @@ Abnormal HTTP Header Example:: content:"User-Agent|3a 20|Mozilla/5.0|0d 0a|"; \ content:"User-Agent|3a 20|Chrome|0d 0a|";` classtype:bad-unknown; sid:73; rev:1;) +.. note:: ``http.header.raw`` can have additional formatting applied to buffer + contents, see :ref:`http.normalization` for additional details. + .. _http.header_names: http.header_names