Merge pull request #3041 in SNORT/snort3 from ~STECHEW/snort3:build_3.1.11.0 to master

Squashed commit of the following:

commit b6adb6b8f275d005823b0932758e45fd42424650
Author: Steve Chew <stechew@cisco.com>
Date:   Thu Aug 26 11:34:45 2021 -0400

    build: generate and tag 3.1.11.0

diff --git a/CMakeLists.txt b/CMakeLists.txt
index cf1f337eac34739173ea59a01c9e4cb62a558201..8b012afa1b27121e4dc1ec4bf2d9485b73cee50b 100644 (file)
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -3,7 +3,7 @@ project (snort CXX C)
 
 set (VERSION_MAJOR 3)
 set (VERSION_MINOR 1)
-set (VERSION_PATCH 10)
+set (VERSION_PATCH 11)
 set (VERSION_SUBLEVEL 0)
 set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}")
 

diff --git a/ChangeLog b/ChangeLog
index edcec6ba6dffec31b35fdcbcb84361d23b254e3f..998a44c89d012b8964e03d20070e7f8ac894e20a 100644 (file)
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,42 @@
+2021/08/26 - 3.1.11.0
+
+build: update help for --enable-tsc-clock to include arm. Thanks to liangxwa01 for reporting the issue.
+codec: geneve: fix incorrect parsing of option header length
+data_bus: support ordered call of handlers
+dns, ssh: remove obsolete stream insert checks
+doc: Add js_norm_max_template_nesting description
+flow: introduce bidirectional flag for expected session.
+flow: set the client initiated flag before publishing the flow state setup event
+framework: update base API version to 8
+framework: version rollback
+http_inspect: add builtin rule for consecutive commas in accept-encoding header
+http_inspect: Add JavaScript template literals normalization
+http_inspect: check if Normalizer has consumed input
+http_inspect: hard-code infraction enum numbers
+http_inspect: http_raw_header, http_raw_trailer field support
+http_inspect: refactor NormalizedHeader
+http_inspect: support more infractions and events
+http_inspect: two new built-in rules
+inspection: process wizard matches on defragged packets
+ips: add action_map table to map rule types, eg block -> alert
+ips: add action_override which applies to all rules
+lua: update comments in the default config
+modbus: check record length for write file record command
+normalize: remove tcp.trim config
+payload_injector: check if stream is established on flow rather than the packet flag to handle retries
+policy: put inspection policy accessors in public space
+policy: reorganize for sanity
+README: mention vars in default config
+sip: deprecate max_requestName_len in favor of max_request_name_len
+smb: Invoke SMB debug in destructor when packet thread available
+stream_tcp: update API called by payload_injector to check for unflushed queued TCP segments
+style: remove crufty comments
+style: remove C style (void) arglists
+style: remove or update crufty preprocessor comments
+utils: address compiler warning
+utils: support streamed processing of JS text
+wizard: support more HTTP and SIP methods
+
 2021/08/11 - 3.1.10.0
 
 appid: update netbios-ss (SMB) detector to extract SMB domain from SMBv2, and more intelligently handle payload appid detection

diff --git a/doc/reference/snort_reference.text b/doc/reference/snort_reference.text
index 48e97c58937c70d18b396e736882780e7a87d124..c19762a0bbfcbb0487e9fd7f1bd78c2ca59e0c36 100644 (file)
--- a/doc/reference/snort_reference.text
+++ b/doc/reference/snort_reference.text
@@ -8,7 +8,7 @@ Snort 3 Reference Manual
 The Snort Team
 
 Revision History
-Revision 3.1.10.0 2021-08-11 07:53:39 EDT TST
+Revision 3.1.11.0 2021-08-26 11:41:00 EDT TST
 
 ---------------------------------------------------------------------
 
@@ -890,6 +890,10 @@ Usage: detect
 
 Configuration:
 
+  * string ips.action_map[].replace: action you want to change
+  * string ips.action_map[].with: action you want to use instead
+  * string ips.action_override: use this action for all rules
+    (applied before action_map)
   * enum ips.default_rule_state = inherit: enable or disable ips
     rules { no | yes | inherit }
   * bool ips.enable_builtin_rules = false: enable events from builtin
@@ -2747,8 +2751,8 @@ Rules:
   * 133:19 (dce_smb) SMB - excessive read requests with pending read
     responses
   * 133:20 (dce_smb) SMB - excessive command chaining
-  * 133:21 (dce_smb) SMB - multiple chained tree connect requests
-  * 133:22 (dce_smb) SMB - multiple chained tree connect requests
+  * 133:21 (dce_smb) SMB - Multiple chained login requests
+  * 133:22 (dce_smb) SMB - Multiple chained tree connect requests
   * 133:23 (dce_smb) SMB - chained/compounded login followed by
     logoff
   * 133:24 (dce_smb) SMB - chained/compounded tree connect followed
@@ -3656,6 +3660,10 @@ Configuration:
     body bytes to examine (-1 no limit) { -1:max53 }
   * bool http_inspect.unzip = true: decompress gzip and deflate
     message bodies
+  * int http_inspect.maximum_host_length = -1: maximum allowed length
+    for Host header value (-1 no limit) { -1:max53 }
+  * int http_inspect.maximum_chunk_length = 4294967295: maximum
+    allowed length for a message body chunk { 0:4294967295 }
   * bool http_inspect.normalize_utf = true: normalize charset utf
     encodings in response bodies
   * bool http_inspect.decompress_pdf = false: decompress pdf files in
@@ -3673,6 +3681,9 @@ Configuration:
     normalize (-1 unlimited) (experimental) { -1:max53 }
   * int http_inspect.js_norm_identifier_depth = 260000: max number of
     unique JavaScript identifiers to normalize { 0:260000 }
+  * int http_inspect.js_norm_max_tmpl_nest = 32: maximum depth of
+    template literal nesting that enhanced javascript normalizer will
+    process (experimental) { 0:255 }
   * int http_inspect.max_javascript_whitespaces = 200: maximum
     consecutive whitespaces allowed within the JavaScript obfuscated
     data { 1:65535 }
@@ -3729,12 +3740,14 @@ Rules:
     CR
   * 119:14 (http_inspect) non-RFC defined char
   * 119:15 (http_inspect) oversize request-uri directory
+  * 119:16 (http_inspect) oversize chunk encoding
   * 119:18 (http_inspect) webroot directory traversal
   * 119:19 (http_inspect) long header
   * 119:20 (http_inspect) max header fields
   * 119:21 (http_inspect) multiple content length
   * 119:24 (http_inspect) Host header field appears more than once or
     has multiple values
+  * 119:25 (http_inspect) Host header value is too long
   * 119:28 (http_inspect) POST or PUT w/o content-length or chunks
   * 119:31 (http_inspect) unknown method
   * 119:32 (http_inspect) simple request
@@ -3855,6 +3868,10 @@ Rules:
   * 119:269 (http_inspect) script opening tag in a short form
   * 119:270 (http_inspect) max number of unique JavaScript
     identifiers reached
+  * 119:271 (http_inspect) JavaScript template literal nesting is
+    over capacity
+  * 119:272 (http_inspect) Consecutive commas in HTTP Accept-Encoding
+    header
 
 Peg counts:
 
@@ -4249,8 +4266,6 @@ Configuration:
     packet
   * bool normalizer.tcp.trim_win = false: trim data to window
   * bool normalizer.tcp.trim_mss = false: trim data to MSS
-  * bool normalizer.tcp.trim = false: enable all of the TCP trim
-    options
   * bool normalizer.tcp.opts = false: clear all options except mss,
     wscale, timestamp, and any explicitly allowed
   * bool normalizer.tcp.req_urg = false: clear the urgent pointer if
@@ -4967,8 +4982,10 @@ Configuration:
   * int sip.max_dialogs = 4: maximum number of dialogs within one
     stream session { 1:max32 }
   * int sip.max_from_len = 256: maximum from field size { 0:65535 }
-  * int sip.max_requestName_len = 20: maximum request name field size
-    { 0:65535 }
+  * int sip.max_request_name_len = 20: maximum request name field
+    size { 0:65535 }
+  * int sip.max_requestName_len = 20: deprecated - use
+    max_request_name_len instead { 0:65535 }
   * int sip.max_to_len = 256: maximum to field size { 0:65535 }
   * int sip.max_uri_len = 256: maximum request uri field size {
     0:65535 }
@@ -6774,6 +6791,8 @@ Usage: detect
 
 Configuration:
 
+  * string http_raw_header.field: restrict to given header. Header
+    name is case insensitive.
   * implied http_raw_header.request: match against the headers from
     the request message even when examining the response
   * implied http_raw_header.with_header: this rule is limited to
@@ -6860,6 +6879,8 @@ Usage: detect
 
 Configuration:
 
+  * string http_raw_trailer.field: restrict to given trailer. Trailer
+    name is case insensitive.
   * implied http_raw_trailer.request: match against the trailers from
     the request message even when examining the response
   * implied http_raw_trailer.with_header: parts of this rule examine
@@ -9149,6 +9170,13 @@ these libraries see the Getting Started section of the manual.
     normalize (-1 unlimited) (experimental) { -1:max53 }
   * int http_inspect.js_norm_identifier_depth = 260000: max number of
     unique JavaScript identifiers to normalize { 0:260000 }
+  * int http_inspect.js_norm_max_tmpl_nest = 32: maximum depth of
+    template literal nesting that enhanced javascript normalizer will
+    process (experimental) { 0:255 }
+  * int http_inspect.maximum_chunk_length = 4294967295: maximum
+    allowed length for a message body chunk { 0:4294967295 }
+  * int http_inspect.maximum_host_length = -1: maximum allowed length
+    for Host header value (-1 no limit) { -1:max53 }
   * int http_inspect.max_javascript_whitespaces = 200: maximum
     consecutive whitespaces allowed within the JavaScript obfuscated
     data { 1:65535 }
@@ -9207,6 +9235,8 @@ these libraries see the Getting Started section of the manual.
     limited to examining HTTP message headers
   * implied http_raw_header_complete.with_trailer: parts of this rule
     examine HTTP message trailers
+  * string http_raw_header.field: restrict to given header. Header
+    name is case insensitive.
   * implied http_raw_header.request: match against the headers from
     the request message even when examining the response
   * implied http_raw_header.with_body: parts of this rule examine
@@ -9225,6 +9255,8 @@ these libraries see the Getting Started section of the manual.
     HTTP message body
   * implied http_raw_status.with_trailer: parts of this rule examine
     HTTP message trailers
+  * string http_raw_trailer.field: restrict to given trailer. Trailer
+    name is case insensitive.
   * implied http_raw_trailer.request: match against the trailers from
     the request message even when examining the response
   * implied http_raw_trailer.with_body: parts of this rule examine
@@ -9323,6 +9355,10 @@ these libraries see the Getting Started section of the manual.
   * select ipopts.~opt: output format { rr|eol|nop|ts|sec|esec|lsrr|
     lsrre|ssrr|satid|any }
   * string ip_proto.~proto: [!|>|<] name or number
+  * string ips.action_map[].replace: action you want to change
+  * string ips.action_map[].with: action you want to use instead
+  * string ips.action_override: use this action for all rules
+    (applied before action_map)
   * enum ips.default_rule_state = inherit: enable or disable ips
     rules { no | yes | inherit }
   * bool ips.enable_builtin_rules = false: enable events from builtin
@@ -9464,8 +9500,6 @@ these libraries see the Getting Started section of the manual.
     urgent pointer is not set
   * bool normalizer.tcp.rsv = false: clear the reserved bits in the
     TCP header
-  * bool normalizer.tcp.trim = false: enable all of the TCP trim
-    options
   * bool normalizer.tcp.trim_mss = false: trim data to MSS
   * bool normalizer.tcp.trim_rst = false: remove any data from RST
     packet
@@ -9926,8 +9960,10 @@ these libraries see the Getting Started section of the manual.
   * int sip.max_dialogs = 4: maximum number of dialogs within one
     stream session { 1:max32 }
   * int sip.max_from_len = 256: maximum from field size { 0:65535 }
-  * int sip.max_requestName_len = 20: maximum request name field size
-    { 0:65535 }
+  * int sip.max_requestName_len = 20: deprecated - use
+    max_request_name_len instead { 0:65535 }
+  * int sip.max_request_name_len = 20: maximum request name field
+    size { 0:65535 }
   * int sip.max_to_len = 256: maximum to field size { 0:65535 }
   * int sip.max_uri_len = 256: maximum request uri field size {
     0:65535 }
@@ -11850,12 +11886,14 @@ these libraries see the Getting Started section of the manual.
     CR
   * 119:14 (http_inspect) non-RFC defined char
   * 119:15 (http_inspect) oversize request-uri directory
+  * 119:16 (http_inspect) oversize chunk encoding
   * 119:18 (http_inspect) webroot directory traversal
   * 119:19 (http_inspect) long header
   * 119:20 (http_inspect) max header fields
   * 119:21 (http_inspect) multiple content length
   * 119:24 (http_inspect) Host header field appears more than once or
     has multiple values
+  * 119:25 (http_inspect) Host header value is too long
   * 119:28 (http_inspect) POST or PUT w/o content-length or chunks
   * 119:31 (http_inspect) unknown method
   * 119:32 (http_inspect) simple request
@@ -11976,6 +12014,10 @@ these libraries see the Getting Started section of the manual.
   * 119:269 (http_inspect) script opening tag in a short form
   * 119:270 (http_inspect) max number of unique JavaScript
     identifiers reached
+  * 119:271 (http_inspect) JavaScript template literal nesting is
+    over capacity
+  * 119:272 (http_inspect) Consecutive commas in HTTP Accept-Encoding
+    header
   * 121:1 (http2_inspect) invalid flag set on HTTP/2 frame
   * 121:2 (http2_inspect) HPACK integer value has leading zeros
   * 121:3 (http2_inspect) HTTP/2 stream initiated with invalid stream
@@ -12157,8 +12199,8 @@ these libraries see the Getting Started section of the manual.
   * 133:19 (dce_smb) SMB - excessive read requests with pending read
     responses
   * 133:20 (dce_smb) SMB - excessive command chaining
-  * 133:21 (dce_smb) SMB - multiple chained tree connect requests
-  * 133:22 (dce_smb) SMB - multiple chained tree connect requests
+  * 133:21 (dce_smb) SMB - Multiple chained login requests
+  * 133:22 (dce_smb) SMB - Multiple chained tree connect requests
   * 133:23 (dce_smb) SMB - chained/compounded login followed by
     logoff
   * 133:24 (dce_smb) SMB - chained/compounded tree connect followed

diff --git a/doc/upgrade/snort_upgrade.text b/doc/upgrade/snort_upgrade.text
index 2759a777db8f32b5dcf5af889e9335f179357fef..34007db40a6b73ce1fbb269ba9b6ea9bb99296b9 100644 (file)
--- a/doc/upgrade/snort_upgrade.text
+++ b/doc/upgrade/snort_upgrade.text
@@ -8,7 +8,7 @@ Snort 3 Upgrade Manual
 The Snort Team
 
 Revision History
-Revision 3.1.10.0 2021-08-11 07:53:28 EDT TST
+Revision 3.1.11.0 2021-08-26 11:40:49 EDT TST
 
 ---------------------------------------------------------------------
 
@@ -965,6 +965,7 @@ change -> rate_filter: 'sig_id' ==> 'sid'
 change -> reputation: 'shared_mem' ==> 'list_dir'
 change -> sfportscan: 'proto' ==> 'protos'
 change -> sfportscan: 'scan_type' ==> 'scan_types'
+change -> sip: 'max_requestName_len' ==> 'max_request_name_len'
 change -> sip: 'ports' ==> 'bindings'
 change -> smtp: 'ports' ==> 'bindings'
 change -> ssh: 'server_ports' ==> 'bindings'

diff --git a/doc/user/snort_user.text b/doc/user/snort_user.text
index 76bffaa9ac02bc141d5e0e67e0bd103cdd52245c..39f0dc3aa2cfc02da12ea55a09568b257ab6fabb 100644 (file)
--- a/doc/user/snort_user.text
+++ b/doc/user/snort_user.text
@@ -8,7 +8,7 @@ Snort 3 User Manual
 The Snort Team
 
 Revision History
-Revision 3.1.10.0 2021-08-11 07:53:28 EDT TST
+Revision 3.1.11.0 2021-08-26 11:40:49 EDT TST
 
 ---------------------------------------------------------------------
 
@@ -3918,7 +3918,21 @@ js_normalization_depth is set to a non-zero value, enabling the
 enhanced normalizer. This is currently experimental and still under
 development.
 
-5.10.2.10. xff_headers
+5.10.2.10. js_norm_max_tmpl_nest
+
+js_norm_max_tmpl_nest = N {0 : 255} (default 32) is an option of the
+enhanced JavaScript normalizer that determines the deepest level of
+nested template literals to be processed. Introduced in ES6, template
+literals provide syntax to define a literal multiline string, which
+can have arbitrary JavaScript substitutions, that will be evaluated
+and inserted into the string. Such substitutions can be nested, and
+require keeping track of every layer for proper normalization. This
+option is present to limit the amount of memory dedicated to this
+tracking. This option is used only when js_normalization_depth is not
+0. This feature is currently experimental and still under
+development.
+
+5.10.2.11. xff_headers
 
 This configuration supports defining custom x-forwarded-for type
 headers. In a multi-vendor world, it is quite possible that the
@@ -3933,7 +3947,23 @@ they are defined, e.g "x-forwarded-for" will be preferred than
 "true-client-ip" if both headers are present in the stream. The
 header names should be delimited by a space.
 
-5.10.2.11. URI processing
+5.10.2.12. maximum_host_length
+
+Setting maximum_host_length causes http_inspect to generate 119:25 if
+the Host header value including optional white space exceeds the
+specified length. In the abnormal case of multiple Host headers, the
+total length of the combined values is used. The default value is -1,
+meaning do not perform this check.
+
+5.10.2.13. maximum_chunk_length
+
+http_inspect strictly limits individual chunks within a chunked
+message body to be less than four gigabytes.
+
+A lower limit may be configured by setting maximum_chunk_length. Any
+chunk longer than maximum chunk length will generate a 119:16 alert.
+
+5.10.2.14. URI processing
 
 Normalization and inspection of the URI in the HTTP request message
 is a key aspect of what http_inspect does. The best way to normalize
@@ -4208,8 +4238,8 @@ mixture of upper and lower case.
 With http_header the individual header value is normalized in a way
 that is appropriate for that header.
 
-Specifying an individual header is not available for http_raw_header
-and http_raw_header_complete.
+Specifying an individual header is not available for
+http_raw_header_complete, use http_raw_header instead.
 
 If you don’t specify a header you get all of the headers.
 http_raw_header_complete includes cookie headers Cookie and