From: Mike Stepanek (mstepane) Date: Wed, 11 Aug 2021 14:13:56 +0000 (+0000) Subject: Merge pull request #3018 in SNORT/snort3 from ~MSTEPANE/snort3:build_3.1.10.0 to... X-Git-Tag: 3.1.10.0 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=270295cc76b822ebc7ad46647593dd5d9e077d26;p=thirdparty%2Fsnort3.git Merge pull request #3018 in SNORT/snort3 from ~MSTEPANE/snort3:build_3.1.10.0 to master Squashed commit of the following: commit 806d5a1481672022a70738ee17506341f1c09ceb Author: Mike Stepanek Date: Wed Aug 11 07:48:19 2021 -0400 build: generate and tag 3.1.10.0 --- diff --git a/CMakeLists.txt b/CMakeLists.txt index 4c349f5a2..cf1f337ea 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3,7 +3,7 @@ project (snort CXX C) set (VERSION_MAJOR 3) set (VERSION_MINOR 1) -set (VERSION_PATCH 9) +set (VERSION_PATCH 10) set (VERSION_SUBLEVEL 0) set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}") diff --git a/ChangeLog b/ChangeLog index 160e2198a..edcec6ba6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,22 @@ +2021/08/11 - 3.1.10.0 + +appid: update netbios-ss (SMB) detector to extract SMB domain from SMBv2, and more intelligently handle payload appid detection +appid: use packet thread odp context while creating SIP session +build: install DAQ modules and Snort plugins in separate folders +dce_smb: restore file tracker size post deletion +dns: add DNS splitter +doc: update user manual for identifier normalization +file_api: add infra and file debugs to existing debugging framework +ftp: remove unused defines and crufty comments +http_inspect: add JavaScript identifiers normalization +http_inspect: change the default value of request_body_app_detection config parameter to true +smtp: remove unused defines +ssh: handle traffic with invalid version string +ssh: handle version string packets that also contain key exchange data +stream_tcp: skip unordered segments if last flushed position already moved past +telnet: correct help for ayt_attack_thresh +wizard: add wizard max_pattern option and update HTTP/SIP aware methods patterns + 2021/07/28 - 3.1.9.0 actions: allow session data to stay accessible for loggers for reject rule action diff --git a/doc/reference/snort_reference.text b/doc/reference/snort_reference.text index 9a4d7db3b..48e97c589 100644 --- a/doc/reference/snort_reference.text +++ b/doc/reference/snort_reference.text @@ -8,7 +8,7 @@ Snort 3 Reference Manual The Snort Team Revision History -Revision 3.1.9.0 2021-07-28 06:22:26 EDT TST +Revision 3.1.10.0 2021-08-11 07:53:39 EDT TST --------------------------------------------------------------------- @@ -1636,6 +1636,7 @@ Configuration: * int trace.modules.all: enable trace for all modules { 0:255 } * int trace.modules.dce_smb.all: enable all trace options { 0:255 } * int trace.modules.dpx.all: enable all trace options { 0:255 } + * int trace.modules.file_id.all: enable all trace options { 0:255 } * int trace.modules.snort.all: enable all trace options { 0:255 } * int trace.modules.snort.inspector_manager: enable inspector manager trace logging { 0:255 } @@ -3667,9 +3668,11 @@ Configuration: immediately upon script end * bool http_inspect.normalize_javascript = false: use legacy normalizer to normalize JavaScript in response bodies - * int http_inspect.js_normalization_depth = 0: number of input - JavaScript bytes to normalize with enhanced normalizer (-1 max - allowed value) (experimental) { -1:max53 } + * int http_inspect.js_normalization_depth = 0: enable enhanced + normalizer (0 is disabled); number of input JavaScript bytes to + normalize (-1 unlimited) (experimental) { -1:max53 } + * int http_inspect.js_norm_identifier_depth = 260000: max number of + unique JavaScript identifiers to normalize { 0:260000 } * int http_inspect.max_javascript_whitespaces = 200: maximum consecutive whitespaces allowed within the JavaScript obfuscated data { 1:65535 } @@ -3705,7 +3708,7 @@ Configuration: * string http_inspect.xff_headers = x-forwarded-for true-client-ip: specifies the xff type headers to parse and consider in the same order of preference as defined - * bool http_inspect.request_body_app_detection = false: make HTTP/2 + * bool http_inspect.request_body_app_detection = true: make HTTP/2 request message bodies available for application detection (detection requires AppId) @@ -3850,6 +3853,8 @@ Rules: * 119:268 (http_inspect) JavaScript code under the external script tags * 119:269 (http_inspect) script opening tag in a short form + * 119:270 (http_inspect) max number of unique JavaScript + identifiers reached Peg counts: @@ -3905,6 +3910,10 @@ Peg counts: JavaScripts processed (sum) * http_inspect.js_bytes: total number of JavaScript bytes processed (sum) + * http_inspect.js_identifiers: total number of unique JavaScript + identifiers processed (sum) + * http_inspect.js_identifier_overflows: total number of unique + JavaScript identifier limit overflows (sum) 5.25. iec104 @@ -5681,8 +5690,8 @@ Instance Type: multiton Configuration: - * int telnet.ayt_attack_thresh = -1: alert on this number of - consecutive Telnet AYT commands { -1:max31 } + * int telnet.ayt_attack_thresh = -1: alert beyond this number of + consecutive Telnet AYT commands (-1 is disabled) { -1:max31 } * bool telnet.check_encrypted = false: check for end of encryption * bool telnet.encrypted_traffic = false: check for encrypted Telnet * bool telnet.normalize = false: eliminate escape sequences @@ -5737,6 +5746,8 @@ Configuration: wild cards (*) * multi wizard.curses: enable service identification based on internal algorithm { dce_smb | dce_udp | dce_tcp | sslv2 } + * int wizard.max_pattern = 64: maximum scan depth per segment (0 is + unlimited) { 0:65535 } Peg counts: @@ -9133,9 +9144,11 @@ these libraries see the Getting Started section of the manual. mapping to normalize characters * string http_inspect.iis_unicode_map_file: file containing code points for IIS unicode. { (optional) } - * int http_inspect.js_normalization_depth = 0: number of input - JavaScript bytes to normalize with enhanced normalizer (-1 max - allowed value) (experimental) { -1:max53 } + * int http_inspect.js_normalization_depth = 0: enable enhanced + normalizer (0 is disabled); number of input JavaScript bytes to + normalize (-1 unlimited) (experimental) { -1:max53 } + * int http_inspect.js_norm_identifier_depth = 260000: max number of + unique JavaScript identifiers to normalize { 0:260000 } * int http_inspect.max_javascript_whitespaces = 200: maximum consecutive whitespaces allowed within the JavaScript obfuscated data { 1:65535 } @@ -9149,7 +9162,7 @@ these libraries see the Getting Started section of the manual. encodings * bool http_inspect.plus_to_space = true: replace + with when normalizing URIs - * bool http_inspect.request_body_app_detection = false: make HTTP/2 + * bool http_inspect.request_body_app_detection = true: make HTTP/2 request message bodies available for application detection (detection requires AppId) * int http_inspect.request_depth = -1: maximum request message body @@ -10352,8 +10365,8 @@ these libraries see the Getting Started section of the manual. * string tcp_connector[].connector: connector name * enum tcp_connector[].setup: stream establishment { call | answer } - * int telnet.ayt_attack_thresh = -1: alert on this number of - consecutive Telnet AYT commands { -1:max31 } + * int telnet.ayt_attack_thresh = -1: alert beyond this number of + consecutive Telnet AYT commands (-1 is disabled) { -1:max31 } * bool telnet.check_encrypted = false: check for end of encryption * bool telnet.encrypted_traffic = false: check for encrypted Telnet * bool telnet.normalize = false: eliminate escape sequences @@ -10370,6 +10383,7 @@ these libraries see the Getting Started section of the manual. * int trace.modules.all: enable trace for all modules { 0:255 } * int trace.modules.dce_smb.all: enable all trace options { 0:255 } * int trace.modules.dpx.all: enable all trace options { 0:255 } + * int trace.modules.file_id.all: enable all trace options { 0:255 } * int trace.modules.snort.all: enable all trace options { 0:255 } * int trace.modules.snort.inspector_manager: enable inspector manager trace logging { 0:255 } @@ -10407,6 +10421,8 @@ these libraries see the Getting Started section of the manual. chars (?) * string wizard.hexes[].to_server[].hex: sequence of data with wild chars (?) + * int wizard.max_pattern = 64: maximum scan depth per segment (0 is + unlimited) { 0:65535 } * bool wizard.spells[].client_first = true: which end initiates data transfer * select wizard.spells[].proto = tcp: protocol to scan { tcp | udp @@ -10966,6 +10982,10 @@ these libraries see the Getting Started section of the manual. (sum) * http_inspect.js_external_scripts: total number of external JavaScripts processed (sum) + * http_inspect.js_identifier_overflows: total number of unique + JavaScript identifier limit overflows (sum) + * http_inspect.js_identifiers: total number of unique JavaScript + identifiers processed (sum) * http_inspect.js_inline_scripts: total number of inline JavaScripts processed (sum) * http_inspect.max_concurrent_sessions: maximum concurrent http @@ -11954,6 +11974,8 @@ these libraries see the Getting Started section of the manual. * 119:268 (http_inspect) JavaScript code under the external script tags * 119:269 (http_inspect) script opening tag in a short form + * 119:270 (http_inspect) max number of unique JavaScript + identifiers reached * 121:1 (http2_inspect) invalid flag set on HTTP/2 frame * 121:2 (http2_inspect) HPACK integer value has leading zeros * 121:3 (http2_inspect) HTTP/2 stream initiated with invalid stream diff --git a/doc/upgrade/snort_upgrade.text b/doc/upgrade/snort_upgrade.text index a00f298b4..2759a777d 100644 --- a/doc/upgrade/snort_upgrade.text +++ b/doc/upgrade/snort_upgrade.text @@ -8,7 +8,7 @@ Snort 3 Upgrade Manual The Snort Team Revision History -Revision 3.1.9.0 2021-07-28 06:22:15 EDT TST +Revision 3.1.10.0 2021-08-11 07:53:28 EDT TST --------------------------------------------------------------------- diff --git a/doc/user/snort_user.text b/doc/user/snort_user.text index c6b642ae1..76bffaa9a 100644 --- a/doc/user/snort_user.text +++ b/doc/user/snort_user.text @@ -8,7 +8,7 @@ Snort 3 User Manual The Snort Team Revision History -Revision 3.1.9.0 2021-07-28 06:22:15 EDT TST +Revision 3.1.10.0 2021-08-11 07:53:28 EDT TST --------------------------------------------------------------------- @@ -3895,11 +3895,30 @@ means that normalizer is disabled. The enhanced normalizer provides more precise whitespace normalization of JavaScript, that removes all redundant whitespaces and line terminators from the JavaScript syntax point of view (between identifier and punctuator, between identifier -and operator, etc.) according to ECMAScript 5.1 standard. The -normalized data is available through the script_data rule option. -This is currently experimental and still under development. - -5.10.2.9. xff_headers +and operator, etc.) according to ECMAScript 5.1 standard. +Additionally, it performs normalization of JavaScript identifiers +making a substitution of unique names with unified names +representation: a0 → z9999. The identifiers are variables and +function names. The normalized data is available through the +script_data rule option. This is currently experimental and still +under development. + +5.10.2.9. js_norm_identifier_depth + +js_norm_identifier_depth = N {0 : 260000} will set a number of unique +JavaScript identifiers to normalize. When the depth is reached, a +built-in alert is generated. It’s implemented per HTTP transaction +(request/response), so the context of identifier substitutions is +shared between all the scripts in the payload. By default, the value +is set to 260000, which is the max allowed number of unique +identifiers. The generated names are in the range from a0 to z9999. +Thus, the number of unique identifiers cannot be greater than 26 * +10000 = 260000. This option takes effect only if +js_normalization_depth is set to a non-zero value, enabling the +enhanced normalizer. This is currently experimental and still under +development. + +5.10.2.10. xff_headers This configuration supports defining custom x-forwarded-for type headers. In a multi-vendor world, it is quite possible that the @@ -3914,7 +3933,7 @@ they are defined, e.g "x-forwarded-for" will be preferred than "true-client-ip" if both headers are present in the stream. The header names should be delimited by a space. -5.10.2.10. URI processing +5.10.2.11. URI processing Normalization and inspection of the URI in the HTTP request message is a key aspect of what http_inspect does. The best way to normalize