From: Steve Chew (stechew) Date: Thu, 17 Nov 2022 16:22:55 +0000 (+0000) Subject: Pull request #3670: build: generate and tag 3.1.47.0 X-Git-Tag: 3.1.47.0 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=913bb5773d2995716c26b7f1046ec2cc939bbf0a;p=thirdparty%2Fsnort3.git Pull request #3670: build: generate and tag 3.1.47.0 Merge in SNORT/snort3 from ~PRBG/snort3:build_3.1.47.0 to master Squashed commit of the following: commit fe159caeea79e73e48063207c5b0a8cf70594ee2 Author: Priyanka Gurudev Date: Wed Nov 16 21:49:48 2022 -0500 build: generate and tag 3.1.47.0 --- diff --git a/CMakeLists.txt b/CMakeLists.txt index 98739843a..05681c109 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3,7 +3,7 @@ project (snort CXX C) set (VERSION_MAJOR 3) set (VERSION_MINOR 1) -set (VERSION_PATCH 46) +set (VERSION_PATCH 47) set (VERSION_SUBLEVEL 0) set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}") diff --git a/ChangeLog.md b/ChangeLog.md index 6322750e5..e1f52bb86 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,3 +1,25 @@ +2022-11-17: 3.1.47.0 + +* appid: add a changed bit for discovery finished +* appid: ntp detection improvements +* appid: service, client and payload detection by lua detectors and third-party when first packet re-inspection is enabled +* doc: add JavaScript Normalization section to user manual +* doc: add js_norm alerts to builtin_stubs.txt +* http_inspect: subdivide dev_notes into topics +* http_inspect: move Enhanced JS Normalizer from NHI to a standalone component +* js_norm: implement standalone Enhanced JavaScript Normalizer +* main: dump packet trace after publishing finalize event since verdict could be modified. +* main: update to improve performance by making packet tracer checks before calling function. +* netflow: implement deferred trust, cleanup +* packet_io: allow ACT_TRUST to be used as a delayed action. +* packet_io: the most strict delayed action takes precedence. +* smtp: do not accumulate cmds across policies and reloads. Avoids memory and performance problem. +* stream: add info about the splitter lifetime to dev_notes +* stream: ignore flushing from meta-ack if sent after FIN +* stream: remove splitter from session before inspectors +* stream: set splitter only on initialized tcp sessions or if midstream sessions are allowed +* wizard: remove inspector's ref counter increments from MagicSplitter + 2022-11-04: 3.1.46.0 * appid: check for empty patterns in lua detector api input diff --git a/doc/reference/snort_reference.text b/doc/reference/snort_reference.text index 76a85cff9..3a6ceaf55 100644 --- a/doc/reference/snort_reference.text +++ b/doc/reference/snort_reference.text @@ -8,7 +8,7 @@ Snort 3 Reference Manual The Snort Team Revision History -Revision 3.1.46.0 2022-11-06 23:55:31 EST TST +Revision 3.1.47.0 2022-11-16 21:37:26 EST TST --------------------------------------------------------------------- @@ -33,22 +33,23 @@ Table of Contents 2.14. hosts 2.15. inspection 2.16. ips - 2.17. latency - 2.18. memory - 2.19. network - 2.20. output - 2.21. packet_tracer - 2.22. packets - 2.23. payload_injector - 2.24. process - 2.25. profiler - 2.26. rate_filter - 2.27. references - 2.28. search_engine - 2.29. side_channel - 2.30. snort - 2.31. suppress - 2.32. trace + 2.17. js_norm + 2.18. latency + 2.19. memory + 2.20. network + 2.21. output + 2.22. packet_tracer + 2.23. packets + 2.24. payload_injector + 2.25. process + 2.26. profiler + 2.27. rate_filter + 2.28. references + 2.29. search_engine + 2.30. side_channel + 2.31. snort + 2.32. suppress + 2.33. trace 3. Codec Modules @@ -965,7 +966,57 @@ Configuration: * string ips.variables.ports.$var: IPS policy variable -2.17. latency +2.17. js_norm + +-------------- + +Help: JavaScript normalizer + +Type: basic + +Usage: inspect + +Configuration: + + * int js_norm.bytes_depth = -1: number of input JavaScript bytes to + normalize (-1 unlimited) { -1:max53 } + * int js_norm.identifier_depth = 65536: max number of unique + JavaScript identifiers to normalize { 0:65536 } + * int js_norm.max_tmpl_nest = 32: maximum depth of template literal + nesting that enhanced JavaScript normalizer will process { 0:255 + } + * int js_norm.max_bracket_depth = 256: maximum depth of bracket + nesting that enhanced JavaScript normalizer will process { + 1:65535 } + * int js_norm.max_scope_depth = 256: maximum depth of scope nesting + that enhanced JavaScript normalizer will process { 1:65535 } + * string js_norm.ident_ignore[].ident_name: name of the identifier + to ignore + * string js_norm.prop_ignore[].prop_name: name of the object + property to ignore + +Rules: + + * 154:1 (js_norm) nested unescape functions + * 154:2 (js_norm) mixed unescape sequence + * 154:3 (js_norm) bad token + * 154:4 (js_norm) unexpected HTML script opening tag + * 154:5 (js_norm) unexpected HTML script closing tag + * 154:6 (js_norm) max number of unique identifiers reached + * 154:7 (js_norm) excessive bracket nesting + * 154:8 (js_norm) data gaps during normalization + * 154:9 (js_norm) excessive scope nesting + +Peg counts: + + * js_norm.bytes: total number of bytes processed (sum) + * js_norm.identifiers: total number of unique identifiers processed + (sum) + * js_norm.identifier_overflows: total number of unique identifier + limit overflows (sum) + + +2.18. latency -------------- @@ -1008,7 +1059,7 @@ Peg counts: * latency.rule_tree_enables: rule tree re-enables (sum) -2.18. memory +2.19. memory -------------- @@ -1036,7 +1087,7 @@ Peg counts: * memory.max_in_use: maximum memory used (max) -2.19. network +2.20. network -------------- @@ -1074,7 +1125,7 @@ Commands: the user policy id -2.20. output +2.21. output -------------- @@ -1112,7 +1163,7 @@ Rules: * 2:1 (output) tagged packet -2.21. packet_tracer +2.22. packet_tracer -------------- @@ -1136,7 +1187,7 @@ Commands: * packet_tracer.disable(): disable packet tracer -2.22. packets +2.23. packets -------------- @@ -1162,7 +1213,7 @@ Configuration: are used to track fragments and connections -2.23. payload_injector +2.24. payload_injector -------------- @@ -1184,7 +1235,7 @@ Peg counts: inject mid-frame (sum) -2.24. process +2.25. process -------------- @@ -1216,7 +1267,7 @@ Configuration: (seconds, 0 to disable) { 0:60 } -2.25. profiler +2.26. profiler -------------- @@ -1251,7 +1302,7 @@ Configuration: avg_match | avg_no_match } -2.26. rate_filter +2.27. rate_filter -------------- @@ -1283,7 +1334,7 @@ Peg counts: memory (sum) -2.27. references +2.28. references -------------- @@ -1299,7 +1350,7 @@ Configuration: * string references[].url: where this reference is defined -2.28. search_engine +2.29. search_engine -------------- @@ -1363,7 +1414,7 @@ Peg counts: * search_engine.searched_bytes: total bytes searched (sum) -2.29. side_channel +2.30. side_channel -------------- @@ -1385,7 +1436,7 @@ Peg counts: * side_channel.packets: total packets (sum) -2.30. snort +2.31. snort -------------- @@ -1664,7 +1715,7 @@ Peg counts: failed due to attribute table full (sum) -2.31. suppress +2.32. suppress -------------- @@ -1684,7 +1735,7 @@ Configuration: according to track -2.32. trace +2.33. trace -------------- @@ -1700,12 +1751,10 @@ Configuration: * int trace.modules.dce_smb.all: enable all trace options { 0:255 } * int trace.modules.dpx.all: enable all trace options { 0:255 } * int trace.modules.file_id.all: enable all trace options { 0:255 } - * int trace.modules.http_inspect.all: enable all trace options { - 0:255 } - * int trace.modules.http_inspect.js_proc: enable JavaScript - processing logging { 0:255 } - * int trace.modules.http_inspect.js_dump: enable JavaScript data - logging { 0:255 } + * int trace.modules.js_norm.all: enable all trace options { 0:255 } + * int trace.modules.js_norm.proc: enable processing logging { 0:255 + } + * int trace.modules.js_norm.dump: enable data logging { 0:255 } * int trace.modules.snort.all: enable all trace options { 0:255 } * int trace.modules.snort.inspector_manager: enable inspector manager trace logging { 0:255 } @@ -3757,23 +3806,6 @@ Configuration: immediately upon script end * bool http_inspect.normalize_javascript = false: use legacy normalizer to normalize JavaScript in response bodies - * int http_inspect.js_norm_bytes_depth = -1: number of input - JavaScript bytes to normalize (-1 unlimited) { -1:max53 } - * int http_inspect.js_norm_identifier_depth = 65536: max number of - unique JavaScript identifiers to normalize { 0:65536 } - * int http_inspect.js_norm_max_tmpl_nest = 32: maximum depth of - template literal nesting that enhanced javascript normalizer will - process { 0:255 } - * int http_inspect.js_norm_max_bracket_depth = 256: maximum depth - of bracket nesting that enhanced JavaScript normalizer will - process { 1:65535 } - * int http_inspect.js_norm_max_scope_depth = 256: maximum depth of - scope nesting that enhanced JavaScript normalizer will process { - 1:65535 } - * string http_inspect.js_norm_ident_ignore[].ident_name: name of - the identifier to ignore - * string http_inspect.js_norm_prop_ignore[].prop_name: name of the - object property to ignore * int http_inspect.max_javascript_whitespaces = 200: maximum consecutive whitespaces allowed within the JavaScript obfuscated data { 1:65535 } @@ -3977,21 +4009,11 @@ Rules: * 119:262 (http_inspect) HTTP URI scheme longer than 10 characters * 119:263 (http_inspect) HTTP/1 client requested HTTP/2 upgrade * 119:264 (http_inspect) HTTP/1 server granted HTTP/2 upgrade - * 119:265 (http_inspect) bad token in JavaScript - * 119:266 (http_inspect) unexpected script opening tag in - JavaScript - * 119:267 (http_inspect) unexpected script closing tag in - JavaScript * 119:268 (http_inspect) JavaScript code under the external script tags * 119:269 (http_inspect) script opening tag in a short form - * 119:270 (http_inspect) max number of unique JavaScript - identifiers reached - * 119:271 (http_inspect) excessive JavaScript bracket nesting * 119:272 (http_inspect) Consecutive commas in HTTP Accept-Encoding header - * 119:273 (http_inspect) data gaps during JavaScript normalization - * 119:274 (http_inspect) excessive JavaScript scope nesting * 119:275 (http_inspect) HTTP/1 version other than 1.0 or 1.1 * 119:276 (http_inspect) HTTP version in start line is 0 * 119:277 (http_inspect) HTTP version in start line is higher than @@ -4065,12 +4087,6 @@ Peg counts: JavaScripts processed (sum) * http_inspect.js_external_scripts: total number of external JavaScripts processed (sum) - * http_inspect.js_bytes: total number of JavaScript bytes processed - (sum) - * http_inspect.js_identifiers: total number of unique JavaScript - identifiers processed (sum) - * http_inspect.js_identifier_overflows: total number of unique - JavaScript identifier limit overflows (sum) * http_inspect.skip_mime_attach: total number of HTTP requests with too many MIME attachments to inspect (sum) @@ -9572,23 +9588,6 @@ libraries see the Getting Started section of the manual. mapping to normalize characters * string http_inspect.iis_unicode_map_file: file containing code points for IIS unicode. { (optional) } - * int http_inspect.js_norm_bytes_depth = -1: number of input - JavaScript bytes to normalize (-1 unlimited) { -1:max53 } - * int http_inspect.js_norm_identifier_depth = 65536: max number of - unique JavaScript identifiers to normalize { 0:65536 } - * string http_inspect.js_norm_ident_ignore[].ident_name: name of - the identifier to ignore - * int http_inspect.js_norm_max_bracket_depth = 256: maximum depth - of bracket nesting that enhanced JavaScript normalizer will - process { 1:65535 } - * int http_inspect.js_norm_max_scope_depth = 256: maximum depth of - scope nesting that enhanced JavaScript normalizer will process { - 1:65535 } - * int http_inspect.js_norm_max_tmpl_nest = 32: maximum depth of - template literal nesting that enhanced javascript normalizer will - process { 0:255 } - * string http_inspect.js_norm_prop_ignore[].prop_name: name of the - object property to ignore * int http_inspect.maximum_chunk_length = 4294967295: maximum allowed length for a message body chunk { 0:4294967295 } * int http_inspect.maximum_header_length = 4096: alert when the @@ -9856,6 +9855,22 @@ libraries see the Getting Started section of the manual. buffer * interval itype.~range: check if ICMP type is in given range { 0:255 } + * int js_norm.bytes_depth = -1: number of input JavaScript bytes to + normalize (-1 unlimited) { -1:max53 } + * int js_norm.identifier_depth = 65536: max number of unique + JavaScript identifiers to normalize { 0:65536 } + * string js_norm.ident_ignore[].ident_name: name of the identifier + to ignore + * int js_norm.max_bracket_depth = 256: maximum depth of bracket + nesting that enhanced JavaScript normalizer will process { + 1:65535 } + * int js_norm.max_scope_depth = 256: maximum depth of scope nesting + that enhanced JavaScript normalizer will process { 1:65535 } + * int js_norm.max_tmpl_nest = 32: maximum depth of template literal + nesting that enhanced JavaScript normalizer will process { 0:255 + } + * string js_norm.prop_ignore[].prop_name: name of the object + property to ignore * bool latency.packet.fastpath = false: fastpath expensive packets (max_time exceeded) * int latency.packet.max_time = 500: set timeout for packet latency @@ -10899,12 +10914,10 @@ libraries see the Getting Started section of the manual. * int trace.modules.dce_smb.all: enable all trace options { 0:255 } * int trace.modules.dpx.all: enable all trace options { 0:255 } * int trace.modules.file_id.all: enable all trace options { 0:255 } - * int trace.modules.http_inspect.all: enable all trace options { - 0:255 } - * int trace.modules.http_inspect.js_dump: enable JavaScript data - logging { 0:255 } - * int trace.modules.http_inspect.js_proc: enable JavaScript - processing logging { 0:255 } + * int trace.modules.js_norm.all: enable all trace options { 0:255 } + * int trace.modules.js_norm.dump: enable data logging { 0:255 } + * int trace.modules.js_norm.proc: enable processing logging { 0:255 + } * int trace.modules.snort.all: enable all trace options { 0:255 } * int trace.modules.snort.inspector_manager: enable inspector manager trace logging { 0:255 } @@ -11514,14 +11527,8 @@ libraries see the Getting Started section of the manual. * http_inspect.get_requests: GET requests inspected (sum) * http_inspect.head_requests: HEAD requests inspected (sum) * http_inspect.inspections: total message sections inspected (sum) - * http_inspect.js_bytes: total number of JavaScript bytes processed - (sum) * http_inspect.js_external_scripts: total number of external JavaScripts processed (sum) - * http_inspect.js_identifier_overflows: total number of unique - JavaScript identifier limit overflows (sum) - * http_inspect.js_identifiers: total number of unique JavaScript - identifiers processed (sum) * http_inspect.js_inline_scripts: total number of inline JavaScripts processed (sum) * http_inspect.max_concurrent_sessions: maximum concurrent http @@ -11590,6 +11597,11 @@ libraries see the Getting Started section of the manual. * imap.uu_decoded_bytes: total uu decoded bytes (sum) * ipv4.bad_checksum: nonzero ip checksums (sum) * ipv4.checksum_bypassed: checksum calculations bypassed (sum) + * js_norm.bytes: total number of bytes processed (sum) + * js_norm.identifier_overflows: total number of unique identifier + limit overflows (sum) + * js_norm.identifiers: total number of unique identifiers processed + (sum) * latency.max_usecs: maximum usecs elapsed (sum) * latency.packet_timeouts: packets that timed out (sum) * latency.rule_eval_timeouts: rule evals that timed out (sum) @@ -12185,6 +12197,7 @@ libraries see the Getting Started section of the manual. * 150: file_id * 151: iec104 * 152: mms + * 154: js_norm * 175: domain_filter * 256: dpx @@ -12767,12 +12780,12 @@ session. The TCP packet is invalid because it doesn’t have a SYN, ACK, or RST flag set. -116:424 (pbb) truncated ethernet header +116:424 (eth) truncated ethernet header The packet length is less than the minimum ethernet header size (14 bytes) -116:424 (pbb) truncated ethernet header +116:424 (eth) truncated ethernet header A truncated ethernet header was detected. @@ -13594,25 +13607,6 @@ A client sent a request to upgrade an HTTP/1 connection to HTTP/2. A server granted a request to upgrade a connection from HTTP/1 to HTTP/2. -119:265 (http_inspect) bad token in JavaScript - -Enhanced JavaScript normalizer has encountered a symbol that is not -expected as a part of a valid JavaScript statement, making further -normalization impossible. - -119:266 (http_inspect) unexpected script opening tag in JavaScript - -HTML end-tag is encountered inside a -JavaScript comment or literal, which is a syntax error, as the last -comment or literal is not closed before script end. This alert is -raised by the enhanced JavaScript normalizer. - 119:268 (http_inspect) JavaScript code under the external script tags When HTML end-tag is encountered inside a +JavaScript comment or literal, which is a syntax error, as the last +comment or literal is not closed before script end. This alert is +raised by the enhanced JavaScript normalizer. + +154:6 (js_norm) max number of unique identifiers reached + +JavaScript normalization includes identifier substitution, which +brings arbitrary JavaScript identifiers to a common form. Amount of +unique identifiers to normalize is limited, for memory +considerations, with http_inspect.js_norm_identifier_depth parameter. +When this threshold is reached, a corresponding alert is raised. This +alert is not expected for typical network traffic and may be an +indication that an attacker is trying to exhaust resources. This +alert is raised by the enhanced JavaScript normalizer. + +154:7 (js_norm) excessive bracket nesting + +In JavaScript, template literals can have substitutions, that in turn +can have nested template literals, which requires a stack to track +for proper whitespace normalization. Also, the normalization tracks +the current bracket scope, which requires a stack as well. When the +depth of nesting exceeds limit set in +http_inspect.js_norm_max_tmpl_nest or in +http_inspect.js_norm_max_bracket_depth, this alert is raised. This +alert is not expected for typical network traffic and may be an +indication that an attacker is trying to exhaust resources. This +alert is raised by the enhanced JavaScript normalizer. + +154:8 (js_norm) data gaps during normalization + +This alert is raised for the following situation. During JavaScript +normalization some data can be lost and not normalized. Usually it +happens when rules have file_data and js_data ips options and +fast-pattern (FP) search is applying to file_data. Some data doesn’t +match file_data FP search and JavaScript normalization won’t be +executed for it. The following normalization for inline/external +scripts will be stopped for current request within the flow. This +alert is raised by the enhanced JavaScript normalizer. + +154:9 (js_norm) excessive scope nesting + +To resolve variable names in JavaScript, a current stack of variable +scopes has to be tracked. When the depth of nesting exceeds the limit +set in http_inspect.js_norm_max_scope_depth, this alert is raised. +This alert is not expected for typical network traffic and may be an +indication that an attacker is trying to exhaust resources. This +alert is raised by the enhanced JavaScript normalizer. + 175:1 (domain_filter) configured domain detected (domain_filter) configured domain detected @@ -15742,6 +15764,7 @@ and are not applicable elsewhere. * itype (ips_option): rule option to check ICMP type * js_data (ips_option): rule option to set detection cursor to normalized JavaScript data + * js_norm (basic): JavaScript normalizer * latency (basic): packet and rule latency monitoring and control * llc (codec): support for logical link control * log_codecs (logger): log protocols in packet by layer diff --git a/doc/upgrade/snort_upgrade.text b/doc/upgrade/snort_upgrade.text index fb541970a..520a3ffb1 100644 --- a/doc/upgrade/snort_upgrade.text +++ b/doc/upgrade/snort_upgrade.text @@ -8,7 +8,7 @@ Snort 3 Upgrade Manual The Snort Team Revision History -Revision 3.1.46.0 2022-11-06 23:55:16 EST TST +Revision 3.1.47.0 2022-11-16 21:38:32 EST TST --------------------------------------------------------------------- diff --git a/doc/user/snort_user.text b/doc/user/snort_user.text index ad411dd08..d6cb87134 100644 --- a/doc/user/snort_user.text +++ b/doc/user/snort_user.text @@ -8,7 +8,7 @@ Snort 3 User Manual The Snort Team Revision History -Revision 3.1.46.0 2022-11-06 23:55:16 EST TST +Revision 3.1.47.0 2022-11-16 21:37:48 EST TST --------------------------------------------------------------------- @@ -67,15 +67,16 @@ Table of Contents 5.10. HTTP Inspector 5.11. HTTP/2 Inspector 5.12. IEC104 Inspector - 5.13. MMS Inspector - 5.14. Performance Monitor - 5.15. POP and IMAP - 5.16. Port Scan - 5.17. Sensitive Data Filtering - 5.18. SMTP - 5.19. Telnet - 5.20. Trace - 5.21. Wizard + 5.13. JavaScript Normalization + 5.14. MMS Inspector + 5.15. Performance Monitor + 5.16. POP and IMAP + 5.17. Port Scan + 5.18. Sensitive Data Filtering + 5.19. SMTP + 5.20. Telnet + 5.21. Trace + 5.22. Wizard 6. DAQ Configuration and Modules @@ -3903,7 +3904,13 @@ normalization. Both normalizers are independent and can be configured separately. The Legacy normalizer should be considered deprecated. The Enhanced Normalizer is encouraged to use for JavaScript normalization in the first place as we continue improving -functionality and quality. +functionality and quality. The Enhanced JavaScript Normalizer has to +be configured as a separate module: + +js_norm = {} + +Refer to JavaScript Normalization section for Enhanced Normalizer +specifics. 5.10.2.1. Legacy Normalizer @@ -3917,28 +3924,6 @@ option. Legacy Normalizer is deprecated preferably to use Enhanced Normalizer. After supporting backward compatibility in the Enhanced Normalizer, Legacy Normalizer will be removed. -5.10.2.2. Enhanced Normalizer - -Having ips option js_data in the rules automatically enables Enhanced -Normalizer. The Enhanced Normalizer can normalize inline/external -scripts. It supports scripts over multiple PDUs. It is a stateful -JavaScript whitespace and identifiers normalizer. Normalizer -concatenates string literals whenever it’s possible to do. This also -works with any other normalizations that result in string literals. -All JavaScript identifier names, except those from the ignore lists, -will be substituted with unified names in the following format: -var_0000 → var_ffff. But the unescape-like function names will be -removed from the normalized data. The Normalizer tries to expand an -escaped text, so it will appear in a usual form in the output. -Moreover, Normalizer validates the syntax concerning ECMA-262 -Standard, including scope tracking and restrictions for script -elements. For more information on how additionally configure Enhanced -Normalizer check with the following configuration options: -js_norm_bytes_depth, js_norm_identifier_depth, js_norm_max_tmpl_nest, -js_norm_max_bracket_depth, js_norm_max_scope_depth, -js_norm_ident_ignore, js_norm_prop_ignore. Eventually Enhanced -Normalizer will completely replace Legacy Normalizer. - 5.10.3. Configuration Configuration can be as simple as adding: @@ -4085,137 +4070,7 @@ http_inspect also replaces consecutive whitespaces with a single space and normalizes the plus by concatenating the strings. Such normalizations refer to basic JavaScript normalization. -5.10.3.12. js_norm_bytes_depth - -js_norm_bytes_depth = N {-1 : max53} will set a number of input -JavaScript bytes to normalize. When the depth is reached, -normalization will be stopped. It’s implemented per-script. By -default js_norm_bytes_depth = -1, will set unlimited depth. The -enhanced normalizer provides more precise whitespace normalization of -JavaScript, that removes all redundant whitespaces and line -terminators from the JavaScript syntax point of view (between -identifier and punctuator, between identifier and operator, etc.) -according to ECMAScript 5.1 standard. Additionally, it performs -normalization of JavaScript identifiers making a substitution of -unique names with unified names representation: var_0000:var_ffff. -The identifiers are variables and function names. The normalized data -is available through the js_data rule option. - -5.10.3.13. js_norm_identifier_depth - -js_norm_identifier_depth = N {0 : 65536} will set a number of unique -JavaScript identifiers to normalize. When the depth is reached, a -built-in alert is generated. Every HTTP response has its own -identifier substitution context, which means that identifier will -retain same normal form in multiple scripts, if they are a part of -the same HTTP response, and that this limit is set for a single HTTP -response and not a single script. By default, the value is set to -65536, which is the max allowed number of unique identifiers. The -generated names are in the range from var_0000 to var_ffff. - -5.10.3.14. js_norm_max_tmpl_nest - -js_norm_max_tmpl_nest = N {0 : 255} (default 32) is an option of the -enhanced JavaScript normalizer that determines the deepest level of -nested template literals to be processed. Introduced in ES6, template -literals provide syntax to define a literal multiline string, which -can have arbitrary JavaScript substitutions, that will be evaluated -and inserted into the string. Such substitutions can be nested, and -require keeping track of every layer for proper normalization. This -option is present to limit the amount of memory dedicated to template -nesting tracking. - -5.10.3.15. js_norm_max_bracket_depth - -js_norm_max_bracket_depth = N {1 : 65535} (default 256) is an option -of the enhanced JavaScript normalizer that determines the maximum -depth of nesting brackets, i.e. parentheses, braces and square -brackets, nested within a matching pair, in any combination. This -option is present to limit the amount of memory dedicated to bracket -tracking. - -5.10.3.16. js_norm_max_scope_depth - -js_norm_max_scope_depth = N {1 : 65535} (default 256) is an option of -the enhanced JavaScript normalizer that determines the deepest level -of nested variable scope, i.e. functions, code blocks, etc. including -the global scope. This option is present to limit the amount of -memory dedicated to scope tracking. - -5.10.3.17. js_norm_ident_ignore - -js_norm_ident_ignore = {} is an option -of the enhanced JavaScript normalizer that defines a list of -identifiers to keep intact. - -Identifiers in this list will not be put into normal form (var_0000). -Subsequent accessors, after dot, in square brackets or after function -call, will not be normalized as well. - -For example: - -console.log("bar") -document.getElementById("id").text -eval("script") -console["log"] - -Every entry has to be a simple identifier, i.e. not include dots, -brackets, etc. For example: - -http_inspect.js_norm_ident_ignore = { 'console', 'document', 'eval', 'foo' } - -When a variable assignment that aliases an identifier from the list -is found, the assignment will be tracked, and subsequent occurrences -of the variable will be replaced with the stored value. This -substitution will follow JavaScript variable scope limits. - -For example: - -var a = console.log -a("hello") // will be substituted to 'console.log("hello")' - -For class names and constructors in the list, when the class is used -with the keyword new, created object will be tracked, and its -properties will be kept intact. Identifier of the object itself, -however, will be brought to unified form. - -For example: - -var o = new Array() // normalized to 'var var_0000=new Array()' -o.push(10) // normalized to 'var_0000.push(10)' - -The default list of ignore-identifiers is present in -"snort_defaults.lua". - -Unescape function names should remain intact in the output. They -ought to be included in the ignore list. If for some reason the user -wants to disable unescape related features, then removing function’s -name from the ignore list does the trick. - -5.10.3.18. js_norm_prop_ignore - -js_norm_prop_ignore = {} is an option of -the enhanced JavaScript normalizer that defines a list of object -properties and methods that will be kept intact during the -identifiers normalization. This list should include methods and -properties of objects that will not be tracked by assignment -substitution functionality, for example, those that can be created -implicitly. - -Subsequent accessors, after dot, in square brackets or after function -call, will not be normalized as well. - -For example: - -http_inspect.js_norm_prop_ignore = { 'split' } - -in: "string".toUpperCase().split("").reverse().join(""); -out: "string".var_0000().split("").reverse().join(""); - -The default list of ignored properties is present in -"snort_defaults.lua". - -5.10.3.19. xff_headers +5.10.3.12. xff_headers This configuration supports defining custom x-forwarded-for type headers. In a multi-vendor world, it is quite possible that the @@ -4230,7 +4085,7 @@ they are defined, e.g "x-forwarded-for" will be preferred than "true-client-ip" if both headers are present in the stream. The header names should be delimited by a space. -5.10.3.20. maximum_host_length +5.10.3.13. maximum_host_length Setting maximum_host_length causes http_inspect to generate 119:25 if the Host header value including optional white space exceeds the @@ -4238,7 +4093,7 @@ specified length. In the abnormal case of multiple Host headers, the total length of the combined values is used. The default value is -1, meaning do not perform this check. -5.10.3.21. maximum_chunk_length +5.10.3.14. maximum_chunk_length http_inspect strictly limits individual chunks within a chunked message body to be less than four gigabytes. @@ -4246,17 +4101,17 @@ message body to be less than four gigabytes. A lower limit may be configured by setting maximum_chunk_length. Any chunk longer than maximum chunk length will generate a 119:16 alert. -5.10.3.22. maximum_header_length +5.10.3.15. maximum_header_length http_inspect generates 119:19 when the length of a header exceeds maximum_header_length = N {0 : 65535} (default 4096). -5.10.3.23. maximum_headers +5.10.3.16. maximum_headers http_inspect generates 119:20 when the number of headers exceeds maximum_headers = N {0 : 65535} (default 200). -5.10.3.24. maximum_pipelined_requests +5.10.3.17. maximum_pipelined_requests http_inspect generates 119:34 when the number of pipelined requests exceeds maximum_pipelined_requests = N {0 : 99} (default 99). This @@ -4266,7 +4121,7 @@ the case of an alternating sequence of requests and responses. It will trigger the alert once the client issue a request before getting the response to a previous request. -5.10.3.25. URI processing +5.10.3.18. URI processing Normalization and inspection of the URI in the HTTP request message is a key aspect of what http_inspect does. The best way to normalize @@ -4393,33 +4248,7 @@ tactic, the HTTP inspector will not cut over to the wizard if it sees any early client-to-server traffic, but will continue normal HTTP processing of the flow regardless of the eventual server response. -5.10.5. Trace messages - -When a user needs help to sort out things going on inside HTTP -inspector, Trace module becomes handy. - -$ snort --help-module trace | grep http_inspect - -Messages for the enhanced JavaScript Normalizer follow (more -verbosity available in debug build): - -5.10.5.1. trace.module.http_inspect.js_proc - -Messages from script processing flow and their verbosity levels: - - 1. Script opening tag location. - 2. Attributes of the detected script. - 3. Return codes from Normalizer. - -5.10.5.2. trace.module.http_inspect.js_dump - -JavaScript data dump and verbosity levels: - - 1. js_data buffer as it is passed to detection. - 2. (no messages available currently) - 3. Current script as it is passed to Normalizer. - -5.10.6. Detection rules +5.10.5. Detection rules http_inspect parses HTTP messages into their components and makes them available to the detection engine through rule options. Let’s @@ -4541,7 +4370,7 @@ headers or more and 200 headers or less: alert tcp any any -> any any ( msg:"between [100,200] headers"; http_num_headers: 100<=>200; sid:95; rev:1; ) -5.10.6.1. http_uri and http_raw_uri +5.10.5.1. http_uri and http_raw_uri These provide the URI of the request message. The raw form is exactly as it appeared in the message and the normalized form is determined @@ -4604,7 +4433,7 @@ Note: this section uses informal language to explain some things. Nothing here is intended to conflict with the technical language of the HTTP RFCs and the implementation follows the RFCs. -5.10.6.2. http_header and http_raw_header +5.10.5.2. http_header and http_raw_header These cover all the header lines except the first one. You may specify an individual header by name using the field option as shown @@ -4633,7 +4462,7 @@ In most cases specifying individual headers creates a more efficient and accurate rule. It is recommended that new rules be written using individual headers whenever possible. -5.10.6.3. http_trailer and http_raw_trailer +5.10.5.3. http_trailer and http_raw_trailer HTTP permits header lines to appear after a chunked body ends. Typically they contain information about the message content that was @@ -4645,7 +4474,7 @@ counterparts except they apply to these end headers. If you want a rule to inspect both kinds of headers you need to write two rules, one using header and one using trailer. -5.10.6.4. http_cookie and http_raw_cookie +5.10.5.4. http_cookie and http_raw_cookie These provide the value of the Cookie header for a request message and the Set-Cookie for a response message. If multiple cookies are @@ -4654,7 +4483,7 @@ present they will be concatenated into a comma-separated list. Normalization for http_cookie is the same URI-style normalization applied to http_header when no specific header is specified. -5.10.6.5. http_true_ip +5.10.5.5. http_true_ip This provides the original IP address of the client sending the request as it was stored by a proxy in the request message headers. @@ -4663,42 +4492,42 @@ True-Client-IP or any other custom x-forwarded-for type header. If multiple headers are present the preference defined in xff_headers configuration is considered. -5.10.6.6. http_client_body +5.10.5.6. http_client_body This is the body of a request message such as POST or PUT. Normalization for http_client_body is the same URI-like normalization applied to http_header when no specific header is specified. -5.10.6.7. http_raw_body +5.10.5.7. http_raw_body This is the body of a request or response message. It will be dechunked and unzipped if applicable but will not be normalized in any other way. -5.10.6.8. http_method +5.10.5.8. http_method The method field of a request message. Common values are "GET", "POST", "OPTIONS", "HEAD", "DELETE", "PUT", "TRACE", and "CONNECT". -5.10.6.9. http_stat_code +5.10.5.9. http_stat_code The status code field of a response message. This is normally a 3-digit number between 100 and 599. In this example it is 200. HTTP/1.1 200 OK -5.10.6.10. http_stat_msg +5.10.5.10. http_stat_msg The reason phrase field of a response message. This is the human-readable text following the status code. "OK" in the previous example. -5.10.6.11. http_version +5.10.5.11. http_version The protocol version information that appears on the first line of an HTTP message. This is usually "HTTP/1.0" or "HTTP/1.1". -5.10.6.12. http_raw_request and http_raw_status +5.10.5.12. http_raw_request and http_raw_status These are the unmodified first header line of the HTTP request and response messages respectively. These rule options are a safety valve @@ -4708,38 +4537,37 @@ first header line. For a request message those are http_method, http_raw_uri, and http_version. For a response message those are http_version, http_stat_code, and http_stat_msg. -5.10.6.13. file_data +5.10.5.13. file_data The file_data contains the normalized message body. This is the normalization described above under gzip, normalize_utf, decompress_pdf, decompress_swf, and normalize_javascript. -5.10.6.14. js_data +5.10.5.14. js_data The js_data contains normalized JavaScript text collected from the whole PDU (inline or external scripts). It requires the Enhanced -Normalizer enabled: http_inspect = { js_norm_bytes_depth = N }, -js_norm_bytes_depth option is described above. Despite what js_data -has, file_data still contains the whole HTTP body with an original +Normalizer enabled: js_norm = { }, Despite what js_data has, +file_data still contains the whole HTTP body with an original JavaScript in it. -5.10.6.15. vba_data +5.10.5.15. vba_data The vba_data will contain the decompressed Visual Basic for Applications (vba) macro data embedded in MS office files. It requires decompress_zip and decompress_vba options enabled. -5.10.6.16. http_num_headers and http_num_trailers +5.10.5.16. http_num_headers and http_num_trailers These are range-based rule options used to check the number of headers and trailers, respectively. -5.10.6.17. http_max_header_line and http_max_trailer_line +5.10.5.17. http_max_header_line and http_max_trailer_line These are range-based rule options used to check the longest line in request and response headers and trailers, respectively. -5.10.6.18. http_num_cookies +5.10.5.18. http_num_cookies This is a range-based rule option that checks the number of cookies. In a request all the individual cookies found in Cookie header are @@ -4757,7 +4585,7 @@ Content-Length: 5 Set-Cookie: lang=en-US; Path=/; Domain=example.com Set-Cookie: id=a3fWa; Expires=Thu, 21 Oct 2021 07:28:00 GMT; Secure; HttpOnly -5.10.6.19. http_version_match +5.10.5.19. http_version_match Rule option that matches HTTP version to one of the listed version values. Possible match values: 1.0, 1.1, 2.0, 0.9, other, and @@ -4773,7 +4601,7 @@ be HTTP/2.0 or HTTP/0.9 will match "other" as described above. The http_version rule option is available to examine the actual bytes in the version field. -5.10.6.20. http_header_test and http_trailer_test +5.10.5.20. http_header_test and http_trailer_test Rule options that perform various tests against a specific header and trailer field, respectively. It can perform a range test, check @@ -4781,7 +4609,7 @@ whether the value is numeric or whether it is absent. Negative values are considered non-numeric. Values with more than 18 digits are considered non-numeric. -5.10.7. Timing issues and combining rule options +5.10.6. Timing issues and combining rule options HTTP inspector is stateful. That means it is aware of a bigger picture than the packet in front of it. It knows what all the pieces @@ -5049,14 +4877,244 @@ option can be specified in one of two ways: the uppercase function name, or the lowercase function name. -5.13. MMS Inspector +5.13. JavaScript Normalization + +-------------- + +One of the improvements in Snort 3 is Enhanced JavaScript Normalizer +which has its own module and can be used with any service inspectors +where JavaScript code might occur. Currently it is only used by HTTP +inspector. + +5.13.1. Overview + +You can configure it by adding: + +js_norm = {} + +to your snort.lua configuration file. Or you can read about it in the +source code under src/js_norm. + +Having js_norm module configured and ips option js_data in the rules +automatically enables Enhanced Normalizer. The Enhanced Normalizer +can normalize inline/external scripts. It supports scripts over +multiple PDUs. It is a stateful JavaScript whitespace and identifiers +normalizer. Normalizer concatenates string literals whenever it’s +possible to do. This also works with any other normalizations that +result in string literals. All JavaScript identifier names, except +those from the ignore lists, will be substituted with unified names +in the following format: var_0000 → var_ffff. But the unescape-like +function names will be removed from the normalized data. The +Normalizer tries to expand an escaped text, so it will appear in a +usual form in the output. Moreover, Normalizer validates the syntax +concerning ECMA-262 Standard, including scope tracking and +restrictions for script elements. For more information on how +additionally configure Enhanced Normalizer check with the following +configuration options: bytes_depth, identifier_depth, max_tmpl_nest, +max_bracket_depth, max_scope_depth, ident_ignore, prop_ignore. +Eventually Enhanced Normalizer will completely replace Legacy +Normalizer from HTTP inspector. + +5.13.2. Configuration + +Configuration can be as simple as adding: + +js_norm = {} + +to your snort.lua file. The default configuration provides a thorough +normalization and may be all that you need. But there are some +options that provide extra features, tweak how things are done, or +conserve resources by doing less. + +Also, there are default lists of ignored identifiers and object +properties provided. To get a complete default configuration, use +default_js_norm from lua/snort_default.lua by adding: + +js_norm = default_js_norm + +to your snort.lua file. + +Enhanced JavaScript Normalizer implements JIT approach. Actual +normalization takes place only when js_data option is evaluated. This +option also used as a buffer selector for normalized JavaScript data. + +5.13.2.1. bytes_depth + +bytes_depth = N {-1 : max53} will set a number of input JavaScript +bytes to normalize. When the depth is reached, normalization will be +stopped. It’s implemented per-script. By default bytes_depth = -1, +will set unlimited depth. The enhanced normalizer provides more +precise whitespace normalization of JavaScript, that removes all +redundant whitespaces and line terminators from the JavaScript syntax +point of view (between identifier and punctuator, between identifier +and operator, etc.) according to ECMAScript 5.1 standard. +Additionally, it performs normalization of JavaScript identifiers +making a substitution of unique names with unified names +representation: var_0000:var_ffff. The identifiers are variables and +function names. The normalized data is available through the js_data +rule option. + +5.13.2.2. identifier_depth + +identifier_depth = N {0 : 65536} will set a number of unique +JavaScript identifiers to normalize. When the depth is reached, a +built-in alert is generated. Every response has its own identifier +substitution context, which means that identifier will retain same +normal form in multiple scripts, if they are a part of the same +response, and that this limit is set for a single response and not a +single script. By default, the value is set to 65536, which is the +max allowed number of unique identifiers. The generated names are in +the range from var_0000 to var_ffff. + +5.13.2.3. max_tmpl_nest + +max_tmpl_nest = N {0 : 255} (default 32) is an option of the enhanced +JavaScript normalizer that determines the deepest level of nested +template literals to be processed. Introduced in ES6, template +literals provide syntax to define a literal multiline string, which +can have arbitrary JavaScript substitutions, that will be evaluated +and inserted into the string. Such substitutions can be nested, and +require keeping track of every layer for proper normalization. This +option is present to limit the amount of memory dedicated to template +nesting tracking. + +5.13.2.4. max_bracket_depth + +max_bracket_depth = N {1 : 65535} (default 256) is an option of the +enhanced JavaScript normalizer that determines the maximum depth of +nesting brackets, i.e. parentheses, braces and square brackets, +nested within a matching pair, in any combination. This option is +present to limit the amount of memory dedicated to bracket tracking. + +5.13.2.5. max_scope_depth + +max_scope_depth = N {1 : 65535} (default 256) is an option of the +enhanced JavaScript normalizer that determines the deepest level of +nested variable scope, i.e. functions, code blocks, etc. including +the global scope. This option is present to limit the amount of +memory dedicated to scope tracking. + +5.13.2.6. ident_ignore + +ident_ignore = {} is an option of the +enhanced JavaScript normalizer that defines a list of identifiers to +keep intact. + +Identifiers in this list will not be put into normal form (var_0000). +Subsequent accessors, after dot, in square brackets or after function +call, will not be normalized as well. + +For example: + +console.log("bar") +document.getElementById("id").text +eval("script") +console["log"] + +Every entry has to be a simple identifier, i.e. not include dots, +brackets, etc. For example: + +js_norm.ident_ignore = { 'console', 'document', 'eval', 'foo' } + +When a variable assignment that aliases an identifier from the list +is found, the assignment will be tracked, and subsequent occurrences +of the variable will be replaced with the stored value. This +substitution will follow JavaScript variable scope limits. + +For example: + +var a = console.log +a("hello") // will be substituted to 'console.log("hello")' + +For class names and constructors in the list, when the class is used +with the keyword new, created object will be tracked, and its +properties will be kept intact. Identifier of the object itself, +however, will be brought to unified form. + +For example: + +var o = new Array() // normalized to 'var var_0000=new Array()' +o.push(10) // normalized to 'var_0000.push(10)' + +The default list of ignore-identifiers is present in +"snort_defaults.lua". + +Unescape function names should remain intact in the output. They +ought to be included in the ignore list. If for some reason the user +wants to disable unescape related features, then removing function’s +name from the ignore list does the trick. + +5.13.2.7. prop_ignore + +prop_ignore = {} is an option of the +enhanced JavaScript normalizer that defines a list of object +properties and methods that will be kept intact during the +identifiers normalization. This list should include methods and +properties of objects that will not be tracked by assignment +substitution functionality, for example, those that can be created +implicitly. + +Subsequent accessors, after dot, in square brackets or after function +call, will not be normalized as well. + +For example: + +js_norm.prop_ignore = { 'split' } + +in: "string".toUpperCase().split("").reverse().join(""); +out: "string".var_0000().split("").reverse().join(""); + +The default list of ignored properties is present in +"snort_defaults.lua". + +5.13.3. Detection rules + +Enhanced JavaScript Normalizer follows JIT approach which require to +have rules with js_data IPS option loaded. An example rule: + +alert tcp any any -> any any (msg:"JavaScript"; js_data; content:"var var_0000=1;"; sid:1;) + +5.13.3.1. js_data + +The js_data IPS contains normalized JavaScript text collected from +the whole PDU. It requires the Enhanced JavaScript Normalizer +configured. + +5.13.4. Trace messages + +When a user needs help to sort out things going on inside Enhanced +JavaScript Normalizer, Trace module becomes handy. + +$ snort --help-module trace | grep js_norm + +Messages for the enhanced JavaScript Normalizer follow (more +verbosity available in debug build): + +5.13.4.1. trace.module.js_norm.proc + +Messages from script processing flow and their verbosity levels: + + 1. Script opening tag location. + 2. Attributes of the detected script. + 3. Return codes from Normalizer. + +5.13.4.2. trace.module.js_norm.dump + +JavaScript data dump and verbosity levels: + + 1. js_data buffer as it is passed to detection. + 2. (no messages available currently) + 3. Current script as it is passed to Normalizer. + + +5.14. MMS Inspector -------------- MMS inspector is a service inspector for the MMS protocol within the IEC 61850 specification. -5.13.1. Overview +5.14.1. Overview IEC 61850 is a family of protocols, including MMS, distributed by the International Electrotechnical Commission (IEC) that provide a @@ -5073,7 +5131,7 @@ protocol and provides rule writers access to certain protocol fields and data content through rule options. This allows the user to write rules for MMS messages without decoding the protocol. -5.13.2. Configuration +5.14.2. Configuration MMS messages can be sent in a variety of ways including multiple PDUs within one TCP packet, one PDU split across multiple TCP packets, or @@ -5083,7 +5141,7 @@ presented to the user. No manual configuration other than enabling the MMS service inspector is necessary to leverage this functionality. -5.13.3. Quick Guide +5.14.3. Quick Guide A typical MMS configuration looks like this: @@ -5099,14 +5157,14 @@ binder = In this example, the mms inspector is defined based on patterns known to be consistent with MMS messages. -5.13.4. Rule Options +5.14.4. Rule Options New rule options are supported by enabling the MMS inspector: * mms_data * mms_func -5.13.4.1. mms_data +5.14.4.1. mms_data mms_data moves the cursor to the start of the MMS message, bypassing all of the OSI encapsulation layers and allowing subsequent rule @@ -5127,7 +5185,7 @@ alert tcp ( \ sid:1000000; \ ) -5.13.4.2. mms_func +5.14.4.2. mms_func mms_func takes the supplied function name or number and compares it with the Confirmed Service Request/Response in the message being @@ -5162,7 +5220,7 @@ alert tcp ( \ ) -5.14. Performance Monitor +5.15. Performance Monitor -------------- @@ -5171,14 +5229,14 @@ down by too many flows? perf_monitor! Why are certain TCP segments being dropped without hitting a rule? perf_monitor! Why is a sensor leaking water? Not perf_monitor, check with stream… -5.14.1. Overview +5.15.1. Overview The Snort performance monitor is the built-in utility for monitoring system and traffic statistics. All statistics are separated by processing thread. perf_monitor supports several trackers for monitoring such data: -5.14.2. Base Tracker +5.15.2. Base Tracker The base tracker is used to gather running statistics about Snort and its running modules. All Snort modules gather, at the very least, @@ -5235,7 +5293,7 @@ perf_monitor = Note: Event stats from prior Snorts are now located within base statistics. -5.14.3. Flow Tracker +5.15.3. Flow Tracker Flow tracks statistics regarding traffic and L3/L4 protocol distributions. This data can be used to build a profile of traffic @@ -5245,7 +5303,7 @@ To enable: perf_monitor = { flow = true } -5.14.4. FlowIP Tracker +5.15.4. FlowIP Tracker FlowIP provides statistics for individual hosts within a network. This data can be used for identifying communication habits, such as @@ -5257,7 +5315,7 @@ To enable: perf_monitor = { flow_ip = true } -5.14.5. CPU Tracker +5.15.5. CPU Tracker This tracker monitors the CPU and wall time spent by a given processing thread. @@ -5266,21 +5324,21 @@ To enable: perf_monitor = { cpu = true } -5.14.6. Formatters +5.15.6. Formatters Performance monitor allows statistics to be output in a few formats. Along with human readable text (as seen at shutdown) and csv formats, a JSON format format is also available. -5.15. POP and IMAP +5.16. POP and IMAP -------------- POP inspector is a service inspector for POP3 protocol and IMAP inspector is for IMAP4 protocol. -5.15.1. Overview +5.16.1. Overview POP and IMAP inspectors examine data traffic and find POP and IMAP commands and responses. The inspectors also identify the command, @@ -5288,7 +5346,7 @@ header, body sections and extract the MIME attachments and decode it appropriately. The pop and imap also identify and whitelist the pop and imap traffic. -5.15.2. Configuration +5.16.2. Configuration POP inspector and IMAP inspector offer same set of configuration options for MIME decoding depth. These depths range from 0 to 65535 @@ -5299,27 +5357,27 @@ be decoded. If you do not specify the default value is -1 The depth limits apply per attachment. They are: -5.15.2.1. b64_decode_depth +5.16.2.1. b64_decode_depth Set the base64 decoding depth used to decode the base64-encoded MIME attachments. -5.15.2.2. qp_decode_depth +5.16.2.2. qp_decode_depth Set the Quoted-Printable (QP) decoding depth used to decode QP-encoded MIME attachments. -5.15.2.3. bitenc_decode_depth +5.16.2.3. bitenc_decode_depth Set the non-encoded MIME extraction depth used for non-encoded MIME attachments. -5.15.2.4. uu_decode_depth +5.16.2.4. uu_decode_depth Set the Unix-to-Unix (UU) decoding depth used to decode UU-encoded attachments. -5.15.2.5. Examples +5.16.2.5. Examples stream = { } @@ -5353,13 +5411,13 @@ pop = } -5.16. Port Scan +5.17. Port Scan -------------- A module to detect port scanning -5.16.1. Overview +5.17.1. Overview This module is designed to detect the first phase in a network attack: Reconnaissance. In the Reconnaissance phase, an attacker @@ -5459,7 +5517,7 @@ however, Portscan will only track open ports after the alert has been triggered. Open port events are not individual alerts, but tags based off the original scan alert. -5.16.2. Scan levels +5.17.2. Scan levels There are 3 default scan levels that can be set. @@ -5513,7 +5571,7 @@ setting will catch some slow scans because of the continuous monitoring, but is very sensitive to active hosts. This most definitely will require the user to tune Portscan. -5.16.3. Tuning Portscan +5.17.3. Tuning Portscan The most important aspect in detecting portscans is tuning the detection engine for your network(s). Here are some tuning tips: @@ -5590,7 +5648,7 @@ require the least tuning. The low sensitivity level does not catch filtered scans, since these are more prone to false positives. -5.17. Sensitive Data Filtering +5.18. Sensitive Data Filtering -------------- @@ -5600,21 +5658,21 @@ credit card numbers, U.S. Social Security numbers, phone numbers, and email addresses. A rich regular expression syntax is available for defining your own PII. -5.17.1. Hyperscan +5.18.1. Hyperscan The sd_pattern rule option is powered by the open source Hyperscan library from Intel. It provides a regex grammar which is mostly PCRE compatible. To learn more about Hyperscan see https://intel.github.io /hyperscan/dev-reference/ -5.17.2. Syntax +5.18.2. Syntax Snort provides sd_pattern as IPS rule option with no additional inspector overhead. The Rule option takes the following syntax. sd_pattern: ""[, threshold ]; -5.17.2.1. Pattern +5.18.2.1. Pattern Pattern is the most important and is the only required parameter to sd_pattern. It supports 5 built-in patterns which are configured by @@ -5664,7 +5722,7 @@ but would not match 1@ourdomain.com ab12@ourdomain.com or Note: This is just an example, this pattern is not suitable to detect many correctly formatted emails. -5.17.2.2. Threshold +5.18.2.2. Threshold Threshold is an optional parameter allowing you to change built-in default value (default value is 1). The following two instances are @@ -5682,7 +5740,7 @@ This example requires 300 matches of the pattern "This is a string literal" to qualify as a positive match. That is, if the string only occurred 299 times in a packet, you will not see an event. -5.17.2.3. Obfuscating built-in patterns +5.18.2.3. Obfuscating built-in patterns Snort provides discreet logging for the built-in patterns "credit_card", "us_social", "us_social_nodashes", "us_phone" and @@ -5695,7 +5753,7 @@ ips = obfuscate_pii = true } -5.17.3. Example +5.18.3. Example A complete Snort IPS rule @@ -5711,7 +5769,7 @@ Logged output when running Snort in "cmg" alert format. 58 58 58 58 58 58 58 58 58 58 58 58 39 32 39 34 XXXXXXXXXXXX9294 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -5.17.4. Caveats +5.18.4. Caveats 1. Snort currently requires setting the fast pattern engine to use "hyperscan" in order for sd_pattern ips option to function @@ -5728,13 +5786,13 @@ Logged output when running Snort in "cmg" alert format. (This is a known bug). -5.18. SMTP +5.19. SMTP -------------- SMTP inspector is a service inspector for SMTP protocol. -5.18.1. Overview +5.19.1. Overview The SMTP inspector examines SMTP connections looking for commands and responses. It also identifies the command, header and body sections, @@ -5744,7 +5802,7 @@ identifies and whitelists the SMTP traffic. SMTP inspector logs the filename, email addresses, attachment names when configured. -5.18.2. Configuration +5.19.2. Configuration SMTP command lines can be normalized to remove extraneous spaces. TLS-encrypted traffic can be ignored, which improves performance. In @@ -5753,7 +5811,7 @@ performance boost. The configuration options are described below: -5.18.2.1. normalize and normalize_cmds +5.19.2.1. normalize and normalize_cmds Normalization checks for more than one space character after a command. Space characters are defined as space (ASCII 0x20) or tab @@ -5764,34 +5822,34 @@ example: smtp = { normalize = 'cmds', normalize_cmds = 'RCPT VRFY EXPN' } -5.18.2.2. ignore_data +5.19.2.2. ignore_data Set it to true to ignore data section of mail (except for mail headers) when processing rules. -5.18.2.3. ignore_tls_data +5.19.2.3. ignore_tls_data Set it to true to ignore TLS-encrypted data when processing rules. -5.18.2.4. max_command_line_len +5.19.2.4. max_command_line_len Alert if an SMTP command line is longer than this value. Absence of this option or a "0" means never alert on command line length. RFC 2821 recommends 512 as a maximum command line length. -5.18.2.5. max_header_line_len +5.19.2.5. max_header_line_len Alert if an SMTP DATA header line is longer than this value. Absence of this option or a "0" means never alert on data header line length. RFC 2821 recommends 1024 as a maximum data header line length. -5.18.2.6. max_response_line_len +5.19.2.6. max_response_line_len Alert if an SMTP response line is longer than this value. Absence of this option or a "0" means never alert on response line length. RFC 2821 recommends 512 as a maximum response line length. -5.18.2.7. alt_max_command_line_len +5.19.2.7. alt_max_command_line_len Overrides max_command_line_len for specific commands For example: @@ -5807,11 +5865,11 @@ alt_max_command_line_len = }, } -5.18.2.8. invalid_cmds +5.19.2.8. invalid_cmds Alert if this command is sent from client side. -5.18.2.9. valid_cmds +5.19.2.9. valid_cmds List of valid commands. We do not alert on commands in this list. @@ -5821,36 +5879,36 @@ HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT RSET SAML SEND SIZE STARTTLS SOML TICK TIME TURN TURNME VERB VRFY X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR ]] -5.18.2.10. data_cmds +5.19.2.10. data_cmds List of commands that initiate sending of data with an end of data delimiter the same as that of the DATA command per RFC 5321 - " .". -5.18.2.11. binary_data_cmds +5.19.2.11. binary_data_cmds List of commands that initiate sending of data and use a length value after the command to indicate the amount of data to be sent, similar to that of the BDAT command per RFC 3030. -5.18.2.12. auth_cmds +5.19.2.12. auth_cmds List of commands that initiate an authentication exchange between client and server. -5.18.2.13. xlink2state +5.19.2.13. xlink2state Enable/disable xlink2state alert, options are {disable | alert | drop}. See CVE-2005-0560 for a description of the vulnerability. -5.18.2.14. MIME processing depth parameters +5.19.2.14. MIME processing depth parameters These four MIME processing depth parameters are identical to their POP and IMAP counterparts. See that section for further details. b64_decode_depth qp_decode_depth bitenc_decode_depth uu_decode_depth -5.18.2.15. Log Options +5.19.2.15. Log Options Following log options allow SMTP inspector to log email addresses and filenames. Please note, this is logged only with the unified2 output @@ -5893,7 +5951,7 @@ This option specifies the depth for logging email headers. The allowed range for this option is 0 - 20480. A value of 0 will disable email headers logging. The default value for this option is 1464. -5.18.3. Example +5.19.3. Example smtp = { @@ -5946,7 +6004,7 @@ smtp = } -5.19. Telnet +5.20. Telnet -------------- @@ -5956,7 +6014,7 @@ command sequences per RFC 854. It will also determine when a telnet connection is encrypted, per the use of the telnet encryption option per RFC 2946. -5.19.1. Configuring the inspector to block exploits and attacks +5.20.1. Configuring the inspector to block exploits and attacks ayt_attack_thresh number @@ -5965,7 +6023,7 @@ the threshold number specified. This addresses a few specific vulnerabilities relating to bsd-based implementations of telnet. -5.20. Trace +5.21. Trace -------------- @@ -5978,7 +6036,7 @@ enable debug tracing, Snort must be configured at build time with wizard and snort.inspector_manager) are providing non-debug trace messages in normal production builds. -5.20.1. Trace module +5.21.1. Trace module The trace module is responsible for configuring traces and supports the following parameters: @@ -6018,7 +6076,7 @@ The trace module supports config reloading. Also, it’s possible to set or clear modules traces and packet filter constraints via the control channel command. -5.20.2. Trace module - configuring traces +5.21.2. Trace module - configuring traces The trace module has the modules option - a table with trace configuration for specific modules. The following lines placed in @@ -6100,7 +6158,7 @@ trace = } } -5.20.3. Trace module - configuring packet filter constraints for +5.21.3. Trace module - configuring packet filter constraints for packet related trace messages There is a capability to filter traces by the packet constraints. The @@ -6155,7 +6213,7 @@ trace = } } -5.20.4. Trace module - configuring trace output method +5.21.4. Trace module - configuring trace output method There is a capability to configure the output method for trace messages. The trace module has the output option with two acceptable @@ -6184,7 +6242,7 @@ trace = As a result, each trace message will be printed into syslog (the Snort run-mode will be ignored). -5.20.5. Configuring traces via control channel command +5.21.5. Configuring traces via control channel command There is a capability to configure module trace options and packet constraints via the control channel command by using a Snort shell. @@ -6219,7 +6277,7 @@ trace.set({modules = {...}}) - set only module trace options keeping old filteri trace.set({}) - disable traces and constraints (set to empty) -5.20.6. Trace messages format +5.21.6. Trace messages format Each tracing message has a standard format: @@ -6268,7 +6326,7 @@ m – minutes s – seconds S – milliseconds -5.20.7. Example - Debugging rules using detection trace +5.21.7. Example - Debugging rules using detection trace The detection engine is responsible for rule evaluation. Turning on the trace for it can help with debugging new rules. @@ -6396,7 +6454,7 @@ detection:rule_eval:1: Matched rule gid:sid:rev 1:3:0 detection:rule_vars:1: Rule options variables: var[0]=1 var[1]=10 var[2]=0 04/22-20:21:40.905630, 1, TCP, raw, 56, C2S, 127.0.0.1:1234, 127.0.0.1:5678, 1:3:0, allow -5.20.8. Example - Protocols decoding trace +5.21.8. Example - Protocols decoding trace Turning on decode trace will print out information about the packets decoded protocols. Can be useful in case of tunneling. @@ -6420,7 +6478,7 @@ decode:all:1: Codec ipv6 (protocol_id: 1) ip header starts at: 0x7f70800110f0, l decode:all:1: Codec icmp4 (protocol_id: 256) ip header starts at: 0x7f70800110f0, length is 8 decode:all:1: Codec unknown (protocol_id: 256) ip header starts at: 0x7f70800110f0, length is 0 -5.20.9. Example - Track the time packet spends in each inspector +5.21.9. Example - Track the time packet spends in each inspector There is a capability to track which inspectors evaluate a packet, and how much time the inspector consumes doing so. These trace @@ -6461,7 +6519,7 @@ snort:inspector_manager:1: post detection inspection, raw, packet 1, context 1 snort:inspector_manager:1: end inspection, raw, packet 1, context 1, total time: 0 usec snort:main:1: [0] Destroying completed command RUN -5.20.10. Example - trace filtering by packet constraints: +5.21.10. Example - trace filtering by packet constraints: In snort.lua, the following lines were added: @@ -6523,7 +6581,7 @@ detection:rule_eval:1: packet 4 UNK 10.1.1.2:200 10.2.1.1:100 (non-fast-patterns The trace messages for two last packets (numbers 5 and 6) weren’t printed. -5.20.11. Example - configuring traces via trace.set() command +5.21.11. Example - configuring traces via trace.set() command In snort.lua, the following lines were added: @@ -6606,7 +6664,7 @@ The new configuration was applied. decode:all:1 messages aren’t filtered because they don’t include a packet (a packet isn’t well-formed at the point when the message is printing). -5.20.12. Other available traces +5.21.12. Other available traces There are more trace options supported by detection: @@ -6633,7 +6691,7 @@ developer. Some are for corner cases, others for complex data structures. -5.21. Wizard +5.22. Wizard -------------- @@ -6646,7 +6704,7 @@ the session can be handed off to the appropriate inspector. The wizard is still under development; if you find you need to tweak the defaults please let us know. -5.21.1. Wizard patterns +5.22.1. Wizard patterns Wizard supports 3 kinds of patterns: @@ -6672,7 +6730,7 @@ looks at the first arriving packet from the meta-flow. If no pattern matches that packet or wizard’s max_search_depth is reached, the meta-flow is abandoned by wizard. -5.21.2. Wizard patterns - Spells +5.22.2. Wizard patterns - Spells Spell is a text based pattern. The best area of usage - text protocols: http, smtp, sip, etc. Spells are: @@ -6711,7 +6769,7 @@ contain following options: to_client = { '220*SMTP', '220*MAIL' } } -5.21.3. Wizard patterns - Hexes +5.22.3. Wizard patterns - Hexes Hexes can be used to match binary protocols: dnp3, http2, ssl, etc. Hexes use hexadecimal representation of the data for pattern @@ -6736,7 +6794,7 @@ Example of a hex definition in Lua: to_client = { '|05 64|' } } -5.21.4. Wizard patterns - Curses +5.22.4. Wizard patterns - Curses Curses are internal algorithms of service identification. They are implemented as state machines in C++ code and can have their own @@ -6748,7 +6806,7 @@ A list of available services can be obtained using snort A configuration which enables some curses: curses = {'dce_udp', 'dce_tcp', 'dce_smb', 'sslv2'} -5.21.5. Additional Details: +5.22.5. Additional Details: * Note that usually more specific patterns have higher precedence.