From: Mike Stepanek (mstepane) Date: Wed, 30 Jun 2021 15:07:06 +0000 (+0000) Subject: Merge pull request #2963 in SNORT/snort3 from ~MSTEPANE/snort3:build_3.1.7.0 to master X-Git-Tag: 3.1.7.0 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=b74e347d374b1043ffb9f1fc5fdb9a895a72d36d;p=thirdparty%2Fsnort3.git Merge pull request #2963 in SNORT/snort3 from ~MSTEPANE/snort3:build_3.1.7.0 to master Squashed commit of the following: commit 85517e1fb597d0d159fcf7728de68c86fcee39da Author: Mike Stepanek Date: Wed Jun 30 09:53:05 2021 -0400 build: generate and tag 3.1.7.0 --- diff --git a/CMakeLists.txt b/CMakeLists.txt index 22361253b..d0c3670e9 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3,7 +3,7 @@ project (snort CXX C) set (VERSION_MAJOR 3) set (VERSION_MINOR 1) -set (VERSION_PATCH 6) +set (VERSION_PATCH 7) set (VERSION_SUBLEVEL 0) set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}") diff --git a/ChangeLog b/ChangeLog index 406df0389..156d2be2c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,20 @@ +2021/06/30 - 3.1.7.0 + +appid: enhance netbios service detector to identify SMB versions as web app +appid: update documentation +appid: update the DNS detector to support the all record request +control: resolve socket issues due to race conditions +doc: updates for http2_inspect +framework: update base API version to 3 +main: implement test_features run flag to enable debug-like output +mime: track memory for mime sessions +payload_injector: don't inject if there are unflushed S2C TCP packets queued +reputation: include list id for daq trace log +sfip: fix unit tests for non-regtest builds +snort2lua: fix lua conversion of unsupported http preproc options without parameters +snort2lua: remove footprint size config +stream: fix is_ack_valid to return true even when current ack is to the left of snd_una, per RFC793 + 2021/06/16 - 3.1.6.0 appid: extract auxiliary ip when uri is provided by third-party diff --git a/doc/reference/snort_reference.text b/doc/reference/snort_reference.text index a0e042f9d..d24cff441 100644 --- a/doc/reference/snort_reference.text +++ b/doc/reference/snort_reference.text @@ -8,7 +8,7 @@ Snort 3 Reference Manual The Snort Team Revision History -Revision 3.1.6.0 2021-06-16 07:30:59 EDT TST +Revision 3.1.7.0 2021-06-30 09:59:01 EDT TST --------------------------------------------------------------------- @@ -1429,6 +1429,8 @@ Configuration: version, and only the version * implied snort.--enable-inline-test: enable Inline-Test Mode Operation + * implied snort.--enable-test-features: enable features used in + testing * implied snort.--gen-msg-map: dump configured rules in gen-msg.map format for use by other tools * implied snort.--help: show help overview @@ -8310,6 +8312,7 @@ these libraries see the Getting Started section of the manual. * --dump-version output the version, the whole version, and only the version * --enable-inline-test enable Inline-Test Mode Operation + * --enable-test-features enable features used in testing * --gen-msg-map dump configured rules in gen-msg.map format for use by other tools * --help show help overview @@ -9980,6 +9983,8 @@ these libraries see the Getting Started section of the manual. * implied snort.-e: display the second layer header info * implied snort.--enable-inline-test: enable Inline-Test Mode Operation + * implied snort.--enable-test-features: enable features used in + testing * implied snort.-f: turn off fflush() calls after binary log writes * int snort.-G: <0xid> (same as --logid) { 0:65535 } * implied snort.--gen-msg-map: dump configured rules in gen-msg.map diff --git a/doc/upgrade/snort_upgrade.text b/doc/upgrade/snort_upgrade.text index dee4d11e5..78ebfabea 100644 --- a/doc/upgrade/snort_upgrade.text +++ b/doc/upgrade/snort_upgrade.text @@ -8,7 +8,7 @@ Snort 3 Upgrade Manual The Snort Team Revision History -Revision 3.1.6.0 2021-06-16 07:30:48 EDT TST +Revision 3.1.7.0 2021-06-30 09:58:50 EDT TST --------------------------------------------------------------------- @@ -1086,6 +1086,8 @@ deleted -> ftp_telnet_protocol: 'detect_anomalies' deleted -> full: ' can no longer be specific' deleted -> http_inspect: 'detect_anomalous_servers' deleted -> http_inspect: 'disabled' +deleted -> http_inspect: 'fast_blocking' +deleted -> http_inspect: 'normalize_random_nulls_in_text' deleted -> http_inspect: 'proxy_alert' deleted -> http_inspect_server: 'allow_proxy_use' deleted -> http_inspect_server: 'enable_cookie' @@ -1163,6 +1165,7 @@ deleted -> stream5_tcp: 'ignore_any_rules' deleted -> stream5_tcp: 'log_asymmetric_traffic' deleted -> stream5_tcp: 'policy noack' deleted -> stream5_tcp: 'policy unknown' +deleted -> stream5_tcp: 'use_static_footprint_sizes' deleted -> stream5_udp: 'ignore_any_rules' deleted -> tcpdump: ' can no longer be specific' deleted -> test: 'file' diff --git a/doc/user/snort_user.text b/doc/user/snort_user.text index b9c7c6842..a54c6786e 100644 --- a/doc/user/snort_user.text +++ b/doc/user/snort_user.text @@ -8,7 +8,7 @@ Snort 3 User Manual The Snort Team Revision History -Revision 3.1.6.0 2021-06-16 07:30:48 EDT TST +Revision 3.1.7.0 2021-06-30 09:58:50 EDT TST --------------------------------------------------------------------- @@ -1921,18 +1921,20 @@ networks by providing the following features: For proper functioning of the AppId inspector, at a minimum stream flow tracking must be enabled. In addition, to identify TCP-based or -UDP-based applications then the appropriate stream inspector must be +UDP-based applications, the appropriate stream inspector must be enabled, e.g. stream_tcp or stream_udp. -In addition, in order to identify HTTP-based applications, the HTTP -inspector must be enabled. Otherwise, only non-HTTP applications will -be identified. +In order to identify HTTP-based applications, the HTTP inspector must +be enabled. Otherwise, only non-HTTP applications will be identified. AppId subscribes to the inspection events published by other inspectors, such as the HTTP and SSL inspectors, to gain access to the data needed. It uses that data to help determine the application ID. +AppId subscribes to the events published by SIP and DCE/RPC +inspectors to detect applications on expected flows. + 6.2.3. Configuration The AppId feature can be enabled via configuration. To enable it with @@ -2040,14 +2042,11 @@ from snort.org. ODP is a package that contains the following artifacts: * Application detectors in the Lua language. - * Port detectors, which are port only application detectors, in - meta-data in YAML format. * appMapping.data file containing application metadata. This file should not be modified. The first column contains application identifier and second column contains application name. Other columns contain internal information. - * Lua library files DetectorCommon.lua, flowTrackerModule.lua and - hostServiceTrackerModule.lua + * Lua library file DetectorCommon.lua. A user can install the ODP package in any directory and configure this directory via the app_detector_dir option in the appid @@ -2056,7 +2055,6 @@ subdirectory named custom, where user-created detectors are located. When installed, ODP will create following sub-directories: - * odp/port //Cisco port-only detectors * odp/lua //Cisco Lua detectors * odp/libs //Cisco Lua modules @@ -2071,7 +2069,6 @@ in the next section. Users must organize their Lua detectors and libraries by creating the following directory structure, under the ODP installation directory. - * custom/port //port-only detectors * custom/lua //Lua detectors * custom/libs //Lua modules @@ -2089,7 +2086,14 @@ openappid/custom/lua/ None of the directories below /usr/local/lib/openappid/ would be added for you. -6.2.8. Application Detector Creation Tool +6.2.8. Application Detector Reload + +Both ODP detectors and user created detectors can be reloaded using +the command appid.reload_detectors(). Detectors are expected to be +updated in the path appid.app_detector_dir before this command is +issued. The command takes no parameters. + +6.2.9. Application Detector Creation Tool For rudimentary Lua detectors, there is a tool provided called appid_detector_builder.sh. This is a simple, menu-driven bash script @@ -3787,15 +3791,6 @@ processing. Want to ask questions that involve both the client request and the server response? Or different requests in the same session? These things are possible. -Another new feature on the horizon is HTTP/2 analysis. HTTP/2 derives -from Google’s SPDY project and is in the process of being -standardized. Despite the name, it is better to think of HTTP/2 not -as a newer version of HTTP/1.1, but rather a separate protocol layer -that runs under HTTP/1.1 and on top of TLS or TCP. It’s a perfect fit -for the new Snort 3 architecture because a new HTTP/2 inspector would -naturally output HTTP/1.1 messages but not any underlying packets. -Exactly what http_inspect wants to input. - http_inspect is taking a very different approach to HTTP header fields. The classic preprocessor divides all the HTTP headers following the start line into cookies and everything else. It @@ -4431,17 +4426,52 @@ cannot. -------------- -Snort 3 is developing an inspector for HTTP/2. +New in Snort 3, the HTTP/2 inspector enables Snort to process HTTP/2 +traffic. -You can configure it by adding: +6.11.1. Overview + +Despite the name, it is better to think of HTTP/2 not as a newer +version of HTTP/1.1, but rather a separate protocol layer that runs +under HTTP/1.1 and on top of TLS or TCP. It supports several new +features with the goal of improving the performance of HTTP requests, +notably the ability to multiplex many requests over a single TCP +connection, HTTP header compression, and server push. + +HTTP/2 is a perfect fit for the new Snort 3 PDU-based inspection +architecture. The HTTP/2 inspector parses and strips the HTTP/2 +protocol framing and outputs HTTP/1.1 messages, exactly what +http_inspect wants to input. The HTTP/2 traffic then undergoes the +same processing as regular HTTP/1.1 traffic discussed above. So if +you haven’t already, take a look at the HTTP Inspector section; those +features also apply to HTTP/2 traffic. + +6.11.2. Configuration + +You can configure the HTTP/2 inspector with the default configuration +by adding: http2_inspect = {} -to your snort.lua configuration file. +to your snort.lua configuration file. Since processing HTTP/2 traffic +relies on the HTTP inspector, http_inspect must also be configured. +Keep in mind that the http_inspect configuration will also impact +HTTP/2 traffic. + +6.11.2.1. concurrent_streams_limit + +This limits the maximum number of HTTP/2 streams Snort will process +concurrently in a single HTTP/2 flow. The default and minimum +configurable value is 100. It can be configured up to a maximum of +1000. + +6.11.3. Detection rules -To smooth the transition to inspecting HTTP/2, rules that specify -service:http will be treated as if they also specify service:http2. -Thus: +Since HTTP/2 traffic is processed through the HTTP inspector, all of +the rule options discussed above are also available for HTTP/2 +traffic. To smooth the transition to inspecting HTTP/2, rules that +specify service:http will be treated as if they also specify +service:http2. Thus: alert tcp any any -> any any (flow:established, to_server; http_uri; content:"/foo";