From: Russ Combs (rucombs) Date: Tue, 14 Feb 2017 20:30:53 +0000 (-0500) Subject: Merge pull request #809 in SNORT/snort3 from doc_ha_sc_conn to master X-Git-Tag: 3.0.0-233~73 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=e0b8be3a7cbafaba2abdc49fd4de9d2598656eb4;p=thirdparty%2Fsnort3.git Merge pull request #809 in SNORT/snort3 from doc_ha_sc_conn to master Squashed commit of the following: commit f3884b9e20e574158bb97ffed7bd9311907cf8ac Author: Ed Borgoyn Date: Thu Feb 9 17:44:49 2017 -0500 Documentation for HA, side_channel, and connectors. --- diff --git a/doc/CMakeLists.txt b/doc/CMakeLists.txt index b91cb5c43..354816a84 100644 --- a/doc/CMakeLists.txt +++ b/doc/CMakeLists.txt @@ -37,6 +37,7 @@ set ( bugs.txt building.txt concepts.txt + connector.txt daq.txt dcerpc.txt differences.txt @@ -45,12 +46,14 @@ set ( extending.txt features.txt file_processing.txt + high_availability.txt http_inspect.txt overview.txt params.txt perf_monitor.txt reference.txt sensitive_data.txt + side_channel.txt snort2lua.txt snort2x.png snort3x.png diff --git a/doc/Makefile.am b/doc/Makefile.am index 40af54c8c..358ee1062 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -30,6 +30,7 @@ bugs.txt \ building.txt \ concepts.txt \ config_changes.txt \ +connector.txt \ daq.txt \ dcerpc.txt \ differences.txt \ @@ -38,12 +39,14 @@ errors.txt \ extending.txt \ features.txt \ file_processing.txt \ +high_availability.txt \ http_inspect.txt \ overview.txt \ params.txt \ perf_monitor.txt \ reference.txt \ sensitive_data.txt \ +side_channel.txt \ snort2lua.txt \ snort2x.png \ snort3x.png \ diff --git a/doc/connector.txt b/doc/connector.txt new file mode 100644 index 000000000..61756a59a --- /dev/null +++ b/doc/connector.txt @@ -0,0 +1,98 @@ +Connectors are a set of modules that are used to exchange message-oriented +data among snort threads and the external world. A typical use-case is +HA (High Availability) message exchange. Connectors serve to decouple the +message transport from the message creation/consumption. Connectors expose +a common API for several forms of message transport. + +Connectors are a Snort plugin type. + +===== Connector (parent plugin class): + +Connectors may either be a simplex channel and perform unidirectional +communications. Or may be duplex and perform bidirectional communications. +The TcpConnector is duplex while the FileConnector is simplex. + +All subtypes of Connector have a 'direction' configuration element and a +'connector' element. The 'connector' string is the key used to identify the +element for sidechannel configiration. The 'direction' element may have a +default value, for instance TcpConnector's are 'duplex'. + + +There are currently two implementations of Connectors: + +* TcpConnector - Exchange messages over a tcp channel. + +* FileConnector - Write messages to files and read messages from files. + + +===== TcpConnector: + +TcpConnector is a subclass of Connector and implements a DUPLEX type Connector, +able to send and receive messages over a tcp session. + +TcpConnector adds a few session setup configuration elements: + +* setup = 'call' or 'answer' - 'call' is used to have TcpConnector initiate + the connection. 'answer' is used to have TcpConnector accept incoming + connections. + +* address = '' - used for 'call' setup to specify the partner + +* base_port = port - used to contruct the actual port number for 'call' and + 'answer' modes. Actual port used is (base_port + instance_id). + +An example segment of TcpConnector configuration: + + tcp_connector = + { + { + connector = 'tcp_1', + address = '127.0.0.1', + setup = 'call', + base_port = 11000 + }, + } + + +===== FileConnector: + +FileConnector implements a Connector that can either read from files or write +to files. FileConnector's are simplex and must be configured to be +CONN_TRANSMIT or CONN_RECEIVE. + +FileConnector configuration adds two additional element: + +* name = string - used as part of the messsage file name + +* format = 'text' or 'binary' - FileConnector supports two file types + +The configured 'name' string is used to construct the actual names as in: + +* file_connector_NAME_transmit and file_connector_NAME_receive + +All messages for one snort invocation are read and written to one file. + +In the case of a receive FileConnector, all messages are read from the file +prior to the start of packet processing. This allows the messages to +establish state information for all processed packets. + +Connectors are used solely by SideChannel + +An example segment of FileConnector configuration: + + file_connector = + { + { + connector = 'file_tx_1', + direction = 'transmit', + format = 'text', + name = 'HA' + }, + { + connector = 'file_rx_1', + direction = 'receive', + format = 'text', + name = 'HA' + }, + } + diff --git a/doc/features.txt b/doc/features.txt index 644db46e7..c997e5916 100644 --- a/doc/features.txt +++ b/doc/features.txt @@ -14,6 +14,23 @@ include::dcerpc.txt[] include::file_processing.txt[] +=== High Availability + +High Availability includes the HA flow synchronization and the SideChannel +messaging subsystems. + +==== HA + +include::high_availability.txt[] + +==== Connector + +include::connector.txt[] + +==== Side Channel + +include::side_channel.txt[] + === HTTP Inspector include::http_inspect.txt[] diff --git a/doc/high_availability.txt b/doc/high_availability.txt new file mode 100644 index 000000000..15b0b025f --- /dev/null +++ b/doc/high_availability.txt @@ -0,0 +1,40 @@ +HighAvailability (or HA) is a snort module that provides state coherancy +between two partner snort instances. It uses SideChannel for messaging. + +There can be multiple types of HA within snort and snort plugins. HA +implements an extensible architecture to enable plugins to subscribe to the +base flow HA messaging. These plugins can then include their own messages +along with the flow cache HA messages. + +HA produces and consumes two type of messages: + +* Update - Update flow status. Plugins may add their own data to the messages + +* Delete - A flow has been removed from the cache + +The HA module is configured with these items: + + high_availability = + { + ports = "1", + enable = true, + min_age = 0.0, + min_sync = 0.0 + } + +The 'ports' item maps to the SideChannel port to use for the HA messaging. + +The 'enabled' item controls the overall HA operation. + +The items min_age and min_sync are used in the stream HA logic. min_age is +the number of seconds that a flow must exist in the flow cache before sending +HA messages to the partner. min_sync is the minimum time between HA status +updates. HA messages for a particular flow will not be sent faster than +min_sync. Both are expressed as a floating point number of seconds. + +HA messages are composed of the base 'stream' information plus any content +from additional modules. Modules subscribe HA in order to add message +content. The 'stream' HA content is always present in the messages while +the ancillary module content is only present when requested via a status +change request. + diff --git a/doc/side_channel.txt b/doc/side_channel.txt new file mode 100644 index 000000000..fcb336a1c --- /dev/null +++ b/doc/side_channel.txt @@ -0,0 +1,64 @@ +SideChannel is a snort module that uses Connectors to implement a messaging +infrastructure that is used to communicate between snort threads and the +outside world. + +SideChannel adds functionality onto the Connector as: + +* message multiplexing/demultiplexing - An additional protocol layer is + added to the messages. This port number is used to direct message to/from + various SideClass instancs. + +* application receive processing - handler for received messages on a + specific port. + +SideChannel's are always implement a duplex (bidirectional) messaging model +and can map to separate transmit and receive Connectors. + +The message handling model leverages the underlying Connector handling. So +please refer to the Connector documentation. + +SideChannel's are instantiated by various applications. The SideChannel port +numbers are the configuration element used to map SideChannel's to +applications. + +The SideChannel configuration mostly serves to map a port number to a Connector +or set of connectors. Each port mapping can have at most one transmit plus +one receive connector or one duplex connector. Multiple SideChannel's +may be configured and instatiated to support multiple applications. + +An example SideChannel configuration along with the corresponing Connector +configuration: + + side_channel = + { + { + ports = '1', + connectors = + { + { + connector = 'file_rx_1', + }, + { + connector = 'file_tx_1', + } + }, + }, + } + + file_connector = + { + { + connector = 'file_tx_1', + direction = 'transmit', + format = 'text', + name = 'HA' + }, + { + connector = 'file_rx_1', + direction = 'receive', + format = 'text', + name = 'HA' + }, + } + +