project (snort CXX C)
set (VERSION_MAJOR 3)
-set (VERSION_MINOR 2)
-set (VERSION_PATCH 1)
+set (VERSION_MINOR 3)
+set (VERSION_PATCH 6)
set (VERSION_SUBLEVEL 0)
set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}")
+2024-09-05: 3.3.5.0
+
+* appid: added new logs for reload third party
+* extractor: add field name to logging function
+* extractor: add json logger
+* extractor: add unit tests for enum types
+* extractor: fix guard-macro names
+* extractor: fix local variable
+* extractor: mention a field in initialization list
+* extractor: remove unused headers
+* extractor: take a note of FIXIT-P in key points
+* file_api: set file name for file processing
+* http_inspect: when cutting chunks check for MAX_OCTETS too
+* packet_tracer: add tcp window size, options and meta-ack info
+
+2024-08-26: 3.3.4.0
+
+* appid: notify binder on service change
+* appid: replaced hsessions vector of raw pointers into vector of smart pointers
+* ftp_telnet: refactoring ftp-data
+* latency, dce, stream_ip: fix max pegs incorrectly declared sum
+* telnet: avoid flush when cr or lf is between commands
+
+2024-08-13: 3.3.3.0
+
+* control: code cleanup
+* control: handle control commands after packet threads are fully initialised
+* daq: add outstanding packets counter
+* extractor: add flow hash key
+* file_api: max depth is set as part of initial config
+* file: remove unused variable in FileFlows destructor
+* filters: update dev_notes.txt with details for event_filter
+* flow: optimize timeout handling for different packet type
+* http_inspect: add peg counts for gzip, known-not-supported, and unknown
+* http_inspect: log normalized URI in extra data
+* ips_options: separate main thread pcre counts from packet threads stats
+* memory: account memory for profiler only when packet thread is involved
+* src: resolve various warnings
+* stream_tcp: make sure ports are correctly swapped when filling a meta-ACK packet
+
+2024-07-29: 3.3.2.0
+
+* appid: fixing cpp warnings and cosmetic changes for appid cpu profiler
+* appid: removing trailing whitespaces
+* daq: added outstanding packets counter
+* doc: builtin rule documentation updates
+* flow: added compile-time option to disable tenant_id
+* flow: clear deferred trust after the flow is trusted to stop repeated trusting
+* js_norm: address pdf tokenizer issues
+* kaizen: fix verbose mode output for unlimited options
+* main: fix coverage
+* sip: fallback functionality for sip inspector
+* stream: refactor paf logic into a c++ class
+* stream_tcp: delete lws_init, it was redundant with tcp_init; delete FIXITs that are no longer relevant
+* stream_tcp: improve variable and function names for overlap processing
+* stream_tcp: integrate and streamline setting of flush policy and splitter
+* stream_tcp: merge TcpStreamSession into TcpSession
+* stream_tcp: refactor segment nodes to implement reassembly cursor and eliminate tracking variables
+* stream_tcp: refactor TcpReassembler into a virtual base class and subclasses for each mode: ignore, IPS and IDS
+* stream_tcp: refactor to move alert functions to their own class
+* stream_tcp: refactor to move tcp overlap processing out of reassembly class
+
+2024-07-15: 3.3.1.0
+
+* appid: restructure the appid code to make it easier to follow and maintain
+* appid: updating appid cpu profiler cli
+* dce_rpc: correct the session counters post the upgrade to smb v2 from v1
+* detection: include OPT_TREE traces in release build
+* detection: make print of fast pattern as a trace module
+* extractor: support trans_depth, origin and referrer fields
+* file: fixing file context reuse
+* flow: clear flow stash when freeing the flow data
+* flow: handle significant groups with unknown group value as non-group flow keys
+* http_inspect: add origin header
+* parser: do not skip symbols while expanding variables
+* perf_monitor: introducing new parameters for ip flow profiling
+* stream_tcp: move prev_norm object from TcpNormalizer to TcpNormalizerState
+* stream_tcp: set daq_msg field in meta-ack pseudo-packet header to the value from the wire packet.
+* stream_tcp: support tracing without compilation flags
+* wizard: expand MMS curse
+
+2024-06-18: 3.3.0.0
+
+* appid: display rows limit of table and totals
+* appid: using different api for picking appids for appid cpu profiler
+* build: bump version to 3.2.0
+* codecs: add handling of NDP types
+* dns: set Flow timeout after getting DNS response
+* extractor: add protocol logging for HTTP
+* framework: add new Cursor Action Type
+* http_inspect: set CAT_SET_SUB_SECTION for buffer with a sub-selector configured
+* js_norm: fix prerequisites for FlexLexer includes
+* main: add CLI command to show snort cpu percentage
+* stream_tcp: use default size atomsplitter on fallback
+* utils: remove duplication of definition. Thanks to xxxx81 for reporting the issue.
+
+2024-06-02: 3.2.2.0
+
+* appid: appid cpu profiler max columns
+* appid: re-enabling appid cpu profiler making it thread safe
+* appid: store and retrieve only SNI in AppIdSession
+* appid: updating file_magic.rules with some new file types added to the VDB.
+* dce_smb: do not prune from LRU cache during file tracker update
+* doc: fix formatting in dev_notes.txt
+* flow: add the newly-created flow to p->flow to avoid segv
+* js_norm: stop PDF processing on syntax error
+* main: apply loaded configuration only once
+* packet_capture: make sure packet_capture executed before detection
+* service_inspectors: fix get_buf handling
+* sip: flow clean-up based on lina configured timeout
+* src: remove repetitive words. Thanks @gopherorg for finding those typos
+* src: udpate to resolve new issues
+* stream_tcp: don't attempt to verify or process keep-alive probes with data
+* stream_tcp: fix infinite recursion cases. Thanks to scloder-ut-iso for helping with debug information that uncovered a case of infinite recursion
+* utils: add explicit include
+
2024-05-16: 3.2.1.0
* framework: supply directories to system headers to plug_gen.sh
#]=======================================================================]
find_package(PkgConfig)
-pkg_check_modules(PC_DAQ libdaq>=3.0.13)
+pkg_check_modules(PC_DAQ libdaq>=3.0.16)
# Use DAQ_INCLUDE_DIR_HINT and DAQ_LIBRARIES_DIR_HINT from configure_cmake.sh as primary hints
# and then package config information after that.
set ( DEEP_PROFILING ${ENABLE_DEEP_PROFILING} )
set ( ENABLE_MEMORY_PROFILER ${ENABLE_MEMORY_PROFILER} )
set ( ENABLE_RULE_PROFILER ${ENABLE_RULE_PROFILER} )
+set ( DISABLE_TENANT_ID ${DISABLE_TENANT_ID} )
if ( ENABLE_LARGE_PCAP )
set ( _FILE_OFFSET_BITS 64 )
option ( ENABLE_SHELL "enable shell support" OFF )
option ( ENABLE_UNIT_TESTS "enable unit tests" OFF )
option ( ENABLE_BENCHMARK_TESTS "enable benchmark tests" OFF )
+option ( DISABLE_TENANT_ID "disable tenant ID in the FlowKey structure" OFF )
option ( ENABLE_COREFILES "Prevent Snort from generating core files" ON )
option ( ENABLE_LARGE_PCAP "Enable support for pcaps larger than 2 GB" OFF )
set(MEMORY_PROFILER_CPPFLAGS "-DENABLE_MEMORY_PROFILER")
endif()
+if(DISABLE_TENANT_ID)
+ set (TENANT_ID_CPPFLAGS "-DDISABLE_TENANT_ID")
+ message(STATUS "Tenant ID support in FlowKey is disabled")
+endif()
+
+
if(ENABLE_RULE_PROFILER)
set(RULE_PROFILER_CPPFLAGS "-DENABLE_RULE_PROFILER")
endif()
/* enable memory profiler */
#cmakedefine ENABLE_MEMORY_PROFILER 1
+/* disable tenant_id */
+#cmakedefine DISABLE_TENANT_ID 1
+
/* enable rule profiler */
#cmakedefine ENABLE_RULE_PROFILER 1
--enable-jemalloc enable using jemalloc for dynamic memory management
--enable-jemalloc-static
same as --enable-jemalloc but linked statically
+ --disable-tenant-id disable tenant ID in the FlowKey
--enable-luajit-static enable luajit linked statically
--enable-appid-third-party
enable third party appid
--disable-jemalloc-static)
append_cache_entry ENABLE_JEMALLOC BOOL false
;;
+ --disable-tenant-id)
+ append_cache_entry DISABLE_TENANT_ID BOOL true
+ ;;
--enable-appid-third-party)
;;
--enable-unit-tests)
#include "daq_user.h"
#include <arpa/inet.h>
+#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <inttypes.h>
while (*s && *s != '"' && desc->msg.data_len < hc->snaplen)
{
if (unescape(*s++, &t))
+ {
+ assert(desc->data);
desc->data[desc->msg.data_len++] = t;
+ }
}
desc->pkthdr.pktlen = desc->msg.data_len;
}
116:250
-The ICMP error message's original IP header is truncated.
+The ICMP or IP in ICMP error message's original IP header is truncated.
116:251
-The ICMP error message's original IP packet's version and original IP header versions differ.
+The ICMP or IP in ICMP error message's original IP packet's version and original IP header versions differ.
116:252
-The ICMP error message's original datagram's length is less than the original IP's header length.
+The ICMP or IP in ICMP error message's original datagram's length is less than the original IP's header length.
116:253
116:462
-The ERSpan2 version is not equal to 1 (the value of 1 signals that it's ERSpan2).
+Incorrect ERSpan version. ERSpan2 should have version 1, ERSpan3 should have version 2.
116:463
The IPv6 packet has a reserved destination address.
+116:478
+
+ICMPv6 option length field is set to 0.
+
119:1
URI has percent encoding of an unreserved character. The ignore_unreserved option designates
The Snort Team
Revision History
-Revision 3.2.1.0 2024-05-16 22:50:33 EDT TST
+Revision 3.3.6.0 2024-09-05 12:57:08 EDT TST
---------------------------------------------------------------------
5.15. dns
5.16. domain_filter
5.17. dpx
- 5.18. file_id
- 5.19. file_log
- 5.20. ftp_client
- 5.21. ftp_data
- 5.22. ftp_server
- 5.23. gtp_inspect
- 5.24. http2_inspect
- 5.25. http_inspect
- 5.26. iec104
- 5.27. imap
- 5.28. mem_test
- 5.29. mms
- 5.30. modbus
- 5.31. netflow
- 5.32. normalizer
- 5.33. null_trace_logger
- 5.34. packet_capture
- 5.35. perf_monitor
- 5.36. pop
- 5.37. port_scan
- 5.38. reputation
- 5.39. rna
- 5.40. rpc_decode
- 5.41. s7commplus
- 5.42. sip
- 5.43. smtp
- 5.44. so_proxy
- 5.45. ssh
- 5.46. ssl
- 5.47. stream
- 5.48. stream_file
- 5.49. stream_icmp
- 5.50. stream_ip
- 5.51. stream_tcp
- 5.52. stream_udp
- 5.53. stream_user
- 5.54. telnet
- 5.55. wizard
+ 5.18. extractor
+ 5.19. file_id
+ 5.20. file_log
+ 5.21. ftp_client
+ 5.22. ftp_data
+ 5.23. ftp_server
+ 5.24. gtp_inspect
+ 5.25. http2_inspect
+ 5.26. http_inspect
+ 5.27. iec104
+ 5.28. imap
+ 5.29. mem_test
+ 5.30. mms
+ 5.31. modbus
+ 5.32. netflow
+ 5.33. normalizer
+ 5.34. null_trace_logger
+ 5.35. packet_capture
+ 5.36. perf_monitor
+ 5.37. pop
+ 5.38. port_scan
+ 5.39. reputation
+ 5.40. rna
+ 5.41. rpc_decode
+ 5.42. s7commplus
+ 5.43. sip
+ 5.44. smtp
+ 5.45. snort_ml
+ 5.46. snort_ml_engine
+ 5.47. so_proxy
+ 5.48. ssh
+ 5.49. ssl
+ 5.50. stream
+ 5.51. stream_file
+ 5.52. stream_icmp
+ 5.53. stream_ip
+ 5.54. stream_tcp
+ 5.55. stream_udp
+ 5.56. stream_user
+ 5.57. telnet
+ 5.58. wizard
6. IPS Action Modules
* latency.total_packets: total packets monitored (sum)
* latency.total_usecs: total usecs elapsed (sum)
- * latency.max_usecs: maximum usecs elapsed (sum)
+ * latency.max_usecs: maximum usecs elapsed (max)
* latency.packet_timeouts: packets that timed out (sum)
* latency.total_rule_evals: total rule evals monitored (sum)
* latency.rule_eval_timeouts: rule evals that timed out (sum)
ac_full | hyperscan | lowmem }
* string search_engine.rule_db_dir: deserialize rule databases from
given directory
- * bool search_engine.show_fast_patterns = false: print fast pattern
- info for each rule
* bool search_engine.split_any_any = true: evaluate any-any rules
separately to save memory
* int search_engine.queue_limit = 0: maximum number of fast pattern
* snort.log_command(command, logging): enable or disable command
logging
* snort.show_config_generation(): show loaded configuration ID
+ * snort.show_snort_cpu(): show snort cpu usage
* snort.pause(): suspend packet processing
* snort.resume(pkt_num): continue packet processing. If number of
packets is specified, will resume for n packets and pause
* int trace.modules.all: enable trace for all modules { 0:255 }
* int trace.modules.appid.all: enable all trace options { 0:255 }
* int trace.modules.dce_smb.all: enable all trace options { 0:255 }
+ * int trace.modules.detection.all: enable all trace options { 0:255
+ }
+ * int trace.modules.detection.opt_tree: enable tree option trace
+ logging { 0:255 }
+ * int trace.modules.detection.fp_info: enable fast pattern info
+ logging { 0:255 }
* int trace.modules.dpx.all: enable all trace options { 0:255 }
* int trace.modules.file_id.all: enable all trace options { 0:255 }
* int trace.modules.js_norm.all: enable all trace options { 0:255 }
* int trace.modules.snort.all: enable all trace options { 0:255 }
* int trace.modules.snort.inspector_manager: enable inspector
manager trace logging { 0:255 }
+ * int trace.modules.stream_tcp.all: enable all trace options {
+ 0:255 }
+ * int trace.modules.stream_tcp.segments: enable stream TCP segments
+ trace logging { 0:255 }
+ * int trace.modules.stream_tcp.state: enable stream TCP state trace
+ logging { 0:255 }
* int trace.modules.vba_data.all: enable all trace options { 0:255
}
* int trace.modules.wizard.all: enable all trace options { 0:255 }
* 116:460 (icmp6) ICMPv6 node info query/response packet with a
code greater than 2
* 116:474 (icmp6) ICMPv6 not encapsulated in IPv6
+ * 116:478 (icmp6) ICMPv6 option length field is set to 0
Peg counts:
* appid.reload_third_party(): reload appid third-party module
* appid.reload_detectors(): reload appid detectors
* appid.print_appid_config(): print appid configs
- * appid.show_cpu_profiler_stats(appid): show appid cpu profiling
- stats
+ * appid.show_cpu_profiler_stats(appid, display_rows_limit): show
+ appid cpu profiling stats
* appid.show_cpu_profiler_status(): show appid cpu profiling status
Peg counts:
* string binder[].when.tenants: list of tenants
* enum binder[].when.role = any: use the given configuration on one
or any end of a session { client | server | any }
- * string binder[].when.service: override default configuration
+ * string binder[].when.service: space separated list of services
* enum binder[].use.action = inspect: what to do with matching
traffic { reset | block | allow | inspect }
* string binder[].use.file: use configuration in given file
* dce_smb.cache_misses: smbv2 cache did not find entry (sum)
* dce_smb.cache_replaces: smbv2 cache found entry and replaced its
value (sum)
- * dce_smb.cache_max: smbv2 cache’s maximum byte usage (sum)
+ * dce_smb.cache_max: smbv2 cache’s maximum byte usage (max)
* dce_smb.cache_prunes: smbv2 cache pruned entry to make space for
new entry (sum)
* dce_smb.cache_removes: smbv2 cache removed existing entry (sum)
* dce_smb.response_fragments: total connection-oriented response
fragments (sum)
* dce_smb.client_max_fragment_size: connection-oriented client
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_smb.client_min_fragment_size: connection-oriented client
minimum fragment size (sum)
* dce_smb.client_segs_reassembled: total connection-oriented client
* dce_smb.client_frags_reassembled: total connection-oriented
client fragments reassembled (sum)
* dce_smb.server_max_fragment_size: connection-oriented server
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_smb.server_min_fragment_size: connection-oriented server
minimum fragment size (sum)
* dce_smb.server_segs_reassembled: total connection-oriented server
* dce_tcp.response_fragments: total connection-oriented response
fragments (sum)
* dce_tcp.client_max_fragment_size: connection-oriented client
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_tcp.client_min_fragment_size: connection-oriented client
minimum fragment size (sum)
* dce_tcp.client_segs_reassembled: total connection-oriented client
* dce_tcp.client_frags_reassembled: total connection-oriented
client fragments reassembled (sum)
* dce_tcp.server_max_fragment_size: connection-oriented server
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_tcp.server_min_fragment_size: connection-oriented server
minimum fragment size (sum)
* dce_tcp.server_segs_reassembled: total connection-oriented server
(sum)
* dce_udp.fragments: total connection-less fragments (sum)
* dce_udp.max_fragment_size: connection-less maximum fragment size
- (sum)
+ (max)
* dce_udp.frags_reassembled: total connection-less fragments
reassembled (sum)
* dce_udp.max_seqnum: max connection-less seqnum (sum)
* dpx.packets: total packets (sum)
-5.18. file_id
+5.18. extractor
+
+--------------
+
+Help: extracts protocol specific data
+
+Type: inspector (passive)
+
+Usage: global
+
+Instance Type: global
+
+Configuration:
+
+ * enum extractor.formatting = csv: output format for extractor {
+ csv | json }
+ * enum extractor.output = stdout: output destination for extractor
+ { stdout }
+ * enum extractor.protocols[].service: service to extract from {
+ http }
+ * int extractor.protocols[].tenant_id = 0: tenant_id of target
+ tenant { 0:max32 }
+ * string extractor.protocols[].on_events: specify events to log
+ * string extractor.protocols[].fields: specify fields to log
+
+Peg counts:
+
+ * extractor.total_events: total extractor events (sum)
+
+
+5.19. file_id
--------------
concurrently on a flow (max)
-5.19. file_log
+5.20. file_log
--------------
* file_log.total_events: total file events (sum)
-5.20. ftp_client
+5.21. ftp_client
--------------
sequences on FTP control channel
-5.21. ftp_data
+5.22. ftp_data
--------------
* ftp_data.packets: total packets (sum)
-5.22. ftp_server
+5.23. ftp_server
--------------
sessions with segment size change (sum)
-5.23. gtp_inspect
+5.24. gtp_inspect
--------------
* gtp_inspect.unknown_infos: unknown information elements (sum)
-5.24. http2_inspect
+5.25. http2_inspect
--------------
concurrent streams (sum)
-5.25. http_inspect
+5.26. http_inspect
--------------
(sum)
* http_inspect.skip_mime_attach: total number of HTTP requests with
too many MIME attachments to inspect (sum)
+ * http_inspect.compressed_gzip: total number of HTTP bodies
+ compressed with GZIP (sum)
+ * http_inspect.compressed_not_supported: total number of HTTP
+ bodies compressed with known but not supported methods (sum)
+ * http_inspect.compressed_unknown: total number of HTTP bodies
+ compressed with unknown methods (sum)
-5.26. iec104
+5.27. iec104
--------------
sessions (max)
-5.27. imap
+5.28. imap
--------------
* imap.js_pdf_scripts: total number of PDF files processed (sum)
-5.28. mem_test
+5.29. mem_test
--------------
* mem_test.packets: total packets (sum)
-5.29. mms
+5.30. mms
--------------
(max)
-5.30. modbus
+5.31. modbus
--------------
sessions (max)
-5.31. netflow
+5.32. netflow
--------------
* netflow.cache_misses: netflow cache did not find entry (sum)
* netflow.cache_replaces: netflow cache found entry and replaced
its value (sum)
- * netflow.cache_max: netflow cache’s maximum byte usage (sum)
+ * netflow.cache_max: netflow cache’s maximum byte usage (max)
* netflow.cache_prunes: netflow cache pruned entry to make space
for new entry (sum)
* netflow.cache_removes: netflow cache removed existing entry (sum)
template cache (now)
-5.32. normalizer
+5.33. normalizer
--------------
* normalizer.tcp_block: blocked segments (sum)
-5.33. null_trace_logger
+5.34. null_trace_logger
--------------
Instance Type: global
-5.34. packet_capture
+5.35. packet_capture
--------------
Help: raw packet dumping facility
-Type: inspector (probe)
+Type: inspector (probe_first)
Usage: global
(sum)
-5.35. perf_monitor
+5.36. perf_monitor
--------------
* bool perf_monitor.flow = false: enable traffic statistics
* bool perf_monitor.flow_ip = false: enable statistics on host
pairs
+ * bool perf_monitor.flow_ip_all = false: enable every stat of
+ flow_ip profiling on host pairs
* int perf_monitor.packets = 10000: minimum packets to report {
0:max32 }
* int perf_monitor.seconds = 60: report interval { 0:max32 }
Commands:
- * perf_monitor.enable_flow_ip_profiling(seconds, packets): enable
- statistics on host pairs
+ * perf_monitor.enable_flow_ip_profiling(seconds, packets,
+ flow_ip_all): enable all statistics on host pairs
* perf_monitor.disable_flow_ip_profiling(): disable statistics on
host pairs
* perf_monitor.show_flow_ip_profiling(): show status of statistics
by new flows (sum)
-5.36. pop
+5.37. pop
--------------
* pop.js_pdf_scripts: total number of PDF files processed (sum)
-5.37. port_scan
+5.38. port_scan
--------------
portscan (now)
-5.38. reputation
+5.39. reputation
--------------
monitored (sum)
-5.39. rna
+5.40. rna
--------------
* rna.total_bytes_in_interval: count of bytes processed (sum)
-5.40. rpc_decode
+5.41. rpc_decode
--------------
sessions (max)
-5.41. s7commplus
+5.42. s7commplus
--------------
sessions (max)
-5.42. sip
+5.43. sip
--------------
* int sip.max_via_len = 1024: maximum via field size { 0:65535 }
* string sip.methods = invite cancel ack bye register options: list
of methods to check in SIP messages
+ * int sip.sip_timeout = 0: SIP Timeout value in milliseconds { 0: }
+ * int sip.sip_media_timeout = 0: SIP Media timeout milliseconds {
+ 0: }
+ * int sip.sip_invite_timeout = 0: SIP Invite timeout milliseconds {
+ 0: }
+ * int sip.sip_disconnect_timeout = 0: SIP Disconnect timeout
+ milliseconds { 0: }
Rules:
* sip.concurrent_sessions: total concurrent SIP sessions (now)
* sip.max_concurrent_sessions: maximum concurrent SIP sessions
(max)
+ * sip.aborted_sessions: total session aborted (sum)
* sip.events: events generated (sum)
* sip.dialogs: total dialogs (sum)
* sip.ignored_channels: total channels ignored (sum)
* sip.code_9xx: 9xx (sum)
-5.43. smtp
+5.44. smtp
--------------
* smtp.js_pdf_scripts: total number of PDF files processed (sum)
-5.44. so_proxy
+5.45. snort_ml
+
+--------------
+
+Help: machine learning based exploit detector
+
+Type: inspector (passive)
+
+Usage: inspect
+
+Instance Type: singleton
+
+Configuration:
+
+ * int snort_ml.uri_depth = -1: number of input HTTP URI bytes to
+ scan (-1 unlimited) { -1:max31 }
+ * int snort_ml.client_body_depth = 0: number of input HTTP client
+ body bytes to scan (-1 unlimited) { -1:max31 }
+ * real snort_ml.http_param_threshold = 0.95: alert threshold for
+ http_param_model { 0:1 }
+
+Rules:
+
+ * 411:1 (snort_ml) potential threat found in HTTP parameters via
+ Neural Network Based Exploit Detection
+
+Peg counts:
+
+ * snort_ml.uri_alerts: total number of alerts triggered on HTTP URI
+ (sum)
+ * snort_ml.client_body_alerts: total number of alerts triggered on
+ HTTP client body (sum)
+ * snort_ml.uri_bytes: total number of HTTP URI bytes processed
+ (sum)
+ * snort_ml.client_body_bytes: total number of HTTP client body
+ bytes processed (sum)
+ * snort_ml.libml_calls: total libml calls (sum)
+
+
+5.46. snort_ml_engine
+
+--------------
+
+Help: configure machine learning engine settings
+
+Type: inspector (passive)
+
+Usage: global
+
+Instance Type: global
+
+Configuration:
+
+ * string snort_ml_engine.http_param_model: path to the model file
+
+
+5.47. so_proxy
--------------
Instance Type: global
-5.45. ssh
+5.48. ssh
--------------
(max)
-5.46. ssl
+5.49. ssl
--------------
(max)
-5.47. stream
+5.50. stream
--------------
* stream.uni_ip_flows: number of uni ip flows in cache (now)
-5.48. stream_file
+5.51. stream_file
--------------
* bool stream_file.upload = false: indicate file transfer direction
-5.49. stream_icmp
+5.52. stream_icmp
--------------
* stream_icmp.prunes: icmp session prunes (sum)
-5.50. stream_ip
+5.53. stream_ip
--------------
* stream_ip.total_bytes: total number of bytes processed (sum)
* stream_ip.total_frags: total fragments (sum)
* stream_ip.current_frags: current fragments (now)
- * stream_ip.max_frags: max fragments (sum)
+ * stream_ip.max_frags: max fragments (max)
* stream_ip.reassembled: reassembled datagrams (sum)
* stream_ip.discards: fragments discarded (sum)
* stream_ip.frag_timeouts: datagrams abandoned (sum)
* stream_ip.fragmented_bytes: total fragmented bytes (sum)
-5.51. stream_tcp
+5.54. stream_tcp
--------------
(sum)
* stream_tcp.zero_win_probes: number of tcp zero window probes
(sum)
+ * stream_tcp.keep_alive_probes: number of tcp keep-alive probes
+ (sum)
* stream_tcp.proxy_mode_flows: number of flows set to proxy
normalization policy (sum)
* stream_tcp.full_retransmits: number of fully retransmitted
one-way traffic only (sum)
-5.52. stream_udp
+5.55. stream_udp
--------------
* stream_udp.ignored: udp packets ignored (sum)
-5.53. stream_user
+5.56. stream_user
--------------
1:max31 }
-5.54. telnet
+5.57. telnet
--------------
sessions (max)
-5.55. wizard
+5.58. wizard
--------------
Peg counts:
- * pcre.pcre_rules: total rules processed with pcre option (sum)
- * pcre.pcre_to_hyper: total pcre rules by hyperscan engine (sum)
- * pcre.pcre_native: total pcre rules compiled by pcre engine (sum)
- * pcre.pcre_negated: total pcre rules using negation syntax (sum)
* pcre.pcre_match_limit: total number of times pcre hit the match
limit (sum)
* pcre.pcre_recursion_limit: total number of times pcre hit the
* bool alert_fast.file = false: output to alert_fast.txt instead of
stdout
* bool alert_fast.packet = false: output packet dump with alert
- * bool alert_fast.buffers = false: output IPS buffer dump
+ * enum alert_fast.buffers = none: output IPS buffer dump (evaluated
+ by IPS rule or an inspector) { none | rule | inspector | both }
* int alert_fast.buffers_depth = 0: number of IPS buffer bytes to
dump per buffer (0 is unlimited) { 0:maxSZ }
* int alert_fast.limit = 0: set maximum size in MB before rollover
case
* int alert_fast.buffers_depth = 0: number of IPS buffer bytes to
dump per buffer (0 is unlimited) { 0:maxSZ }
- * bool alert_fast.buffers = false: output IPS buffer dump
+ * enum alert_fast.buffers = none: output IPS buffer dump (evaluated
+ by IPS rule or an inspector) { none | rule | inspector | both }
* bool alert_fast.file = false: output to alert_fast.txt instead of
stdout
* int alert_fast.limit = 0: set maximum size in MB before rollover
| user | file }
* enum binder[].when.role = any: use the given configuration on one
or any end of a session { client | server | any }
- * string binder[].when.service: override default configuration
+ * string binder[].when.service: space separated list of services
* string binder[].when.src_groups: list of source interface group
IDs
* string binder[].when.src_intfs: list of source interface IDs
ordering incoming events { priority|content_length }
* bool event_queue.process_all_events = false: process just first
action group or all action groups
+ * enum extractor.formatting = csv: output format for extractor {
+ csv | json }
+ * enum extractor.output = stdout: output destination for extractor
+ { stdout }
+ * string extractor.protocols[].fields: specify fields to log
+ * string extractor.protocols[].on_events: specify events to log
+ * enum extractor.protocols[].service: service to extract from {
+ http }
+ * int extractor.protocols[].tenant_id = 0: tenant_id of target
+ tenant { 0:max32 }
* string file_connector[].connector: connector name
* enum file_connector[].direction: usage { receive | transmit |
duplex }
* bool perf_monitor.base = true: enable base statistics
* bool perf_monitor.cpu = false: enable cpu statistics
* bool perf_monitor.flow = false: enable traffic statistics
+ * bool perf_monitor.flow_ip_all = false: enable every stat of
+ flow_ip profiling on host pairs
* bool perf_monitor.flow_ip = false: enable statistics on host
pairs
* int perf_monitor.flow_ip_memcap = 52428800: maximum memory in
* dynamic search_engine.search_method = ac_bnfa: set fast pattern
algorithm - choose available search engine { ac_bnfa | ac_full |
hyperscan | lowmem }
- * bool search_engine.show_fast_patterns = false: print fast pattern
- info for each rule
* bool search_engine.split_any_any = true: evaluate any-any rules
separately to save memory
* interval seq.~range: check if TCP sequence number is in given
* string sip_method.*method: sip method
* string sip.methods = invite cancel ack bye register options: list
of methods to check in SIP messages
+ * int sip.sip_disconnect_timeout = 0: SIP Disconnect timeout
+ milliseconds { 0: }
+ * int sip.sip_invite_timeout = 0: SIP Invite timeout milliseconds {
+ 0: }
+ * int sip.sip_media_timeout = 0: SIP Media timeout milliseconds {
+ 0: }
+ * int sip.sip_timeout = 0: SIP Timeout value in milliseconds { 0: }
* int sip_stat_code.*code: status code { 1:999 }
* string smtp.alt_max_command_line_len[].command: command string
* int smtp.alt_max_command_line_len[].length = 0: specify
engines
* string snort.--metadata-filter: <filter> load only rules
containing filter string in metadata if set
+ * int snort_ml.client_body_depth = 0: number of input HTTP client
+ body bytes to scan (-1 unlimited) { -1:max31 }
+ * string snort_ml_engine.http_param_model: path to the model file
+ * real snort_ml.http_param_threshold = 0.95: alert threshold for
+ http_param_model { 0:1 }
* implied snort.-M: log messages to syslog (not alerts)
+ * int snort_ml.uri_depth = -1: number of input HTTP URI bytes to
+ scan (-1 unlimited) { -1:max31 }
* int snort.-m: <umask> set the process file mode creation mask {
0x000:0x1FF }
* int snort.-n: <count> stop after count packets { 0:max53 }
* int trace.modules.all: enable trace for all modules { 0:255 }
* int trace.modules.appid.all: enable all trace options { 0:255 }
* int trace.modules.dce_smb.all: enable all trace options { 0:255 }
+ * int trace.modules.detection.all: enable all trace options { 0:255
+ }
+ * int trace.modules.detection.fp_info: enable fast pattern info
+ logging { 0:255 }
+ * int trace.modules.detection.opt_tree: enable tree option trace
+ logging { 0:255 }
* int trace.modules.dpx.all: enable all trace options { 0:255 }
* int trace.modules.file_id.all: enable all trace options { 0:255 }
* int trace.modules.js_norm.all: enable all trace options { 0:255 }
* int trace.modules.snort.all: enable all trace options { 0:255 }
* int trace.modules.snort.inspector_manager: enable inspector
manager trace logging { 0:255 }
+ * int trace.modules.stream_tcp.all: enable all trace options {
+ 0:255 }
+ * int trace.modules.stream_tcp.segments: enable stream TCP segments
+ trace logging { 0:255 }
+ * int trace.modules.stream_tcp.state: enable stream TCP state trace
+ logging { 0:255 }
* int trace.modules.vba_data.all: enable all trace options { 0:255
}
* int trace.modules.wizard.all: enable all trace options { 0:255 }
* dce_smb.binds: total connection-oriented binds (sum)
* dce_smb.cache_adds: smbv2 cache added new entry (sum)
* dce_smb.cache_hits: smbv2 cache found existing entry (sum)
- * dce_smb.cache_max: smbv2 cache’s maximum byte usage (sum)
+ * dce_smb.cache_max: smbv2 cache’s maximum byte usage (max)
* dce_smb.cache_misses: smbv2 cache did not find entry (sum)
* dce_smb.cache_prunes: smbv2 cache pruned entry to make space for
new entry (sum)
* dce_smb.client_frags_reassembled: total connection-oriented
client fragments reassembled (sum)
* dce_smb.client_max_fragment_size: connection-oriented client
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_smb.client_min_fragment_size: connection-oriented client
minimum fragment size (sum)
* dce_smb.client_segs_reassembled: total connection-oriented client
* dce_smb.server_frags_reassembled: total connection-oriented
server fragments reassembled (sum)
* dce_smb.server_max_fragment_size: connection-oriented server
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_smb.server_min_fragment_size: connection-oriented server
minimum fragment size (sum)
* dce_smb.server_segs_reassembled: total connection-oriented server
* dce_tcp.client_frags_reassembled: total connection-oriented
client fragments reassembled (sum)
* dce_tcp.client_max_fragment_size: connection-oriented client
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_tcp.client_min_fragment_size: connection-oriented client
minimum fragment size (sum)
* dce_tcp.client_segs_reassembled: total connection-oriented client
* dce_tcp.server_frags_reassembled: total connection-oriented
server fragments reassembled (sum)
* dce_tcp.server_max_fragment_size: connection-oriented server
- maximum fragment size (sum)
+ maximum fragment size (max)
* dce_tcp.server_min_fragment_size: connection-oriented server
minimum fragment size (sum)
* dce_tcp.server_segs_reassembled: total connection-oriented server
* dce_udp.max_concurrent_sessions: maximum concurrent sessions
(max)
* dce_udp.max_fragment_size: connection-less maximum fragment size
- (sum)
+ (max)
* dce_udp.max_seqnum: max connection-less seqnum (sum)
* dce_udp.no_calls: total connection-less no calls (sum)
* dce_udp.other_requests: total connection-less other requests
out of global memory (sum)
* event_filter.no_memory_local: number of times event filter ran
out of local memory (sum)
+ * extractor.total_events: total extractor events (sum)
* file_connector.messages: total messages (sum)
* file_id.cache_failures: number of file cache add failures (sum)
* file_id.files_not_processed: number of files not processed due to
* http2_inspect.total_bytes: total HTTP/2 data bytes inspected
(sum)
* http_inspect.chunked: chunked message bodies (sum)
+ * http_inspect.compressed_gzip: total number of HTTP bodies
+ compressed with GZIP (sum)
+ * http_inspect.compressed_not_supported: total number of HTTP
+ bodies compressed with known but not supported methods (sum)
+ * http_inspect.compressed_unknown: total number of HTTP bodies
+ compressed with unknown methods (sum)
* http_inspect.concurrent_sessions: total concurrent http sessions
(now)
* http_inspect.connect_requests: CONNECT requests inspected (sum)
limit overflows (sum)
* js_norm.identifiers: total number of unique identifiers processed
(sum)
- * latency.max_usecs: maximum usecs elapsed (sum)
+ * latency.max_usecs: maximum usecs elapsed (max)
* latency.packet_timeouts: packets that timed out (sum)
* latency.rule_eval_timeouts: rule evals that timed out (sum)
* latency.rule_tree_enables: rule tree re-enables (sum)
* modbus.sessions: total sessions processed (sum)
* netflow.cache_adds: netflow cache added new entry (sum)
* netflow.cache_hits: netflow cache found existing entry (sum)
- * netflow.cache_max: netflow cache’s maximum byte usage (sum)
+ * netflow.cache_max: netflow cache’s maximum byte usage (max)
* netflow.cache_misses: netflow cache did not find entry (sum)
* netflow.cache_prunes: netflow cache pruned entry to make space
for new entry (sum)
* pcre.pcre_error: total number of times pcre returns error (sum)
* pcre.pcre_match_limit: total number of times pcre hit the match
limit (sum)
- * pcre.pcre_native: total pcre rules compiled by pcre engine (sum)
- * pcre.pcre_negated: total pcre rules using negation syntax (sum)
* pcre.pcre_recursion_limit: total number of times pcre hit the
recursion limit (sum)
- * pcre.pcre_rules: total rules processed with pcre option (sum)
- * pcre.pcre_to_hyper: total pcre rules by hyperscan engine (sum)
* perf_monitor.flow_tracker_creates: total number of flow trackers
created (sum)
* perf_monitor.flow_tracker_prunes: flow trackers pruned for reuse
to overflow (sum)
* search_engine.total_unique: total unique fast pattern hits (sum)
* side_channel.packets: total packets (sum)
+ * sip.aborted_sessions: total session aborted (sum)
* sip.ack: ack (sum)
* sip.bye: bye (sum)
* sip.cancel: cancel (sum)
* snort.inspector_deletions: number of times inspectors were
deleted (sum)
* snort.local_commands: total local commands processed (sum)
+ * snort_ml.client_body_alerts: total number of alerts triggered on
+ HTTP client body (sum)
+ * snort_ml.client_body_bytes: total number of HTTP client body
+ bytes processed (sum)
+ * snort_ml.libml_calls: total libml calls (sum)
+ * snort_ml.uri_alerts: total number of alerts triggered on HTTP URI
+ (sum)
+ * snort_ml.uri_bytes: total number of HTTP URI bytes processed
+ (sum)
* snort.policy_reloads: number of times policies were reloaded
(sum)
* snort.remote_commands: total remote commands processed (sum)
* stream_ip.drops: fragments dropped (sum)
* stream_ip.fragmented_bytes: total fragmented bytes (sum)
* stream_ip.frag_timeouts: datagrams abandoned (sum)
- * stream_ip.max_frags: max fragments (sum)
+ * stream_ip.max_frags: max fragments (max)
* stream_ip.max: max ip sessions (max)
* stream.ip_memcap_prunes: number of IP flows pruned due to memcap
(sum)
number (sum)
* stream_tcp.invalid_seq_num: tcp packets received with an invalid
sequence number (sum)
+ * stream_tcp.keep_alive_probes: number of tcp keep-alive probes
+ (sum)
* stream_tcp.max_bytes: maximum number of bytes queued in any flow
(max)
* stream_tcp.max: max tcp sessions (max)
* 154: js_norm
* 175: domain_filter
* 256: dpx
+ * 411: snort_ml
11.7. Builtin Rules
116:250 (icmp4) ICMP original IP header truncated
-The ICMP error message’s original IP header is truncated.
+The ICMP or IP in ICMP error message’s original IP header is
+truncated.
116:251 (icmp4) ICMP version and original IP header versions differ
-The ICMP error message’s original IP packet’s version and original IP
-header versions differ.
+The ICMP or IP in ICMP error message’s original IP packet’s version
+and original IP header versions differ.
116:252 (icmp4) ICMP original datagram length < original IP header
length
-The ICMP error message’s original datagram’s length is less than the
-original IP’s header length.
+The ICMP or IP in ICMP error message’s original datagram’s length is
+less than the original IP’s header length.
116:253 (icmp4) ICMP original IP payload < 64 bits
116:462 (erspan2) ERSpan header version mismatch
-The ERSpan2 version is not equal to 1 (the value of 1 signals that
-it’s ERSpan2).
+Incorrect ERSpan version. ERSpan2 should have version 1, ERSpan3
+should have version 2.
116:463 (erspan2) captured length < ERSpan type2 header length
The IPv6 packet has a reserved destination address.
+116:478 (icmp6) ICMPv6 option length field is set to 0
+
+ICMPv6 option length field is set to 0.
+
119:1 (http_inspect) URI has percent-encoding of an unreserved
character
* appid.reload_third_party(): reload appid third-party module
* appid.reload_detectors(): reload appid detectors
* appid.print_appid_config(): print appid configs
- * appid.show_cpu_profiler_stats(appid): show appid cpu profiling
- stats
+ * appid.show_cpu_profiler_stats(appid, display_rows_limit): show
+ appid cpu profiling stats
* appid.show_cpu_profiler_status(): show appid cpu profiling status
* host_cache.dump(file_name): dump host cache
* host_cache.delete_host(host_ip): delete host from host cache
* packet_tracer.enable(proto, src_ip, src_port, dst_ip, dst_port,
tenants): enable packet tracer debugging
* packet_tracer.disable(): disable packet tracer
- * perf_monitor.enable_flow_ip_profiling(seconds, packets): enable
- statistics on host pairs
+ * perf_monitor.enable_flow_ip_profiling(seconds, packets,
+ flow_ip_all): enable all statistics on host pairs
* perf_monitor.disable_flow_ip_profiling(): disable statistics on
host pairs
* perf_monitor.show_flow_ip_profiling(): show status of statistics
* snort.log_command(command, logging): enable or disable command
logging
* snort.show_config_generation(): show loaded configuration ID
+ * snort.show_snort_cpu(): show snort cpu usage
* snort.pause(): suspend packet processing
* snort.resume(pkt_num): continue packet processing. If number of
packets is specified, will resume for n packets and pause
* eth (codec): support for ethernet protocol (DLT 1) (DLT 51)
* event_filter (basic): configure thresholding of events
* event_queue (basic): configure event queue parameters
+ * extractor (inspector): extracts protocol specific data
* fabricpath (codec): support for fabricpath
* file_connector (connector): implement the file based connector
* file_data (ips_option): rule option to set detection cursor to
* sip_stat_code (ips_option): detection option for sip stat code
* smtp (inspector): smtp inspection
* snort (basic): command line configuration and shell commands
+ * snort_ml (inspector): machine learning based exploit detector
+ * snort_ml_engine (inspector): configure machine learning engine
+ settings
* so (ips_option): rule option to call custom eval function
* so_proxy (inspector): a proxy inspector to track flow data from
SO rules (internal use only)
* inspector::dns: dns inspection
* inspector::domain_filter: alert on configured HTTP domains
* inspector::dpx: dynamic inspector example
+ * inspector::extractor: extracts protocol specific data
* inspector::file_id: configure file identification
* inspector::file_log: log file event to file.log
* inspector::ftp_client: FTP inspector client module
* inspector::s7commplus: s7commplus inspection
* inspector::sip: sip inspection
* inspector::smtp: smtp inspection
+ * inspector::snort_ml: machine learning based exploit detector
+ * inspector::snort_ml_engine: configure machine learning engine
+ settings
* inspector::so_proxy: a proxy inspector to track flow data from SO
rules (internal use only)
* inspector::ssh: ssh inspection
The Snort Team
Revision History
-Revision 3.2.1.0 2024-05-16 22:51:40 EDT TST
+Revision 3.3.6.0 2024-09-05 12:57:44 EDT TST
---------------------------------------------------------------------
The Snort Team
Revision History
-Revision 3.2.1.0 2024-05-16 22:50:55 EDT TST
+Revision 3.3.6.0 2024-09-05 12:57:20 EDT TST
---------------------------------------------------------------------
file_id (msg:"Microsoft Office Open XML Format (OOXML) Document (XLSX)"; file_meta:type XLSX, id 328, category "Office Documents,Dynamic Analysis Capable,Local Malware Analysis Capable", group "office"; file_data; content:"| 50 4B 03 04 |", depth 4, offset 0; content:"| 78 6c 2f |", depth 3, offset 30; gid:4; sid:206; rev:1;)
file_id (msg:"Microsoft Office Open XML Format (OOXML) Document (DOCX, PPTX, XLSX)"; file_meta:type NEW_OFFICE, id 329, category "Office Documents,Dynamic Analysis Capable,Local Malware Analysis Capable", group "office"; file_data; content:"| 50 4B 03 04 |", depth 4, offset 0; content:"| 5b 43 6f 6e 74 65 6e 74 5f 54 79 70 65 73 5d 2e |", depth 16, offset 30; gid:4; sid:207; rev:1;)
file_id (msg:"Microsoft Office Open XML Format (OOXML) Document (DOCX, PPTX, XLSX)"; file_meta:type NEW_OFFICE, id 330, category "Office Documents,Dynamic Analysis Capable,Local Malware Analysis Capable", group "office"; file_data; content:"| 50 4B 03 04 |", depth 4, offset 0; content:"| 5f 72 65 6c 73 2f |", depth 6, offset 30; gid:4; sid:208; rev:1;)
-file_id (msg:"Microsoft Office Open XML Format (OOXML) Document (DOCX, PPTX, XLSX)"; file_meta:type NEW_OFFICE, id 331, category "Office Documents,Dynamic Analysis Capable,Local Malware Analysis Capable", group "office"; file_data; content:"| 50 4B 03 04 |", depth 4, offset 0; content:"| 64 6f 63 50 72 6f 70 73 2f |", depth 9, offset 30; gid:4; sid:209; rev:1;)
\ No newline at end of file
+file_id (msg:"Microsoft Office Open XML Format (OOXML) Document (DOCX, PPTX, XLSX)"; file_meta:type NEW_OFFICE, id 331, category "Office Documents,Dynamic Analysis Capable,Local Malware Analysis Capable", group "office"; file_data; content:"| 50 4B 03 04 |", depth 4, offset 0; content:"| 64 6f 63 50 72 6f 70 73 2f |", depth 9, offset 30; gid:4; sid:209; rev:1;)
+file_id (msg:"Audio Interchange File Format"; file_meta:type AIF, id 345, category "Multimedia"; file_data; content:"| 46 4F 52 40 |", depth 4, offset 0; gid:4; sid:210; rev:1;)
+file_id (msg:"Audio Interchange File Format"; file_meta:type AIF, id 346, category "Multimedia"; file_data; content:"| 41 49 46 46 |", depth 4, offset 8; gid:4; sid:211; rev:1;)
+file_id (msg:"Debian package file"; file_meta:type DEB, id 333, category "System files"; file_data; content:"| 21 3C 61 72 |", depth 4, offset 0; gid:4; sid:212; rev:1;)
+file_id (msg:"Windows Cursor file"; file_meta:type CUR, id 337, category "System files"; file_data; content:"| 00 00 02 00 02 00 30 30 00 00 01 |", depth 11, offset 0; gid:4; sid:216; rev:1;)
+file_id (msg:"Audio Video Interleave"; file_meta:type AVI, id 339, category "Multimedia"; file_data; content:"| 41 56 49 |", depth 3, offset 8; gid:4; sid:218; rev:1;)
+file_id (msg:"Sound file"; file_meta:type SND, id 340, category "Multimedia"; file_data; content:"| 73 6E 64 |", depth 3, offset 1; gid:4; sid:219; rev:1;)
+file_id (msg:"MPEG-4 Audio"; file_meta:type M4A, id 341, category "Multimedia"; file_data; content:"| 4D 34 41 |", depth 3, offset 8; gid:4; sid:220; rev:1;)
+file_id (msg:"Video Object file, audio container in DVD media"; file_meta:type VOB, id 342, category "Multimedia"; file_data; content:"| 00 00 01 BA 44 |", depth 5, offset 0; gid:4; sid:221; rev:1;)
+file_id (msg:"Windows Media Video"; file_meta:type WMV, id 343, category "Multimedia"; file_data; content:"| 30 26 B2 75 |", depth 4, offset 0; gid:4; sid:222; rev:1;)
+file_id (msg:"iTunes video file"; file_meta:type M5V, id 344, category "Multimedia"; file_data; content:"| 4D 34 56 |", depth 3, offset 8; gid:4; sid:223; rev:1;)
+file_id (msg:"Executable script"; file_meta:type UNIX_SCRIPT, id 347, category "Executables"; file_data; content:"| 23 21 2F |", depth 3, offset 0; gid:4; sid:224; rev:1;)
URL: www.snort.org
Version: @VERSION@
Libs: -L${libdir}/snort
-Cflags: -I${includedir}/snort @DEEP_PROFILING_CPPFLAGS@ @MEMORY_OVERLOADS_CPPFLAGS@ @MEMORY_PROFILER_CPPFLAGS@ @RULE_PROFILER_CPPFLAGS@ @NO_PROFILER_CPPFLAGS@ @TP_APPID_CPPFLAGS@ @TSC_CPPFLAGS@
+Cflags: -I${includedir}/snort @DEEP_PROFILING_CPPFLAGS@ @MEMORY_OVERLOADS_CPPFLAGS@ @MEMORY_PROFILER_CPPFLAGS@ @RULE_PROFILER_CPPFLAGS@ @NO_PROFILER_CPPFLAGS@ @TP_APPID_CPPFLAGS@ @TSC_CPPFLAGS@ @TENANT_ID_CPPFLAGS@
DECODE_MIPV6_BAD_PAYLOAD_PROTO,
DECODE_IPV6_SRC_RESERVED,
DECODE_IPV6_DST_RESERVED,
+ DECODE_ICMP6_OPT_ZERO_LENGTH,
DECODE_INDEX_MAX
};
{ DECODE_ICMPV6_NODE_INFO_BAD_CODE,
"ICMPv6 node info query/response packet with a code greater than 2" },
{ DECODE_ICMP6_NOT_IP6, "ICMPv6 not encapsulated in IPv6" },
+ { DECODE_ICMP6_OPT_ZERO_LENGTH, "ICMPv6 option length field is set to 0" },
{ 0, nullptr }
};
private:
bool valid_checksum_from_daq(const RawData&);
+ template<typename OptionType>
+ inline void validate_ndp_option(CodecData &codec, const OptionType* const icmp_pkt,
+ const uint16_t min_len, const uint16_t dsize);
};
} // anonymous namespace
return true;
}
+template <typename NDPType>
+void Icmp6Codec::validate_ndp_option(CodecData &codec, const NDPType* const icmp_pkt,
+ const uint16_t min_len, const uint16_t dsize)
+{
+ const uint16_t real_size = dsize + icmp::ICMP6_HEADER_MIN_LEN;
+ bool option_field_not_exists = real_size == min_len;
+ if (option_field_not_exists)
+ return;
+
+ assert(real_size > min_len - icmp::ICMP6_HEADER_MIN_LEN);
+ // Based on RFC2461, for types 133-137, "option" field depends on version
+ // of implementation, so there's no unified format for all types.
+ assert(icmp_pkt->type >= 133 && icmp_pkt->type <= 137);
+
+ const uint8_t* const icmp_pkt_ptr = (const uint8_t* const)icmp_pkt;
+ const uint8_t* curr_option_ptr = (const uint8_t* const)&icmp_pkt->options_start;
+
+ while (real_size > curr_option_ptr - icmp_pkt_ptr )
+ {
+ assert(curr_option_ptr > icmp_pkt_ptr);
+ bool field_incomplete = real_size == curr_option_ptr - icmp_pkt_ptr + 1;
+ if (field_incomplete)
+ return; // FIXIT-L good candidate for new builtin rule
+
+ const icmp::NDPOptionFormatBasic* curr_option = (const icmp::NDPOptionFormatBasic*)curr_option_ptr;
+
+ // Right now, only option types explicitly specified in RFC4861 are supported
+ if (curr_option->type == 0 || curr_option->type > 5)
+ return; // FIXIT-L good candidate for non-supported option field
+
+ if (curr_option->length == 0)
+ {
+ codec_event(codec, DECODE_ICMP6_OPT_ZERO_LENGTH);
+ return;
+ }
+
+ curr_option_ptr += curr_option->length * 8;
+ }
+}
+
bool Icmp6Codec::decode(const RawData& raw, CodecData& codec, DecodeData& snort)
{
if (raw.len < icmp::ICMP6_HEADER_MIN_LEN)
}
break;
+ case icmp::Icmp6Types::ROUTER_SOLICITATION:
+ if (dsize >= (ICMPv6_RS_MIN_LEN - icmp::ICMP6_HEADER_MIN_LEN))
+ {
+ const icmp::ICMP6RouterSolicitation* rs = (const icmp::ICMP6RouterSolicitation*)raw.data;
+
+ if (rs->code != 0)
+ codec_event(codec, DECODE_ICMPV6_SOLICITATION_BAD_CODE);
+
+ if (ntohl(rs->reserved) != 0)
+ codec_event(codec, DECODE_ICMPV6_SOLICITATION_BAD_RESERVED);
+
+ validate_ndp_option(codec, rs, ICMPv6_RS_MIN_LEN, dsize);
+ }
+ else
+ {
+ codec_event(codec, DECODE_ICMP_DGRAM_LT_ICMPHDR);
+ return false;
+ }
+ break;
+
case icmp::Icmp6Types::ROUTER_ADVERTISEMENT:
- if (dsize >= (sizeof(icmp::ICMP6RouterAdvertisement) - icmp::ICMP6_HEADER_MIN_LEN))
+ if (dsize >= (ICMPv6_RA_MIN_LEN - icmp::ICMP6_HEADER_MIN_LEN))
{
const icmp::ICMP6RouterAdvertisement* ra = (const icmp::ICMP6RouterAdvertisement*)raw.data;
if (ntohl(ra->reachable_time) > 3600000)
codec_event(codec, DECODE_ICMPV6_ADVERT_BAD_REACHABLE);
+
+ validate_ndp_option(codec, ra, ICMPv6_RA_MIN_LEN, dsize);
}
else
{
}
break;
- case icmp::Icmp6Types::ROUTER_SOLICITATION:
- if (dsize >= (sizeof(icmp::ICMP6RouterSolicitation) - icmp::ICMP6_HEADER_MIN_LEN))
+ case icmp::Icmp6Types::NEIGHBOR_SOLICITATION:
+ if (dsize >= (ICMPv6_NS_MIN_LEN - icmp::ICMP6_HEADER_MIN_LEN))
{
- const icmp::ICMP6RouterSolicitation* rs = (const icmp::ICMP6RouterSolicitation*)raw.data;
- if (rs->code != 0)
- codec_event(codec, DECODE_ICMPV6_SOLICITATION_BAD_CODE);
+ const icmp::ICMP6NeighborSolicitation* ns = (const icmp::ICMP6NeighborSolicitation*)raw.data;
- if (ntohl(rs->reserved) != 0)
- codec_event(codec, DECODE_ICMPV6_SOLICITATION_BAD_RESERVED);
+ validate_ndp_option(codec, ns, ICMPv6_NS_MIN_LEN, dsize);
+ }
+ else
+ {
+ codec_event(codec, DECODE_ICMP_DGRAM_LT_ICMPHDR);
+ return false;
+ }
+ break;
+
+ case icmp::Icmp6Types::NEIGHBOR_ADVERTISEMENT:
+ if (dsize >= (ICMPv6_NA_MIN_LEN - icmp::ICMP6_HEADER_MIN_LEN))
+ {
+ const icmp::ICMP6NeighborAdvertisement* na = (const icmp::ICMP6NeighborAdvertisement*)raw.data;
+
+ validate_ndp_option(codec, na, ICMPv6_NA_MIN_LEN, dsize);
+ }
+ else
+ {
+ codec_event(codec, DECODE_ICMP_DGRAM_LT_ICMPHDR);
+ return false;
+ }
+ break;
+
+ case icmp::Icmp6Types::REDIRECT6:
+ if (dsize >= (ICMPv6_RD_MIN_LEN - icmp::ICMP6_HEADER_MIN_LEN))
+ {
+ const icmp::ICMP6Redirect* rd = (const icmp::ICMP6Redirect*)raw.data;
+
+ validate_ndp_option(codec, rd, ICMPv6_RD_MIN_LEN, dsize);
}
else
{
case icmp::Icmp6Types::MULTICAST_LISTENER_QUERY:
case icmp::Icmp6Types::MULTICAST_LISTENER_REPORT:
case icmp::Icmp6Types::MULTICAST_LISTENER_DONE:
- case icmp::Icmp6Types::NEIGHBOR_SOLICITATION:
- case icmp::Icmp6Types::NEIGHBOR_ADVERTISEMENT:
- case icmp::Icmp6Types::REDIRECT6:
case icmp::Icmp6Types::INVERSE_NEIGHBOR_DISCOVERY_SOLICITATION:
case icmp::Icmp6Types::INVERSE_NEIGHBOR_DISCOVERY_ADVERTISEMENT:
case icmp::Icmp6Types::VERSION_2_MULTICAST_LISTENER_REPORT:
if (errno != EAGAIN && errno != EINTR)
{
shutdown();
+ ErrorMessage("ControlConn: Error in writing response, closing the connection: %s\n", get_error(errno));
return false;
}
}
}
touch();
-
return true;
}
get_pattern_info(pmd, hex, txt, opts);
debug_logf(detection_trace, TRACE_RULE_EVAL, p,
- "Fast pattern %s[%u] = '%s' |%s| %s\n",
- pmd->sticky_buf, pmd->pattern_size, txt.c_str(), hex.c_str(), opts.c_str());
+ "Fast pattern %s[%u] = %s %s %s\n",
+ pmd->sticky_buf, pmd->fp_length, txt.c_str(), hex.c_str(), opts.c_str());
}
void dump_buffer(const uint8_t* buff, unsigned len, Packet* p)
enum
{
- TRACE_DETECTION_ENGINE = 0,
+ TRACE_OPTION_TREE = 0,
+ TRACE_FP_INFO,
+ TRACE_DETECTION_ENGINE,
TRACE_RULE_EVAL,
TRACE_BUFFER,
TRACE_RULE_VARS,
TRACE_FP_SEARCH,
TRACE_PKT_DETECTION,
- TRACE_OPTION_TREE,
TRACE_TAG,
TRACE_CONT,
};
{
PacketLatency::Context pkt_latency_ctx { p };
+ InspectorManager::probe_first(p);
if ( p->ptrs.decode_flags & DECODE_ERR_FLAGS )
{
if ( p->context->conf->ips_inline_mode() and
THREAD_LOCAL const Trace* detection_trace = nullptr;
-#ifdef DEBUG_MSGS
static const TraceOption detection_trace_options[] =
{
+ { "opt_tree", TRACE_OPTION_TREE, "enable tree option trace logging" },
+ { "fp_info", TRACE_FP_INFO, "enable fast pattern info logging" },
+#ifdef DEBUG_MSGS
{ "detect_engine", TRACE_DETECTION_ENGINE, "enable detection engine trace logging" },
{ "rule_eval", TRACE_RULE_EVAL, "enable rule evaluation trace logging" },
{ "buffer", TRACE_BUFFER, "enable buffer trace logging" },
{ "rule_vars", TRACE_RULE_VARS, "enable rule variables trace logging" },
{ "fp_search", TRACE_FP_SEARCH, "enable fast pattern search trace logging" },
{ "pkt_detect", TRACE_PKT_DETECTION, "enable packet detection trace logging" },
- { "opt_tree", TRACE_OPTION_TREE, "enable tree option trace logging" },
{ "tag", TRACE_TAG, "enable tag trace logging" },
{ "cont", TRACE_CONT, "enable rule continuation trace logging" },
+#endif
{ nullptr, 0, nullptr }
};
-#endif
static const Parameter extend_to_service[] =
const TraceOption* DetectionModule::get_trace_options() const
{
-#ifndef DEBUG_MSGS
- return nullptr;
-#else
return detection_trace_options;
-#endif
}
bool DetectionModule::begin(const char* fqn, int idx, SnortConfig* sc)
int get_debug_print_rule_groups_uncompiled() const
{ return portlists_flags & PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED; }
- void set_debug_print_fast_patterns(bool b)
- { debug_print_fast_pattern = b; }
-
- bool get_debug_print_fast_patterns() const
- { return debug_print_fast_pattern; }
-
void set_split_any_any(bool enable)
{ split_any_any = enable; }
bool inspect_stream_insert = true;
bool split_any_any = false;
- bool debug_print_fast_pattern = false;
bool debug = false;
bool dedup = true;
fixup_tree(root->children[i], true, 0);
}
- debug_logf(detection_trace, TRACE_OPTION_TREE, nullptr, "%3d %3d %p %4s\n",
+ trace_logf(detection_trace, TRACE_OPTION_TREE, nullptr, "%3d %3d %p %4s\n",
0, root->num_children, (void*)root, "root");
print_option_tree(root->children[i], 0);
{
bool found_child_match = child->option_data == option_data;
- for (i = 1; !found_child_match && i < bud->num_children; i++)
+ for (i = 1; !found_child_match and i < bud->num_children; i++)
{
child = bud->children[i];
if (child->option_data == option_data)
{
pattern = pmd->pattern_buf;
pattern_length = pmd->pattern_size;
+
+ // alt buffer's pmd isn't guaranteed to be filled fully,
+ // so on first pass, setting fp_length to correct value
+ if (pmd->fp_length == 0)
+ {
+ pmd->fp_length = pmd->pattern_size;
+ pmd->fp_offset = 0;
+ }
}
if (pmd->pattern_size > otn->longestPatternLen)
{
for ( auto& it : pg->pm_list[sect] )
{
- if ( it->group.normal_mpse && !it->group.normal_is_dup)
+ if ( it->group.normal_mpse and !it->group.normal_is_dup)
{
queue_mpse(it->group.normal_mpse);
has_rules = true;
}
- if ( it->group.offload_mpse && !it->group.offload_is_dup)
+ if ( it->group.offload_mpse and !it->group.offload_is_dup)
{
queue_mpse(it->group.offload_mpse);
has_rules = true;
static bool srvc_supports_section(const char* srvc)
{
- return (srvc && ( !strcmp("http", srvc) || !strcmp("http2",srvc) || !strcmp("http3",srvc)));
+ return (srvc and ( !strcmp("http", srvc) or !strcmp("http2",srvc) or !strcmp("http3",srvc)));
}
static int fpAddRuleGroupRule(
{
PatternMatcher* pm = pg->get_pattern_matcher(pmt, s, (PduSection)sect);
MpseGroup* mpg = &pm->group;
- const bool update_mpse = srvc || is_first_sect;
+ const bool update_mpse = srvc or is_first_sect;
if ( !mpg->normal_mpse )
{
}
}
- if ( add_to_offload && !mpg->offload_mpse )
+ if ( add_to_offload and !mpg->offload_mpse )
{
// Keep the created mpse alongside the same pm type as the main pmd
if (!update_mpse)
}
}
- if ( mpg->normal_mpse && update_mpse )
+ if ( mpg->normal_mpse and update_mpse )
{
add_rule = true;
if ( main_pmd->is_negated() )
main_pmd->sticky_buf = pm->name;
- if ( fp->get_debug_print_fast_patterns() and !otn->soid )
+ if ( !otn->soid )
print_fp_info(s_group, otn, main_pmd, sect);
// Add Alternative patterns
fpFinishRuleGroupRule(mpg->normal_mpse, otn, alt_pmd, fp, false);
alt_pmd->sticky_buf = pm->name;
- if ( fp->get_debug_print_fast_patterns() and !otn->soid )
+ if ( !otn->soid )
print_fp_info(s_group, otn, alt_pmd, sect);
}
}
}
- if ( ol_pmd and mpg->offload_mpse && update_mpse )
+ if ( ol_pmd and mpg->offload_mpse and update_mpse )
{
add_rule = true;
if ( ol_pmd->is_negated() )
main_pmd->sticky_buf = pm->name;
- if ( fp->get_debug_print_fast_patterns() and !otn->soid )
+ if ( !otn->soid )
print_fp_info(s_group, otn, main_pmd, sect);
// Add Alternative patterns
fpFinishRuleGroupRule(mpg->offload_mpse, otn, alt_pmd, fp, false);
alt_pmd->sticky_buf = pm->name;
- if ( fp->get_debug_print_fast_patterns() and !otn->soid )
+ if ( !otn->soid )
print_fp_info(s_group, otn, alt_pmd, sect);
}
}
{
ret_pattern = pattern;
ret_bytes = bytes;
+ pmd->fp_length = pmd->pattern_size;
+
return 0;
}
- if ( pmd->is_fast_pattern() && (pmd->fp_offset || pmd->fp_length) )
+ if ( pmd->is_fast_pattern() and (pmd->fp_offset or pmd->fp_length) )
{
/* (offset + length) potentially being larger than the pattern itself
* is taken care of during parsing */
ret_pattern = pattern;
ret_bytes = fp->set_max(bytes);
- if ( ret_bytes < pmd->pattern_size )
+ if ( ret_bytes < pmd->pattern_size or !pmd->fp_length )
pmd->fp_length = ret_bytes;
return 0;
{
GHashNode* n;
- if ( !p || !p->get_count() )
+ if ( !p or !p->get_count() )
return;
std::string label = "service rule counts - ";
void get_pattern_info(const PatternMatchData* pmd, string& hex, string& txt, string& opts)
{
char buf[8];
-
- for ( unsigned i = 0; i < pmd->pattern_size; ++i )
+ hex = "| ";
+ txt = "'";
+ for ( unsigned i = pmd->fp_offset; i < pmd->fp_offset + pmd->fp_length; ++i )
{
snprintf(buf, sizeof(buf), "%2.02X ", (uint8_t)pmd->pattern_buf[i]);
hex += buf;
txt += isprint(pmd->pattern_buf[i]) ? pmd->pattern_buf[i] : '.';
}
+ hex += "|";
+ txt += "'";
+
opts = "(";
if ( pmd->is_fast_pattern() )
opts += " user";
opts += " )";
}
-static void print_fp_info(const char* group, const OptTreeNode* otn, const PatternMatchData* pmd, int sect)
+static void print_fp_info(
+ const char* group, const OptTreeNode* otn, const PatternMatchData* pmd, int sect)
{
std::string hex, txt, opts;
get_pattern_info(pmd, hex, txt, opts);
- LogMessage("FP %s %u:%u:%u %s[%d] = '%s' |%s| %s, section %s\n",
- group, otn->sigInfo.gid, otn->sigInfo.sid, otn->sigInfo.rev,
- pmd->sticky_buf, pmd->pattern_size, txt.c_str(), hex.c_str(), opts.c_str(), section_to_str[sect]);
-}
+ trace_logf(detection_trace, TRACE_FP_INFO, nullptr, "%s group, %u:%u:%u %s[%u] = %s %s %s, section %s\n",
+ group, otn->sigInfo.gid, otn->sigInfo.sid, otn->sigInfo.rev, pmd->sticky_buf, pmd->fp_length,
+ txt.c_str(), hex.c_str(), opts.c_str(), section_to_str[sect]);
+}
if ( cat > CAT_ADJUST )
{
- if ( cat == CAT_SET_FAST_PATTERN or cat == CAT_SET_RAW )
+ if ( cat >= CAT_SET_RAW )
curr_opt = ofl->ips_opt;
curr_cat = cat;
if ( !tmp )
continue;
+ if (curr_cat == CAT_SET_SUB_SECTION)
+ tmp->set_sub_section();
+
content = true;
FpSelector curr(curr_cat, ofl->ips_opt, tmp);
return false;
// FIXIT-L no_case consideration is mpse specific, delegate
- if ( !pmd->is_relative() and !pmd->is_negated() and
- !pmd->offset and !pmd->depth and pmd->is_no_case() )
- {
- ofp = ofp->next;
- if ( !ofp || !ofp->ips_opt || !ofp->ips_opt->is_relative() )
- return true;
- }
- return false;
+ if ( pmd->is_relative() or pmd->is_negated() or pmd->offset or pmd->depth or !pmd->is_no_case() or
+ pmd->is_sub_section())
+ return false;
+
+ ofp = ofp->next;
+
+ if ( ofp and ofp->ips_opt and ofp->ips_opt->is_relative() )
+ return false;
+
+ return true;
}
bool is_fast_pattern_only(const OptTreeNode* otn, const OptFpList* ofp, Mpse::MpseType mpse_type)
LITERAL = 0x08,
FAST_PAT = 0x10,
NO_FP = 0x20,
+ SUB_SECT = 0x40,
};
uint16_t flags = 0; // from above enum
uint16_t mpse_flags; // passed through to mpse
- uint16_t fp_offset;
- uint16_t fp_length;
+ uint16_t fp_offset = 0;
+ uint16_t fp_length = 0;
bool is_unbounded() const
{ return !depth; }
void set_literal()
{ flags |= LITERAL; }
+ void set_sub_section()
+ { flags |= SUB_SECT; }
+
bool is_fast_pattern() const
{ return (flags & FAST_PAT) != 0; }
bool is_literal() const
{ return (flags & LITERAL) != 0; }
+ bool is_sub_section() const
+ { return (flags & SUB_SECT) != 0; }
+
bool can_be_fp() const;
bool has_alpha() const;
This directory contains classes related to Snort configuration dump.
-Snort supports dumping config into a file for JSON and text formats.
+Snort can dump configuration (it read from all config sources) in JSON or text formats.
-* ConfigData
+==== ConfigData
- The ConfigData structure represents the internal config data of a particular Lua file.
- A unique shell is associated with every base and targeted policy file.
- Every module in a Lua file is represented by the General Tree.
- The Tree node can be one of the following types: TreeConfigNode, ValueConfigNode.
- The TreeConfigNode represents a table or list.
- The ValueConfigNode represents a config value itself.
+ConfigData structure represents the internal config data of a particular Lua file.
+An unique shell is associated with every base and targeted policy files.
+Every module in a Lua file is represented by the General Tree.
+Tree node can be one of the following types: TreeConfigNode, ValueConfigNode.
+TreeConfigNode represents a table or a list.
+ValueConfigNode represents a configured value itself.
-* ConfigOutput
+==== ConfigOutput
- The ConfigOutput class is a base class that encapsulates dumping config into a file.
- Pure virtual function dump() should be overridden by derived classes for particular
- output format.
+ConfigOutput class is a base class that encapsulates dumping config into a file.
+Pure virtual function dump() should be overridden by derived classes for particular
+output format.
-* JsonAllConfigOutput
+==== JsonAllConfigOutput
- The JsonAllConfigOutput class inherits from ConfigOutput and dumps base and targeted
- policy file in JSON format.
+The JsonAllConfigOutput class inherits from ConfigOutput and dumps base and targeted
+policy file in JSON format.
-* TextConfigOutput
+==== TextConfigOutput
- The TextConfigOutput class inherits from ConfigOutput and dumps base and targeted
- policy file in text format.
+The TextConfigOutput class inherits from ConfigOutput and dumps base and targeted
+policy file in text format.
#include "file_service.h"
#include "file_stats.h"
+#define DEFAULT_FILE_LOOKUP_TIMEOUT_CACHED_ITEM 3600 // 1 hour
+
using namespace snort;
class ExpectedFileCache : public XHash
}
FileContext* FileCache::get_file(Flow* flow, uint64_t file_id, bool to_create,
- int64_t timeout)
+ int64_t timeout, bool using_cache_entry)
{
FileHashKey hashKey;
hashKey.dip = flow->client_ip;
hashKey.file_id = file_id;
hashKey.asid = flow->key->addressSpaceId;
hashKey.padding[0] = hashKey.padding[1] = hashKey.padding[2] = 0;
- FileContext* file = find(hashKey, timeout);
+
+ FileContext* file = nullptr;
+ if (using_cache_entry)
+ file = find(hashKey, DEFAULT_FILE_LOOKUP_TIMEOUT_CACHED_ITEM);
+ else
+ file = find(hashKey, timeout);
+
if (to_create and !file)
file = add(hashKey, timeout);
return file;
}
-FileContext* FileCache::get_file(Flow* flow, uint64_t file_id, bool to_create)
+FileContext* FileCache::get_file(Flow* flow, uint64_t file_id, bool to_create, bool using_cache_entry)
{
- return get_file(flow, file_id, to_create, lookup_timeout);
+ return get_file(flow, file_id, to_create, lookup_timeout, using_cache_entry);
}
FileVerdict FileCache::check_verdict(Packet* p, FileInfo* file,
return 0;
}
- FileContext* file_got = get_file(flow, file_id, true, timeout);
+ FileContext* file_got = get_file(flow, file_id, true, timeout, false);
if (file_got)
{
*((FileInfo*)(file_got)) = *file;
return verdict;
}
- FileContext* file_found = get_file(flow, file_id, false);
+ FileContext* file_found = get_file(flow, file_id, false, false);
if (file_found)
{
void set_lookup_timeout(int64_t);
void set_max_files(int64_t);
- snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create);
+ snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create, bool using_cache_entry);
FileVerdict cached_verdict_lookup(snort::Packet*, snort::FileInfo*,
snort::FilePolicyBase*);
bool apply_verdict(snort::Packet*, snort::FileContext*, FileVerdict, bool resume,
snort::FileContext* find(const FileHashKey&, int64_t);
snort::FileContext* find_add(const FileHashKey&, int64_t);
snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create,
- int64_t timeout);
+ int64_t timeout, bool using_cache_entry);
FileVerdict check_verdict(snort::Packet*, snort::FileInfo*, snort::FilePolicyBase*);
int store_verdict(snort::Flow*, snort::FileInfo*, int64_t timeout);
if (!file->get_file_data())
{
if (file_cache)
- file_got = file_cache->get_file(flow, pending_file_id, false);
+ file_got = file_cache->get_file(flow, pending_file_id, false, false);
if (file_got and file_got->get_file_data() and file_got->verdict == FILE_VERDICT_PENDING)
{
file_got->user_file_data_mutex.lock();
current_context_delete_pending = false;
FileCache* file_cache = FileService::get_file_cache();
assert(file_cache);
- FileContext* file_got = file_cache->get_file(flow, file_id, false);
+ FileContext* file_got = file_cache->get_file(flow, file_id, false, false);
if (file_got and file_got->verdict == FILE_VERDICT_PENDING and current_context != file_got)
{
file_got->user_file_data_mutex.lock();
FileFlows::~FileFlows()
{
- FileCache* file_cache = FileService::get_file_cache();
- assert(file_cache);
- uint64_t file_id = 0;
- if (current_context)
- file_id = current_context->get_file_id();
- else if (main_context)
- file_id = main_context->get_file_id();
-
- FileContext* file_got = file_cache->get_file(flow, file_id, false);
-
- if (file_got and (file_got->verdict == FILE_VERDICT_PENDING))
- {
- file_got->user_file_data_mutex.lock();
- delete (file_got->get_file_data());
- file_got->set_file_data(nullptr);
- file_got->user_file_data_mutex.unlock();
- }
-
delete(main_context);
if (current_context_delete_pending)
delete(current_context);
{
FileCache* file_cache = FileService::get_file_cache();
assert(file_cache);
- context = file_cache->get_file(flow, file_id, false);
+ context = file_cache->get_file(flow, file_id, false, true);
FILE_DEBUG(file_trace, DEFAULT_TRACE_OPTION_ID, TRACE_DEBUG_LEVEL, GET_CURRENT_PACKET,
"get_file_context:trying to get context from cache\n");
}
*/
bool FileFlows::file_process(Packet* p, uint64_t file_id, const uint8_t* file_data,
int data_size, uint64_t offset, FileDirection dir, uint64_t multi_file_processing_id,
- FilePosition position)
+ FilePosition position, const uint8_t* fname, uint32_t name_size)
{
int64_t file_depth = FileService::get_max_file_depth();
bool continue_processing;
}
FileContext* context = get_file_context(file_id, true, is_new_context, multi_file_processing_id);
-
if (!context)
{
FILE_DEBUG(file_trace , DEFAULT_TRACE_OPTION_ID, TRACE_CRITICAL_LEVEL, p,
"file_process:context missing, returning \n");
return false;
}
+ context->set_file_name((const char*)fname, name_size, false);
if (PacketTracer::is_daq_activated())
PacketTracer::restart_timer();
* false: ignore this file
*/
bool FileFlows::file_process(Packet* p, const uint8_t* file_data, int data_size,
- FilePosition position, bool upload, size_t file_index)
+ FilePosition position, bool upload, size_t file_index, const uint8_t* fname, uint32_t name_size)
{
FileContext* context;
FileDirection direction = upload ? FILE_UPLOAD : FILE_DOWNLOAD;
context = find_main_file_context(position, direction, file_index);
set_current_file_context(context);
+ context->set_file_name((const char*)fname, name_size, false);
context->set_signature_state(gen_signature);
bool file_process_ret = context->process(p, file_data, data_size, position, file_policy);
// This is used when there is only one file per session
bool file_process(Packet* p, const uint8_t* file_data, int data_size, FilePosition,
- bool upload, size_t file_index = 0);
+ bool upload, size_t file_index = 0, const uint8_t* fname = nullptr, uint32_t name_size = 0);
// This is used for each file context. Support multiple files per session
bool file_process(Packet* p, uint64_t file_id, const uint8_t* file_data,
int data_size, uint64_t offset, FileDirection, uint64_t multi_file_processing_id=0,
- FilePosition=SNORT_FILE_POSITION_UNKNOWN);
+ FilePosition=SNORT_FILE_POSITION_UNKNOWN, const uint8_t* fname = nullptr, uint32_t name_size = 0);
static unsigned file_flow_data_id;
//--------------------------------------------------------------------------
-// Copyright (C) 2014-2023 Cisco and/or its affiliates. All rights reserved.
+// Copyright (C) 2014-2024 Cisco and/or its affiliates. All rights reserved.
// Copyright (C) 2012-2013 Sourcefire, Inc.
//
// This program is free software; you can redistribute it and/or modify it
delete config;
}
-bool FileInspect::configure(SnortConfig*)
+bool FileInspect::configure(SnortConfig* sc)
{
if (!config)
return true;
file_cache->set_max_files(config->max_files_cached);
}
+ FileService::set_max_file_depth(sc);
+
return true;
}
/*File properties*/
-void FileInfo::set_file_name(const char* name, uint32_t name_size)
+void FileInfo::set_file_name(const char* name, uint32_t name_size, bool fn_set)
{
if (name and name_size)
- {
file_name.assign(name, name_size);
- }
- file_name_set = true;
+ if (fn_set)
+ file_name_set = fn_set;
}
void FileInfo::set_url(const char* url_name, uint32_t url_size)
FileInfo& operator=(const FileInfo& other);
uint32_t get_file_type() const;
void set_file_type(uint64_t index);
- void set_file_name(const char* file_name, uint32_t name_size);
+ void set_file_name(const char* file_name, uint32_t name_size, bool fn_set = true);
void set_url(const char* url, uint32_t url_size);
std::string& get_file_name();
std::string& get_url();
/* Get maximal file depth based on configuration
* This function must be called after all file services are configured/enabled.
*/
-int64_t FileService::get_max_file_depth()
+int64_t FileService::get_max_file_depth(FileConfig *fc)
{
- FileConfig* file_config = get_file_config();
+ FileConfig* file_config = fc ? fc : get_file_config();
if (!file_config)
return -1;
- if (file_config->file_depth)
+ if (file_config->file_depth > 0)
return file_config->file_depth;
- file_config->file_depth = -1;
+ return -1;
+}
+
+void FileService::set_max_file_depth(const SnortConfig* sc)
+{
+ FileConfig* file_config = get_file_config(sc);
+
+ if (!file_config)
+ return;
if (file_type_id_enabled)
{
file_config->file_depth = file_config->file_type_depth;
}
- if (file_signature_enabled)
+ if ((file_signature_enabled) and
+ (file_config->file_signature_depth > file_config->file_depth))
{
- if (file_config->file_signature_depth > file_config->file_depth)
- file_config->file_depth = file_config->file_signature_depth;
+ file_config->file_depth = file_config->file_signature_depth;
}
if (file_config->file_depth > 0)
{
/*Extra byte for deciding whether file data will be over limit*/
file_config->file_depth++;
- return (file_config->file_depth);
}
- else
- {
- return -1;
- }
-}
-
-void FileService::reset_depths()
-{
- FileConfig* file_config = get_file_config();
-
- if (file_config)
- file_config->file_depth = 0;
- decode_conf.sync_all_depths();
+ return;
}
namespace snort
class FileEnforcer;
class FileCache;
+class FileConfig;
namespace snort
{
static bool is_file_signature_enabled() { return file_signature_enabled; }
static bool is_file_capture_enabled() { return file_capture_enabled; }
static bool is_file_service_enabled();
- static int64_t get_max_file_depth();
- static void reset_depths();
+ static int64_t get_max_file_depth(FileConfig* = nullptr);
+ static void set_max_file_depth(const SnortConfig*);
static FileCache* get_file_cache() { return file_cache; }
static DecodeConfig decode_conf;
Event Filter - After the rules engine generates whatever actions it needs
to, the Event Filter is then invoked to filter the logging of these events.
Once again, tracking by event/address tuples, block the logging of events
-if the configured counts per time is exceeded. This will tend to reduce
-the logging system load for rules that fire too often.
+if the configured counts per time is exceeded. This will tend to reduce
+the logging system load for rules that fire too often. Due to technical
+difficulties of a multi-threaded hash table, a thread local table is used.
+Thus, the modules work within a packet thread. A user might see events
+from different packet threads, even if they would be suppressed be it a
+single packet thread.
All of the filters in this area are a collection of similar services
brought together to share the same event tracking logic. sfthreshold.cc
deferred_trust = TRUST_DEFER_ON;
}
auto element = deferred_trust_modules.begin();
- for (; element != deferred_trust_modules.end() && *element != module_id;
- ++element);
+ for (; element != deferred_trust_modules.end() && *element != module_id; ++element);
if (element == deferred_trust_modules.end())
deferred_trust_modules.emplace_front(module_id);
}
if (active.session_was_blocked())
clear();
else if (TRUST_DEFER_DO_TRUST == deferred_trust && active.session_was_allowed())
+ {
active.set_trust();
+ clear();
+ }
else if ((TRUST_DEFER_ON == deferred_trust || TRUST_DEFER_DEFERRING == deferred_trust)
&& active.session_was_trusted())
{
// This code assumes that the expected session is in the opposite direction of the control session
// when groups are significant
- bool reversed_key = (ctrlPkt->pkth->flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS)
- ? key.init(ctrlPkt->context->conf, type, ip_proto, cliIP, cliPort,
- srvIP, srvPort, vlanId, mplsId, ctrlPkt->pkth->address_space_id, ctrlPkt->pkth->tenant_id, ctrlPkt->pkth->egress_group,
- ctrlPkt->pkth->ingress_group)
- : key.init(ctrlPkt->context->conf, type, ip_proto, cliIP, cliPort,
- srvIP, srvPort, vlanId, mplsId, *ctrlPkt->pkth);
-
+ bool reversed_key = key.init(ctrlPkt->context->conf, type, ip_proto, cliIP, cliPort,
+ srvIP, srvPort, vlanId, mplsId, ctrlPkt->pkth->address_space_id,
+#ifndef DISABLE_TENANT_ID
+ ctrlPkt->pkth->tenant_id,
+#endif
+ 0 != (ctrlPkt->pkth->flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS),
+ ctrlPkt->pkth->egress_group, ctrlPkt->pkth->ingress_group);
bool new_node = false;
ExpectNode* node = static_cast<ExpectNode*> ( hash_table->get_user_data(&key) );
if ( !node )
void Flow::free_flow_data(FlowData* fd)
{
- if ( !fd ) return;
- flow_data.erase(fd->get_id());
+ if ( fd )
+ flow_data.erase(fd->get_id());
}
void Flow::free_flow_data(uint32_t proto)
void Flow::free_flow_data()
{
if ( flow_data.empty() )
+ {
+ if (stash)
+ stash->reset();
return;
+ }
const SnortConfig* sc = SnortConfig::get_conf();
PolicySelector* ps = sc->policy_map->get_policy_selector();
NetworkPolicy* np = nullptr;
{
_daq_pkt_hdr pkthdr = {};
pkthdr.address_space_id = key->addressSpaceId;
- pkthdr.tenant_id = tenant;
+#ifndef DISABLE_TENANT_ID
+ pkthdr.tenant_id = key->tenant_id;
+#else
+ pkthdr.tenant_id = 0;
+#endif
select_default_policy(pkthdr, sc);
}
}
flow_data.clear();
+ if (stash)
+ stash->reset();
if (ps)
{
{
for (auto& fd_pair : flow_data)
{
- FlowData* fd = fd_pair.second.get();
+ FlowData* fd = fd_pair.second.get();
if ( eof )
fd->handle_eof(p);
else
uint64_t server_bytes;
struct timeval start_time;
uint64_t total_flow_latency;
+ uint64_t total_rule_latency;
};
struct LwState
unsigned inspection_policy_id = 0;
unsigned ips_policy_id = 0;
unsigned reload_id = 0;
- uint32_t tenant = 0;
uint32_t default_session_timeout = 0;
uint32_t idle_timeout = 0;
int32_t client_intf = 0;
bool ips_block_event_suppressed : 1; // Set if event filters have suppressed a block ips event
bool ips_wblock_event_suppressed : 1; // set if event filters have suppressed a would block/drop ips event
bool ips_pblock_event_suppressed : 1; // set if event filters have suppressed a partial block ips event
+ bool binder_action_allow : 1;
+ bool binder_action_block : 1;
} flags = {};
FlowState flow_state = FlowState::SETUP;
#include "packet_io/active.h"
#include "packet_io/packet_tracer.h"
#include "stream/base/stream_module.h"
-#include "stream/tcp/tcp_stream_session.h"
+#include "stream/tcp/tcp_session.h"
#include "stream/tcp/tcp_trace.h"
#include "time/packet_time.h"
#include "trace/trace_api.h"
constexpr uint8_t MAX_PROTOCOLS = (uint8_t)to_utype(PktType::MAX) - 1; //removing PktType::NONE from count
constexpr uint64_t max_skip_protos = (1ULL << MAX_PROTOCOLS) - 1;
+constexpr uint8_t first_proto = to_utype(PktType::NONE) + 1;
uint8_t DumpFlows::dump_code = 0;
uni_flows = new FlowUniList;
uni_ip_flows = new FlowUniList;
flags = 0x0;
+ empty_proto = ( 1 << MAX_PROTOCOLS ) - 1;
+ timeout_idx = first_proto;
assert(prune_stats.get_total() == 0);
}
link_uni(flow);
flow->last_data_seen = timestamp;
flow->set_idle_timeout(config.proto[to_utype(flow->key->pkt_type)].nominal_timeout);
+ empty_proto &= ~(1ULL << to_utype(key->pkt_type)); // clear the bit for this protocol
return flow;
}
ActiveSuspendContext act_susp(Active::ASP_PRUNE);
unsigned pruned = 0;
- uint64_t skip_protos = 0;
+ uint64_t skip_protos = empty_proto;
assert(MAX_PROTOCOLS < 8 * sizeof(skip_protos));
skip_protos != max_skip_protos )
{
// Round-robin through the proto types
- for( uint8_t proto_idx = 0; proto_idx < MAX_PROTOCOLS; ++proto_idx )
+ for( uint8_t proto_idx = first_proto; proto_idx < MAX_PROTOCOLS; ++proto_idx )
{
if( pruned > cleanup_flows )
break;
if ( !flow )
{
skip_protos |= proto_mask;
+ empty_proto |= proto_mask;
continue;
}
assert(MAX_PROTOCOLS < 8 * sizeof(skip_protos));
-
{
PacketTracerSuspend pt_susp;
unsigned blocks = 0;
ignore_offloads == 0 or skip_protos == max_skip_protos )
break;
- for( uint8_t proto_idx = 0; proto_idx < MAX_PROTOCOLS; ++proto_idx )
+ for( uint8_t proto_idx = first_proto; proto_idx < MAX_PROTOCOLS; ++proto_idx )
{
num_nodes = hash_table->get_num_nodes();
if ( num_nodes <= max_cap or num_nodes <= blocks )
ActiveSuspendContext act_susp(Active::ASP_TIMEOUT);
unsigned retired = 0;
- uint64_t skip_protos = 0;
+ uint64_t skip_protos = empty_proto;
assert(MAX_PROTOCOLS < 8 * sizeof(skip_protos));
while ( retired < num_flows and skip_protos != max_skip_protos )
{
- for( uint8_t proto_idx = 0; proto_idx < MAX_PROTOCOLS; ++proto_idx )
+ for( ; timeout_idx < MAX_PROTOCOLS; ++timeout_idx )
{
- if( retired >= num_flows )
- break;
-
- const uint64_t proto_mask = 1ULL << proto_idx;
+
+ const uint64_t proto_mask = 1ULL << timeout_idx;
if ( skip_protos & proto_mask )
continue;
- auto flow = static_cast<Flow*>(hash_table->lru_current(proto_idx));
- if ( !flow )
- flow = static_cast<Flow*>(hash_table->lru_first(proto_idx));
+ auto flow = static_cast<Flow*>(hash_table->lru_current(timeout_idx));
if ( !flow )
{
- skip_protos |= proto_mask;
- continue;
+ flow = static_cast<Flow*>(hash_table->lru_first(timeout_idx));
+ if ( !flow )
+ {
+ skip_protos |= proto_mask;
+ empty_proto |= proto_mask;
+ continue;
+ }
}
if ( flow->is_hard_expiration() )
flow->ssn_state.session_flags |= SSNFLAG_TIMEDOUT;
if ( release(flow, PruneReason::IDLE_PROTOCOL_TIMEOUT) )
- ++retired;
+ {
+ if( ++retired >= num_flows )
+ break;
+ }
}
+
+ timeout_idx = first_proto;
}
}
unsigned FlowCache::delete_active_flows(unsigned mode, unsigned num_to_delete, unsigned &deleted)
{
- uint64_t skip_protos = 0;
+ uint64_t skip_protos = empty_proto;
uint64_t undeletable = 0;
assert(MAX_PROTOCOLS < 8 * sizeof(skip_protos));
while ( num_to_delete and skip_protos != max_skip_protos and
undeletable < hash_table->get_num_nodes() )
{
- for( uint8_t proto_idx = 0; proto_idx < MAX_PROTOCOLS; ++proto_idx )
+ for( uint8_t proto_idx = first_proto; proto_idx < MAX_PROTOCOLS; ++proto_idx )
{
if( num_to_delete == 0)
break;
if ( !flow )
{
skip_protos |= proto_mask;
+ empty_proto |= proto_mask;
continue;
}
unsigned retired = 0;
- for( uint8_t proto_idx = 0; proto_idx < MAX_PROTOCOLS; ++proto_idx )
+ for( uint8_t proto_idx = first_proto; proto_idx < MAX_PROTOCOLS; ++proto_idx )
{
while ( auto flow = static_cast<Flow*>(hash_table->lru_first(proto_idx)) )
{
<< dst_ip << "/" << dst_port;
if (flow.session)
{
- TcpStreamSession* tcp_session = static_cast<TcpStreamSession*>(flow.session);
+ TcpSession* tcp_session = static_cast<TcpSession*>(flow.session);
proto << " state client " << stream_tcp_state_to_str(tcp_session->client)
<< " server " << stream_tcp_state_to_str(tcp_session->server);
}
void output_flow(std::fstream&, const snort::Flow&, const struct timeval&) const;
private:
+ uint8_t timeout_idx;
static const unsigned cleanup_flows = 1;
FlowCacheConfig config;
uint32_t flags;
PruneStats prune_stats;
FlowDeleteStats delete_stats;
+ uint64_t empty_proto;
};
#endif
flow->server_group = p->pkth->egress_group;
}
- flow->tenant = p->pkth->tenant_id;
-
flow->flags.app_direction_swapped = false;
if ( flow->ssn_state.direction == FROM_CLIENT )
p->packet_flags |= PKT_FROM_CLIENT;
if ( ignore )
{
- flow->ssn_state.ignore_direction = ignore;
+ flow->ssn_state.ignore_direction = SSN_DIR_BOTH;
DetectionEngine::disable_all(p);
}
}
PktType type, IpProtocol ip_proto,
const SfIp *srcIP, uint16_t srcPort,
const SfIp *dstIP, uint16_t dstPort,
- uint16_t vlanId, uint32_t mplsId,
- uint32_t addrSpaceId, uint32_t tid,
+ uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId,
+#ifndef DISABLE_TENANT_ID
+ uint32_t tid,
+#endif
+ bool significant_groups,
int16_t ingress_group, int16_t egress_group)
{
bool reversed;
pkt_type = type;
ip_protocol = (uint8_t)ip_proto;
+#ifndef DISABLE_TENANT_ID
tenant_id = tid;
+#endif
init_vlan(sc, vlanId);
init_address_space(sc, addrSpaceId);
padding = flags.padding_bits = 0;
- flags.group_used = (ingress_group != DAQ_PKTHDR_UNKNOWN and egress_group != DAQ_PKTHDR_UNKNOWN);
+ flags.group_used = significant_groups;
init_groups(ingress_group, egress_group, reversed);
return reversed;
pkt_type = type;
ip_protocol = (uint8_t)ip_proto;
+#ifndef DISABLE_TENANT_ID
tenant_id = pkt_hdr.tenant_id;
+#endif
init_vlan(sc, vlanId);
init_address_space(sc, pkt_hdr.address_space_id);
init_mpls(sc, mplsId);
padding = flags.padding_bits = 0;
- flags.group_used = ((pkt_hdr.flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS) != 0);
+ flags.group_used = 0 != (pkt_hdr.flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS);
init_groups(pkt_hdr.ingress_group, pkt_hdr.egress_group, reversed);
return reversed;
}
-bool FlowKey::init(
- const SnortConfig* sc,
- PktType type, IpProtocol ip_proto,
- const SfIp *srcIP, const SfIp *dstIP,
- uint32_t id, uint16_t vlanId,
- uint32_t mplsId, uint32_t addrSpaceId,
- uint32_t tid, int16_t ingress_group,
- int16_t egress_group)
-{
- // to avoid confusing 2 different datagrams or confusing a datagram
- // with a session, we don't order the addresses and we set version
-
- uint16_t srcPort = id & 0xFFFF;
- uint16_t dstPort = id >> 16;
- bool reversed;
-
- if (srcIP->is_ip4() && dstIP->is_ip4())
- {
- version = 4;
- reversed = init4(ip_proto, srcIP, srcPort, dstIP, dstPort, false);
- ip_protocol = (uint8_t)ip_proto;
- }
- else
- {
- version = 6;
- reversed = init6(ip_proto, srcIP, srcPort, dstIP, dstPort, false);
- ip_protocol = 0;
- }
-
- pkt_type = type;
- tenant_id = tid;
-
- init_vlan(sc, vlanId);
- init_address_space(sc, addrSpaceId);
- init_mpls(sc, mplsId);
-
- padding = flags.padding_bits = 0;
-
- flags.group_used = (ingress_group != DAQ_PKTHDR_UNKNOWN and egress_group != DAQ_PKTHDR_UNKNOWN);
- init_groups(ingress_group, egress_group, reversed);
-
- return reversed;
-}
-
bool FlowKey::init(
const SnortConfig* sc,
PktType type, IpProtocol ip_proto,
}
pkt_type = type;
+#ifndef DISABLE_TENANT_ID
tenant_id = pkt_hdr.tenant_id;
+#endif
init_vlan(sc, vlanId);
init_address_space(sc, pkt_hdr.address_space_id);
padding = flags.padding_bits = 0;
- flags.group_used = ((pkt_hdr.flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS) != 0);
+ flags.group_used = 0 != (pkt_hdr.flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS);
init_groups(pkt_hdr.ingress_group, pkt_hdr.egress_group, reversed);
return reversed;
mix(a, b, c);
a += d[9]; // addressSpaceId
+
+#ifndef DISABLE_TENANT_ID
b += d[10]; // tenant_id
c += d[11]; // port lo & port hi
a += d[12]; // group lo & group hi
b += d[13]; // vlan & padding
c += d[14]; // ip_protocol & pkt_type, version, flags
+#else
+ b += d[10]; // port lo & port hi
+ c += d[11]; // group lo & group hi
+
+ mix(a, b, c);
+
+ b += d[12]; // vlan & padding
+ c += d[13]; // ip_protocol & pkt_type, version, flags
+#endif
finalize(a, b, c);
uint32_t ip_h[4]; /* High IP */
uint32_t mplsLabel;
uint32_t addressSpaceId;
- uint32_t tenant_id;
+#ifndef DISABLE_TENANT_ID
+ uint32_t tenant_id; // included by default
+#endif
uint16_t port_l; /* Low Port - 0 if ICMP */
uint16_t port_h; /* High Port - 0 if ICMP */
int16_t group_l;
uint8_t padding_bits : 7;
} flags;
- /* The init() functions return true if the key IP/port fields were actively
- normalized, reversing the source and destination addresses internally.
- The IP-only init() will always return false as we will not reorder its
- addresses at this time. */
+ // The init() functions return true if the key IP/port fields were actively
+ // normalized, reversing the source and destination addresses internally.
+ // The IP-only init() will always return false as we will not reorder its
+ // addresses at this time.
bool init(
const SnortConfig*, PktType, IpProtocol,
const snort::SfIp *srcIP, uint16_t srcPort,
const snort::SfIp *dstIP, uint16_t dstPort,
- uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId, uint32_t tid,
- int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);
-
- bool init(
- const SnortConfig*, PktType, IpProtocol,
- const snort::SfIp *srcIP, const snort::SfIp *dstIP,
- uint32_t id, uint16_t vlanId,
- uint32_t mplsId, uint32_t addrSpaceId, uint32_t tid,
+ uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId,
+#ifndef DISABLE_TENANT_ID
+ uint32_t tid,
+#endif
+ bool significant_groups,
int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);
bool init(
const snort::SfIp *dstIP, uint16_t dstPort,
uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);
+ // IP fragment key
bool init(
const SnortConfig*, PktType, IpProtocol,
const snort::SfIp *srcIP, const snort::SfIp *dstIP,
bool use_daq_channel;
};
+
+// Ensure to increment both versions simultaneously to maintain consistency
+#ifndef DISABLE_TENANT_ID
static constexpr uint8_t HA_MESSAGE_VERSION = 4;
+#else
+static constexpr uint8_t HA_MESSAGE_VERSION = 5;
+#endif
// define message size and content constants.
static constexpr uint8_t KEY_SIZE_IP6 = sizeof(FlowKey);
if( p && no_flow_found && flow && flow->session )
{
+ p->flow = flow;
flow->session->setup(p);
flow->set_direction(p);
flow->set_client_initiate(p);
deferred_trust.set_deferred_trust(1, false);
CHECK_TEXT(!deferred_trust.is_active(), "Deferred trust should not be active");
// State should be do trust
- deferred_trust.finalize(active);
- CHECK_TEXT(active.session_was_trusted(), "Session was not trusted from do trust");
-
// Enable with state do trust goes to deferring
- deferred_trust.set_deferred_trust(1, true);
+ deferred_trust.set_deferred_trust(2, true);
CHECK_TEXT(deferred_trust.is_active(), "Deferred trust should be active");
CHECK_TEXT(deferred_trust.is_deferred(), "Deferred trust should be deferring");
+ deferred_trust.set_deferred_trust(2, false);
+ CHECK_TEXT(!deferred_trust.is_active(), "Deferred trust should not be active");
+ // State should be do trust
+ deferred_trust.finalize(active);
+ CHECK_TEXT(active.session_was_trusted(), "Session was not trusted from do trust");
- deferred_trust.clear();
// Enable
deferred_trust.set_deferred_trust(1, true);
CHECK_TEXT(deferred_trust.is_active(), "Deferred trust should be active");
const SfIp*, uint16_t,
const SfIp*, uint16_t,
uint16_t, uint32_t,
- uint32_t, uint32_t, int16_t, int16_t)
+ uint32_t,
+#ifndef DISABLE_TENANT_ID
+ uint32_t,
+#endif
+ bool, int16_t, int16_t)
{
return true;
}
return true;
}
-bool FlowKey::init(
- const SnortConfig*,
- PktType, IpProtocol,
- const SfIp*, const SfIp*,
- uint32_t, uint16_t,
- uint32_t, uint32_t, uint32_t, int16_t,
- int16_t)
-{
- return true;
-}
-
bool FlowKey::init(
const SnortConfig*,
PktType, IpProtocol,
/* .ip_h = */ { 5, 6, 7, 8 },
/* .mplsLabel = */ 9,
/* .addressSpaceId = */ 0,
+#ifndef DISABLE_TENANT_ID
/* .tenant_id = */ 0,
+#endif
/* .port_l = */ 10,
/* .port_h = */ 11,
/* .group_l = */ 0,
{
HA_DELETE_EVENT,
HA_MESSAGE_VERSION,
+#ifndef DISABLE_TENANT_ID
65,
+#else
+ 61,
+#endif
KEY_TYPE_IP6
},
s_test_key
{
HA_UPDATE_EVENT,
HA_MESSAGE_VERSION,
+#ifndef DISABLE_TENANT_ID
77,
+#else
+ 73,
+#endif
KEY_TYPE_IP6
},
s_test_key,
"control",
"probe",
"file",
+ "probe_first",
};
const char* InspectApi::get_type(InspectorType type)
IT_CONTROL, // process all packets before detection (eg appid)
IT_PROBE, // process all packets after detection (eg perf_monitor, port_scan)
IT_FILE, // file identification inspector
+ IT_PROBE_FIRST, // process all packets before detection (eg packet_capture)
IT_MAX
};
class Module;
// this is the current version of the api
-#define IPSAPI_VERSION ((BASE_API_VERSION << 16) | 1)
+#define IPSAPI_VERSION ((BASE_API_VERSION << 16) | 2)
enum CursorActionType
{
CAT_SET_OTHER,
CAT_SET_RAW,
CAT_SET_FAST_PATTERN,
+ CAT_SET_SUB_SECTION,
};
enum RuleDirection
{ CountType::SUM, "cache_hits", module " cache found existing entry" }, \
{ CountType::SUM, "cache_misses", module " cache did not find entry" }, \
{ CountType::SUM, "cache_replaces", module " cache found entry and replaced its value" }, \
- { CountType::SUM, "cache_max", module " cache's maximum byte usage"}, \
+ { CountType::MAX, "cache_max", module " cache's maximum byte usage"}, \
{ CountType::SUM, "cache_prunes", module " cache pruned entry to make space for new entry" }, \
{ CountType::SUM, "cache_removes", module " cache removed existing entry"}
ips_js_data.cc
ips_metadata.cc
ips_msg.cc
- ips_pcre.cc
ips_pkt_data.cc
ips_priority.cc
ips_raw_data.cc
ips_luajit.cc
ips_options.cc
ips_options.h
+ ips_pcre.cc
ips_replace.cc
ips_so.cc
ips_vba_data.cc
add_dynamic_module(ips_js_data ips_options ips_js_data.cc)
add_dynamic_module(ips_metadata ips_options ips_metadata.cc)
add_dynamic_module(ips_msg ips_options ips_msg.cc)
- add_dynamic_module(ips_pcre ips_options ips_pcre.cc)
add_dynamic_module(ips_pkt_data ips_options ips_pkt_data.cc)
add_dynamic_module(ips_priority ips_options ips_priority.cc)
add_dynamic_module(ips_raw_data ips_options ips_raw_data.cc)
if ( cd->pmd.is_negated() )
{
+ if (cd->pmd.fp_length || cd->pmd.fp_offset)
+ {
+ ParseWarning(WARN_RULES,
+ "Fast pattern constraints for negated "
+ "content will be ignored");
+
+ cd->pmd.fp_length = cd->pmd.pattern_size;
+ cd->pmd.fp_offset = 0;
+ }
+
cd->pmd.last_check = (PmdLastCheck*)snort_calloc(
ThreadConfig::get_instance_max(), sizeof(*cd->pmd.last_check));
}
// these have various dependencies:
extern const BaseApi* ips_detection_filter[]; // perf stats
extern const BaseApi* ips_flowbits[]; // public methods like flowbits_setter
+extern const BaseApi* ips_pcre[]; // FIXIT-L called directly from parser
extern const BaseApi* ips_replace[]; // needs snort::SFDAQ::can_replace
extern const BaseApi* ips_so[]; // needs SO manager
extern const BaseApi* ips_vba_data[]; // FIXIT-L some trace dependency
extern const BaseApi* ips_js_data[];
extern const BaseApi* ips_metadata[];
extern const BaseApi* ips_msg[];
-extern const BaseApi* ips_pcre[];
extern const BaseApi* ips_pkt_data[];
extern const BaseApi* ips_priority[];
extern const BaseApi* ips_raw_data[];
{
PluginManager::load_plugins(ips_detection_filter);
PluginManager::load_plugins(ips_flowbits);
+ PluginManager::load_plugins(ips_pcre);
PluginManager::load_plugins(ips_replace);
PluginManager::load_plugins(ips_so);
PluginManager::load_plugins(ips_vba_data);
PluginManager::load_plugins(ips_js_data);
PluginManager::load_plugins(ips_metadata);
PluginManager::load_plugins(ips_msg);
- PluginManager::load_plugins(ips_pcre);
PluginManager::load_plugins(ips_pkt_data);
PluginManager::load_plugins(ips_priority);
PluginManager::load_plugins(ips_raw_data);
#include "framework/pig_pen.h"
#include "hash/hash_key_operations.h"
#include "helpers/scratch_allocator.h"
+#include "log/log_stats.h"
#include "log/messages.h"
#include "main/snort_config.h"
#include "managers/ips_manager.h"
#define s_name "pcre"
#define mod_regex_name "regex"
+void show_pcre_counts();
+
struct PcreData
{
pcre* re; /* compiled regex */
static THREAD_LOCAL ProfileStats pcrePerfStats;
+struct PcreCounts
+{
+ unsigned pcre_rules;
+#ifdef HAVE_HYPERSCAN
+ unsigned pcre_to_hyper;
+#endif
+ unsigned pcre_native;
+};
+
+PcreCounts pcre_counts;
+
+void show_pcre_counts()
+{
+ if (pcre_counts.pcre_rules == 0)
+ return;
+
+ LogLabel("pcre counts");
+ LogCount("pcre_rules", pcre_counts.pcre_rules);
+#ifdef HAVE_HYPERSCAN
+ LogCount("pcre_to_hyper", pcre_counts.pcre_to_hyper);
+#endif
+ LogCount("pcre_native", pcre_counts.pcre_native);
+}
+
//-------------------------------------------------------------------------
// stats foo
//-------------------------------------------------------------------------
struct PcreStats
{
- PegCount pcre_rules;
-#ifdef HAVE_HYPERSCAN
- PegCount pcre_to_hyper;
-#endif
- PegCount pcre_native;
- PegCount pcre_negated;
PegCount pcre_match_limit;
PegCount pcre_recursion_limit;
PegCount pcre_error;
const PegInfo pcre_pegs[] =
{
- { CountType::SUM, "pcre_rules", "total rules processed with pcre option" },
-#ifdef HAVE_HYPERSCAN
- { CountType::SUM, "pcre_to_hyper", "total pcre rules by hyperscan engine" },
-#endif
- { CountType::SUM, "pcre_native", "total pcre rules compiled by pcre engine" },
- { CountType::SUM, "pcre_negated", "total pcre rules using negation syntax" },
{ CountType::SUM, "pcre_match_limit", "total number of times pcre hit the match limit" },
{ CountType::SUM, "pcre_recursion_limit", "total number of times pcre hit the recursion limit" },
{ CountType::SUM, "pcre_error", "total number of times pcre returns error" },
{ CountType::END, nullptr, nullptr }
};
-PcreStats pcre_stats;
+THREAD_LOCAL PcreStats pcre_stats;
//-------------------------------------------------------------------------
// implementation foo
PcreData* get_data();
- bool global_stats() const override
- { return true; }
-
Usage get_usage() const override
{ return DETECT; }
static IpsOption* pcre_ctor(Module* p, IpsInfo& info)
{
- pcre_stats.pcre_rules++;
+ pcre_counts.pcre_rules++;
PcreModule* m = (PcreModule*)p;
#ifdef HAVE_HYPERSCAN
Module* mod_regex = m->get_mod_regex();
if ( mod_regex )
{
- pcre_stats.pcre_to_hyper++;
+ pcre_counts.pcre_to_hyper++;
const IpsApi* opt_api = IpsManager::get_option_api(mod_regex_name);
return opt_api->ctor(mod_regex, info);
}
UNUSED(info);
#endif
{
- pcre_stats.pcre_native++;
+ pcre_counts.pcre_native++;
PcreData* d = m->get_data();
return new PcreOption(d);
}
JSNormModule::increment_peg_counts(PEG_BYTES, next - src_ptr);
src_ptr = next;
- alive = post_proc(ret);
+ alive = alive and post_proc(ret);
}
if (jsn_ctx != nullptr)
#include "main/snort_types.h"
+// This follows the prefix from js_tokenizer.l
+#undef yyFlexLexer
+#define yyFlexLexer jsFlexLexer
#include <FlexLexer.h>
#include "helpers/streambuf.h"
{
trace_logf(2, js_trace, TRACE_PROC, DetectionEngine::get_current_packet(),
"pdf processing failed: %d\n", (int)r);
- return false;
+ alive = false;
}
src_ptr = (const uint8_t*)buf_pdf_out.data();
#ifndef JS_PDF_NORM_H
#define JS_PDF_NORM_H
-#include <FlexLexer.h>
#include <cstring>
+// This follows the prefix from pdf_tokenizer.l
+#undef yyFlexLexer
+#define yyFlexLexer pdfFlexLexer
+#include <FlexLexer.h>
+
#include "helpers/streambuf.h"
#include "js_norm/js_norm.h"
#include "js_norm/pdf_tokenizer.h"
INCOMPLETE_ARRAY_IN_DICTIONARY,
STREAM_NO_LENGTH,
UNEXPECTED_SYMBOL,
+ TOKEN_TOO_LONG,
MAX
};
PDFRet h_lit_u16_unescape();
PDFRet h_stream_open();
PDFRet h_stream();
+ PDFRet h_array_nesting();
bool h_stream_close();
void h_stream_length();
void h_ref();
char key[PDFTOKENIZER_NAME_MAX_SIZE] = {0};
};
+ struct IndirectObject
+ {
+ void clear()
+ { ref_met = false; }
+
+ bool ref_met = false;
+ };
+
struct Stream
{
int rem_length = -1;
bool is_js = false;
+ bool is_ref_len = false;
};
ObjectString obj_string;
ObjectDictionary obj_dictionary;
DictionaryEntry obj_entry;
Stream obj_stream;
+ IndirectObject indirect_obj;
std::unordered_set<unsigned int> js_stream_refs;
// represents UTF-16BE code point
void PDFTokenizer::h_ind_obj_close()
{
+ indirect_obj.clear();
obj_stream.is_js = false;
+ obj_stream.is_ref_len = false;
}
}
#include "js_norm/pdf_tokenizer.h"
#include "log/messages.h"
#include "trace/trace_api.h"
+#include "utils/util_cstring.h"
extern THREAD_LOCAL const snort::Trace* js_trace;
GRP_REGULAR [^\x00\x09\x0a\x0c\x0d\x20\(\)\<\>\[\]\{\}\/\%]
/* 7.2.3 Comments */
-COMMENT %{GRP_NOT_NEWLINE}*{EOL_MARKER}
+COMMENT_START %
+COMMENT_CONTENT {GRP_NOT_NEWLINE}{1,16}
+COMMENT_END {EOL_MARKER}
/* 7.3.2 Boolean Objects */
OBJ_BOOLEAN true|false
INDIRECT_OBJ_CLOSE endobj
-OBJ_REFERENCE {OBJ_INT_NUM}{GRP_WHITESPACE}+{OBJ_INT_NUM}{GRP_WHITESPACE}+R
+OBJ_REFERENCE {OBJ_INT_NUM}{GRP_WHITESPACE}+{OBJ_INT_NUM}{GRP_WHITESPACE}+R
/* Not object start, not comments */
-SKIP [^[:digit:]%]{1,16}|.
-WHITESPACE {GRP_WHITESPACE}{1,16}
+SKIP [^[:digit:]%]{1,16}|.
+WHITESPACE {GRP_WHITESPACE}{1,16}
-/* Start conditions: INITIAL or inside dictionary, literal string, hexadecimal string, stream */
+/* Start conditions: structures: comment, indirect object, dictionary or array */
+%x comment
%x indobj
-%x stream
%x dictnr
+
+/* Start conditions: literals: regular, hexadecimal, stream */
%x litstr
%x hexstr
+%x stream
%x jslstr
%x jshstr
%x jsstream
-/* Start conditions: UTF-16BE BOM, UTF-16BE literal string, UTF-16BE hexadecimal string, UTF-16BE stream */
+/* Start conditions: UTF-16BE: BOM, hex BOM, regular, hexadecimal, stream */
%x u16
%x u16hex
%x jsstru16
%%
-{SKIP} { }
-{COMMENT} { }
+
+<INITIAL,indobj,dictnr>{COMMENT_START} { PUSH(comment); }
+<comment>{COMMENT_CONTENT} { }
+<comment>{COMMENT_END} { POP(); }
<INITIAL>{INDIRECT_OBJ_OPEN} { PUSH(indobj); h_ind_obj_open(); }
-<indobj>{COMMENT} { }
<indobj>{WHITESPACE} { }
-<indobj>{INDIRECT_OBJ_CLOSE} { POP(); h_ind_obj_close(); }
+<indobj>{INDIRECT_OBJ_CLOSE} { POP(); h_ind_obj_close(); EXEC(h_array_nesting()) }
+<indobj>{OBJ_ARRAY_OPEN} { ++obj_array.nesting_level; }
+<indobj>{OBJ_ARRAY_CLOSE} { --obj_array.nesting_level; }
+<indobj>{OBJ_REFERENCE} { indirect_obj.ref_met = true; }
+<indobj>{OBJ_BOOLEAN} { }
+<indobj>{OBJ_INT_NUM} { }
+<indobj>{OBJ_REL_NUM} { }
+<indobj>{OBJ_NULL} { }
+<indobj>{OBJ_NAME} { }
<indobj>{OBJ_STREAM_OPEN} { EXEC(h_stream_open()) PUSH(obj_stream.is_js ? u16 : stream); }
<stream>{OBJ_STREAM_SKIP} { EXEC(h_stream()) }
<dictnr>{OBJ_DICT_OPEN} { PUSH(dictnr); EXEC(h_dict_open()) }
<indobj>{OBJ_DICT_OPEN} { PUSH(dictnr); EXEC(h_dict_open()) }
<dictnr>{OBJ_DICT_CLOSE} { POP(); EXEC(h_dict_close()) }
-<dictnr>{COMMENT} { }
<dictnr>{WHITESPACE} { }
<dictnr>{OBJ_REFERENCE} { EXEC(h_dict_other()) h_ref(); }
<dictnr>{OBJ_BOOLEAN} { EXEC(h_dict_other()) }
<dictnr>{OBJ_LIT_STR_OPEN} { EXEC(h_dict_other()) if (h_lit_str()) PUSH(jslstr); else PUSH(litstr); yyless(0); }
<dictnr>{OBJ_HEX_STR_OPEN} { EXEC(h_dict_other()) if (h_hex_str()) PUSH(jshstr); else PUSH(hexstr); yyless(0); }
<dictnr>{OBJ_DICT_SKIP} { }
+<dictnr>{INDIRECT_OBJ_CLOSE} { return PDFRet::UNEXPECTED_SYMBOL; }
<indobj>{OBJ_LIT_STR_OPEN} { if (h_lit_open()) PUSH(litstr); }
<litstr>{OBJ_LIT_STR_OPEN} { h_lit_open(); }
<*><<EOF>> { return PDFRet::EOS; }
+{SKIP} { }
<*>.|\n { return PDFRet::UNEXPECTED_SYMBOL; }
%%
PDFTokenizer::PDFRet PDFTokenizer::h_dict_close()
{
- obj_dictionary.clear();
-
debug_logf(6, js_trace, TRACE_PDF_PROC, nullptr,
"dictionary close, at array level %d\n", obj_array.nesting_level);
- if (obj_dictionary.array_level != obj_array.nesting_level)
+ auto dict_arr_lvl = obj_dictionary.array_level;
+ obj_dictionary.clear();
+
+ if (dict_arr_lvl != obj_array.nesting_level)
return PDFRet::INCOMPLETE_ARRAY_IN_DICTIONARY;
return PDFRet::EOS;
return PDFRet::EOS;
}
+PDFTokenizer::PDFRet PDFTokenizer::h_array_nesting()
+{
+ if (obj_array.nesting_level)
+ return PDFRet::UNEXPECTED_SYMBOL;
+ else
+ return PDFRet::EOS;
+}
+
PDFTokenizer::PDFRet PDFTokenizer::h_stream_open()
{
- if (obj_stream.rem_length < 0)
+ if (obj_stream.rem_length < 0 and !obj_stream.is_ref_len)
return PDFRet::STREAM_NO_LENGTH;
+ if (indirect_obj.ref_met)
+ {
+ indirect_obj.clear();
+ return PDFRet::UNEXPECTED_SYMBOL; // indirect streams must have direct dictionaries
+ }
+
debug_logf(6, js_trace, TRACE_PDF_PROC, nullptr,
"Starting %s stream, length %d\n", obj_stream.is_js ? "JavaScript" : "skipping", obj_stream.rem_length);
bool PDFTokenizer::h_stream_close()
{
obj_stream.rem_length -= yyleng;
+
if (obj_stream.rem_length <= 0)
{
if (YY_START == jsstream)
if (YY_START == jsstream)
ECHO;
- return false;
+ return obj_stream.is_ref_len;
}
void PDFTokenizer::h_stream_length()
{
if (!strcmp(obj_entry.key, "/Length"))
- obj_stream.rem_length = atoi(yytext);
+ obj_stream.rem_length = snort::SnortStrtol(yytext, nullptr, 10);
}
void PDFTokenizer::h_ref()
{
if (!strcmp(obj_entry.key, "/JS"))
- js_stream_refs.insert(atoi(yytext));
+ js_stream_refs.insert(snort::SnortStrtoul(yytext, nullptr, 10));
+ else if (!strcmp(obj_entry.key, "/Length"))
+ {
+ obj_stream.is_ref_len = true;
+ obj_stream.rem_length = -1;
+ }
}
void PDFTokenizer::h_ind_obj_open()
{
- int value = atoi(yytext);
+ unsigned int value = snort::SnortStrtoul(yytext, nullptr, 10);
if (js_stream_refs.count(value) > 0)
obj_stream.is_js = true;
}
PDFTokenizer::PDFRet PDFTokenizer::process()
{
- auto r = yylex();
- return static_cast<PDFTokenizer::PDFRet>(r);
+ auto r = static_cast<PDFTokenizer::PDFRet>(yylex());
+
+ if (!yy_buffer_stack or !YY_CURRENT_BUFFER_LVALUE)
+ return r;
+
+ if (YY_CURRENT_BUFFER_LVALUE->yy_buf_size > YY_BUF_SIZE)
+ r = PDFTokenizer::TOKEN_TOO_LONG;
+
+ if (r != PDFTokenizer::EOS)
+ yy_flush_buffer(YY_CURRENT_BUFFER);
+
+ return r;
}
""
);
}
- SECTION("indirect object")
+ SECTION("indirect dictionary")
{
test_pdf_proc(
"19 0 obj"
""
);
}
+
+ SECTION("indirect array")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "null 1 2 3.14 (string) << /SubDict [/Sub /Array] >> true 2 0 R"
+ "]"
+ "endobj",
+ ""
+ );
+ }
+
+ SECTION("indirect imbalanced array")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "1 2 3\n"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+
+ SECTION("indirect number")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "1\n"
+ "endobj\n"
+ "2 0 obj\n"
+ "3.14\n"
+ "endobj",
+ ""
+ );
+ }
+
+ SECTION("indirect ref")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "2 0 R\n"
+ "endobj",
+ ""
+ );
+ }
+
+ SECTION("indirect bool")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "false\n"
+ "endobj\n",
+ ""
+ );
+ }
+
+ SECTION("indirect name")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "/name\n"
+ "endobj",
+ ""
+ );
+ }
+
+ SECTION("indirect null")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "null\n"
+ "endobj\n",
+ ""
+ );
+ }
+
SECTION("records")
{
test_pdf_proc(
"", PDFTokenizer::PDFRet::NOT_NAME_IN_DICTIONARY_KEY
);
}
- SECTION("incomplete array")
+ SECTION("token too long")
+ {
+ test_pdf_proc(
+ "1"s + std::string(16 * 1024,' ') + " 0 obj"
+ "<< >>"
+ "endobj"s,
+ "", PDFTokenizer::PDFRet::TOKEN_TOO_LONG
+ );
+ }
+}
+
+TEST_CASE("brackets balancing", "[PDFTokenizer]")
+{
+ SECTION("imbalanced array")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "[ 0 "
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "[ 0 ]]"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("imbalanced dictionary")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<< /dict "
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<< /dict >> >>"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("balanced array in array")
{
test_pdf_proc(
"1 0 obj"
- "<<"
- "/K1 [ /V1 /V2 /V3 "
- ">>"
+ "["
+ "[ /nested /array ]"
+ "]"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("imbalanced array in array")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "[ /nested /array "
+ "]"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "[ /nested /array ] ]"
+ "]"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("balanced dictionary in array")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "<< /nested /dict >>"
+ "]"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("imbalanced dictionary in array")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "<< /nested /dict "
+ "]"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "["
+ "<< /nested /dict >> >>"
+ "]"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("balanced array in dictionary")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<< /array [] >>"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("imbalanced array in dictionary")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<<"
+ "/K1 [ /V1 /V2 /V3 "
+ ">>"
+ "endobj",
+ "", PDFTokenizer::PDFRet::INCOMPLETE_ARRAY_IN_DICTIONARY
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<<"
+ "/K1 [ /V1 /V2 /V3 ]]"
+ ">>"
+ "endobj",
+ "", PDFTokenizer::PDFRet::INCOMPLETE_ARRAY_IN_DICTIONARY
+ );
+ }
+ }
+ SECTION("balanced strings")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "( a string with ( parentheses ) in it )"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("imbalanced strings")
+ {
+ SECTION("missing end")
+ {
+ // NOTE: such syntax doesn't generate an error, because it's possible
+ // to have a string continuation in next PDUs. Same holds true for
+ // hex strings too
+ test_pdf_proc(
+ "1 0 obj"
+ "( a string with ( parentheses in it )"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "( a string with ( parentheses in it )))"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("balanced hex strings")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<FE FF 00 66 006F 00 6F>"
"endobj",
- "", PDFTokenizer::PDFRet::INCOMPLETE_ARRAY_IN_DICTIONARY
+ ""
);
}
+ SECTION("imbalanced hex strings")
+ {
+ SECTION("missing end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<FE FF 00 66 006F 00 6F "
+ "endobj",
+ ""
+ );
+ }
+ SECTION("redundant end")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<FE FF 00 66 006F 00 6F>>"
+ "endobj",
+ "", PDFTokenizer::PDFRet::UNEXPECTED_SYMBOL
+ );
+ }
+ }
+ SECTION("multiple tokens inter-nesting")
+ {
+ SECTION("array-array-array")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "[ [ [ null ] ] ]"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("array-array-dict")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "[ [ << /key /value >> ] ]"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("dict-dict-array")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<< /key1 << /key2 [ null ] >> >>"
+ "endobj",
+ ""
+ );
+ }
+ SECTION("dict-dict-dict")
+ {
+ test_pdf_proc(
+ "1 0 obj"
+ "<< /key1 << /key2 << /key3 /val3 >> >> >>"
+ "endobj",
+ ""
+ );
+ }
+ }
}
TEST_CASE("JS location", "[PDFTokenizer]")
"bar\n"
);
}
+ SECTION("reference as length")
+ {
+ test_pdf_proc(
+ "1 0 obj\n"
+ "<</S /JavaScript /JS 2 0 R>>\n"
+ "endobj\n"
+ "2 0 obj\n"
+ "<<"
+ "/Length 3 0 R"
+ ">>\n"
+ "stream\n"
+ "foo\n"
+ "endstream\n"
+ "endobj\n"
+ "3 0 obj\n"
+ "3\n"
+ "endobj\n",
+ "foo\n", PDFTokenizer::PDFRet::EOS
+ );
+ }
SECTION("special symbols in a stream")
{
test_pdf_proc(
{
{ CountType::SUM, "total_packets", "total packets monitored" },
{ CountType::SUM, "total_usecs", "total usecs elapsed" },
- { CountType::SUM, "max_usecs", "maximum usecs elapsed" },
+ { CountType::MAX, "max_usecs", "maximum usecs elapsed" },
{ CountType::SUM, "packet_timeouts", "packets that timed out" },
{ CountType::SUM, "total_rule_evals", "total rule evals monitored" },
{ CountType::SUM, "rule_eval_timeouts", "rule evals that timed out" },
bool LatencyModule::end(const char*, int, SnortConfig* sc)
{
- PacketLatencyConfig& config = sc->latency->packet_latency;
+ PacketLatencyConfig& packet_config = sc->latency->packet_latency;
+ RuleLatencyConfig& rule_config = sc->latency->rule_latency;
- if (config.max_time > CLOCK_ZERO)
- config.force_enable = true;
+ if ( packet_config.max_time > CLOCK_ZERO )
+ packet_config.force_enable = true;
+
+ if ( rule_config.max_time > CLOCK_ZERO )
+ rule_config.force_enable = true;
return true;
}
{
assert(!timers.empty());
const auto& timer = timers.back();
+ if ( timer.packet->flow )
+ timer.packet->flow->flowstats.total_rule_latency += clock_usecs(TO_USECS(timer.elapsed()));
bool timed_out = false;
void RuleLatency::push(const detection_option_tree_root_t& root, Packet* p)
{
- if ( rule_latency::config->enabled() )
+ if ( rule_latency::config->force_enabled() )
{
if ( rule_latency::get_impl().push(root, p) )
++latency_stats.rule_tree_enables;
void RuleLatency::pop()
{
- if ( rule_latency::config->enabled() )
+ if ( rule_latency::config->force_enabled() )
{
if ( rule_latency::get_impl().pop() )
++latency_stats.rule_eval_timeouts;
struct RuleLatencyConfig
{
hr_duration max_time = 0_ticks;
+ bool force_enable = false;
bool suspend = false;
unsigned suspend_threshold = 0;
hr_duration max_suspend_time = 0_ticks;
#endif
return max_time > 0_ticks;
}
+
+ bool force_enabled() const
+ {
+ return force_enable;
+ }
+
bool allow_reenable() const { return max_suspend_time > 0_ticks; }
};
#include "detection/detection_engine.h"
#include "events/event.h"
+#include "flow/flow_key.h"
#include "framework/logger.h"
#include "framework/module.h"
#include "log/messages.h"
const IpsContext* c = DetectionEngine::get_context();
Obfuscator* obf = (c and c->packet) ? c->packet->obfuscator : nullptr;
uint32_t tenant_id = 0;
+
+#ifndef DISABLE_TENANT_ID
if (flow)
- tenant_id = flow->tenant;
- else if (c and c->packet)
+ tenant_id = flow->key->tenant_id;
+ else
+#endif
+ if (c and c->packet)
tenant_id = c->packet->pkth->tenant_id;
while ( xid && (xid <= max_count) )
using namespace snort;
-static bool exit_requested = false;
+bool exit_requested = false;
static int main_exit_code = 0;
static bool paused = false;
static bool pthreads_started = false;
+static bool pthreads_running = false;
static std::queue<AnalyzerCommand*> orphan_commands;
static std::mutex poke_mutex;
static bool* pigs_started = nullptr;
+static bool* pigs_running = nullptr;
static Pig* pigs = nullptr;
static unsigned max_pigs = 0;
static unsigned pigs_failed = 0;
return 0;
}
+int show_snort_cpu(lua_State* L)
+{
+ ControlConn* ctrlconn = ControlConn::query_from_lua(L);
+ send_response(ctrlconn, "Id \tPid \t30sec \t2min \t5min\n\n");
+ main_broadcast_command(new ACShowSnortCPU(ctrlconn), ctrlconn);
+ return 0;
+}
+
//-------------------------------------------------------------------------
// housekeeping foo
//-------------------------------------------------------------------------
static void service_check()
{
#ifdef SHELL
- if (pthreads_started && ControlMgmt::service_users())
+ if (pthreads_running && ControlMgmt::service_users())
return;
#endif
if ( pig.analyzer )
{
handle(pig, swine, pending_privileges);
- if (!pigs_started[idx] && pig.analyzer && (pig.analyzer->get_state() ==
- Analyzer::State::STARTED))
+
+ if (!pigs_started[idx] && pig.analyzer && ((pig.analyzer->get_state() == Analyzer::State::STARTED)))
pigs_started[idx] = true;
+
+ if (!pigs_running[idx] && pig.analyzer && ((pig.analyzer->get_state() == Analyzer::State::PAUSED) ||
+ (pig.analyzer->get_state() == Analyzer::State::RUNNING)))
+ pigs_running[idx] = true;
+
if (pigs_started[idx] && (!pig.analyzer || pig.analyzer->get_state() ==
Analyzer::State::FAILED))
pigs_started[idx] = false;
}
pthreads_started = pigs_started_count && num_threads <= pigs_started_count + pigs_failed;
-
+
if (pthreads_started)
{
#ifdef REG_TEST
LogMessage("All pthreads started\n");
#endif
+
#ifdef SHELL
if (use_shell(SnortConfig::get_conf()))
{
}
}
+ if (!pthreads_running)
+ {
+ const unsigned num_threads = (!Trough::has_next()) ? max_swine : max_pigs;
+ unsigned pigs_running_count = 0;
+
+ for (unsigned i = 0; i < num_threads; i++)
+ {
+ if(pigs_running[i])
+ pigs_running_count++;
+ }
+
+ pthreads_running = pigs_running_count && num_threads <= pigs_running_count + pigs_failed;
+ }
+
if ( !exit_requested and (swine < max_pigs) and (src = Trough::get_next()) )
{
Pig* pig = get_lazy_pig(max_pigs);
pig_poke = new Ring<unsigned>((max_pigs*max_grunts)+1);
pigs = new Pig[max_pigs];
pigs_started = new bool[max_pigs];
+ pigs_running = new bool[max_pigs];
for (unsigned idx = 0; idx < max_pigs; idx++)
{
Pig& pig = pigs[idx];
pig.set_index(idx);
pigs_started[idx] = false;
+ pigs_running[idx] = false;
}
main_loop();
pigs = nullptr;
delete[] pigs_started;
pigs_started = nullptr;
+ delete[] pigs_running;
+ pigs_running = nullptr;
#ifdef SHELL
ControlMgmt::socket_term();
#define MAIN_H
struct lua_State;
-
+extern bool exit_requested;
const char* get_prompt();
// commands provided by the snort module
int main_quit(lua_State* = nullptr);
int main_help(lua_State* = nullptr);
int convert_counter_type(const char* type);
+int show_snort_cpu(lua_State* = nullptr);
#ifdef SHELL
int main_dump_plugins(lua_State* = nullptr);
#include "protocols/packet_manager.h"
#include "target_based/host_attributes.h"
#include "utils/stats.h"
+#include "packet_io/sfdaq_instance.h"
#include "analyzer.h"
#include "reload_tracker.h"
#include "reload_tuner.h"
#include "snort.h"
#include "snort_config.h"
+#include "thread_config.h"
#include "swapper.h"
using namespace snort;
{
return analyzer.get_daq_instance();
}
+
+ACShowSnortCPU::~ACShowSnortCPU()
+{
+ if (DAQ_SUCCESS == status)
+ {
+ LogRespond(ctrlcon, "\nSummary \t%.1f%% \t%.1f%% \t%.1f%%\n",
+ cpu_usage_30s/instance_num, cpu_usage_120s/instance_num,
+ cpu_usage_300s/instance_num);
+ }
+}
+
+bool ACShowSnortCPU::execute(Analyzer& analyzer, void**)
+{
+ DIOCTL_GetCpuProfileData get_data = {};
+
+ {
+ std::lock_guard<std::mutex> lock(cpu_usage_mutex);
+ assert(DAQ_SUCCESS == status);
+
+ SFDAQInstance* instance = get_daq_instance(analyzer);
+ ThreadConfig *thread_config = SnortConfig::get_conf()->thread_config;
+ int tid = thread_config->get_instance_tid(get_instance_id());
+
+ status = instance->ioctl(
+ (DAQ_IoctlCmd)DIOCTL_GET_CPU_PROFILE_DATA,
+ (void *)(&get_data),
+ sizeof(DIOCTL_GetCpuProfileData));
+
+ if (DAQ_SUCCESS != status)
+ {
+ LogRespond(ctrlcon, "Fetching profile data failed from DAQ instance\n");
+ return true;
+ }
+
+ double cpu_30s = static_cast<double> (get_data.cpu_usage_percent_30s);
+ double cpu_120s = static_cast<double> (get_data.cpu_usage_percent_120s);
+ double cpu_300s = static_cast<double> (get_data.cpu_usage_percent_300s);
+
+ // Print CPU usage
+ LogRespond(ctrlcon, "%-3d \t%-6d \t%.1f%% \t%.1f%% \t%.1f%%\n",
+ instance_num, tid, cpu_30s, cpu_120s, cpu_300s);
+
+ // Add CPU usage data
+ cpu_usage_30s += cpu_30s;
+ cpu_usage_120s += cpu_120s;
+ cpu_usage_300s += cpu_300s;
+ instance_num++;
+
+ }
+
+ return true;
+}
#ifndef ANALYZER_COMMANDS_H
#define ANALYZER_COMMANDS_H
+#include <daq_common.h>
+
#include <cstdarg>
#include <vector>
+#include <mutex>
#include "main/snort_types.h"
std::vector<snort::ScratchAllocator*>& handlers;
};
+class ACShowSnortCPU : public snort::AnalyzerCommand
+{
+public:
+ explicit ACShowSnortCPU(ControlConn* conn) : AnalyzerCommand(conn)
+ { }
+ bool execute(Analyzer&, void**) override;
+ const char* stringify() override { return "SHOW_SNORT_CPU"; }
+ ~ACShowSnortCPU() override;
+
+private:
+ int status = DAQ_SUCCESS;
+ double cpu_usage_30s = 0.0;
+ double cpu_usage_120s = 0.0;
+ double cpu_usage_300s = 0.0;
+ int instance_num = 0;
+ std::mutex cpu_usage_mutex;
+};
+
namespace snort
{
// from main.cc
{ "rule_db_dir", Parameter::PT_STRING, nullptr, nullptr,
"deserialize rule databases from given directory" },
- { "show_fast_patterns", Parameter::PT_BOOL, nullptr, "false",
- "print fast pattern info for each rule" },
-
{ "split_any_any", Parameter::PT_BOOL, nullptr, "true",
"evaluate any-any rules separately to save memory" },
if ( !fp->set_offload_search_method(v.get_string()) )
return false;
}
- else if ( v.is("show_fast_patterns") )
- fp->set_debug_print_fast_patterns(v.get_bool());
else if ( v.is("split_any_any") )
fp->set_split_any_any(v.get_bool());
overlay_trace_config = tc;
}
-bool SnortConfig::set_latency_enable()
+bool SnortConfig::set_packet_latency(bool is_enabled) const
{
- if (latency)
+ if ( latency )
{
- latency->packet_latency.force_enable = true;
+ latency->packet_latency.force_enable = is_enabled;
+ return is_enabled;
+ }
+ return false;
+}
+
+bool SnortConfig::get_packet_latency() const
+{
+ if ( latency->packet_latency.force_enabled() )
return true;
+ return false;
+}
+
+bool SnortConfig::set_rule_latency(bool is_enabled) const
+{
+ if ( latency )
+ {
+ latency->rule_latency.force_enable = is_enabled;
+ return is_enabled;
}
return false;
}
+bool SnortConfig::get_rule_latency() const
+{
+ if ( latency->rule_latency.force_enabled() )
+ return true;
+ return false;
+}
+
void SnortConfig::set_tunnel_verdicts(const char* args)
{
char* tmp, * tok;
void set_utc(bool);
void set_watchdog(uint16_t);
void set_watchdog_min_thread_count(uint16_t);
- SO_PUBLIC bool set_latency_enable();
+ SO_PUBLIC bool set_packet_latency(bool) const;
+ SO_PUBLIC bool get_packet_latency() const;
+ SO_PUBLIC bool set_rule_latency(bool) const;
+ SO_PUBLIC bool get_rule_latency() const;
//------------------------------------------------------
// accessor methods
{ "reload_hosts", main_reload_hosts, s_reload, "load a new hosts table" },
{ "log_command", main_log_command,main_log_command_param, "enable or disable command logging"},
{ "show_config_generation", main_show_config_generation, nullptr, "show loaded configuration ID"},
+ { "show_snort_cpu", show_snort_cpu, nullptr, "show snort cpu usage"},
// FIXIT-M rewrite trough to permit updates on the fly
//{ "process", main_process, nullptr, "process given pcap" },
{
PHVector passive;
PHVector probe;
+ PHVector probe_first;
PHVector control;
void vectorize(SnortConfig*) override;
{
passive.alloc(ilist.size());
probe.alloc(ilist.size());
+ probe_first.alloc(ilist.size());
control.alloc(ilist.size());
for ( auto* p : ilist )
{
probe.add(p);
break;
+ case IT_PROBE_FIRST:
+ probe_first.add(p);
+ break;
+
case IT_CONTROL:
control.add_control(p);
break;
case IT_PROBE:
return get_instance_from_vector(key, probe.vec, probe.total_num);
+ case IT_PROBE_FIRST:
+ return get_instance_from_vector(key, probe_first.vec, probe_first.total_num);
+
case IT_CONTROL:
return get_instance_from_vector(key, control.vec, control.total_num);
}
}
+void InspectorManager::probe_first(Packet* p)
+{
+ GlobalInspectorPolicy* pp = p->context->conf->policy_map->get_global_inspector_policy();
+ assert(pp);
+ if ( !trace_enabled(snort_trace, TRACE_INSPECTOR_MANAGER, DEFAULT_TRACE_LOG_LEVEL, p) )
+ ::execute<false>(p, pp->probe_first.vec, pp->probe_first.num, true);
+ else
+ ::execute<true>(p, pp->probe_first.vec, pp->probe_first.num, true);
+}
+
void InspectorManager::clear(Packet* p)
{
if ( !p->context->clear_inspectors )
static void execute(Packet*);
static void probe(Packet*);
+ static void probe_first(Packet*);
static void clear(Packet*);
static void empty_trash();
#include <cassert>
#include "profiler/memory_profiler_active_context.h"
-
+#include "main.h"
#include "memory_allocator.h"
#ifdef UNIT_TEST
// number of requested bytes
size_t payload_size;
- uint32_t thread_id;
+ bool is_packet_thread;
// stat used to keep track of allocation/deallocation
MemoryTracker* mp_inspector_stats = nullptr;
#if defined(REG_TEST) || defined(UNIT_TEST)
sanity(SANITY_CHECK_VALUE),
#endif
- payload_size(n), thread_id(instance_id),
+ payload_size(n), is_packet_thread(snort::is_packet_thread()),
mp_inspector_stats(&mp_active_context.get_default())
{ }
assert(meta);
#ifdef ENABLE_MEMORY_PROFILER
- if (!snort::Snort::is_exiting())
+ if ( !snort::Snort::is_exiting() and !exit_requested )
{
- if (meta->mp_inspector_stats and meta->thread_id == instance_id)
+ if ( meta->mp_inspector_stats and meta->is_packet_thread == true )
meta->mp_inspector_stats->update_deallocs(meta->total_size());
else
- mp_active_context.update_deallocs(meta->total_size());
+ mp_active_context.get_fallback().update_deallocs(meta->total_size());
}
#endif
#include "log/messages.h"
#include "file_api/file_service.h"
+#include "file_api/file_config.h"
using namespace snort;
}
// update file depth and max_depth etc
-void DecodeConfig::sync_all_depths()
+void DecodeConfig::sync_all_depths(const SnortConfig* sc)
{
- file_depth = FileService::get_max_file_depth();
+ file_depth = FileService::get_max_file_depth(get_file_config(sc));
decode_enabled = (file_depth >= 0) or (b64_depth >= 0) or (qp_depth >= 0) or
(bitenc_depth >= 0) or (uu_depth >= 0);
}
// List of MIME decode and log configuration functions
#include "main/snort_types.h"
+#include "main/snort_config.h"
/*These are temporary values*/
#define DEFAULT_MIME_MEMCAP 838860
int64_t get_file_depth() const;
bool is_decoding_enabled() const;
- void sync_all_depths();
+ void sync_all_depths(const SnortConfig*);
void show(bool = false) const;
int get_max_depth(int) const;
{
const FileDirection dir = upload ? FILE_UPLOAD : FILE_DOWNLOAD;
continue_inspecting_file = file_flows->file_process(p, get_file_cache_file_id(), data,
- data_size, file_offset, dir, get_multiprocessing_file_id(), position);
+ data_size, file_offset, dir, get_multiprocessing_file_id(), position, (const uint8_t*)filename.c_str(),
+ filename.length());
}
else
{
continue_inspecting_file = file_flows->file_process(p, data, data_size, position,
- upload);
+ upload, 0, (const uint8_t*)filename.c_str(),
+ filename.length());
}
file_offset += data_size;
if (continue_inspecting_file and (isFileStart(position)) && log_state)
add_subdirectory(appid)
add_subdirectory(arp_spoof)
add_subdirectory(binder)
+add_subdirectory(extractor)
add_subdirectory(kaizen)
add_subdirectory(normalize)
add_subdirectory(packet_capture)
set(STATIC_NETWORK_INSPECTOR_PLUGINS
$<TARGET_OBJECTS:appid>
$<TARGET_OBJECTS:binder>
+ $<TARGET_OBJECTS:extractor>
$<TARGET_OBJECTS:kaizen>
$<TARGET_OBJECTS:normalize>
$<TARGET_OBJECTS:port_scan>
tp_appid_session_api.h
tp_appid_module_api.h
tp_appid_module_api.cc
- app_cpu_profile_table.cc
- app_cpu_profile_table.h
+ appid_cpu_profile_table.cc
+ appid_cpu_profile_table.h
)
#if (STATIC_INSPECTORS)
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2014-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// app_cpu_profiling_table.cc author Umang Sharma <umasharm@cisco.com>
-
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
-#include "log/text_log.h"
-#include "time/packet_time.h"
-#include <iomanip>
-#include <sstream>
-#include <queue>
-
-#include "appid_session.h"
-#include "app_cpu_profile_table.h"
-
-using namespace snort;
-
-const char* table_header = "AppId Performance Statistics (all)\n========================================================================================================================\n";
-const char* columns = " AppId App Name Microsecs Packets Avg/Packet Sessions Avg/Session \n";
-const char* partition = "------------------------------------------------------------------------------------------------------------------------\n";
-
-std::string FormatWithCommas(uint64_t value)
-{
- std::string numStr = std::to_string(value);
- int insertPosition = numStr.length() - 3;
- while (insertPosition > 0)
- {
- numStr.insert(insertPosition, ",");
- insertPosition -= 3;
- }
- return numStr;
-}
-
-// Comparator for priority queue based on avg_processing_time/session
-struct CompareByAvgProcessingTime {
- bool operator()(const std::pair<AppId, AppidCPUProfilerStats>& a, const std::pair<AppId, AppidCPUProfilerStats>& b) const {
- if (a.second.processed_packets == 0 or b.second.processed_packets == 0) {
- return false;
- }
- return a.second.processing_time/a.second.per_appid_sessions < b.second.processing_time/b.second.per_appid_sessions ;
- }
-};
-
-void AppidCPUProfilingManager::display_appid_cpu_profiler_table(AppId appid)
-{
- if (appid_cpu_profiling_table.empty())
- {
- appid_log(nullptr, TRACE_INFO_LEVEL,"Appid CPU Profiler Table is empty\n", appid);
- return;
- }
- auto bucket = appid_cpu_profiling_table.find(appid);
-
- if (bucket != appid_cpu_profiling_table.end())
- {
- appid_log(nullptr, TRACE_INFO_LEVEL, table_header);
- appid_log(nullptr, TRACE_INFO_LEVEL, columns);
- appid_log(nullptr, TRACE_INFO_LEVEL, partition);
-
- appid_log(nullptr, TRACE_INFO_LEVEL, " %5d %-25.25s %18s %12s %15s %12s %12s\n",
- appid, bucket->second.app_name.c_str(), FormatWithCommas(bucket->second.processing_time).c_str(), FormatWithCommas(bucket->second.processed_packets).c_str(), FormatWithCommas(bucket->second.processing_time/bucket->second.processed_packets).c_str(),
- FormatWithCommas(bucket->second.per_appid_sessions).c_str(), FormatWithCommas(bucket->second.processing_time/bucket->second.per_appid_sessions).c_str());
- }
- else
- {
- appid_log(nullptr, TRACE_INFO_LEVEL,"Appid %d not found in the table\n", appid);
- }
-}
-
-void AppidCPUProfilingManager::display_appid_cpu_profiler_table()
-{
- if (appid_cpu_profiling_table.empty())
- {
- appid_log(nullptr, TRACE_INFO_LEVEL,"Appid CPU Profiler Table is empty\n");
- return;
- }
-
- std::priority_queue<std::pair<AppId, AppidCPUProfilerStats>, std::vector<std::pair<AppId, AppidCPUProfilerStats>>, CompareByAvgProcessingTime> sorted_appid_cpu_profiler_table;
-
- for (const auto& entry : appid_cpu_profiling_table)
- {
- sorted_appid_cpu_profiler_table.push(entry); // Push the pair (AppId, AppidCPUProfilerStats)
- }
-
- appid_log(nullptr, TRACE_INFO_LEVEL, table_header);
- appid_log(nullptr, TRACE_INFO_LEVEL, columns);
- appid_log(nullptr, TRACE_INFO_LEVEL, partition);
-
- while (!sorted_appid_cpu_profiler_table.empty())
- {
- auto entry = sorted_appid_cpu_profiler_table.top();
- sorted_appid_cpu_profiler_table.pop();
- if (!entry.second.processed_packets)
- continue;
-
- appid_log(nullptr, TRACE_INFO_LEVEL, " %5d %-25.25s %18s %12s %15s %12s %12s\n",
- entry.first, entry.second.app_name.c_str(), FormatWithCommas(entry.second.processing_time).c_str(), FormatWithCommas(entry.second.processed_packets).c_str(), FormatWithCommas(entry.second.processing_time/entry.second.processed_packets).c_str(),
- FormatWithCommas(entry.second.per_appid_sessions).c_str(), FormatWithCommas(entry.second.processing_time/entry.second.per_appid_sessions).c_str());
- }
-}
-
-AppidCPUProfilingManager::AppidCPUProfilingManager()
-{
- appid_cpu_profiling_table.clear();
-}
-
-void AppidCPUProfilingManager::cleanup_appid_cpu_profiler_table()
-{
- appid_cpu_profiling_table.clear();
-}
-
-void AppidCPUProfilingManager::insert_appid_cpu_profiler_record(AppId appId, const AppidCPUProfilerStats& stats)
-{
- auto it = appid_cpu_profiling_table.find(appId);
- if (it == appid_cpu_profiling_table.end())
- {
- appid_cpu_profiling_table.emplace(appId, stats);
- }
- else
- {
- it->second.processing_time += stats.processing_time;
- it->second.processed_packets += stats.processed_packets;
- it->second.per_appid_sessions += 1;
- }
-}
-
-void AppidCPUProfilingManager::check_appid_cpu_profiler_table_entry(const AppIdSession* asd, AppId payload_id)
-{
- if (payload_id > APP_ID_NONE)
- {
- const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(payload_id);
- if (app_name == nullptr)
- app_name = "unknown";
-
- stats_bucket_insert(payload_id, app_name, asd->stats.prev_payload_processing_time, asd->stats.prev_payload_processing_packets);
- }
-}
-
-void AppidCPUProfilingManager::stats_bucket_insert(AppId appid, const char* app_name, uint64_t processing_time, uint64_t processed_packets)
-{
- if (!processed_packets or !processing_time)
- {
- appid_log(nullptr, TRACE_INFO_LEVEL, "appid: processed packets/time are NULL for appid : %d , app_name : %s , processing time :%lu \n", appid, app_name, processing_time);
- return;
- }
-
- AppidCPUProfilerStats stats(app_name, processing_time, processed_packets, 1);
- insert_appid_cpu_profiler_record(appid, stats);
-}
-
-void AppidCPUProfilingManager::check_appid_cpu_profiler_table_entry(const AppIdSession* asd,AppId service_id, AppId client_id, AppId payload_id, AppId misc_id)
-{
- if (!asd->stats.processing_time or !asd->stats.cpu_profiler_pkt_count)
- return;
-
- if (service_id > APP_ID_NONE)
- {
- const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(service_id);
- if (app_name == nullptr)
- app_name = "unknown";
-
- stats_bucket_insert(service_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
- }
- if (client_id > APP_ID_NONE and client_id != service_id){
- const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(client_id);
- if (app_name == nullptr)
- app_name = "unknown";
-
- stats_bucket_insert(client_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
- }
- if (payload_id > APP_ID_NONE and payload_id != service_id and payload_id != client_id)
- {
- const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(payload_id);
- if (app_name == nullptr)
- app_name = "unknown";
-
- stats_bucket_insert(payload_id, app_name, asd->stats.processing_time - asd->stats.prev_payload_processing_time, asd->stats.cpu_profiler_pkt_count - asd->stats.prev_payload_processing_packets);
- }
- if (misc_id > APP_ID_NONE and misc_id != service_id and misc_id != client_id and misc_id != payload_id)
- {
- const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(misc_id);
- if (app_name == nullptr)
- app_name = "unknown";
-
- stats_bucket_insert(misc_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
- }
-}
{
odp_ctxt.appid_cpu_profiler = false;
}
- else if (!(strcasecmp(conf_val, "enabled")))
- {
- odp_ctxt.appid_cpu_profiler = true;
- }
}
else
ParseWarning(WARN_CONF, "appid: unsupported configuration: %s\n", conf_key);
if (!asd->tsession)
asd->tsession = new TlsSession();
else if (sni_mismatch)
- asd->tsession->set_tls_host(nullptr, 0, change_bits);
+ {
+ asd->tsession->process_sni_mismatch();
+ }
+
if (sni_mismatch)
asd->scan_flags |= SCAN_SPOOFED_SNI_FLAG;
return false;
SnortProtocolId id = inspector.get_service();
- const AppIdConfig& config = appid_inspector->get_ctxt().config;
+ const AppIdConfig& config = appid_inspector->get_config();
if (id == config.snort_proto_ids[PROTO_INDEX_HTTP2] or id == config.snort_proto_ids[PROTO_INDEX_SSH]
or id == config.snort_proto_ids[PROTO_INDEX_CIP])
return true;
OdpContext* AppIdContext::odp_ctxt = nullptr;
uint32_t OdpContext::next_version = 0;
-static void map_app_names_to_snort_ids(SnortConfig* sc, AppIdConfig& config)
+AppIdConfig::~AppIdConfig()
{
- // Have to create SnortProtocolIds during configuration initialization.
- config.snort_proto_ids[PROTO_INDEX_UNSYNCHRONIZED] = sc->proto_ref->add("unsynchronized");
- config.snort_proto_ids[PROTO_INDEX_FTP_DATA] = sc->proto_ref->add("ftp-data");
- config.snort_proto_ids[PROTO_INDEX_HTTP2] = sc->proto_ref->add("http2");
- config.snort_proto_ids[PROTO_INDEX_REXEC] = sc->proto_ref->add("rexec");
- config.snort_proto_ids[PROTO_INDEX_RSH_ERROR] = sc->proto_ref->add("rsh-error");
- config.snort_proto_ids[PROTO_INDEX_SNMP] = sc->proto_ref->add("snmp");
- config.snort_proto_ids[PROTO_INDEX_SUNRPC] = sc->proto_ref->add("sunrpc");
- config.snort_proto_ids[PROTO_INDEX_TFTP] = sc->proto_ref->add("tftp");
- config.snort_proto_ids[PROTO_INDEX_SIP] = sc->proto_ref->add("sip");
- config.snort_proto_ids[PROTO_INDEX_SSH] = sc->proto_ref->add("ssh");
- config.snort_proto_ids[PROTO_INDEX_CIP] = sc->proto_ref->add("cip");
+ snort_free((void*)app_detector_dir);
}
-AppIdConfig::~AppIdConfig()
+void AppIdConfig::map_app_names_to_snort_ids(SnortConfig& sc)
{
- snort_free((void*)app_detector_dir);
+ // Have to create SnortProtocolIds during configuration initialization.
+ snort_proto_ids[PROTO_INDEX_UNSYNCHRONIZED] = sc.proto_ref->add("unsynchronized");
+ snort_proto_ids[PROTO_INDEX_FTP_DATA] = sc.proto_ref->add("ftp-data");
+ snort_proto_ids[PROTO_INDEX_HTTP2] = sc.proto_ref->add("http2");
+ snort_proto_ids[PROTO_INDEX_REXEC] = sc.proto_ref->add("rexec");
+ snort_proto_ids[PROTO_INDEX_RSH_ERROR] = sc.proto_ref->add("rsh-error");
+ snort_proto_ids[PROTO_INDEX_SNMP] = sc.proto_ref->add("snmp");
+ snort_proto_ids[PROTO_INDEX_SUNRPC] = sc.proto_ref->add("sunrpc");
+ snort_proto_ids[PROTO_INDEX_TFTP] = sc.proto_ref->add("tftp");
+ snort_proto_ids[PROTO_INDEX_SIP] = sc.proto_ref->add("sip");
+ snort_proto_ids[PROTO_INDEX_SSH] = sc.proto_ref->add("ssh");
+ snort_proto_ids[PROTO_INDEX_CIP] = sc.proto_ref->add("cip");
}
void AppIdConfig::show() const
void AppIdContext::pterm()
{
- if (odp_thread_local_ctxt)
- {
- delete odp_thread_local_ctxt;
- odp_thread_local_ctxt = nullptr;
- }
+ delete odp_control_thread_ctxt;
+ odp_control_thread_ctxt = nullptr;
if (odp_ctxt)
{
odp_ctxt->get_app_info_mgr().cleanup_appid_info_table();
- if (odp_ctxt->is_appid_cpu_profiler_running())
- odp_ctxt->get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table();
+ if (odp_ctxt->is_appid_cpu_profiler_enabled())
+ odp_ctxt->get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table(*odp_ctxt, APPID_CPU_PROFILER_DEFAULT_DISPLAY_ROWS, true);
odp_ctxt->get_appid_cpu_profiler_mgr().cleanup_appid_cpu_profiler_table();
delete odp_ctxt;
bool AppIdContext::init_appid(SnortConfig* sc, AppIdInspector& inspector)
{
// do not reload ODP on reload_config()
- if (!odp_ctxt)
- odp_ctxt = new OdpContext(config, sc);
-
- if (!odp_thread_local_ctxt)
- odp_thread_local_ctxt = new OdpThreadContext;
-
if (!once)
{
+ assert(!odp_ctxt);
+ odp_ctxt = new OdpContext(config, sc);
odp_ctxt->get_client_disco_mgr().initialize(inspector);
odp_ctxt->get_service_disco_mgr().initialize(inspector);
odp_ctxt->set_client_and_service_detectors();
appidDebug->set_enabled(config.log_all_sessions);
}
- odp_thread_local_ctxt->initialize(sc, *this, true);
+ assert(!odp_control_thread_ctxt);
+ odp_control_thread_ctxt = new OdpControlContext;
+ odp_control_thread_ctxt->initialize(sc, *this);
+
odp_ctxt->initialize(inspector);
// do not reload third party on reload_config()
}
else
{
+ assert(odp_ctxt);
odp_ctxt->get_client_disco_mgr().reload();
odp_ctxt->get_service_disco_mgr().reload();
odp_ctxt->reload();
}
- map_app_names_to_snort_ids(sc, config);
if (config.enable_rna_filter)
discovery_filter = new DiscoveryFilter(config.rna_conf_path);
return true;
appid_log(nullptr, TRACE_INFO_LEVEL, "Appid Config: max_packet_before_service_fail %" PRIu16" \n", max_packet_before_service_fail);
appid_log(nullptr, TRACE_INFO_LEVEL, "Appid Config: max_packet_service_fail_ignore_bytes %" PRIu16" \n", max_packet_service_fail_ignore_bytes);
appid_log(nullptr, TRACE_INFO_LEVEL, "Appid Config: eve_http_client %s\n", (eve_http_client ? "True" : "False"));
- appid_log(nullptr, TRACE_INFO_LEVEL, "Appid Config: appid_cpu_profiler %s\n", (appid_cpu_profiler ? "True" : "False"));
+ appid_log(nullptr, TRACE_INFO_LEVEL, "Appid Config: appid_cpu_profiler %s\n", (appid_cpu_profiler ? "True" : "False"));
}
bool OdpContext::is_appid_cpu_profiler_running()
{
return (TimeProfilerStats::is_enabled() and appid_cpu_profiler);
-}
+}
bool OdpContext::is_appid_cpu_profiler_enabled()
{
return appid_cpu_profiler;
-}
+}
OdpContext::OdpContext(const AppIdConfig& config, SnortConfig* sc)
{
return ip_protocol[(uint16_t)proto];
}
-void OdpThreadContext::initialize(const SnortConfig* sc, AppIdContext& ctxt, bool is_control,
- bool reload_odp)
+void OdpControlContext::initialize(const SnortConfig* sc, AppIdContext& ctxt)
{
- if (!is_control and reload_odp)
- LuaDetectorManager::init_thread_manager(sc, ctxt);
- else
- LuaDetectorManager::initialize(sc, ctxt, is_control, reload_odp);
+ lua_detector_mgr = std::make_shared<ControlLuaDetectorManager>(ctxt);
+ lua_detector_mgr->initialize(sc);
}
-OdpThreadContext::~OdpThreadContext()
+void OdpPacketThreadContext::initialize(const SnortConfig* sc)
{
+ lua_detector_mgr = ControlLuaDetectorManager::get_packet_lua_detector_manager();
assert(lua_detector_mgr);
- delete lua_detector_mgr;
+ lua_detector_mgr->initialize(sc);
}
#define APP_ID_CONFIG_H
#include <array>
+#include <memory>
#include <string>
#include "helpers/discovery_filter.h"
#include "detector_plugins/ssh_patterns.h"
#include "tp_appid_module_api.h"
#include "utils/sflsq.h"
-#include "app_cpu_profile_table.h"
+#include "appid_cpu_profile_table.h"
#include "profiler/profiler_defs.h"
#define APP_ID_PORT_ARRAY_SIZE 65536
AppIdConfig() = default;
~AppIdConfig();
+ void map_app_names_to_snort_ids(snort::SnortConfig&);
+
// FIXIT-L: DECRYPT_DEBUG - Move this to ssl-module
#ifdef REG_TEST
// To manually restart appid detection for an SSL-decrypted flow (single session only),
uint16_t max_packet_service_fail_ignore_bytes = DEFAULT_MAX_PKT_BEFORE_SERVICE_FAIL_IGNORE_BYTES;
FirstPktAppIdDiscovered first_pkt_appid_prefix = NO_APPID_FOUND;
bool eve_http_client = true;
- bool appid_cpu_profiler = false;
+ bool appid_cpu_profiler = true;
OdpContext(const AppIdConfig&, snort::SnortConfig*);
void initialize(AppIdInspector& inspector);
class OdpThreadContext
{
public:
- ~OdpThreadContext();
- void initialize(const snort::SnortConfig*, AppIdContext& ctxt, bool is_control=false,
- bool reload_odp=false);
+ virtual ~OdpThreadContext() = default;
- void set_lua_detector_mgr(LuaDetectorManager& mgr)
+ lua_State* get_lua_state() const
{
- lua_detector_mgr = &mgr;
+ assert(lua_detector_mgr);
+ return lua_detector_mgr->L;
}
- LuaDetectorManager& get_lua_detector_mgr() const
+ bool insert_cb_detector(AppId app_id, LuaObject* ud)
{
assert(lua_detector_mgr);
- return *lua_detector_mgr;
+ return lua_detector_mgr->insert_cb_detector(app_id, ud);
}
-private:
- LuaDetectorManager* lua_detector_mgr = nullptr;
+ LuaObject* get_cb_detector(AppId app_id)
+ {
+ assert(lua_detector_mgr);
+ return lua_detector_mgr->get_cb_detector(app_id);
+ }
+
+protected:
+ std::shared_ptr<LuaDetectorManager> lua_detector_mgr;
+};
+
+class OdpControlContext : public OdpThreadContext
+{
+public:
+ ~OdpControlContext() override = default;
+ void initialize(const snort::SnortConfig*, AppIdContext&);
+ void set_ignore_chp_cleanup()
+ {
+ assert(lua_detector_mgr);
+ static_cast<ControlLuaDetectorManager*>(lua_detector_mgr.get())->set_ignore_chp_cleanup();
+ }
+};
+
+class OdpPacketThreadContext : public OdpThreadContext
+{
+public:
+ ~OdpPacketThreadContext() override = default;
+ void initialize(const snort::SnortConfig*);
+
+ void set_detector_flow(DetectorFlow* df)
+ {
+ assert(lua_detector_mgr);
+ static_cast<PacketLuaDetectorManager*>(lua_detector_mgr.get())->set_detector_flow(df);
+ }
+
+ DetectorFlow* get_detector_flow()
+ {
+ assert(lua_detector_mgr);
+ return static_cast<PacketLuaDetectorManager*>(lua_detector_mgr.get())->get_detector_flow();
+ }
+
+ void free_detector_flow()
+ {
+ assert(lua_detector_mgr);
+ static_cast<PacketLuaDetectorManager*>(lua_detector_mgr.get())->free_detector_flow();
+ }
};
class AppIdContext
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2014-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// app_cpu_profiling_table.cc author Umang Sharma <umasharm@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "log/text_log.h"
+#include "time/packet_time.h"
+#include <iomanip>
+#include <sstream>
+#include <queue>
+#include <algorithm>
+#include <cstdarg>
+
+#include "appid_session.h"
+#include "appid_cpu_profile_table.h"
+#include "control/control.h"
+
+using namespace snort;
+
+
+#define TABLE_HEADER(num_rows) "AppId Performance Statistics (top %d appids)\n===================================================================================================================================================\n", num_rows
+static const char* columns = " AppId App Name Usecs Pkts AvgUsecs/Pkt Sessions AvgUsecs/Sess MaxPkts/Sess MaxUsecs/Sess %%/Total\n";
+static const char* partition = "---------------------------------------------------------------------------------------------------------------------------------------------------\n";
+
+static std::string FormatWithCommas(uint64_t value)
+{
+ std::string numStr = std::to_string(value);
+ int insertPosition = numStr.length() - 3;
+ while (insertPosition > 0)
+ {
+ numStr.insert(insertPosition, ",");
+ insertPosition -= 3;
+ }
+ return numStr;
+}
+
+static void print_log(ControlConn* ctrlcon, AppidCPUProfilerOutputType output_type, int level, const char* format, ...)
+{
+ static std::vector<char> buffer(STD_BUF);
+
+ va_list args;
+ va_start(args, format);
+
+ int response_len = std::vsnprintf(buffer.data(), buffer.size(), format, args);
+
+ va_end(args);
+
+ if (response_len < 0 || static_cast<size_t>(response_len) >= buffer.size())
+ return;
+
+ switch (output_type)
+ {
+ case OUTPUT_CONSOLE:
+ LogRespond(ctrlcon, "%s", buffer.data());
+ break;
+ case OUPUT_LOGFILE:
+ appid_log(nullptr, level, "%s", buffer.data());
+ break;
+ default:
+ break;
+ }
+}
+
+// Comparator for priority queue based on avg_processing_time/session
+struct CompareByAvgProcessingTime {
+ bool operator()(const std::pair<AppId, AppidCPUProfilerStats>& a, const std::pair<AppId, AppidCPUProfilerStats>& b) const {
+ if (!a.second.per_appid_sessions or !b.second.per_appid_sessions)
+ return false;
+
+ return a.second.processing_time/a.second.per_appid_sessions < b.second.processing_time/b.second.per_appid_sessions;
+ }
+};
+
+AppidCpuTableDisplayStatus AppidCPUProfilingManager::display_appid_cpu_profiler_table(AppId appid, OdpContext& odp_ctxt, ControlConn* ctrlcon)
+{
+ AppidCPUProfilerOutputType output_type = OUPUT_LOGFILE;
+
+ if (odp_ctxt.is_appid_cpu_profiler_running())
+ return DISPLAY_ERROR_APPID_PROFILER_RUNNING;
+ else if (appid_cpu_profiling_table.empty())
+ return DISPLAY_ERROR_TABLE_EMPTY;
+
+ if (ctrlcon)
+ output_type = OUTPUT_CONSOLE;
+
+ auto bucket = appid_cpu_profiling_table.find(appid);
+
+ if (bucket != appid_cpu_profiling_table.end())
+ {
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, TABLE_HEADER(1));
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, columns);
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, partition);
+
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, " %5d %-15.15s %14.14s %10.10s %15.14s %11.11s %16.14s %15.14s %15.14s %10.2f\n",
+ appid, bucket->second.app_name.c_str(), FormatWithCommas(bucket->second.processing_time).c_str(), FormatWithCommas(bucket->second.processed_packets).c_str(),
+ FormatWithCommas(bucket->second.processing_time/bucket->second.processed_packets).c_str(), FormatWithCommas(bucket->second.per_appid_sessions).c_str(),
+ FormatWithCommas(bucket->second.processing_time/bucket->second.per_appid_sessions).c_str(), FormatWithCommas(bucket->second.max_processed_pkts_per_session).c_str(),
+ FormatWithCommas(bucket->second.max_processing_time_per_session).c_str(), static_cast<double>(bucket->second.processing_time) / total_processing_time * 100.0);
+ }
+ else
+ {
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL,"Appid %d not found in the table\n", appid);
+ }
+ return DISPLAY_SUCCESS;
+}
+
+AppidCpuTableDisplayStatus AppidCPUProfilingManager::display_appid_cpu_profiler_table(OdpContext& odp_ctxt, uint32_t display_rows_limit, bool override_running_flag, ControlConn* ctrlcon)
+{
+ AppidCPUProfilerOutputType output_type = OUPUT_LOGFILE;
+
+ if (odp_ctxt.is_appid_cpu_profiler_running() and !override_running_flag)
+ return DISPLAY_ERROR_APPID_PROFILER_RUNNING;
+ else if (appid_cpu_profiling_table.empty())
+ return DISPLAY_ERROR_TABLE_EMPTY;
+
+ std::priority_queue<std::pair<AppId, AppidCPUProfilerStats>, std::vector<std::pair<AppId, AppidCPUProfilerStats>>, CompareByAvgProcessingTime> sorted_appid_cpu_profiler_table;
+
+ for (const auto& entry : appid_cpu_profiling_table)
+ sorted_appid_cpu_profiler_table.push(entry);
+
+ if (ctrlcon)
+ output_type = OUTPUT_CONSOLE;
+
+ display_rows_limit = static_cast<uint32_t>(std::min({static_cast<size_t>(display_rows_limit), sorted_appid_cpu_profiler_table.size(), static_cast<size_t>(APPID_CPU_PROFILER_MAX_DISPLAY_ROWS)}));
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, TABLE_HEADER(display_rows_limit));
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, columns);
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, partition);
+
+ uint32_t rows_displayed = 0;
+
+ while (!sorted_appid_cpu_profiler_table.empty() and rows_displayed < display_rows_limit)
+ {
+ auto entry = sorted_appid_cpu_profiler_table.top();
+ sorted_appid_cpu_profiler_table.pop();
+ if (!entry.second.processed_packets or !entry.second.per_appid_sessions)
+ continue;
+
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, " %5d %-15.15s %14.14s %10.10s %15.14s %11.11s %16.14s %15.14s %15.14s %10.2f\n",
+ entry.first, entry.second.app_name.c_str(), FormatWithCommas(entry.second.processing_time).c_str(), FormatWithCommas(entry.second.processed_packets).c_str(),
+ FormatWithCommas(entry.second.processing_time/entry.second.processed_packets).c_str(), FormatWithCommas(entry.second.per_appid_sessions).c_str(),
+ FormatWithCommas(entry.second.processing_time/entry.second.per_appid_sessions).c_str(), FormatWithCommas(entry.second.max_processed_pkts_per_session).c_str(),
+ FormatWithCommas(entry.second.max_processing_time_per_session).c_str(), static_cast<double>(entry.second.processing_time) / total_processing_time * 100.0);
+
+ rows_displayed += 1;
+ }
+
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, partition);
+
+ print_log(ctrlcon, output_type, TRACE_INFO_LEVEL, "Totals(all_sessions) : %15.15s %10.10s %15.14s %10.10s %15.15s %15.14s %16.15s %9d\n",
+ FormatWithCommas(total_processing_time).c_str(), FormatWithCommas(total_processed_packets).c_str(), FormatWithCommas(total_processing_time/total_processed_packets).c_str(),
+ FormatWithCommas(total_per_appid_sessions).c_str(), FormatWithCommas(total_processing_time/total_per_appid_sessions).c_str(),
+ FormatWithCommas(max_processed_pkts_per_session).c_str(), FormatWithCommas(max_processing_time_per_session).c_str(), 100);
+
+ return DISPLAY_SUCCESS;
+}
+
+void AppidCPUProfilingManager::cleanup_appid_cpu_profiler_table()
+{
+ std::lock_guard<std::mutex> lock(appid_cpu_profiler_mutex);
+ appid_cpu_profiling_table.clear();
+ total_processing_time = 0;
+ total_processed_packets = 0;
+ total_per_appid_sessions = 0;
+ max_processing_time_per_session = 0;
+ max_processed_pkts_per_session = 0;
+}
+
+void AppidCPUProfilingManager::update_totals(const AppidCPUProfilerStats& stats)
+{
+ total_processing_time += stats.processing_time;
+ total_processed_packets += stats.processed_packets;
+ total_per_appid_sessions += stats.per_appid_sessions;
+ if (stats.max_processed_pkts_per_session >= max_processed_pkts_per_session)
+ max_processed_pkts_per_session = stats.max_processed_pkts_per_session;
+ if (stats.max_processing_time_per_session >= max_processing_time_per_session)
+ max_processing_time_per_session = stats.max_processing_time_per_session;
+}
+
+void AppidCPUProfilingManager::insert_appid_cpu_profiler_record(AppId appId, const AppidCPUProfilerStats& stats)
+{
+ std::lock_guard<std::mutex> lock(appid_cpu_profiler_mutex);
+
+ auto it = appid_cpu_profiling_table.find(appId);
+ if (it == appid_cpu_profiling_table.end())
+ {
+ appid_cpu_profiling_table.emplace(appId, stats);
+ }
+ else
+ {
+ it->second.processing_time += stats.processing_time;
+ it->second.processed_packets += stats.processed_packets;
+ it->second.per_appid_sessions += 1;
+ if (stats.processed_packets > it->second.max_processed_pkts_per_session)
+ it->second.max_processed_pkts_per_session = stats.processed_packets;
+
+ if (stats.processing_time > it->second.max_processing_time_per_session)
+ it->second.max_processing_time_per_session = stats.processing_time;
+ }
+ update_totals(stats);
+}
+
+void AppidCPUProfilingManager::check_appid_cpu_profiler_table_entry(const AppIdSession* asd, AppId payload_id)
+{
+ if (payload_id > APP_ID_NONE)
+ {
+ const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(payload_id);
+ if (app_name == nullptr)
+ app_name = "unknown";
+
+ stats_bucket_insert(payload_id, app_name, asd->stats.prev_payload_processing_time, asd->stats.prev_payload_processing_packets);
+ }
+}
+
+void AppidCPUProfilingManager::stats_bucket_insert(AppId appid, const char* app_name, uint64_t processing_time, uint64_t processed_packets)
+{
+ if (!processed_packets or !processing_time)
+ {
+ appid_log(nullptr, TRACE_INFO_LEVEL, "appid: processed packets/time are NULL for appid : %d , app_name : %s , processing time :%lu \n", appid, app_name, processing_time);
+ return;
+ }
+
+ AppidCPUProfilerStats stats(app_name, processing_time, processed_packets, 1);
+ insert_appid_cpu_profiler_record(appid, stats);
+}
+
+void AppidCPUProfilingManager::check_appid_cpu_profiler_table_entry(const AppIdSession* asd,AppId service_id, AppId client_id, AppId payload_id, AppId misc_id)
+{
+ if (!asd->stats.processing_time or !asd->stats.cpu_profiler_pkt_count)
+ return;
+
+ if (service_id > APP_ID_NONE)
+ {
+ const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(service_id);
+ if (app_name == nullptr)
+ app_name = "unknown";
+
+ stats_bucket_insert(service_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
+ }
+ if (client_id > APP_ID_NONE and client_id != service_id){
+ const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(client_id);
+ if (app_name == nullptr)
+ app_name = "unknown";
+
+ stats_bucket_insert(client_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
+ }
+ if (payload_id > APP_ID_NONE and payload_id != service_id and payload_id != client_id)
+ {
+ const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(payload_id);
+ if (app_name == nullptr)
+ app_name = "unknown";
+
+ stats_bucket_insert(payload_id, app_name, asd->stats.processing_time - asd->stats.prev_payload_processing_time, asd->stats.cpu_profiler_pkt_count - asd->stats.prev_payload_processing_packets);
+ }
+ if (misc_id > APP_ID_NONE and misc_id != service_id and misc_id != client_id and misc_id != payload_id)
+ {
+ const char* app_name = asd->get_odp_ctxt().get_app_info_mgr().get_app_name(misc_id);
+ if (app_name == nullptr)
+ app_name = "unknown";
+
+ stats_bucket_insert(misc_id, app_name, asd->stats.processing_time, asd->stats.cpu_profiler_pkt_count);
+ }
+}
#include <unordered_map>
#include <vector>
+#include <mutex>
#include "main/thread.h"
#include "utils/util.h"
class AppIdSession;
class OdpContext;
+class ControlConn;
+
+enum AppidCPUProfilerOutputType
+{
+ OUPUT_LOGFILE = 0,
+ OUTPUT_CONSOLE
+};
+
+#define APPID_CPU_PROFILER_DEFAULT_DISPLAY_ROWS 100
+#define APPID_CPU_PROFILER_MAX_DISPLAY_ROWS 2000
+
+enum AppidCpuTableDisplayStatus {
+ DISPLAY_SUCCESS = 0,
+ DISPLAY_ERROR_TABLE_EMPTY,
+ DISPLAY_ERROR_APPID_PROFILER_RUNNING
+};
struct AppidCPUProfilerStats {
std::string app_name;
uint64_t processing_time = 0;
uint64_t processed_packets = 0;
uint32_t per_appid_sessions = 0;
+ uint64_t max_processing_time_per_session = 0;
+ uint64_t max_processed_pkts_per_session = 0;
AppidCPUProfilerStats(const char* app_name, uint64_t processing_time, uint64_t processed_packets, uint32_t per_appid_sessions) :
- app_name(app_name), processing_time(processing_time), processed_packets(processed_packets), per_appid_sessions (per_appid_sessions)
+ app_name(app_name), processing_time(processing_time), processed_packets(processed_packets), per_appid_sessions(per_appid_sessions),
+ max_processing_time_per_session(processing_time), max_processed_pkts_per_session(processed_packets)
{ }
};
class AppidCPUProfilingManager {
private:
- typedef std::unordered_map<AppId, AppidCPUProfilerStats> AppidCPUProfilingTable;
+ using AppidCPUProfilingTable = std::unordered_map<AppId, AppidCPUProfilerStats>;
AppidCPUProfilingTable appid_cpu_profiling_table;
-
+ std::mutex appid_cpu_profiler_mutex;
+ uint64_t total_processing_time = 0;
+ uint64_t total_processed_packets = 0;
+ uint32_t total_per_appid_sessions = 0;
+ uint64_t max_processing_time_per_session = 0;
+ uint64_t max_processed_pkts_per_session = 0;
+
public:
- AppidCPUProfilingManager();
-
+ AppidCPUProfilingManager() = default;
+
void stats_bucket_insert(AppId appid, const char* app_name, uint64_t processing_time, uint64_t processed_packets);
void insert_appid_cpu_profiler_record(AppId appId, const AppidCPUProfilerStats& stats);
void check_appid_cpu_profiler_table_entry(const AppIdSession* asd, AppId service_id, AppId client_id, AppId payload_id, AppId misc_id);
void check_appid_cpu_profiler_table_entry(const AppIdSession* asd, AppId payload_id);
+ void update_totals(const AppidCPUProfilerStats& stats);
+
+ AppidCpuTableDisplayStatus display_appid_cpu_profiler_table(OdpContext&, uint32_t display_rows_limit = APPID_CPU_PROFILER_DEFAULT_DISPLAY_ROWS,
+ bool override_running_flag = false, ControlConn* control_conn = nullptr);
+ AppidCpuTableDisplayStatus display_appid_cpu_profiler_table(AppId, OdpContext&, ControlConn* control_conn = nullptr);
- void display_appid_cpu_profiler_table();
- void display_appid_cpu_profiler_table(AppId appid);
-
void cleanup_appid_cpu_profiler_table();
};
#endif
// two key->version here to create the proper debug_session string.
activate(key->ip_l, key->ip_h, key->port_l, key->port_h, (IpProtocol)(key->ip_protocol),
key->version, key->addressSpaceId, session, log_all_sessions,
- key->tenant_id, key->group_l, key->group_h, key->flags.group_used);
+#ifndef DISABLE_TENANT_ID
+ key->tenant_id,
+#else
+ 0,
+#endif
+ key->group_l, key->group_h, key->flags.group_used);
}
void AppIdDebug::set_constraints(const char *desc,
return APPID_SUCCESS;
}
-void AppIdDetector::reload()
-{
- do_custom_reload();
-}
-
-void* AppIdDetector::data_get(AppIdSession& asd)
+void* AppIdDetector::data_get(const AppIdSession& asd)
{
return asd.get_flow_data(flow_data_index);
}
AppIdDetector() = default;
virtual ~AppIdDetector() = default;
- virtual int initialize(AppIdInspector&);
- virtual void reload();
+ int initialize(AppIdInspector&);
virtual void do_custom_init() { }
virtual void do_custom_reload() { }
virtual int validate(AppIdDiscoveryArgs&) = 0;
virtual void register_appid(AppId, unsigned extractsInfo, OdpContext& odp_ctxt) = 0;
- virtual void* data_get(AppIdSession&);
- virtual int data_add(AppIdSession&, void*, AppIdFreeFCN);
- virtual void add_user(AppIdSession&, const char*, AppId, bool, AppidChangeBits&);
- virtual void add_payload(AppIdSession&, AppId);
- virtual void add_app(AppIdSession& asd, AppId service_id, AppId client_id, const char* version, AppidChangeBits& change_bits)
+ void* data_get(const AppIdSession&);
+ int data_add(AppIdSession&, void*, AppIdFreeFCN);
+ void add_user(AppIdSession&, const char*, AppId, bool, AppidChangeBits&);
+ void add_payload(AppIdSession&, AppId);
+ void add_app(AppIdSession& asd, AppId service_id, AppId client_id, const char* version, AppidChangeBits& change_bits)
{
if ( version )
asd.set_client_version(version, change_bits);
asd.client_inferred_service_id = service_id;
asd.set_client_id(client_id);
}
- virtual void add_app(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppId, const char*, AppidChangeBits&);
+ void add_app(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppId, const char*, AppidChangeBits&);
const char* get_code_string(APPID_STATUS_CODE) const;
const std::string& get_name() const
#include "appid_http_session.h"
#include "appid_inspector.h"
#include "appid_session.h"
-#include "app_cpu_profile_table.h"
+#include "appid_cpu_profile_table.h"
#include "appid_utils/ip_funcs.h"
#include "client_plugins/client_discovery.h"
#include "detector_plugins/detector_dns.h"
return false;
}
+ const AppIdConfig& config = inspector.get_config();
if (appidDebug->is_enabled())
- appidDebug->activate(p->flow, asd,
- inspector.get_ctxt().config.log_all_sessions);
+ appidDebug->activate(p->flow, asd, config.log_all_sessions);
if (is_packet_ignored(p))
return false;
// FIXIT-L: DECRYPT_DEBUG - Move set_proxied and first_decrypted_packet_debug to ssl-module
// after ssl-module's decryption capability is implemented
#ifdef REG_TEST
- uint32_t fdpd = inspector.get_ctxt().config.first_decrypted_packet_debug;
+ uint32_t fdpd = config.first_decrypted_packet_debug;
if (fdpd and (fdpd == asd->session_packet_count))
{
p->flow->set_proxied();
virtual void initialize(AppIdInspector&) = 0;
virtual void reload() = 0;
- virtual void register_detector(const std::string&, AppIdDetector*, IpProtocol);
- virtual void add_pattern_data(AppIdDetector*, snort::SearchTool&, int position,
+ void register_detector(const std::string&, AppIdDetector*, IpProtocol);
+ void add_pattern_data(AppIdDetector*, snort::SearchTool&, int position,
const uint8_t* const pattern, unsigned size, unsigned nocase);
- virtual void register_tcp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
+ void register_tcp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
int position, unsigned nocase);
- virtual void register_udp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
+ void register_udp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
int position, unsigned nocase);
virtual int add_service_port(AppIdDetector*, const ServiceDetectorPort&);
AppIdSession* asd = new AppIdSession(static_cast<IpProtocol>(key->ip_protocol),
flow.flags.client_initiated ? &flow.client_ip : &flow.server_ip,
flow.flags.client_initiated ? flow.client_port : flow.server_port, inspector,
- *pkt_thread_odp_ctxt, key->addressSpaceId, key->tenant_id);
+ *pkt_thread_odp_ctxt, key->addressSpaceId
+#ifndef DISABLE_TENANT_ID
+ ,flow.key->tenant_id
+#endif
+ );
+
appid_log(CURRENT_PACKET, TRACE_DEBUG_LEVEL, "high-avail - New AppId session created in consume\n");
flow.set_flow_data(asd);
#include "flow/stream_flow.h"
#include "app_info_table.h"
-#include "app_cpu_profile_table.h"
+#include "appid_cpu_profile_table.h"
#include "appid_debug.h"
#include "appid_discovery.h"
#include "appid_http_session.h"
auto direction = event_type == REQUEST_EVENT ? APP_ID_FROM_INITIATOR : APP_ID_FROM_RESPONDER;
bool is_debug_active = false;
+ const AppIdConfig& config = inspector.get_config();
if ( !asd )
{
// The event is received before appid has seen any packet, e.g., data on SYN
per_appid_event_cpu_timer.start();
if (appidDebug->is_enabled() and !is_debug_active)
- appidDebug->activate(flow, asd, inspector.get_ctxt().config.log_all_sessions);
+ appidDebug->activate(flow, asd, config.log_all_sessions);
appid_log(p, TRACE_DEBUG_LEVEL, "Processing HTTP metadata from HTTP Inspector for stream %" PRId64 "\n",
http_event->get_httpx_stream_id());
using namespace snort;
THREAD_LOCAL ThirdPartyAppIdContext* pkt_thread_tp_appid_ctxt = nullptr;
-THREAD_LOCAL OdpThreadContext* odp_thread_local_ctxt = nullptr;
+OdpControlContext* odp_control_thread_ctxt = nullptr;
+THREAD_LOCAL OdpPacketThreadContext* odp_thread_local_ctxt = nullptr;
THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt = nullptr;
unsigned AppIdInspector::cached_global_pub_id = 0;
(misc_name ? misc_name : ""), misc_id);
}
-AppIdInspector::AppIdInspector(AppIdModule& mod)
+AppIdInspector::AppIdInspector(AppIdModule& mod) : config(mod.get_data()), ctxt(*config)
{
- config = mod.get_data();
- assert(config);
}
AppIdInspector::~AppIdInspector()
{
- delete ctxt;
delete config;
}
-AppIdContext& AppIdInspector::get_ctxt() const
-{
- assert(ctxt);
- return *ctxt;
-}
-unsigned AppIdInspector::get_pub_id()
+unsigned AppIdInspector::get_pub_id()
{
return appid_pub_id;
}
bool AppIdInspector::configure(SnortConfig* sc)
{
- assert(!ctxt);
// cppcheck-suppress unreadVariable
Profile profile(appid_perf_stats);
struct rusage ru;
}
#endif
- ctxt = new AppIdContext(const_cast<AppIdConfig&>(*config));
- ctxt->init_appid(sc, *this);
+ assert(sc);
+ config->map_app_names_to_snort_ids(*sc);
+ ctxt.init_appid(sc, *this);
#ifdef REG_TEST
if ( config->log_memory_and_pattern_count )
appid_log(nullptr, TRACE_ERROR_LEVEL, "appid: fetching memory usage failed\n");
else
appid_log(nullptr, TRACE_INFO_LEVEL, "appid: MaxRss diff: %li\n", ru.ru_maxrss - prev_maxrss);
- appid_log(nullptr, TRACE_INFO_LEVEL, "appid: patterns loaded: %u\n", ctxt->get_odp_ctxt().get_pattern_count());
+ appid_log(nullptr, TRACE_INFO_LEVEL, "appid: patterns loaded: %u\n", ctxt.get_odp_ctxt().get_pattern_count());
#ifdef REG_TEST
}
#endif
AppIdStatistics::initialize_manager(*config);
assert(!pkt_thread_odp_ctxt);
- pkt_thread_odp_ctxt = &(ctxt->get_odp_ctxt());
+ pkt_thread_odp_ctxt = &ctxt.get_odp_ctxt();
assert(!odp_thread_local_ctxt);
- odp_thread_local_ctxt = new OdpThreadContext();
- odp_thread_local_ctxt->initialize(SnortConfig::get_conf(), *ctxt);
+ odp_thread_local_ctxt = new OdpPacketThreadContext;
+ odp_thread_local_ctxt->initialize(SnortConfig::get_conf());
AppIdServiceState::initialize(config->memcap);
assert(!pkt_thread_tp_appid_ctxt);
- pkt_thread_tp_appid_ctxt = ctxt->get_tp_appid_ctxt();
+ pkt_thread_tp_appid_ctxt = ctxt.get_tp_appid_ctxt();
if (pkt_thread_tp_appid_ctxt)
pkt_thread_tp_appid_ctxt->tinit();
- if (ctxt->config.log_all_sessions)
+ if (config->log_all_sessions)
appidDebug->set_enabled(true);
if ( snort::HighAvailabilityManager::active() )
AppIdHAManager::tinit();
void tterm() override;
void tear_down(snort::SnortConfig*) override;
void eval(snort::Packet*) override;
- AppIdContext& get_ctxt() const;
- const AppIdConfig& get_config() const { return *config; }
+ AppIdContext& get_ctxt()
+ { return ctxt; }
+
+ const AppIdConfig& get_config() const
+ { return *config; }
static unsigned get_pub_id();
private:
- const AppIdConfig* config = nullptr;
- AppIdContext* ctxt = nullptr;
+ AppIdConfig* config = nullptr;
+ AppIdContext ctxt;
static unsigned cached_global_pub_id;
};
extern const snort::InspectApi appid_inspector_api;
-extern THREAD_LOCAL OdpThreadContext* odp_thread_local_ctxt;
+extern OdpControlContext* odp_control_thread_ctxt;
+extern THREAD_LOCAL OdpPacketThreadContext* odp_thread_local_ctxt;
extern THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt;
extern THREAD_LOCAL ThirdPartyAppIdContext* pkt_thread_tp_appid_ctxt;
#include "appid_inspector.h"
#include "appid_peg_counts.h"
#include "service_state.h"
+#include "appid_cpu_profile_table.h"
using namespace snort;
using namespace std;
{
public:
bool execute(Analyzer&, void**) override;
- ACThirdPartyAppIdContextSwap(const AppIdInspector& inspector, ControlConn* conn)
+ ACThirdPartyAppIdContextSwap(AppIdInspector& inspector, ControlConn* conn)
: AnalyzerCommand(conn), inspector(inspector)
{
appid_log(nullptr, TRACE_INFO_LEVEL, "== swapping third-party configuration\n");
~ACThirdPartyAppIdContextSwap() override;
const char* stringify() override { return "THIRD-PARTY_CONTEXT_SWAP"; }
private:
- const AppIdInspector& inspector;
+ AppIdInspector& inspector;
};
bool ACThirdPartyAppIdContextSwap::execute(Analyzer&, void**)
pkt_thread_tp_appid_ctxt = inspector.get_ctxt().get_tp_appid_ctxt();
pkt_thread_tp_appid_ctxt->tinit();
ThirdPartyAppIdContext::set_tp_reload_in_progress(false);
-
+ appid_log(nullptr, TRACE_INFO_LEVEL, "== third-party context swap in progress\n");
return true;
}
{
public:
bool execute(Analyzer&, void**) override;
- ACThirdPartyAppIdContextUnload(const AppIdInspector& inspector, ThirdPartyAppIdContext* tp_ctxt,
+ ACThirdPartyAppIdContextUnload(AppIdInspector& inspector, ThirdPartyAppIdContext* tp_ctxt,
ControlConn* conn): AnalyzerCommand(conn), inspector(inspector), tp_ctxt(tp_ctxt)
{ }
~ACThirdPartyAppIdContextUnload() override;
const char* stringify() override { return "THIRD-PARTY_CONTEXT_UNLOAD"; }
private:
- const AppIdInspector& inspector;
+ AppIdInspector& inspector;
ThirdPartyAppIdContext* tp_ctxt = nullptr;
};
reload_in_progress = pkt_thread_tp_appid_ctxt->tfini(true);
else
reload_in_progress = pkt_thread_tp_appid_ctxt->tfini();
- if (reload_in_progress)
+
+ if (reload_in_progress) {
+ appid_log(nullptr, TRACE_INFO_LEVEL, "== rescheduling third-party context unload\n");
return false;
+ }
pkt_thread_tp_appid_ctxt = nullptr;
+ appid_log(nullptr, TRACE_INFO_LEVEL, "== third-party context unload in progress\n");
return true;
}
{
public:
bool execute(Analyzer&, void**) override;
- ACOdpContextSwap(const AppIdInspector& inspector, OdpContext& odp_ctxt, ControlConn* conn) :
+ ACOdpContextSwap(AppIdInspector& inspector, OdpContext& odp_ctxt, ControlConn* conn) :
AnalyzerCommand(conn), inspector(inspector), odp_ctxt(odp_ctxt)
{ }
~ACOdpContextSwap() override;
const char* stringify() override { return "ODP_CONTEXT_SWAP"; }
private:
- const AppIdInspector& inspector;
+ AppIdInspector& inspector;
OdpContext& odp_ctxt;
};
bool ACOdpContextSwap::execute(Analyzer&, void**)
{
- AppIdContext& ctxt = inspector.get_ctxt();
- OdpContext& current_odp_ctxt = ctxt.get_odp_ctxt();
- assert(pkt_thread_odp_ctxt != ¤t_odp_ctxt);
-
HostAttributesManager::clear_appid_services();
AppIdServiceState::clean();
AppIdPegCounts::cleanup_pegs();
- AppIdServiceState::initialize(ctxt.config.memcap);
+ const AppIdConfig& config = inspector.get_config();
+ AppIdServiceState::initialize(config.memcap);
AppIdPegCounts::init_pegs();
ServiceDiscovery::set_thread_local_ftp_service();
+ AppIdContext& ctxt = inspector.get_ctxt();
+ OdpContext& current_odp_ctxt = ctxt.get_odp_ctxt();
+ assert(pkt_thread_odp_ctxt != ¤t_odp_ctxt);
pkt_thread_odp_ctxt = ¤t_odp_ctxt;
assert(odp_thread_local_ctxt);
delete odp_thread_local_ctxt;
- odp_thread_local_ctxt = new OdpThreadContext;
- odp_thread_local_ctxt->initialize(SnortConfig::get_conf(), ctxt, false, true);
+ odp_thread_local_ctxt = new OdpPacketThreadContext;
+ odp_thread_local_ctxt->initialize(SnortConfig::get_conf());
return true;
}
delete &odp_ctxt;
AppIdContext& ctxt = inspector.get_ctxt();
- LuaDetectorManager::cleanup_after_swap();
+ ControlLuaDetectorManager::cleanup_after_swap();
if (ctxt.config.app_detector_dir)
{
std::string file_path = std::string(ctxt.config.app_detector_dir) + "/custom/userappid.conf";
}
}
-
static int show_cpu_profiler_stats(lua_State* L)
{
int appid = luaL_optint(L, 1, 0);
+ int display_rows_limit = luaL_optint(L, 2, APPID_CPU_PROFILER_DEFAULT_DISPLAY_ROWS);
+
ControlConn* ctrlcon = ControlConn::query_from_lua(L);
AppIdInspector* inspector = (AppIdInspector*) InspectorManager::get_inspector(MOD_NAME);
if (!inspector)
return 0;
}
const AppIdContext& ctxt = inspector->get_ctxt();
- OdpContext& odp_ctxt = ctxt.get_odp_ctxt();
+ OdpContext& odp_ctxt = ctxt.get_odp_ctxt();
+
if (odp_ctxt.is_appid_cpu_profiler_enabled())
{
+ AppidCpuTableDisplayStatus displayed = DISPLAY_SUCCESS;
ctrlcon->respond("== showing appid cpu profiler table\n");
if (!appid)
- odp_ctxt.get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table();
+ {
+ if (display_rows_limit > APPID_CPU_PROFILER_MAX_DISPLAY_ROWS)
+ ctrlcon->respond("given number of rows exceeds maximum limit of %d, limiting to %d\n",
+ APPID_CPU_PROFILER_MAX_DISPLAY_ROWS, APPID_CPU_PROFILER_MAX_DISPLAY_ROWS);
+ displayed = odp_ctxt.get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table(odp_ctxt, display_rows_limit, false, ctrlcon);
+ }
else
- odp_ctxt.get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table(appid);
+ displayed = odp_ctxt.get_appid_cpu_profiler_mgr().display_appid_cpu_profiler_table(appid, odp_ctxt, ctrlcon);
+
+ switch (displayed){
+ case DISPLAY_ERROR_TABLE_EMPTY:
+ ctrlcon->respond("== appid cpu profiler table is empty\n");
+ break;
+ case DISPLAY_ERROR_APPID_PROFILER_RUNNING:
+ ctrlcon->respond("== appid cpu profiler is still running\n");
+ break;
+ case DISPLAY_SUCCESS:
+ break;
+ }
}
else
ctrlcon->respond("appid cpu profiler is disabled\n");
}
const AppIdContext& ctxt = inspector->get_ctxt();
OdpContext& odp_ctxt = ctxt.get_odp_ctxt();
- ctrlcon->respond("appid cpu profiler enabled: %s , running: %s \n",
+ ctrlcon->respond("appid cpu profiler enabled: %s, running: %s \n",
odp_ctxt.is_appid_cpu_profiler_enabled() ? "yes" : "no", odp_ctxt.is_appid_cpu_profiler_running() ? "yes" : "no");
return 0;
}
clear_dynamic_host_cache_services();
AppIdPegCounts::cleanup_peg_info();
AppIdPegCounts::init_peg_info();
- LuaDetectorManager::clear_lua_detector_mgrs();
+ ControlLuaDetectorManager::clear_lua_detector_mgrs();
ctxt.create_odp_ctxt();
- assert(odp_thread_local_ctxt);
- odp_thread_local_ctxt->get_lua_detector_mgr().set_ignore_chp_cleanup(true);
- delete odp_thread_local_ctxt;
- odp_thread_local_ctxt = new OdpThreadContext;
+ assert(odp_control_thread_ctxt);
+ odp_control_thread_ctxt->set_ignore_chp_cleanup();
+ delete odp_control_thread_ctxt;
+ odp_control_thread_ctxt = new OdpControlContext;
OdpContext& odp_ctxt = ctxt.get_odp_ctxt();
odp_ctxt.get_client_disco_mgr().initialize(*inspector);
odp_ctxt.get_service_disco_mgr().initialize(*inspector);
odp_ctxt.set_client_and_service_detectors();
- odp_thread_local_ctxt->initialize(SnortConfig::get_conf(), ctxt, true, true);
+ odp_control_thread_ctxt->initialize(SnortConfig::get_conf(), ctxt);
odp_ctxt.initialize(*inspector);
ctrlcon->respond("== swapping detectors configuration\n");
static const Parameter appid_cpu_params[] =
{
- { "appid", Parameter::PT_INT, nullptr, nullptr, "show appid cpu profiling stats" },
+ { "appid", Parameter::PT_INT, nullptr, "0", "show appid cpu profiling stats" },
+ { "display_rows_limit", Parameter::PT_INT, "1:2000", "100", "num of rows to be displayed" },
{ nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
};
};
AppIdModule::AppIdModule() : Module(MOD_NAME, MOD_HELP, s_params)
-{ config = nullptr; }
+{
+}
+
+AppIdModule::~AppIdModule()
+{
+ AppIdPegCounts::cleanup_peg_info();
+ delete config;
+}
void AppIdModule::set_trace(const Trace* trace) const
{ appid_trace = trace; }
return nullptr;
}
-const AppIdConfig* AppIdModule::get_data()
+AppIdConfig* AppIdModule::get_data()
{
AppIdConfig* temp = config;
config = nullptr;
{
public:
AppIdModule();
- ~AppIdModule() override = default;
+ ~AppIdModule() override;
bool begin(const char*, int, snort::SnortConfig*) override;
bool set(const char*, snort::Value&, snort::SnortConfig*) override;
snort::ProfileStats* get_profile(
unsigned i, const char*& name, const char*& parent) const override;
- const AppIdConfig* get_data();
+ AppIdConfig* get_data();
void reset_stats() override;
const snort::TraceOption* get_trace_options() const override;
private:
- AppIdConfig* config;
+ AppIdConfig* config = nullptr;
};
class ACThirdPartyAppIdCleanup : public snort::AnalyzerCommand
port = (direction == APP_ID_FROM_INITIATOR) ? p->ptrs.sp : p->ptrs.dp;
AppIdSession* asd = new AppIdSession(proto, ip, port, inspector, odp_context,
- p->pkth->address_space_id, p->pkth->tenant_id);
+ p->pkth->address_space_id
+#ifndef DISABLE_TENANT_ID
+ ,p->pkth->tenant_id
+#endif
+ );
is_session_monitored(asd->flags, p, inspector);
asd->flow = p->flow;
asd->stats.first_packet_second = p->pkth->ts.tv_sec;
}
AppIdSession::AppIdSession(IpProtocol proto, const SfIp* ip, uint16_t port,
- AppIdInspector& inspector, OdpContext& odp_ctxt, uint32_t asid, uint32_t tenant_id)
+ AppIdInspector& inspector, OdpContext& odp_ctxt, uint32_t asid
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t tenant_id
+#endif
+ )
: FlowData(inspector_id, &inspector), config(inspector.get_ctxt().config),
- initiator_port(port), tenant_id(tenant_id), asid(asid), protocol(proto),
+ initiator_port(port),
+#ifndef DISABLE_TENANT_ID
+ tenant_id(tenant_id),
+#endif
+ asid(asid), protocol(proto),
api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(odp_ctxt),
odp_ctxt_version(odp_ctxt.get_version()),
tp_appid_ctxt(pkt_thread_tp_appid_ctxt)
// Skip sessions using old odp context after reload detectors for appid cpu profiling
if ((pkt_thread_odp_ctxt->get_version() == api.asd->get_odp_ctxt_version()) and api.asd->get_odp_ctxt().is_appid_cpu_profiler_running())
{
- api.asd->get_odp_ctxt().get_appid_cpu_profiler_mgr().check_appid_cpu_profiler_table_entry(api.asd, api.service.get_id(), api.client.get_id(), api.payload.get_id(), api.get_misc_app_id());
+ api.asd->get_odp_ctxt().get_appid_cpu_profiler_mgr().check_appid_cpu_profiler_table_entry(api.asd, api.get_service_app_id(), api.get_client_app_id(), api.get_payload_app_id(), api.get_misc_app_id());
}
if (!in_expected_cache)
// FIXIT-RC - port parameter passed in as 0 since we may not know client port, verify
AppIdSession* asd = new AppIdSession(proto, cliIp, 0, *inspector, odp_ctxt,
- ctrlPkt->pkth->address_space_id, ctrlPkt->pkth->tenant_id);
+ ctrlPkt->pkth->address_space_id
+#ifndef DISABLE_TENANT_ID
+ ,ctrlPkt->pkth->tenant_id
+#endif
+ );
is_session_monitored(asd->flags, ctrlPkt, *inspector);
if (Stream::set_snort_protocol_id_expected(ctrlPkt, type, proto, cliIp,
if (odp_ctxt.is_appid_cpu_profiler_running())
{
- odp_ctxt.get_appid_cpu_profiler_mgr().check_appid_cpu_profiler_table_entry(api.asd, api.service.get_id(), api.client.get_id(), api.payload.get_id(), api.get_misc_app_id());
+ odp_ctxt.get_appid_cpu_profiler_mgr().check_appid_cpu_profiler_table_entry(api.asd, api.get_service_app_id(), api.get_client_app_id(), api.get_payload_app_id(), api.get_misc_app_id());
this->stats.processing_time = 0;
this->stats.cpu_profiler_pkt_count = 0;
}
AppIdHttpSession* AppIdSession::get_http_session(uint32_t stream_index) const
{
if (stream_index < api.hsessions.size())
- return api.hsessions[stream_index];
+ return api.hsessions[stream_index].get();
else
return nullptr;
}
AppIdHttpSession* AppIdSession::create_http_session(int64_t stream_id)
{
- AppIdHttpSession* hsession = new AppIdHttpSession(*this, stream_id);
- api.hsessions.push_back(hsession);
- return hsession;
+ auto hsession = std::make_unique<AppIdHttpSession>(*this, stream_id);
+ auto tmp_hsession = hsession.get();
+ api.hsessions.push_back(std::move(hsession));
+ return tmp_hsession;
}
AppIdHttpSession* AppIdSession::get_matching_http_session(int64_t stream_id) const
for (uint32_t stream_index=0; stream_index < api.hsessions.size(); stream_index++)
{
if(stream_id == api.hsessions[stream_index]->get_httpx_stream_id())
- return api.hsessions[stream_index];
+ return api.hsessions[stream_index].get();
}
return nullptr;
}
snort_free(tls_cname);
if (tls_org_unit)
snort_free(tls_org_unit);
+ if (tls_host_mismatch)
+ snort_free(tls_host_mismatch);
}
const char* get_tls_host() const
return nullptr;
}
+ const char* get_tls_sni() const
+ {
+ return tls_host_mismatch ? tls_host_mismatch : tls_host;
+ }
+
+ void process_sni_mismatch()
+ {
+ if(tls_host)
+ {
+ if(tls_host_mismatch)
+ snort_free(tls_host_mismatch);
+ tls_host_mismatch = tls_host;
+ tls_host = nullptr;
+ }
+ }
+
const char* get_tls_first_alt_name() const { return tls_first_alt_name; }
const char* get_tls_cname() const { return tls_cname; }
private:
char* tls_host = nullptr;
+ char* tls_host_mismatch = nullptr;
char* tls_first_alt_name = nullptr;
char* tls_cname = nullptr;
char* tls_org_unit = nullptr;
{
public:
AppIdSession(IpProtocol, const snort::SfIp*, uint16_t port, AppIdInspector&,
- OdpContext&, uint32_t asid, uint32_t tenant_id);
+ OdpContext&, uint32_t asid
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t tenant_id
+#endif
+ );
~AppIdSession() override;
static AppIdSession* allocate_session(const snort::Packet*, IpProtocol,
std::unordered_map<unsigned, AppIdFlowData*> flow_data;
uint64_t flags = 0;
uint16_t initiator_port = 0;
+#ifndef DISABLE_TENANT_ID
uint32_t tenant_id = 0;
+#endif
uint32_t asid = 0;
uint16_t session_packet_count = 0;
void set_tls_host(const AppidChangeBits& change_bits)
{
if (tsession and change_bits[APPID_TLSHOST_BIT])
+ {
api.set_tls_host(tsession->get_tls_host());
+ api.set_tls_sni(tsession->get_tls_sni());
+ }
}
void set_tls_host(const char* tls_host)
void set_tls_host()
{
if (tsession and tsession->is_tls_host_unpublished())
+ {
api.set_tls_host(tsession->get_tls_host());
+ api.set_tls_sni(tsession->get_tls_sni());
+ }
}
void set_netbios_name(AppidChangeBits& change_bits, const char *name)
for (uint32_t stream_index=0; stream_index < hsessions.size(); stream_index++)
{
if(stream_id == hsessions[stream_index]->get_httpx_stream_id())
- return hsessions[stream_index];
+ return hsessions[stream_index].get();
}
return nullptr;
}
AppIdHttpSession* AppIdSessionApi::get_hsession(uint32_t stream_index) const
{
if (stream_index < hsessions.size())
- return hsessions[stream_index];
+ return hsessions[stream_index].get();
else
return nullptr;
}
void clear_user_logged_in() { flags.user_logged_in = false; }
+ const char* get_tls_sni() const { return tls_sni; }
+
protected:
AppIdSessionApi(const AppIdSession* asd, const SfIp& ip);
bool finished : 1;
bool user_logged_in : 1;
} flags = {};
- std::vector<AppIdHttpSession*> hsessions;
+ std::vector<std::unique_ptr<AppIdHttpSession>> hsessions;
AppIdDnsSession* dsession = nullptr;
snort::SfIp initiator_ip;
ServiceAppDescriptor service;
char* tls_host = nullptr;
+ char* tls_sni = nullptr;
char* netbios_name = nullptr;
char* netbios_domain = nullptr;
std::string session_id;
snort_free(tls_host);
snort_free(netbios_name);
snort_free(netbios_domain);
+ snort_free(tls_sni);
delete dsession;
}
void delete_all_http_sessions()
{
- for (auto hsession : hsessions)
- delete hsession;
hsessions.clear();
}
}
}
+ void set_tls_sni(const char* sni)
+ {
+ if (sni and sni != tls_sni)
+ {
+ if (tls_sni)
+ snort_free(tls_sni);
+ tls_sni = snort_strdup(sni);
+ }
+ }
+
friend AppIdSession;
};
void ClientDiscovery::reload()
{
for ( auto& kv : tcp_detectors )
- kv.second->reload();
+ kv.second->do_custom_reload();
for ( auto& kv : udp_detectors )
- kv.second->reload();
+ kv.second->do_custom_reload();
}
void ClientDiscovery::finalize_client_patterns()
return nullptr;
}
-AppIdContext* ctxt;
-AppIdContext& AppIdInspector::get_ctxt() const { return *ctxt; }
void appid_log(const snort::Packet*, unsigned char, char const*, ...) { }
TEST_GROUP(eve_ca_patterns_tests)
#ifndef DETECTOR_PLUGINS_MOCK_H
#define DETECTOR_PLUGINS_MOCK_H
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
#include "log/messages.h"
#include "utils/stats.h"
AppIdConfig::~AppIdConfig() = default;
AppIdModule::AppIdModule()
: Module("a", "b") { }
+AppIdModule::~AppIdModule() = default;
// LCOV_EXCL_START
bool AppIdModule::begin(const char*, int, snort::SnortConfig*)
AppIdContext stub_ctxt(stub_config);
OdpContext stub_odp_ctxt(stub_config, nullptr);
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector& inspector,
- OdpContext& odpctxt, uint32_t, uint32_t) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
+ OdpContext& odpctxt, uint32_t
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t
+#endif
+ ) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
config(stub_config), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(odpctxt)
{
this->set_session_flags(APPID_SESSION_DISCOVER_APP);
return nullptr;
}
-
-AppidCPUProfilingManager::AppidCPUProfilingManager() { }
-
bool AppIdReloadTuner::tinit() { return false; }
bool AppIdReloadTuner::tune_resources(unsigned int)
#include <CppUTest/TestHarness.h>
#include <CppUTestExt/MockSupport.h>
-static AppIdConfig config;
-static AppIdContext context(config);
+static AppIdConfig s_config;
+static AppIdContext context(s_config);
OdpContext* AppIdContext::odp_ctxt = nullptr;
static AppIdModule appid_mod;
static AppIdInspector appid_inspector(appid_mod);
unsigned ThreadConfig::get_instance_max() { return 1; }
}
-AppIdInspector::AppIdInspector(AppIdModule&) { }
+AppIdInspector::AppIdInspector(AppIdModule&) : config(&s_config), ctxt(s_config)
+{ }
bool AppIdInspector::configure(snort::SnortConfig*)
{
- ctxt = &context;
return true;
}
void AppIdInspector::tinit() { }
void AppIdInspector::tterm() { }
void AppIdInspector::tear_down(SnortConfig*) { }
-AppIdContext& AppIdInspector::get_ctxt() const { return *ctxt; }
// LCOV_EXCL_STOP
AppIdInspector::~AppIdInspector() = default;
void AppIdContext::create_odp_ctxt()
{
- odp_ctxt = new OdpContext(config, nullptr);
+ odp_ctxt = new OdpContext(s_config, nullptr);
}
void AppIdContext::pterm() { delete odp_ctxt; }
AppIdSession* AppIdSession::allocate_session(snort::Packet const*, IpProtocol,
AppidSessionDirection, AppIdInspector&, OdpContext& odp_ctxt)
{
- session = new AppIdSession(IpProtocol::IP, &sfip, 0, appid_inspector, odp_ctxt, 0, 0);
+ session = new AppIdSession(IpProtocol::IP, &sfip, 0, appid_inspector, odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0 // tenant_id
+#endif
+ );
return session;
}
// LCOV_EXCL_START
void ClientDetector::register_appid(int, unsigned int, OdpContext&) { }
int AppIdDetector::initialize(AppIdInspector&) { return 1; }
-void AppIdDetector::reload() { }
int AppIdDetector::data_add(AppIdSession&, void*, void (*)(void*)) { return 1; }
void AppIdDetector::add_user(AppIdSession&, char const*, int, bool, AppidChangeBits&) { }
void AppIdDetector::add_payload(AppIdSession&, int) { }
int SipPatternMatchers::get_client_from_ua(char const*, unsigned int, int&, char*&) { return 0; } // LCOV_EXCL_LINE
void SipEventHandler::service_handler(SipEvent&, AppIdSession&, AppidChangeBits&) { }
-void* AppIdDetector::data_get(AppIdSession&)
+void* AppIdDetector::data_get(const AppIdSession&)
{
sip_data = new ClientSIPData();
sip_data->from = "<sip:1001@51.1.1.130:11810>";
static SfIp sfip;
static AppIdModule appid_mod;
static AppIdInspector appid_inspector(appid_mod);
-static AppIdSession session(IpProtocol::IP, &sfip, 0, appid_inspector, odpctxt, 0, 0);
+static AppIdSession session(IpProtocol::IP, &sfip, 0, appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+,0
+#endif
+);
static AppIdHttpSession mock_hsession(session, 0);
static ChpMatchDescriptor cmd_test;
static MatchedCHPAction mchp;
void init_chp_glossary()
{
- if(CHP_glossary)
- old_CHP_glossary = CHP_glossary;
+ assert(!old_CHP_glossary);
+ old_CHP_glossary = CHP_glossary;
CHP_glossary = new CHPGlossary;
}
static void free_chp_glossary(CHPGlossary*& glossary)
{
- if (!glossary)
- return;
-
- for (auto& entry : *glossary)
+ if (glossary)
{
- if (entry.second)
- snort_free(entry.second);
+ for (auto& entry : *glossary)
+ delete entry.second;
+ delete glossary;
+ glossary = nullptr;
}
- delete glossary;
- glossary = nullptr;
}
-void free_current_chp_glossary(){
+void free_current_chp_glossary()
+{
free_chp_glossary(CHP_glossary);
}
// Verify detector user data and that we are in packet context
LuaStateDescriptor* lsd = ud->validate_lua_state(true);
- auto df = odp_thread_local_ctxt->get_lua_detector_mgr().get_detector_flow();
+ auto df = odp_thread_local_ctxt->get_detector_flow();
if (!df)
{
df = new DetectorFlow(L, lsd->ldp.asd);
- odp_thread_local_ctxt->get_lua_detector_mgr().set_detector_flow(df);
+ odp_thread_local_ctxt->set_detector_flow(df);
}
UserData<DetectorFlow>::push(L, DETECTORFLOW, df);
lua_pushvalue(L, -1);
// Note that Lua detector objects are thread local
ud.set_cb_fn_name(callback);
- if (!odp_thread_local_ctxt->get_lua_detector_mgr().insert_cb_detector(app_id, &ud))
+ if (!odp_thread_local_ctxt->insert_cb_detector(app_id, &ud))
{
appid_log(nullptr, TRACE_ERROR_LEVEL, "AppId: detector callback already registered for app %d\n", app_id);
return 1;
return -10;
}
- LuaDetectorManager& lua_detector_mgr = odp_thread_local_ctxt->get_lua_detector_mgr();
- auto my_lua_state = lua_detector_mgr.L;
+ auto my_lua_state = odp_thread_local_ctxt->get_lua_state();
// when an ODP detector triggers the detector callback to be called, there are some elements
// in the stack. Checking here to make sure the number of elements is not too many
if (lua_gettop(my_lua_state) > 20)
}
// detector flows must be destroyed after each packet is processed
- if (lua_detector_mgr.get_detector_flow())
- lua_detector_mgr.free_detector_flow();
+ odp_thread_local_ctxt->free_detector_flow();
// retrieve result
if (!lua_isnumber(my_lua_state, -1))
if (entry->flags & APPINFO_FLAG_CLIENT_DETECTOR_CALLBACK or
entry->flags & APPINFO_FLAG_SERVICE_DETECTOR_CALLBACK)
{
- LuaObject* ud = odp_thread_local_ctxt->get_lua_detector_mgr().get_cb_detector(app_id);
+ LuaObject* ud = odp_thread_local_ctxt->get_cb_detector(app_id);
assert(ud);
if (ud->is_running())
static int create_chp_application(AppId appIdInstance, unsigned app_type_flags, int num_matches)
{
- CHPApp* new_app = (CHPApp*)snort_calloc(sizeof(CHPApp));
+ CHPApp* new_app = new CHPApp();
new_app->appIdInstance = appIdInstance;
new_app->app_type_flags = app_type_flags;
new_app->num_matches = num_matches;
{
appid_log(nullptr, TRACE_ERROR_LEVEL, "LuaDetectorApi:Failed to add CHP for appId %d, instance %d",
CHP_APPIDINSTANCE_TO_ID(appIdInstance), CHP_APPIDINSTANCE_TO_INSTANCE(appIdInstance));
- snort_free(new_app);
+ delete new_app;
return -1;
}
return 0;
int LuaStateDescriptor::lua_validate(AppIdDiscoveryArgs& args)
{
- LuaDetectorManager& lua_detector_mgr = odp_thread_local_ctxt->get_lua_detector_mgr();
- auto my_lua_state = lua_detector_mgr.L;
+ auto my_lua_state = odp_thread_local_ctxt->get_lua_state();
if (!my_lua_state)
{
appid_log(args.pkt, TRACE_ERROR_LEVEL, "lua detector %s: no LUA state\n", package_info.name.c_str());
appid_log(args.pkt, TRACE_ERROR_LEVEL, "lua detector %s: error validating %s\n",
package_info.name.c_str(), lua_tostring(my_lua_state, -1));
ldp.pkt = nullptr;
- lua_detector_mgr.free_detector_flow();
+ odp_thread_local_ctxt->free_detector_flow();
lua_settop(my_lua_state, 0);
return APPID_ENULL;
}
/**detectorFlows must be destroyed after each packet is processed.*/
- if (lua_detector_mgr.get_detector_flow())
- lua_detector_mgr.free_detector_flow();
+ odp_thread_local_ctxt->free_detector_flow();
/* retrieve result */
if (!lua_isnumber(my_lua_state, -1))
int LuaServiceDetector::validate(AppIdDiscoveryArgs& args)
{
- auto my_lua_state = odp_thread_local_ctxt->get_lua_detector_mgr().L;
+ auto my_lua_state = odp_thread_local_ctxt->get_lua_state();
if (lua_gettop(my_lua_state))
appid_log(args.pkt, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before service validate\n",
lua_gettop(my_lua_state));
int LuaClientDetector::validate(AppIdDiscoveryArgs& args)
{
- auto my_lua_state = odp_thread_local_ctxt->get_lua_detector_mgr().L;
+ auto my_lua_state = odp_thread_local_ctxt->get_lua_state();
if (lua_gettop(my_lua_state))
appid_log(args.pkt, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before client validate\n",
lua_gettop(my_lua_state));
#define OPEN_DETECTOR_PACKAGE_VERSION_FILE "version.conf"
#define OPEN_DETECTOR_PACKAGE_VERSION "VERSION="
-static vector<LuaDetectorManager*> lua_detector_mgr_list;
-static unordered_set<string> lua_detectors_w_validate;
+vector<shared_ptr<PacketLuaDetectorManager>> ControlLuaDetectorManager::lua_detector_mgr_list;
bool get_lua_field(lua_State* L, int table, const char* field, string& out)
{
version_file.close();
}
-LuaDetectorManager::LuaDetectorManager(AppIdContext& ctxt, bool is_control) :
- ctxt(ctxt)
+LuaDetectorManager::LuaDetectorManager(AppIdContext& ctxt, bool is_control) : ctxt(ctxt)
{
- allocated_objects.clear();
- cb_detectors.clear();
L = create_lua_state(ctxt.config, is_control);
- if (is_control)
- init_chp_glossary();
+ if (!L)
+ {
+ if (is_control)
+ appid_log(nullptr, TRACE_CRITICAL_LEVEL,
+ "Error - appid: can not create new luaState, control instance\n");
+ else
+ appid_log(nullptr, TRACE_ERROR_LEVEL,
+ "Error - appid: can not create new luaState, instance=%u\n", get_instance_id());
+ }
}
LuaDetectorManager::~LuaDetectorManager()
{
- if (lua_gettop(L))
- appid_log(nullptr, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before detector unload\n",
- lua_gettop(L));
-
if (L)
{
- if (init(L) and !ignore_chp_cleanup)
- free_current_chp_glossary();
+ if (lua_gettop(L))
+ appid_log(nullptr, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before detector unload\n",
+ lua_gettop(L));
for ( auto& lua_object : allocated_objects )
{
lua_close(L);
}
- if (detector_flow)
- free_detector_flow();
- allocated_objects.clear();
- cb_detectors.clear(); // do not free Lua objects in cb_detectors
}
-void LuaDetectorManager::initialize(const SnortConfig* sc, AppIdContext& ctxt, bool is_control,
- bool reload)
+void LuaDetectorManager::initialize(const SnortConfig* sc)
{
- LuaDetectorManager* lua_detector_mgr = new LuaDetectorManager(ctxt, is_control);
- odp_thread_local_ctxt->set_lua_detector_mgr(*lua_detector_mgr);
-
- if (!lua_detector_mgr->L)
- appid_log(nullptr, is_control? TRACE_CRITICAL_LEVEL : TRACE_ERROR_LEVEL,
- "Error - appid: can not create new luaState, instance=%u\n", get_instance_id());
-
- if (reload)
- {
- appid_log(nullptr, TRACE_INFO_LEVEL, "AppId Lua-Detectors : loading lua detectors in control thread\n");
- unsigned max_threads = ThreadConfig::get_instance_max();
- for (unsigned i = 0 ; i < max_threads; i++)
- {
- lua_detector_mgr_list.emplace_back(new LuaDetectorManager(ctxt, 0));
-
- if (!lua_detector_mgr_list[i]->L)
- appid_log(nullptr, TRACE_CRITICAL_LEVEL, "Error - appid: can not create new luaState, instance=%u\n", i);
-
- }
- }
-
- lua_detector_mgr->initialize_lua_detectors(is_control, reload);
- lua_detector_mgr->activate_lua_detectors(sc);
-
+ activate_lua_detectors(sc);
+
if (SnortConfig::log_verbose())
scan_and_print_odp_version(ctxt.config.app_detector_dir);
if (ctxt.config.list_odp_detectors or SnortConfig::log_verbose())
- lua_detector_mgr->list_lua_detectors();
-}
-
-void LuaDetectorManager::init_thread_manager(const SnortConfig* sc, const AppIdContext& ctxt)
-{
- LuaDetectorManager* lua_detector_mgr = lua_detector_mgr_list[get_instance_id()];
- odp_thread_local_ctxt->set_lua_detector_mgr(*lua_detector_mgr);
- lua_detector_mgr->activate_lua_detectors(sc);
- if (ctxt.config.list_odp_detectors)
- lua_detector_mgr->list_lua_detectors();
-}
-
-void LuaDetectorManager::cleanup_after_swap()
-{
- free_old_chp_glossary();
-}
-
-void LuaDetectorManager::clear_lua_detector_mgrs()
-{
- lua_detector_mgr_list.clear();
-}
-
-void LuaDetectorManager::free_detector_flow()
-{
- delete detector_flow;
- detector_flow = nullptr;
+ list_lua_detectors();
}
bool LuaDetectorManager::insert_cb_detector(AppId app_id, LuaObject* cb_detector)
return 0;
}
-bool LuaDetectorManager::load_detector(char* detector_filename, bool is_custom, bool is_control, bool reload, string& buf)
+bool LuaDetectorManager::load_detector(char* detector_filename, bool is_custom, string& buf)
{
- if (reload and !buf.empty())
+ if (!buf.empty())
{
if (luaL_loadbuffer(L, buf.c_str(), buf.length(), detector_filename))
{
}
else
{
- if (!is_control)
- {
- auto iter = lua_detectors_w_validate.find(detector_filename);
- if (iter == lua_detectors_w_validate.end())
- return false;
- }
-
if (luaL_loadfile(L, detector_filename))
{
if (init(L))
lua_pop(L, 1);
return false;
}
- if (reload and lua_dump(L, dump, &buf))
+ if (lua_dump(L, dump, &buf))
{
if (init(L))
appid_log(nullptr, TRACE_ERROR_LEVEL, "Error - appid: can not compile Lua detector, %s\n", lua_tostring(L, -1));
return has_validate;
}
-void LuaDetectorManager::load_lua_detectors(const char* path, bool is_custom, bool is_control, bool reload)
+void LuaDetectorManager::activate_lua_detectors(const SnortConfig* sc)
+{
+ if (lua_gettop(L))
+ appid_log(nullptr, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before detector activate\n",
+ lua_gettop(L));
+
+ uint32_t lua_tracker_size = compute_lua_tracker_size(MAX_MEMORY_FOR_LUA_DETECTORS, allocated_objects.size());
+ list<LuaObject*>::iterator lo = allocated_objects.begin();
+ while (lo != allocated_objects.end())
+ {
+ LuaStateDescriptor* lsd = (*lo)->validate_lua_state(false);
+ lua_getfield(L, LUA_REGISTRYINDEX, lsd->package_info.name.c_str());
+ lua_getfield(L, -1, lsd->package_info.initFunctionName.c_str());
+ if (!lua_isfunction(L, -1))
+ {
+ if (init(L))
+ appid_log(nullptr, TRACE_ERROR_LEVEL, "Error - appid: can not load DetectorInit function from %s\n",
+ (*lo)->get_detector()->get_name().c_str());
+ if (!(*lo)->get_detector()->is_custom_detector())
+ num_odp_detectors--;
+ lua_settop(L, 0);
+ delete *lo;
+ lo = allocated_objects.erase(lo);
+ continue;
+ }
+
+ /*first parameter is DetectorUserData */
+ string name = lsd->package_info.name + "_";
+ lua_getglobal(L, name.c_str());
+
+ /*second parameter is a table containing configuration stuff. */
+ lua_newtable(L);
+ const SnortConfig** sc_ud = static_cast<const SnortConfig**>(lua_newuserdata(L, sizeof(const SnortConfig*)));
+ *(sc_ud) = sc;
+ lua_setglobal(L, LUA_STATE_GLOBAL_SC_ID);
+ if (lua_pcall(L, 2, 1, 0))
+ {
+ if (init(L))
+ appid_log(nullptr, TRACE_ERROR_LEVEL, "Error - appid: can not run DetectorInit, %s\n", lua_tostring(L, -1));
+ if (!(*lo)->get_detector()->is_custom_detector())
+ num_odp_detectors--;
+ lua_settop(L, 0);
+ delete *lo;
+ lo = allocated_objects.erase(lo);
+ continue;
+ }
+ *(sc_ud) = nullptr;
+
+ lua_getfield(L, LUA_REGISTRYINDEX, lsd->package_info.name.c_str());
+ set_lua_tracker_size(L, lua_tracker_size);
+ lua_settop(L, 0);
+ ++lo;
+ }
+}
+
+void ControlLuaDetectorManager::load_lua_detectors(const char* path, bool is_custom)
{
char pattern[PATH_MAX];
snprintf(pattern, sizeof(pattern), "%s/*", path);
appid_log(nullptr, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before detector load\n",
lua_gettop(L));
- string buf;
for (unsigned n = 0; n < globs.gl_pathc; n++)
{
ifstream file(globs.gl_pathv[n], ios::ate);
// During reload, load_lua_detectors() gets called only for control thread. This
// function loads detectors for all the packet threads too during reload. It skips
// loading detectors that don't have validate for packet threads.
- bool has_validate = load_detector(globs.gl_pathv[n], is_custom, is_control, reload, buf);
+ string buf;
+ bool has_validate = load_detector(globs.gl_pathv[n], is_custom, buf);
- if (reload)
+ for (auto& lua_detector_mgr : lua_detector_mgr_list)
{
- for (auto& lua_detector_mgr : lua_detector_mgr_list)
- {
- if (has_validate)
- lua_detector_mgr->load_detector(globs.gl_pathv[n], is_custom, is_control, reload, buf);
- }
- buf.clear();
+ if (has_validate)
+ lua_detector_mgr->load_detector(globs.gl_pathv[n], is_custom, buf);
}
- else if (is_control and has_validate)
- lua_detectors_w_validate.insert(globs.gl_pathv[n]);
lua_settop(L, 0);
}
pattern, rval);
}
-void LuaDetectorManager::initialize_lua_detectors(bool is_control, bool reload)
+void ControlLuaDetectorManager::initialize_lua_detectors()
{
char path[PATH_MAX];
const char* dir = ctxt.config.app_detector_dir;
return;
snprintf(path, sizeof(path), "%s/odp/lua", dir);
- load_lua_detectors(path, false, is_control, reload);
- num_odp_detectors = allocated_objects.size();
+ load_lua_detectors(path, false);
+ set_num_odp_detectors();
+ for (auto& mgr : lua_detector_mgr_list)
+ mgr->set_num_odp_detectors();
- if (reload)
- {
- for (auto& lua_detector_mgr : lua_detector_mgr_list)
- lua_detector_mgr->num_odp_detectors = lua_detector_mgr->allocated_objects.size();
- }
snprintf(path, sizeof(path), "%s/custom/lua", dir);
- load_lua_detectors(path, true, is_control, reload);
+ load_lua_detectors(path, true);
}
-void LuaDetectorManager::activate_lua_detectors(const SnortConfig* sc)
+ControlLuaDetectorManager::ControlLuaDetectorManager(AppIdContext& appid_ctxt) : LuaDetectorManager(appid_ctxt, true)
+{ init_chp_glossary(); }
+
+ControlLuaDetectorManager::~ControlLuaDetectorManager()
{
- uint32_t lua_tracker_size = compute_lua_tracker_size(MAX_MEMORY_FOR_LUA_DETECTORS,
- allocated_objects.size());
- list<LuaObject*>::iterator lo = allocated_objects.begin();
+ clear_lua_detector_mgrs();
+ if (!ignore_chp_cleanup)
+ free_current_chp_glossary();
+}
- if (lua_gettop(L))
- appid_log(nullptr, TRACE_WARNING_LEVEL, "appid: leak of %d lua stack elements before detector activate\n",
- lua_gettop(L));
+void ControlLuaDetectorManager::initialize(const SnortConfig* sc)
+{
+ unsigned max_threads = ThreadConfig::get_instance_max();
+ for (unsigned i = 0 ; i < max_threads; i++)
+ lua_detector_mgr_list.emplace_back(make_shared<PacketLuaDetectorManager>(ctxt));
- while (lo != allocated_objects.end())
- {
- LuaStateDescriptor* lsd = (*lo)->validate_lua_state(false);
- lua_getfield(L, LUA_REGISTRYINDEX, lsd->package_info.name.c_str());
- lua_getfield(L, -1, lsd->package_info.initFunctionName.c_str());
- if (!lua_isfunction(L, -1))
- {
- if (init(L))
- appid_log(nullptr, TRACE_ERROR_LEVEL, "Error - appid: can not load DetectorInit function from %s\n",
- (*lo)->get_detector()->get_name().c_str());
- if (!(*lo)->get_detector()->is_custom_detector())
- num_odp_detectors--;
- lua_settop(L, 0);
- delete *lo;
- lo = allocated_objects.erase(lo);
- continue;
- }
+ initialize_lua_detectors();
+ LuaDetectorManager::initialize(sc);
+}
- /*first parameter is DetectorUserData */
- string name = lsd->package_info.name + "_";
- lua_getglobal(L, name.c_str());
+void ControlLuaDetectorManager::list_lua_detectors()
+{
- /*second parameter is a table containing configuration stuff. */
- lua_newtable(L);
- const SnortConfig** sc_ud = static_cast<const SnortConfig**>(lua_newuserdata(L, sizeof(const SnortConfig*)));
- *(sc_ud) = sc;
- lua_setglobal(L, LUA_STATE_GLOBAL_SC_ID);
- if (lua_pcall(L, 2, 1, 0))
- {
- if (init(L))
- appid_log(nullptr, TRACE_ERROR_LEVEL, "Error - appid: can not run DetectorInit, %s\n", lua_tostring(L, -1));
- if (!(*lo)->get_detector()->is_custom_detector())
- num_odp_detectors--;
- lua_settop(L, 0);
- delete *lo;
- lo = allocated_objects.erase(lo);
- continue;
- }
- *(sc_ud) = nullptr;
+ #ifdef REG_TEST
+ // Lua memory usage is inconsistent, for ease of testing lets print 0 instead.
+ int memory_used_by_lua = 0;
+ #else
+ int memory_used_by_lua = lua_gc(L, LUA_GCCOUNT, 0);
+ #endif
- lua_getfield(L, LUA_REGISTRYINDEX, lsd->package_info.name.c_str());
- set_lua_tracker_size(L, lua_tracker_size);
- lua_settop(L, 0);
- ++lo;
- }
+ appid_log(nullptr, TRACE_INFO_LEVEL, "AppId Lua-Detector Stats: control instance, odp detectors %zu, custom detectors %zu,"
+ " total memory %d kb\n", num_odp_detectors, (allocated_objects.size() - num_odp_detectors), memory_used_by_lua);
}
-void LuaDetectorManager::list_lua_detectors()
+void ControlLuaDetectorManager::cleanup_after_swap()
+{
+ free_old_chp_glossary();
+}
+
+void ControlLuaDetectorManager::clear_lua_detector_mgrs()
+{
+ lua_detector_mgr_list.clear();
+}
+
+std::shared_ptr<LuaDetectorManager> ControlLuaDetectorManager::get_packet_lua_detector_manager()
+{
+ unsigned instance_id = get_instance_id();
+ std::shared_ptr<PacketLuaDetectorManager> mgr = lua_detector_mgr_list[instance_id];
+ return static_cast<std::shared_ptr<LuaDetectorManager>>(mgr);
+}
+
+void PacketLuaDetectorManager::list_lua_detectors()
{
#ifdef REG_TEST
(allocated_objects.size() - num_odp_detectors), memory_used_by_lua);
}
+void PacketLuaDetectorManager::free_detector_flow()
+{
+ delete detector_flow;
+ detector_flow = nullptr;
+}
+
#include <cstdint>
#include <list>
#include <map>
+#include <memory>
#include <string>
+#include <vector>
#include <lua.hpp>
#include <lua/lua.h>
class LuaDetectorManager
{
public:
- LuaDetectorManager(AppIdContext&, bool);
- ~LuaDetectorManager();
- static void initialize(const snort::SnortConfig*, AppIdContext&, bool is_control=false,
- bool reload=false);
- static void init_thread_manager(const snort::SnortConfig*, const AppIdContext&);
- static void cleanup_after_swap();
- static void clear_lua_detector_mgrs();
-
- void set_detector_flow(DetectorFlow* df)
- {
- detector_flow = df;
- }
-
- DetectorFlow* get_detector_flow()
- {
- return detector_flow;
- }
-
- void set_ignore_chp_cleanup(bool value)
- {
- ignore_chp_cleanup = value;
- }
+ LuaDetectorManager(AppIdContext&, bool is_control);
+ virtual ~LuaDetectorManager();
+ virtual void initialize(const snort::SnortConfig*);
- void free_detector_flow();
- lua_State* L;
+ bool load_detector(char* detector_name, bool is_custom, std::string& buf);
+ void set_num_odp_detectors()
+ { num_odp_detectors = allocated_objects.size(); }
bool insert_cb_detector(AppId app_id, LuaObject* ud);
LuaObject* get_cb_detector(AppId app_id);
-private:
- void initialize_lua_detectors(bool is_control, bool reload = false);
+ lua_State* L;
+
+protected:
void activate_lua_detectors(const snort::SnortConfig*);
- void list_lua_detectors();
- bool load_detector(char* detector_name, bool is_custom, bool is_control, bool reload, std::string& buf);
- void load_lua_detectors(const char* path, bool is_custom, bool is_control, bool reload = false);
LuaObject* create_lua_detector(const char* detector_name, bool is_custom,
const char* detector_filename, bool& has_validate);
+ virtual void list_lua_detectors() = 0;
AppIdContext& ctxt;
std::list<LuaObject*> allocated_objects;
size_t num_odp_detectors = 0;
std::map<AppId, LuaObject*> cb_detectors;
+};
+
+class PacketLuaDetectorManager : public LuaDetectorManager
+{
+public:
+ explicit PacketLuaDetectorManager(AppIdContext& appid_ctxt) : LuaDetectorManager(appid_ctxt, false)
+ { }
+ ~PacketLuaDetectorManager() override
+ { free_detector_flow(); }
+
+ void set_detector_flow(DetectorFlow* df)
+ { detector_flow = df; }
+
+ DetectorFlow* get_detector_flow() const
+ { return detector_flow; }
+
+ void free_detector_flow();
+
+private:
+ void list_lua_detectors() override;
+
DetectorFlow* detector_flow = nullptr;
+};
+
+class ControlLuaDetectorManager : public LuaDetectorManager
+{
+public:
+ explicit ControlLuaDetectorManager(AppIdContext&);
+ ~ControlLuaDetectorManager() override;
+ void initialize(const snort::SnortConfig*) override;
+
+ static std::shared_ptr<LuaDetectorManager> get_packet_lua_detector_manager();
+ static void clear_lua_detector_mgrs();
+ static void cleanup_after_swap();
+
+ void set_ignore_chp_cleanup()
+ {
+ ignore_chp_cleanup = true;
+ }
+
+private:
+ static std::vector<std::shared_ptr<PacketLuaDetectorManager>> lua_detector_mgr_list;
+
+ void initialize_lua_detectors();
+ void load_lua_detectors(const char* path, bool is_custom);
+ void list_lua_detectors() override;
+
bool ignore_chp_cleanup = false;
};
void ServiceDiscovery::reload()
{
for ( auto& kv : tcp_detectors )
- kv.second->reload();
+ kv.second->do_custom_reload();
for ( auto& kv : udp_detectors )
- kv.second->reload();
+ kv.second->do_custom_reload();
}
void ServiceDiscovery::finalize_service_patterns()
args.asd.tsession->set_tls_host(ss->client_hello.host_name, 0, args.change_bits);
args.asd.scan_flags |= SCAN_SSL_HOST_FLAG;
}
- else if (ss->server_cert.common_name)
- {
- /* Use common name (from server) if we didn't get host name (from client). */
- args.asd.tsession->set_tls_host(ss->server_cert.common_name, ss->server_cert.common_name_strlen,
- args.change_bits);
- args.asd.scan_flags |= SCAN_SSL_HOST_FLAG;
- }
/* TLS Common Name */
if (ss->server_cert.common_name)
{
args.asd.tsession->set_tls_cname(ss->server_cert.common_name, 0, args.change_bits);
args.asd.scan_flags |= SCAN_SSL_CERTIFICATE_FLAG;
+ args.asd.scan_flags |= SCAN_SSL_HOST_FLAG;
}
/* TLS Org Unit */
if (ss->server_cert.org_unit)
return nullptr;
}
-AppIdContext* ctxt;
-AppIdContext& AppIdInspector::get_ctxt() const { return *ctxt; }
void appid_log(const snort::Packet*, unsigned char, char const*, ...) { }
TEST_GROUP(alpn_patterns_tests)
#include "appid_module.h"
#include "appid_peg_counts.h"
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
#define APPID_UT_ID 1492
namespace snort
FpSMBData* smb_data = nullptr;
int AppIdDetector::initialize(AppIdInspector&){return 0;}
-void AppIdDetector::reload() { }
int AppIdDetector::data_add(AppIdSession&, void*, AppIdFreeFCN){return 0;}
-void* AppIdDetector::data_get(AppIdSession&) {return nullptr;}
+void* AppIdDetector::data_get(const AppIdSession&) {return nullptr;}
void AppIdDetector::add_user(AppIdSession&, const char*, AppId, bool, AppidChangeBits&){}
void AppIdDetector::add_payload(AppIdSession&, AppId){}
void AppIdDetector::add_app(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppId, const char*, AppidChangeBits&){}
static OdpContext stub_odp_ctxt(stub_config, nullptr);
OdpContext* AppIdContext::odp_ctxt = &stub_odp_ctxt;
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector& inspector,
- OdpContext&, uint16_t) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
+ OdpContext&
+#ifndef DISABLE_TENANT_ID
+ ,uint16_t
+#endif
+ ) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
config(stub_config), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt) { }
AppIdSession::~AppIdSession() = default;
DiscoveryFilter::~DiscoveryFilter(){}
{
return nullptr;
}
-AppidCPUProfilingManager::AppidCPUProfilingManager() { }
void ServiceDiscoveryState::set_service_id_valid(ServiceDetector*) { }
#include "appid_http_session.h"
#include "tp_appid_module_api.h"
#include "tp_appid_session_api.h"
-#include "app_cpu_profile_table.h"
+#include "appid_cpu_profile_table.h"
#include "appid_mock_definitions.h"
#include "appid_mock_http_session.h"
mock_init_appid_pegs();
SfIp ip;
mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector,
- dummy_appid_inspector.get_ctxt().get_odp_ctxt(), 0, 0);
+ dummy_appid_inspector.get_ctxt().get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
pkt_thread_odp_ctxt = &mock_session->get_odp_ctxt();
flow = new Flow;
flow->set_flow_data(mock_session);
TEST(appid_api, ssl_app_group_id_lookup)
{
- mock().expectNCalls(6, "publish");
+ mock().expectNCalls(7, "publish");
AppId service, client, payload = APP_ID_NONE;
bool val = false;
STRCMP_EQUAL(mock_session->tsession->get_tls_host(), APPID_UT_TLS_HOST);
STRCMP_EQUAL(mock_session->tsession->get_tls_first_alt_name(), APPID_UT_TLS_HOST);
STRCMP_EQUAL(mock_session->tsession->get_tls_cname(), APPID_UT_TLS_HOST);
+ STRCMP_EQUAL(mock_session->tsession->get_tls_sni(), APPID_UT_TLS_HOST);
STRCMP_EQUAL("Published change_bits == 00000000000100011000", test_log);
// Common name based detection
STRCMP_EQUAL(mock_session->tsession->get_tls_cname(), APPID_UT_TLS_HOST);
STRCMP_EQUAL("Published change_bits == 00000000000100011000", test_log);
+ //check for sni mismatch being stored in sni field
+ change_bits.reset();
+ mock_session->tsession->set_tls_host("mismatchedsni.com", 17, change_bits);
+ service = APP_ID_NONE;
+ client = APP_ID_NONE;
+ payload = APP_ID_NONE;
+ val = appid_api.ssl_app_group_id_lookup(flow, (const char*)APPID_UT_TLS_HOST, (const char*)APPID_UT_TLS_HOST,
+ nullptr, nullptr, true, service, client, payload);
+ CHECK_TRUE(val);
+ STRCMP_EQUAL(APPID_UT_TLS_HOST, mock_session->tsession->get_tls_host());
+ STRCMP_EQUAL("mismatchedsni.com", mock_session->tsession->get_tls_sni());
+
mock().checkExpectations();
// When appid session is not existing
CHECK_FALSE(appid_api.is_service_http_type(APP_ID_SMTP));
}
-TEST(appid_api, get_appid_detector_directory)
-{
- STRCMP_EQUAL(appid_api.get_appid_detector_directory(), "/path/to/appid/detectors/");
-}
-
int main(int argc, char** argv)
{
int rc = CommandLineTestRunner::RunAllTests(argc, argv);
AppIdConfig::~AppIdConfig() = default;
OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*) { }
-AppidCPUProfilingManager::AppidCPUProfilingManager() {}
AppIdConfig stub_config;
AppIdContext stub_ctxt(stub_config);
OdpContext stub_odp_ctxt(stub_config, nullptr);
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector&,
- OdpContext&, uint32_t, uint32_t) : FlowData(0), config(stub_config),
+ OdpContext&, uint32_t
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t
+#endif
+ ) : FlowData(0), config(stub_config),
api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt) { }
AppIdSession::~AppIdSession() = default;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
SfIp dip;
dip.set("10.1.2.3");
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &dip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &dip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
sip.set("10.9.8.7"); // this would be a reply back
uint16_t sport = 80;
sip.set("2001:db8:85a3::8a2e:370:7334"); // IPv6
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("2001:db8:85a3::8a2e:370:7335");
uint16_t sport = 1234;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
SfIp dip;
dip.set("10.1.2.3");
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &dip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &dip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
sip.set("10.9.8.7");
uint16_t sport = 80;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
sip.set("10.1.2.3");
SfIp dip;
AppIdInspector inspector;
- AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession session(IpProtocol::PROTO_NOT_SET, &sip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// This packet...
dip.set("10.9.8.7");
uint16_t sport = 48620;
{
SfIp ip;
mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector,
- dummy_appid_inspector.get_ctxt().get_odp_ctxt(), 0, 0);
+ dummy_appid_inspector.get_ctxt().get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
flow = new Flow;
flow->set_flow_data(mock_session);
}
// Stubs for AppIdModule
AppIdModule::AppIdModule(): Module("appid_mock", "appid_mock_help") {}
+AppIdModule::~AppIdModule() = default;
void AppIdModule::sum_stats(bool) {}
void AppIdModule::show_dynamic_stats() {}
bool AppIdModule::begin(char const*, int, SnortConfig*) { return true; }
// Stubs for config
static AppIdConfig app_config;
-static AppIdContext app_ctxt(app_config);
AppId OdpContext::get_port_service_id(IpProtocol, uint16_t)
{
return APP_ID_NONE;
}
// Stubs for AppIdInspector
-AppIdInspector::AppIdInspector(AppIdModule&) { ctxt = &stub_ctxt; }
+AppIdInspector::AppIdInspector(AppIdModule&) : config(&app_config), ctxt(app_config)
+{ }
AppIdInspector::~AppIdInspector() = default;
void AppIdInspector::eval(Packet*) { }
bool AppIdInspector::configure(SnortConfig*) { return true; }
void AppIdInspector::tinit() { }
void AppIdInspector::tterm() { }
void AppIdInspector::tear_down(SnortConfig*) { }
-AppIdContext& AppIdInspector::get_ctxt() const
-{
- assert(ctxt);
- return *ctxt;
-}
bool DiscoveryFilter::is_app_monitored(const snort::Packet*, uint8_t*){return true;}
// Stubs for AppInfoManager
p.ptrs.ip_api.set(ip, ip);
AppIdModule app_module;
AppIdInspector ins(app_module);
- AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0, 0);
+ AppIdContext& app_ctxt = ins.get_ctxt();
+ AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd->flags |= APPID_SESSION_SPECIAL_MONITORED | APPID_SESSION_DISCOVER_USER |
APPID_SESSION_DISCOVER_APP;
Flow* flow = new Flow;
p.ptrs.tcph = nullptr;
AppIdModule app_module;
AppIdInspector ins(app_module);
- AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0, 0);
+ AppIdContext& app_ctxt = ins.get_ctxt();
+ AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd->flags |= APPID_SESSION_SPECIAL_MONITORED | APPID_SESSION_DISCOVER_USER |
APPID_SESSION_DISCOVER_APP;
Flow* flow = new Flow;
AppIdModule app_module;
AppIdInspector ins(app_module);
SfIp ip;
- AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0, 0);
+ AppIdContext app_ctxt(app_config);
+ AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
const char* version = "3.0";
asd->set_client_version(version, change_bits);
p.ptrs.ip_api.set(ip, ip);
AppIdModule app_module;
AppIdInspector ins(app_module);
- AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0, 0);
+ AppIdContext& app_ctxt = ins.get_ctxt();
+ AppIdSession* asd = new AppIdSession(IpProtocol::TCP, &ip, 21, ins, app_ctxt.get_odp_ctxt(), 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd->flags |= APPID_SESSION_SPECIAL_MONITORED | APPID_SESSION_DISCOVER_USER |
APPID_SESSION_DISCOVER_APP;
Flow* flow = new Flow;
void setup() override
{
SfIp ip;
- session = new AppIdSession(IpProtocol::TCP, &ip, 0, dummy_appid_inspector, stub_odp_ctxt, 0, 0);
+ session = new AppIdSession(IpProtocol::TCP, &ip, 0, dummy_appid_inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
pkt_thread_odp_ctxt = &stub_odp_ctxt;
appidDebug = new AppIdDebug();
appidDebug->activate(nullptr, nullptr, false);
{
if (stream_index < api.hsessions.size())
{
- return api.hsessions[stream_index];
+ return api.hsessions[stream_index].get();
}
return nullptr;
}
void AppIdSession::delete_all_http_sessions()
{
- for (auto hsession : api.hsessions)
- delete hsession;
api.hsessions.clear();
}
{
flow = new Flow;
SfIp ip;
- mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, stub_odp_ctxt, 0, 0);
+ mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
pkt_thread_odp_ctxt = &mock_session->get_odp_ctxt();
mock_session->create_http_session();
flow->set_flow_data(mock_session);
// AppIdSession mock functions
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector& inspector,
- OdpContext&, uint32_t, uint32_t) : FlowData(inspector_id, &inspector), config(stub_config),
+ OdpContext&, uint32_t
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t
+#endif
+ ) : FlowData(inspector_id, &inspector), config(stub_config),
api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt)
{}
void Profiler::show_stats() { }
OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*) { }
-AppidCPUProfilingManager::AppidCPUProfilingManager() { }
AppIdConfig::~AppIdConfig() = default;
void setup() override
{
SfIp sfip;
- session = new AppIdSession(IpProtocol::IP, &sfip, 0, dummy_appid_inspector, stub_odp_ctxt, 0, 0);
+ session = new AppIdSession(IpProtocol::IP, &sfip, 0, dummy_appid_inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
session->flow = &flow;
mock_hsession = new AppIdHttpSession(*session, 0);
appidDebug = new AppIdDebug();
}
-AppIdModule::AppIdModule(): snort::Module("appid_mock", "appid_mock_help") {}
+AppIdModule::AppIdModule(): snort::Module("appid_mock", "appid_mock_help") { }
+AppIdModule::~AppIdModule() = default;
void AppIdModule::sum_stats(bool) {}
void AppIdModule::show_dynamic_stats() {}
bool AppIdModule::begin(char const*, int, snort::SnortConfig*) { return true; }
void AppIdModule::set_trace(const Trace*) const { }
const TraceOption* AppIdModule::get_trace_options() const { return nullptr; }
+AppIdConfig appid_config;
+AppIdInspector::AppIdInspector(AppIdModule&) : config(&appid_config), ctxt(appid_config)
+{ }
AppIdInspector::~AppIdInspector() = default;
void AppIdInspector::eval(snort::Packet*) { }
bool AppIdInspector::configure(snort::SnortConfig*) { return true; }
void AppIdInspector::tinit() { }
void AppIdInspector::tterm() { }
void AppIdInspector::tear_down(snort::SnortConfig*) { }
-AppIdContext& AppIdInspector::get_ctxt() const { return *ctxt; }
AppIdModule appid_mod;
-AppIdConfig appid_config;
-AppIdContext appid_ctxt(appid_config);
-THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt = nullptr;
AppIdInspector dummy_appid_inspector( appid_mod );
-
-AppIdInspector::AppIdInspector(AppIdModule& )
-{
- ctxt = &appid_ctxt;
- appid_config.app_detector_dir = "/path/to/appid/detectors/";
- config = &appid_config;
-}
+THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt = nullptr;
#endif
#ifndef APPID_MOCK_SESSION_H
#define APPID_MOCK_SESSION_H
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
#include "flow/ha.h"
#include "appid_dns_session.h"
void FlowHAState::add(uint8_t) { }
static AppIdConfig stub_config;
-static AppIdContext stub_ctxt(stub_config);
static OdpContext stub_odp_ctxt(stub_config, nullptr);
OdpContext* AppIdContext::odp_ctxt = &stub_odp_ctxt;
AppIdSession::AppIdSession(IpProtocol proto, const SfIp* ip, uint16_t, AppIdInspector& inspector,
- OdpContext&, uint32_t, uint32_t) : FlowData(inspector_id, &inspector), config(stub_config),
+ OdpContext&, uint32_t
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t
+#endif
+ ) : FlowData(inspector_id, &inspector), config(stub_config),
protocol(proto), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt)
{
this->set_session_flags(APPID_SESSION_DISCOVER_APP | APPID_SESSION_SPECIAL_MONITORED);
AppIdHttpSession* AppIdSession::create_http_session(int64_t)
{
- AppIdHttpSession* hsession = new MockAppIdHttpSession(*this);
- AppidChangeBits change_bits;
+ auto hsession = std::make_unique<MockAppIdHttpSession>(*this);
+ auto tmp_hsession = hsession.get();
+ AppidChangeBits change_bits;
hsession->client.set_id(APPID_UT_ID);
hsession->client.set_version(APPID_UT_CLIENT_VERSION);
change_bits.set(APPID_CLIENT_INFO_BIT);
hsession->payload.set_id(APPID_UT_ID);
hsession->misc_app_id = APPID_UT_ID;
hsession->referred_payload_app_id = APPID_UT_ID;
- api.hsessions.push_back(hsession);
- return hsession;
+ api.hsessions.push_back(std::move(hsession));
+ return tmp_hsession;
}
AppIdHttpSession* AppIdSession::get_matching_http_session(int64_t stream_id) const
for (uint64_t stream_index=0; stream_index < api.hsessions.size(); stream_index++)
{
if (stream_id == api.hsessions[stream_index]->get_httpx_stream_id())
- return api.hsessions[stream_index];
+ return api.hsessions[stream_index].get();
}
return nullptr;
}
return true;
}
-AppidCPUProfilingManager::AppidCPUProfilingManager() { }
-
#endif
AppidChangeBits change_bits;
SfIp ip{};
- mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ mock_session = new AppIdSession(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
mock_session->flow = &flow;
pkt_thread_odp_ctxt = &mock_session->get_odp_ctxt();
mock_session->set_ss_application_ids(APPID_UT_ID, APPID_UT_ID, APPID_UT_ID,
TEST(appid_session_api, get_client_app_id_with_eve_for_http2)
{
SfIp ip{};
- AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd.flow = &flow;
AppidChangeBits change_bits;
asd.set_ss_application_ids(APP_ID_HTTP2, APPID_UT_ID, APPID_UT_ID, APPID_UT_ID, APPID_UT_ID, change_bits);
TEST(appid_session_api, get_app_id)
{
SfIp ip{};
- AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd.flow = &flow;
AppidChangeBits change_bits;
asd.set_application_ids_service(APP_ID_HTTP2, change_bits);
TEST(appid_session_api, get_app_id_with_eve_for_http2)
{
SfIp ip{};
- AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd.flow = &flow;
AppidChangeBits change_bits;
asd.set_application_ids_service(APP_ID_HTTP2, change_bits);
TEST(appid_session_api, get_first_stream_appids_for_http2)
{
SfIp ip{};
- AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd.flow = &flow;
AppidChangeBits change_bits;
asd.set_application_ids_service(APP_ID_HTTP2, change_bits);
TEST(appid_session_api, get_client_info_http2)
{
SfIp ip{};
- AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0, 0);
+ AppIdSession asd(IpProtocol::TCP, &ip, 1492, dummy_appid_inspector, odpctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
asd.flow = &flow;
AppidChangeBits change_bits;
asd.set_ss_application_ids(APP_ID_HTTP2, APPID_UT_ID + 1, APPID_UT_ID, APPID_UT_ID, APPID_UT_ID, change_bits);
void ClientAppDescriptor::update_user(AppId, const char*, AppidChangeBits&){}
AppIdConfig::~AppIdConfig() = default;
OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*) { }
-AppidCPUProfilingManager::AppidCPUProfilingManager() {}
AppIdConfig stub_config;
AppIdContext stub_ctxt(stub_config);
OdpContext stub_odp_ctxt(stub_config, nullptr);
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector&,
- OdpContext&, uint32_t, uint32_t) : FlowData(0), config(stub_config),
+ OdpContext&, uint32_t
+#ifndef DISABLE_TENANT_ID
+ ,uint32_t
+#endif
+ ) : FlowData(0), config(stub_config),
api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt) { }
AppIdSession::~AppIdSession() = default;
AppIdDiscovery::~AppIdDiscovery() = default;
AppIdInspector inspector;
SfIp client_ip;
client_ip.set("1.2.3.4");
- AppIdSession asd(IpProtocol::PROTO_NOT_SET, &client_ip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession asd(IpProtocol::PROTO_NOT_SET, &client_ip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// Testing 3+ failures to exceed STATE_ID_NEEDED_DUPE_DETRACT_COUNT with valid_count = 0
sds.set_state(ServiceState::VALID);
AppIdInspector inspector;
SfIp client_ip;
client_ip.set("1.2.3.4");
- AppIdSession asd(IpProtocol::PROTO_NOT_SET, &client_ip, 0, inspector, stub_odp_ctxt, 0, 0);
+ AppIdSession asd(IpProtocol::PROTO_NOT_SET, &client_ip, 0, inspector, stub_odp_ctxt, 0
+#ifndef DISABLE_TENANT_ID
+ ,0
+#endif
+ );
// Testing 3+ failures to exceed STATE_ID_NEEDED_DUPE_DETRACT_COUNT with valid_count > 1
sds.set_state(ServiceState::VALID);
AlpnPatternMatchers::~AlpnPatternMatchers() = default;
CipPatternMatchers::~CipPatternMatchers() = default;
AppIdConfig::~AppIdConfig() = default;
-AppidCPUProfilingManager::AppidCPUProfilingManager() = default;
OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*) { }
void ServiceDiscovery::initialize(AppIdInspector&) { }
void ServiceDiscovery::reload() { }
#define INSPECTION_KEY ".inspection"
#define IPS_KEY ".ips"
+unsigned int BinderModule::module_id = 0;
+
THREAD_LOCAL BindStats bstats;
static const PegInfo bind_pegs[] =
"use the given configuration on one or any end of a session" },
{ "service", Parameter::PT_STRING, nullptr, nullptr,
- "override default configuration" },
+ "space separated list of services" },
{ nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
};
policy_type.clear();
}
+ if (!module_id)
+ module_id = FlowData::create_flow_data_id();
+
return true;
}
// both
if ( !strcmp(fqn, "binder.when.service") )
{
- binding.when.svc = v.get_string();
+ binding.when.parse_service(v.get_string());
binding.when.add_criteria(BindWhen::Criteria::BWC_SVC);
}
else if ( !strcmp(fqn, "binder.use.service") )
void BinderModule::add(const char* svc, const char* type)
{
binding.clear();
- binding.when.svc = svc;
+ binding.when.parse_service(svc);
binding.when.add_criteria(BindWhen::Criteria::BWC_SVC);
binding.use.type = type;
binding.use.name = type;
Usage get_usage() const override
{ return INSPECT; }
+ static unsigned int module_id;
+
private:
Binding binding;
std::vector<Binding> bindings;
#include "config.h"
#endif
+
+#include "appid/appid_session_api.h"
#include "detection/detection_engine.h"
#include "flow/flow.h"
#include "framework/pig_pen.h"
#include "packet_io/active.h"
#include "profiler/profiler.h"
#include "protocols/packet.h"
+#include "pub_sub/appid_events.h"
#include "pub_sub/assistant_gadget_event.h"
#include "pub_sub/intrinsic_event_ids.h"
#include "pub_sub/stream_event_ids.h"
}
if (bw.has_criteria(BindWhen::Criteria::BWC_SVC))
- when += " service = " + bw.svc + ",";
+ when += " service = " + bw.get_service_list() + ",";
if (bw.has_criteria(BindWhen::Criteria::BWC_SPLIT_NETS))
{
void handle_flow_service_change(Flow&);
void handle_assistant_gadget(const char* service, Flow&);
void handle_flow_after_reload(Flow&);
+ void handle_appid_service_change(DataEvent&,Flow&);
private:
void get_policy_bindings(Flow&, const char* service);
}
};
+class AppIDServiceChangeHandler : public DataHandler
+{
+ public:
+ AppIDServiceChangeHandler() : DataHandler(BIND_NAME) { }
+
+ void handle(DataEvent& event, Flow* flow) override
+ {
+ Binder* binder = (Binder*)InspectorManager::get_binder();
+ if (binder && flow)
+ binder->handle_appid_service_change(event, *flow);
+ }
+};
+
// When a flow's service changes, re-evaluate service to inspector mapping.
class FlowServiceChangeHandler : public DataHandler
{
DataBus::subscribe(intrinsic_pub_key, IntrinsicEventIds::FLOW_SERVICE_CHANGE, new FlowServiceChangeHandler());
DataBus::subscribe(intrinsic_pub_key, IntrinsicEventIds::FLOW_ASSISTANT_GADGET, new AssistantGadgetHandler());
DataBus::subscribe(intrinsic_pub_key, IntrinsicEventIds::FLOW_STATE_RELOADED, new RebindFlow());
+ DataBus::subscribe(appid_pub_key, AppIdEventIds::ANY_CHANGE, new AppIDServiceChangeHandler());
DataBus::subscribe(stream_pub_key, StreamEventIds::HA_NEW_FLOW, new StreamHANewFlowHandler());
bstats.verdicts[stuff.action]++;
}
+void Binder::handle_appid_service_change(DataEvent& event, Flow& flow)
+{
+ AppidEvent& appid_event = static_cast<AppidEvent&>(event);
+
+ if(appid_event.get_change_bitset().test(APPID_SERVICE_BIT))
+ {
+ if ((appid_event.get_appid_session_api().get_service_app_id() <= 0)
+ or (flow.ssn_state.snort_protocol_id == 0))
+ return;
+
+ Stuff stuff;
+ const SnortConfig* sc = SnortConfig::get_conf();
+ const char* service = sc->proto_ref->get_name(flow.ssn_state.snort_protocol_id);
+ get_bindings(flow, stuff, service);
+
+ if(stuff.action == BindUse::BA_ALLOW)
+ {
+ flow.set_deferred_trust(BinderModule::module_id, true);
+ flow.flags.binder_action_allow = true;
+ }
+ else if (stuff.action == BindUse::BA_BLOCK)
+ {
+ flow.flags.binder_action_block = true;
+ }
+ }
+
+ if(appid_event.get_change_bitset().test(APPID_DISCOVERY_FINISHED_BIT))
+ {
+ //apply action for delay
+ if(flow.flags.binder_action_allow)
+ {
+ flow.try_trust();
+ flow.set_deferred_trust(BinderModule::module_id, false);
+ }
+ else if(flow.flags.binder_action_block)
+ {
+ flow.block();
+ auto p = const_cast<Packet*>(appid_event.get_packet());
+ p->active->block_session(p, true);
+ }
+ }
+}
+
void Binder::handle_flow_service_change(Flow& flow)
{
bstats.service_changes++;
if (stuff.action != BindUse::BA_INSPECT)
{
stuff.apply_action(flow);
+ flow.disable_inspection();
return;
}
when.ips_id_user = 0;
when.protos = PROTO_BIT__ANY_TYPE;
when.role = BindWhen::BR_EITHER;
- when.svc.clear();
+ when.svc_list.clear();
if (when.src_nets)
{
if (!when.has_criteria(BindWhen::Criteria::BWC_TENANTS))
return true;
- return when.tenants.count(flow.tenant) != 0;
+#ifndef DISABLE_TENANT_ID
+ return when.tenants.count(flow.key->tenant_id) != 0;
+#else
+ return when.tenants.count(0) != 0;
+#endif
}
inline bool Binding::check_tenant(const Packet* p) const
if (!flow.service)
return false;
- return when.svc == flow.service;
+ return when.svc_list.find(flow.service) != when.svc_list.end();
}
inline bool Binding::check_service(const char* service) const
if (!when.has_criteria(BindWhen::Criteria::BWC_SVC))
return false;
- return when.svc == service;
+ return when.svc_list.find(service) != when.svc_list.end();
}
inline bool Binding::check_service() const
#define BINDING_H
#include <string>
+#include <sstream>
#include "main/policy.h"
#include "sfip/sf_ipvar.h"
unsigned ips_id_user;
unsigned protos;
Role role;
- std::string svc;
sfip_var_t* src_nets;
sfip_var_t* dst_nets;
PortBitSet src_ports;
PortBitSet dst_ports;
+ std::unordered_set<std::string> svc_list;
+
std::unordered_set<int32_t> src_intfs;
std::unordered_set<int32_t> dst_intfs;
{ criteria_flags |= flags; }
bool has_criteria(uint16_t flags) const
{ return (criteria_flags & flags) == flags; }
+
+ void parse_service(const std::string& service)
+ {
+ if (service.find(" ") == std::string::npos)
+ {
+ svc_list.emplace(service);
+ return;
+ }
+
+ std::string buf;
+ std::stringstream ss(service);
+ while(getline(ss, buf, ' '))
+ svc_list.emplace(buf);
+ }
+
+ std::string get_service_list() const
+ {
+ std::string res;
+ for(const auto& entry : svc_list)
+ res += entry;
+ return res;
+ }
};
struct BindUse
--- /dev/null
+set( FILE_LIST
+ extractor.cc
+ extractor.h
+ extractor_csv_logger.cc
+ extractor_csv_logger.h
+ extractor_event_handlers.h
+ extractor_http_event_handler.cc
+ extractor_json_logger.cc
+ extractor_json_logger.h
+ extractor_logger.cc
+ extractor_logger.h
+ extractor_service.cc
+ extractor_service.h
+ extractor_writer.cc
+ extractor_writer.h
+)
+
+add_library(extractor OBJECT ${FILE_LIST})
--- /dev/null
+Extractor is a global network inspector that logs flow data upon receiving
+a flow event.
+
+Supported services:
+ * HTTP
+ * HTTP2
+
+Supported events:
+ * end of HTTP transaction (request-response pair)
+
+An example configuration follows:
+
+ extractor =
+ {
+ protocols =
+ {
+ service = 'http',
+ tenant_id = 1,
+ on_events = 'eot',
+ fields = 'ts, uri, host, method'
+ }
+ {
+ service = 'http',
+ tenant_id = 2,
+ on_events = 'eot',
+ fields = 'ts, uri'
+ }
+ }
+
+Each tenant can have its own protocol configuration.
+
+A list of common fields which are logged:
+ * ts (timestamp)
+ * uid (connection id)
+ * id.orig_h (client IP address)
+ * id.orig_p (client TCP port)
+ * id.resp_h (server IP address)
+ * id.resp_p (server TCP port)
+ * pkt_num (packet number)
+
+The following fields are supported for HTTP:
+ * method
+ * host
+ * uri
+ * user_agent
+ * referrer
+ * origin
+ * version
+ * status_code
+ * status_msg
+ * trans_depth
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor.cc author Anna Norokh <anorokh@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor.h"
+
+#include <algorithm>
+
+#include "framework/data_bus.h"
+#include "framework/inspector.h"
+#include "framework/module.h"
+#include "log/messages.h"
+#include "main/snort_config.h"
+#include "protocols/packet.h"
+#include "pub_sub/http_events.h"
+
+#include "extractor_event_handlers.h"
+#include "extractor_logger.h"
+#include "extractor_service.h"
+
+using namespace snort;
+
+THREAD_LOCAL ExtractorStats extractor_stats;
+THREAD_LOCAL ProfileStats extractor_perf_stats;
+
+//-------------------------------------------------------------------------
+// module stuff
+//-------------------------------------------------------------------------
+
+static const Parameter extractor_proto_params[] =
+{
+ { "service", Parameter::PT_ENUM, "http", nullptr,
+ "service to extract from" },
+
+ { "tenant_id", Parameter::PT_INT, "0:max32", "0",
+ "tenant_id of target tenant" },
+
+ { "on_events", Parameter::PT_STRING, nullptr, nullptr,
+ "specify events to log" },
+
+ { "fields", Parameter::PT_STRING, nullptr, nullptr,
+ "specify fields to log" },
+
+ { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
+};
+
+static const Parameter s_params[] =
+{
+ { "formatting", Parameter::PT_ENUM, "csv | json", "csv",
+ "output format for extractor" },
+
+ { "output", Parameter::PT_ENUM, "stdout", "stdout",
+ "output destination for extractor" },
+
+ { "protocols", Parameter::PT_LIST, extractor_proto_params, nullptr,
+ "protocols to extract data" },
+
+ { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
+};
+
+void ServiceConfig::clear()
+{
+ service = ServiceType::UNDEFINED;
+ on_events.clear();
+ tenant_id = 0;
+ fields.clear();
+}
+
+ExtractorModule::ExtractorModule() : Module(S_NAME, s_help, s_params) { }
+
+void ExtractorModule::commit_config()
+{
+ for (const auto& p : extractor_config.protocols)
+ {
+ if (p.tenant_id == service_config.tenant_id and p.service == service_config.service)
+ ParseWarning(WARN_CONF_STRICT, "%s service got multiple configurations", service_config.service.c_str());
+ }
+
+ extractor_config.protocols.push_back(service_config);
+ service_config.clear();
+}
+
+static inline void trim(std::string& str)
+{
+ str.erase(str.find_last_not_of(' ') + 1);
+ str.erase(0, str.find_first_not_of(' '));
+}
+
+void ExtractorModule::store(Value& val, std::vector<std::string>& dst)
+{
+ dst.clear();
+ val.set_first_token();
+ std::string tok;
+ while (val.get_next_csv_token(tok))
+ {
+ trim(tok);
+ dst.push_back(tok);
+ }
+}
+
+bool ExtractorModule::begin(const char*, int idx, SnortConfig*)
+{
+ if (idx == 0)
+ {
+ service_config.clear();
+ extractor_config.protocols.clear();
+ }
+
+ return true;
+}
+
+bool ExtractorModule::set(const char*, Value& v, SnortConfig*)
+{
+ if (v.is("formatting"))
+ extractor_config.formatting = (FormatType)(v.get_uint8());
+
+ else if (v.is("output"))
+ extractor_config.output = (OutputType)(v.get_uint8());
+
+ else if (v.is("service"))
+ service_config.service = (ServiceType)(v.get_uint8());
+
+ else if (v.is("tenant_id"))
+ service_config.tenant_id = v.get_uint32();
+
+ else if (v.is("on_events"))
+ store(v, service_config.on_events);
+
+ else if (v.is("fields"))
+ store(v, service_config.fields);
+
+ return true;
+}
+
+bool ExtractorModule::end(const char* fqn, int idx, SnortConfig*)
+{
+ if (!idx or strcmp(fqn, "extractor.protocols"))
+ return true;
+
+ if (service_config.fields.empty())
+ {
+ ParseError("can't initialize extractor without protocols.fields");
+ return false;
+ }
+
+ commit_config();
+
+ return true;
+}
+
+//-------------------------------------------------------------------------
+// Inspector stuff
+//-------------------------------------------------------------------------
+
+Extractor::Extractor(ExtractorModule* m)
+{
+ auto& cfg = m->get_config();
+
+ format = cfg.formatting;
+ output = cfg.output;
+
+ for (const auto& p : cfg.protocols)
+ {
+ auto s = ExtractorService::make_service(p, format, output);
+
+ if (s)
+ services.push_back(s);
+ }
+}
+
+Extractor::~Extractor()
+{
+ for (const auto& s : services)
+ delete s;
+}
+
+void Extractor::show(const SnortConfig*) const
+{
+ ConfigLogger::log_value("formatting", format.c_str());
+ ConfigLogger::log_value("output", output.c_str());
+
+ bool log_header = true;
+ for (const auto& s : services)
+ {
+ if (log_header)
+ {
+ ConfigLogger::log_option("protocols");
+ log_header = false;
+ }
+ std::string str;
+ s->show(str);
+
+ ConfigLogger::log_list("", str.c_str(), " ");
+ }
+}
+//-------------------------------------------------------------------------
+// api stuff
+//-------------------------------------------------------------------------
+
+static Module* mod_ctor()
+{ return new ExtractorModule; }
+
+static void mod_dtor(Module* m)
+{ delete m; }
+
+static Inspector* extractor_ctor(Module* mod)
+{ return new Extractor((ExtractorModule*)mod); }
+
+static void extractor_dtor(Inspector* p)
+{ delete p; }
+
+static InspectApi extractor_api =
+{
+ {
+ PT_INSPECTOR,
+ sizeof(InspectApi),
+ INSAPI_VERSION,
+ 0,
+ API_RESERVED,
+ API_OPTIONS,
+ S_NAME,
+ s_help,
+ mod_ctor,
+ mod_dtor
+ },
+ IT_PASSIVE,
+ PROTO_BIT__ANY_TYPE,
+ nullptr, // buffers
+ nullptr, // service
+ nullptr, // pinit
+ nullptr, // pterm
+ nullptr, // tinit
+ nullptr, // tterm
+ extractor_ctor,
+ extractor_dtor,
+ nullptr, // ssn
+ nullptr // reset
+};
+
+#ifdef BUILDING_SO
+SO_PUBLIC const BaseApi* snort_plugins[] =
+#else
+const BaseApi* nin_extractor[] =
+#endif
+{
+ &extractor_api.base,
+ nullptr
+};
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor.h author Anna Norokh <anorokh@cisco.com>
+
+#ifndef EXTRACTOR_H
+#define EXTRACTOR_H
+
+#include <cstdint>
+#include <string>
+#include <vector>
+
+#include "framework/inspector.h"
+#include "framework/module.h"
+#include "main/snort_config.h"
+#include "profiler/profiler.h"
+
+#include "extractor_logger.h"
+#include "extractor_service.h"
+#include "extractor_writer.h"
+
+#define S_NAME "extractor"
+#define s_help "extracts protocol specific data"
+
+class ServiceConfig
+{
+public:
+ ServiceConfig() : service(ServiceType::UNDEFINED), tenant_id(0) {}
+ void clear();
+
+ ServiceType service;
+ uint32_t tenant_id;
+ std::vector<std::string> on_events;
+ std::vector<std::string> fields;
+};
+
+struct ExtractorConfig
+{
+ FormatType formatting = FormatType::CSV;
+ OutputType output = OutputType::STD;
+ std::vector<ServiceConfig> protocols;
+};
+
+static const PegInfo extractor_pegs[] =
+{
+ { CountType::SUM, "total_events", "total extractor events" },
+ { CountType::END, nullptr, nullptr }
+};
+
+struct ExtractorStats
+{
+ PegCount total_event;
+};
+
+extern THREAD_LOCAL ExtractorStats extractor_stats;
+extern THREAD_LOCAL snort::ProfileStats extractor_perf_stats;
+
+class ExtractorModule : public snort::Module
+{
+public:
+ ExtractorModule();
+
+ const PegInfo* get_pegs() const override
+ { return extractor_pegs; }
+
+ PegCount* get_counts() const override
+ { return (PegCount*)&extractor_stats; }
+
+ snort::ProfileStats* get_profile() const override
+ { return &extractor_perf_stats; }
+
+ bool begin(const char*, int, snort::SnortConfig*) override;
+ bool set(const char*, snort::Value& v, snort::SnortConfig*) override;
+ bool end(const char*, int, snort::SnortConfig*) override;
+
+ Usage get_usage() const override
+ { return GLOBAL; }
+
+ const ExtractorConfig& get_config()
+ { return extractor_config; }
+
+private:
+ void store(snort::Value& val, std::vector<std::string>& dst);
+ void commit_config();
+
+ ExtractorConfig extractor_config;
+ ServiceConfig service_config;
+};
+
+class Extractor : public snort::Inspector
+{
+public:
+ Extractor(ExtractorModule*);
+ ~Extractor() override;
+
+ void show(const snort::SnortConfig*) const override;
+
+private:
+ std::vector<ExtractorService*> services;
+ FormatType format;
+ OutputType output;
+};
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// csv_logger.cc author Anna Norokh <anorokh@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_csv_logger.h"
+
+#include <cassert>
+#include <string>
+
+static THREAD_LOCAL bool first_write;
+
+void CsvExtractorLogger::add_header()
+{
+ std::string header;
+
+ header += "#";
+ header += fields_name[0];
+ for (size_t i = 1; i < fields_name.size(); ++i)
+ {
+ header += ",";
+ header += fields_name[i];
+ }
+ header += "\n";
+
+ writer->write(header.c_str());
+}
+
+void CsvExtractorLogger::open_record()
+{
+ first_write = true;
+ writer->lock();
+}
+
+void CsvExtractorLogger::close_record()
+{
+ writer->write("\n");
+ writer->unlock();
+}
+
+void CsvExtractorLogger::add_field(const char*, const snort::Value& v)
+{
+ switch (v.get_type())
+ {
+ case snort::Value::ValueType::VT_UNUM:
+ {
+ first_write ? []() { first_write = false; } () : writer->write(",");
+ writer->write(std::to_string(v.get_uint64()).c_str());
+ break;
+ }
+
+ case snort::Value::ValueType::VT_STR:
+ {
+ first_write ? []() { first_write = false; } () : writer->write(",");
+ writer->write(v.get_string());
+ break;
+ }
+
+ case snort::Value::ValueType::VT_BOOL: // fallthrough
+ case snort::Value::ValueType::VT_NUM: // fallthrough
+ case snort::Value::ValueType::VT_REAL: // fallthrough
+ default:
+ assert(false);
+ break;
+ }
+}
+
+CsvExtractorLogger::~CsvExtractorLogger()
+{
+ delete writer;
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// csv_logger.h author Anna Norokh <anorokh@cisco.com>
+
+#ifndef EXTRACTOR_CSV_LOGGER_H
+#define EXTRACTOR_CSV_LOGGER_H
+
+#include "framework/value.h"
+
+#include "extractor_logger.h"
+#include "extractor_writer.h"
+
+class CsvExtractorLogger : public ExtractorLogger
+{
+public:
+ CsvExtractorLogger(OutputType o_type, const std::vector<std::string>& fields)
+ : ExtractorLogger(fields), writer(ExtractorWriter::make_writer(o_type))
+ {
+ if (writer)
+ CsvExtractorLogger::add_header();
+ }
+
+ ~CsvExtractorLogger() override;
+
+ void add_header() override;
+ void add_field(const char*, const snort::Value&) override;
+ void open_record() override;
+ void close_record() override;
+
+private:
+ ExtractorWriter* const writer;
+};
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_event_handlers.h author Maya Dagon <mdagon@cisco.com>
+
+#ifndef EXTRACTOR_EVENT_HANDLERS_H
+#define EXTRACTOR_EVENT_HANDLERS_H
+
+#include "flow/flow_key.h"
+#include "framework/data_bus.h"
+
+#include "extractor.h"
+#include "extractor_logger.h"
+
+namespace snort
+{
+
+class ExtractorEvent
+{
+public:
+ static FlowHashKeyOps& get_hash()
+ {
+ static thread_local FlowHashKeyOps flow_key_ops(0);
+ return flow_key_ops;
+ }
+
+protected:
+ ExtractorEvent(uint32_t tid, const std::vector<std::string>& flds, ExtractorLogger& l)
+ : tenant_id(tid), fields(flds), logger(l) {}
+
+ uint32_t tenant_id;
+ const std::vector<std::string> fields;
+ ExtractorLogger& logger;
+};
+
+class HttpExtractorEventHandler : public DataHandler, public ExtractorEvent
+{
+public:
+ HttpExtractorEventHandler(uint32_t tenant, const std::vector<std::string>& flds,
+ ExtractorLogger& l) : DataHandler(S_NAME), ExtractorEvent(tenant, flds, l) {}
+
+ void handle(DataEvent&, Flow*) override;
+};
+
+}
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_http_event_handler.cc author Maya Dagon <mdagon@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_event_handlers.h"
+
+#include "detection/detection_engine.h"
+#include "flow/flow_key.h"
+#include "framework/value.h"
+#include "profiler/profiler.h"
+#include "pub_sub/http_transaction_end_event.h"
+#include "service_inspectors/http_inspect/http_transaction.h"
+#include "sfip/sf_ip.h"
+#include "utils/util.h"
+#include "utils/util_net.h"
+
+using namespace snort;
+
+// FIXIT-P: inspector's data passes many functions before getting to the logger
+
+typedef Value* (*GetFunc) (DataEvent*, Packet*, Flow*);
+
+// HttpTransactionEnd specific
+Value* get_method(DataEvent*, Packet*, Flow*);
+Value* get_host(DataEvent*, Packet*, Flow*);
+Value* get_user_agent(DataEvent*, Packet*, Flow*);
+Value* get_uri(DataEvent*, Packet*, Flow*);
+Value* get_referrer(DataEvent*, Packet*, Flow*);
+Value* get_origin(DataEvent*, Packet*, Flow*);
+Value* get_version(DataEvent*, Packet*, Flow*);
+Value* get_stat_code(DataEvent*, Packet*, Flow*);
+Value* get_stat_msg(DataEvent*, Packet*, Flow*);
+Value* get_trans_depth(DataEvent*, Packet*, Flow*);
+
+// Common
+Value* get_timestamp(DataEvent*, Packet*, Flow*);
+Value* get_ip_src(DataEvent*, Packet*, Flow*);
+Value* get_ip_dst(DataEvent*, Packet*, Flow*);
+Value* get_ip_src_port(DataEvent*, Packet*, Flow*);
+Value* get_ip_dst_port(DataEvent*, Packet*, Flow*);
+Value* get_pkt_num(DataEvent*, Packet*, Flow*);
+Value* get_uid(DataEvent*, Packet*, Flow*);
+
+static void field_to_string(const Field& field, std::string& value)
+{
+ if (field.length() > 0)
+ value.assign((const char*)field.start(), field.length());
+}
+
+Value* get_method(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_method();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_host(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_host_hdr();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_user_agent(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_user_agent();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_uri(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_uri();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_referrer(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_referer_hdr();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_origin(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_origin_hdr();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_version(DataEvent* event, Packet*, Flow*)
+{
+ HttpEnums::VersionId version = ((HttpTransactionEndEvent*)event)->get_version();
+ const auto& iter = HttpEnums::VersionEnumToStr.find(version);
+ if (iter != HttpEnums::VersionEnumToStr.end())
+ return new Value(iter->second);
+
+ return new Value("");
+}
+
+Value* get_stat_code(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_stat_code();
+ std::string str;
+ field_to_string(field, str);
+
+ return new Value((uint64_t)atoi(str.c_str()));
+}
+
+Value* get_stat_msg(DataEvent* event, Packet*, Flow*)
+{
+ const Field& field = ((HttpTransactionEndEvent*)event)->get_stat_msg();
+ std::string str;
+ field_to_string(field, str);
+ return new Value(str.c_str());
+}
+
+Value* get_trans_depth(DataEvent* event, Packet*, Flow*)
+{
+ const uint64_t trans_depth = ((HttpTransactionEndEvent*)event)->get_trans_depth();
+ return new Value(trans_depth);
+}
+
+Value* get_timestamp(DataEvent*, Packet* p, Flow*)
+{
+ char u_sec[8];
+ SnortSnprintf(u_sec, sizeof(u_sec),".%06d",(unsigned)p->pkth->ts.tv_usec);
+ auto str = std::to_string(p->pkth->ts.tv_sec) + u_sec;
+
+ return new Value(str.c_str());
+}
+
+Value* get_ip_src(DataEvent*, Packet*, Flow* flow)
+{
+ InetBuf src;
+ const SfIp& flow_srcip = flow->flags.client_initiated ? flow->client_ip : flow->server_ip;
+ sfip_ntop(&flow_srcip, src, sizeof(src));
+ std::string str = src;
+ return new Value(str.c_str());
+}
+
+Value* get_ip_dst(DataEvent*, Packet*, Flow* flow)
+{
+ InetBuf dst;
+ const SfIp& flow_dstip = flow->flags.client_initiated ? flow->server_ip : flow->client_ip;
+ sfip_ntop(&flow_dstip, dst, sizeof(dst));
+ std::string str = dst;
+ return new Value(str.c_str());
+}
+
+Value* get_ip_src_port(DataEvent*, Packet*, Flow* flow)
+{
+ return new Value((uint64_t)flow->client_port);
+}
+
+Value* get_ip_dst_port(DataEvent*, Packet*, Flow* flow)
+{
+ return new Value((uint64_t)flow->server_port);
+}
+
+Value* get_pkt_num(DataEvent*, Packet* p, Flow*)
+{
+ return new Value(p->context->packet_number);
+}
+
+Value* get_uid(DataEvent*, Packet*, Flow* f)
+{
+ unsigned key = ExtractorEvent::get_hash().do_hash((const unsigned char*)f->key, 0);
+
+ return new Value((uint64_t)key);
+}
+
+static std::map<std::string, GetFunc> event_getters =
+{
+ {"ts", get_timestamp},
+ {"uid", get_uid},
+ {"id.orig_h", get_ip_src},
+ {"id.resp_h", get_ip_dst},
+ {"id.orig_p", get_ip_src_port},
+ {"id.resp_p", get_ip_dst_port},
+ {"pkt_num", get_pkt_num},
+ {"method", get_method},
+ {"host", get_host},
+ {"uri", get_uri},
+ {"user_agent", get_user_agent},
+ {"referrer", get_referrer},
+ {"origin", get_origin},
+ {"version", get_version},
+ {"status_code", get_stat_code},
+ {"status_msg", get_stat_msg},
+ {"trans_depth", get_trans_depth}
+};
+
+void HttpExtractorEventHandler::handle(DataEvent& event, Flow* flow)
+{
+ // cppcheck-suppress unreadVariable
+ Profile profile(extractor_perf_stats);
+ uint32_t tid;
+
+#ifndef DISABLE_TENANT_ID
+ tid = flow->key->tenant_id;
+#else
+ tid = 0;
+#endif
+
+ if (tenant_id != tid)
+ return;
+
+ Packet* p = DetectionEngine::get_current_packet();
+
+ logger.open_record();
+ for (const auto& field : fields)
+ {
+ // FIXIT-P: this is way too slow (a map with a string key type)
+ auto val = std::unique_ptr<Value>(event_getters[field](&event, p, flow));
+ logger.add_field(field.c_str(), *val.get());
+ }
+ logger.close_record();
+
+ extractor_stats.total_event++;
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// json_logger.cc author Cisco
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_json_logger.h"
+
+#include <cassert>
+
+void JsonExtractorLogger::open_record()
+{
+ oss.str("");
+ js.open();
+}
+
+void JsonExtractorLogger::close_record()
+{
+ js.close();
+
+ writer->lock();
+ writer->write(oss.str().c_str());
+ writer->unlock();
+}
+
+void JsonExtractorLogger::add_field(const char* f, const snort::Value& v)
+{
+ switch (v.get_type())
+ {
+ case snort::Value::ValueType::VT_UNUM:
+ js.uput(f, v.get_uint64());
+ break;
+
+ case snort::Value::ValueType::VT_STR:
+ js.put(f, v.get_string());
+ break;
+
+ case snort::Value::ValueType::VT_BOOL: // fallthrough
+ case snort::Value::ValueType::VT_NUM: // fallthrough
+ case snort::Value::ValueType::VT_REAL: // fallthrough
+ default:
+ assert(false);
+ break;
+ }
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// json_logger.h author Cisco
+
+#ifndef EXTRACTOR_JSON_LOGGER_H
+#define EXTRACTOR_JSON_LOGGER_H
+
+#include <sstream>
+
+#include "framework/value.h"
+#include "helpers/json_stream.h"
+
+#include "extractor_logger.h"
+#include "extractor_writer.h"
+
+class JsonExtractorLogger : public ExtractorLogger
+{
+public:
+ JsonExtractorLogger(OutputType o_type, const std::vector<std::string>& fields)
+ : ExtractorLogger(fields), writer(ExtractorWriter::make_writer(o_type)), oss(), js(oss)
+ { }
+
+ ~JsonExtractorLogger() override
+ { delete writer; }
+
+ void add_field(const char*, const snort::Value&) override;
+ void open_record() override;
+ void close_record() override;
+
+private:
+ ExtractorWriter* const writer;
+ std::ostringstream oss;
+ snort::JsonStream js;
+
+};
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_logger.cc author Anna Norokh <anorokh@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_logger.h"
+
+#include <cassert>
+
+#include "extractor_csv_logger.h"
+#include "extractor_json_logger.h"
+
+ExtractorLogger* ExtractorLogger::make_logger(FormatType f_type, OutputType o_type,
+ const std::vector<std::string>& fields)
+{
+ if (fields.empty())
+ return nullptr;
+
+ ExtractorLogger* logger = nullptr;
+
+ switch (f_type)
+ {
+ case FormatType::CSV:
+ logger = new CsvExtractorLogger(o_type, fields);
+ break;
+ case FormatType::JSON:
+ logger = new JsonExtractorLogger(o_type, fields);
+ break;
+ case FormatType::MAX: // fallthrough
+ default:
+ break;
+ }
+
+ assert(logger);
+
+ return logger;
+}
+
+#ifdef UNIT_TEST
+
+#include "catch/snort_catch.h"
+
+#include <memory.h>
+
+using namespace snort;
+
+TEST_CASE("Format Type", "[extractor]")
+{
+ SECTION("to string")
+ {
+ FormatType csv = FormatType::CSV;
+ FormatType json = FormatType::JSON;
+ FormatType max = FormatType::MAX;
+
+ CHECK_FALSE(strcmp("csv", csv.c_str()));
+ CHECK_FALSE(strcmp("json", json.c_str()));
+ CHECK_FALSE(strcmp("(not set)", max.c_str()));
+ }
+}
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_logger.h author Anna Norokh <anorokh@cisco.com>
+
+#ifndef EXTRACTOR_LOGGER_H
+#define EXTRACTOR_LOGGER_H
+
+#include <string>
+#include <vector>
+
+#include "framework/value.h"
+
+#include "extractor_writer.h"
+
+class FormatType
+{
+public:
+ enum Value : uint8_t
+ {
+ CSV,
+ JSON,
+ MAX
+ };
+
+ FormatType() = default;
+ constexpr FormatType(Value a) : v(a) {}
+ template<typename T> constexpr FormatType(T a) : v((Value)a) {}
+
+ constexpr operator Value() const { return v; }
+ explicit operator bool() const = delete;
+
+ const char* c_str() const
+ {
+ switch (v)
+ {
+ case CSV:
+ return "csv";
+ case JSON:
+ return "json";
+ case MAX: // fallthrough
+ default:
+ return "(not set)";
+ }
+ }
+
+private:
+ Value v = CSV;
+};
+
+class ExtractorLogger
+{
+public:
+ static ExtractorLogger* make_logger(FormatType, OutputType, const std::vector<std::string>&);
+
+ ExtractorLogger() = delete;
+ ExtractorLogger(const ExtractorLogger&) = delete;
+ ExtractorLogger& operator=(const ExtractorLogger&) = delete;
+ ExtractorLogger(ExtractorLogger&&) = delete;
+
+ virtual ~ExtractorLogger() = default;
+
+ virtual void add_header() {}
+ virtual void add_footer() {}
+ // FIXIT-P: replace Value type designed for parsing with a better type
+ virtual void add_field(const char*, const snort::Value&) {}
+
+ virtual void open_record() {}
+ virtual void close_record() {}
+
+protected:
+ ExtractorLogger(const std::vector<std::string>& fns) : fields_name(fns) {}
+
+ const std::vector<std::string>& fields_name;
+};
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_services.cc author Maya Dagon <mdagon@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_service.h"
+
+#include "framework/data_bus.h"
+#include "log/messages.h"
+#include "pub_sub/http_events.h"
+
+#include "extractor.h"
+#include "extractor_event_handlers.h"
+
+using namespace snort;
+
+
+//////////////////////////////////////////////////////////////////////
+//// ExtractorService
+//////////////////////////////////////////////////////////////////////
+
+std::vector<std::string> ExtractorService::common_fields =
+{
+ "ts",
+ "uid",
+ "id.orig_h",
+ "id.orig_p",
+ "id.resp_h",
+ "id.resp_p",
+ "pkt_num"
+};
+
+
+ExtractorService::ExtractorService(uint32_t tenant, const std::vector<std::string>& srv_fields,
+ const std::vector<std::string>& srv_events, const ServiceBlueprint& srv_bp,
+ ServiceType s_type, FormatType f_type, OutputType o_type) : tenant_id(tenant), sbp(srv_bp), type(s_type)
+{
+ add_fields(srv_fields);
+ add_events(srv_events);
+ logger = ExtractorLogger::make_logger(f_type, o_type, get_fields());
+}
+
+void ExtractorService::add_events(const std::vector<std::string>& vals)
+{
+ for (const auto& val : vals)
+ {
+ if (find_event(val))
+ events.push_back(val);
+ else
+ ParseWarning(WARN_CONF_STRICT, "unsupported '%s' event in protocols.on_events", val.c_str());
+ }
+}
+
+void ExtractorService::add_fields(const std::vector<std::string>& vals)
+{
+ for (auto& val : vals)
+ {
+ if (find_field(val))
+ fields.push_back(val);
+ else
+ ParseWarning(WARN_CONF_STRICT, "unsupported '%s' field in protocols.fields", val.c_str());
+ }
+}
+
+ExtractorService* ExtractorService::make_service(const ServiceConfig& cfg, FormatType f_type, OutputType o_type)
+{
+ if (cfg.on_events.empty())
+ {
+ ParseError("%s service misses on_events field", cfg.service.c_str());
+ return nullptr;
+ }
+
+ ExtractorService* srv = nullptr;
+
+ switch (cfg.service)
+ {
+ case ServiceType::HTTP:
+ srv = new HttpExtractorService(cfg.tenant_id, cfg.fields, cfg.on_events, cfg.service, f_type, o_type);
+ break;
+
+ case ServiceType::UNDEFINED: // fallthrough
+ default:
+ ParseError("'%s' service is not supported", cfg.service.c_str());
+ }
+
+ return srv;
+}
+
+bool ExtractorService::find_event(const std::string& event) const
+{
+ return std::find(sbp.supported_events.begin(), sbp.supported_events.end(), event)
+ != sbp.supported_events.end();
+}
+
+bool ExtractorService::find_field(const std::string& field) const
+{
+ return ((std::find(common_fields.begin(), common_fields.end(), field) != common_fields.end()) or
+ (std::find(sbp.supported_fields.begin(), sbp.supported_fields.end(),field)
+ != sbp.supported_fields.end()));
+}
+
+void ExtractorService::show(std::string& str) const
+{
+ str = "{ service = ";
+ str += type.c_str();
+ str += ", tenant_id = ";
+ str += std::to_string(tenant_id);
+ str += ", on_events =";
+ for (const auto& event : get_events())
+ {
+ str += " ";
+ str += event;
+ }
+ str += ", fields = ";
+ for (const auto& field : get_fields())
+ {
+ str += field;
+ str += " ";
+ }
+ str += " }";
+}
+
+//////////////////////////////////////////////////////////////////////
+//// HttpExtractorService
+//////////////////////////////////////////////////////////////////////
+
+ServiceBlueprint HttpExtractorService::blueprint =
+{
+ // events
+ {
+ "eot",
+ },
+ // fields
+ {
+ "method",
+ "host",
+ "uri",
+ "user_agent",
+ "referrer",
+ "origin",
+ "version",
+ "status_code",
+ "status_msg",
+ "trans_depth"
+ },
+};
+
+HttpExtractorService::HttpExtractorService(uint32_t tenant, const std::vector<std::string>& srv_fields,
+ const std::vector<std::string>& srv_events, ServiceType s_type, FormatType f_type, OutputType o_type)
+ : ExtractorService(tenant, srv_fields, srv_events, blueprint, s_type, f_type, o_type)
+{
+ if (!logger)
+ return;
+
+ for (const auto& event : get_events())
+ {
+ if (!strcmp("eot", event.c_str()))
+ {
+ DataBus::subscribe(http_pub_key, HttpEventIds::END_OF_TRANSACTION,
+ new HttpExtractorEventHandler(tenant_id, get_fields(), *logger));
+ }
+ }
+}
+
+#ifdef UNIT_TEST
+
+#include "catch/snort_catch.h"
+
+#include <memory.h>
+
+using namespace snort;
+
+TEST_CASE("Service Type", "[extractor]")
+{
+ SECTION("to string")
+ {
+ ServiceType http = ServiceType::HTTP;
+ ServiceType undef = ServiceType::UNDEFINED;
+ ServiceType max = ServiceType::MAX;
+
+ CHECK_FALSE(strcmp("http", http.c_str()));
+ CHECK_FALSE(strcmp("(not set)", undef.c_str()));
+ CHECK_FALSE(strcmp("(not set)", max.c_str()));
+ }
+}
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_service.h author Maya Dagon <mdagon@cisco.com>
+
+#ifndef EXTRACTOR_SERVICE_H
+#define EXTRACTOR_SERVICE_H
+
+#include <algorithm>
+#include <string>
+#include <vector>
+
+#include "extractor_logger.h"
+
+class ServiceConfig;
+
+class ServiceType
+{
+public:
+ enum Value : uint8_t
+ {
+ HTTP,
+ UNDEFINED,
+ MAX
+ };
+
+ ServiceType() = default;
+ constexpr ServiceType(Value a) : v(a) {}
+ template<typename T> constexpr ServiceType(T a) : v(Value(a)) {}
+
+ constexpr operator Value() const { return v; }
+ explicit operator bool() const = delete;
+
+ const char* c_str() const
+ {
+ switch (v)
+ {
+ case HTTP:
+ return "http";
+ case UNDEFINED: // fallthrough
+ case MAX: // fallthrough
+ default:
+ return "(not set)";
+ }
+ }
+
+private:
+ Value v = UNDEFINED;
+};
+
+struct ServiceBlueprint
+{
+ std::vector<std::string> supported_events;
+ std::vector<std::string> supported_fields;
+};
+
+// FIXIT-P: make a template with Logger and Writer as parameters
+class ExtractorService
+{
+public:
+ static ExtractorService* make_service(const ServiceConfig&, FormatType, OutputType);
+
+ ExtractorService() = delete;
+ ExtractorService(const ExtractorService&) = delete;
+ ExtractorService& operator=(const ExtractorService&) = delete;
+ ExtractorService(ExtractorService&&) = delete;
+
+ virtual ~ExtractorService()
+ { delete logger; }
+
+ void show(std::string&) const;
+ uint32_t get_tenant() const { return tenant_id; }
+ const std::vector<std::string>& get_events() const { return events; }
+ const std::vector<std::string>& get_fields() const { return fields; }
+
+protected:
+ ExtractorService(uint32_t tenant, const std::vector<std::string>& fields, const std::vector<std::string>& events,
+ const ServiceBlueprint& srv_bp, ServiceType, FormatType, OutputType);
+ void add_events(const std::vector<std::string>& vals);
+ void add_fields(const std::vector<std::string>& vals);
+ bool find_event(const std::string&) const;
+ bool find_field(const std::string&) const;
+
+ static std::vector<std::string> common_fields;
+
+ const uint32_t tenant_id;
+ std::vector<std::string> fields;
+ std::vector<std::string> events;
+
+ ExtractorLogger* logger = nullptr;
+
+ const ServiceBlueprint& sbp;
+ const ServiceType type;
+};
+
+class HttpExtractorService : public ExtractorService
+{
+public:
+ HttpExtractorService(uint32_t tenant, const std::vector<std::string>& fields,
+ const std::vector<std::string>& events, ServiceType, FormatType, OutputType);
+
+private:
+ static ServiceBlueprint blueprint;
+};
+
+#endif
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_writer.cc author Anna Norokh <anorokh@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "extractor_writer.h"
+
+ExtractorWriter* ExtractorWriter::make_writer(OutputType o_type)
+{
+ switch (o_type)
+ {
+ case OutputType::STD:
+ return new StdExtractorWriter();
+ case OutputType::MAX: // fallthrough
+ default:
+ return nullptr;
+ }
+}
+
+#ifdef UNIT_TEST
+
+#include "catch/snort_catch.h"
+
+#include <memory.h>
+
+using namespace snort;
+
+TEST_CASE("Output Type", "[extractor]")
+{
+ SECTION("to string")
+ {
+ OutputType std = OutputType::STD;
+ OutputType max = OutputType::MAX;
+
+ CHECK_FALSE(strcmp("stdout", std.c_str()));
+ CHECK_FALSE(strcmp("(not set)", max.c_str()));
+ }
+}
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// extractor_writer.h author Anna Norokh <anorokh@cisco.com>
+
+#ifndef EXTRACTOR_WRITER_H
+#define EXTRACTOR_WRITER_H
+
+#include <mutex>
+#include <string>
+
+class OutputType
+{
+public:
+ enum Value : uint8_t
+ {
+ STD,
+ MAX
+ };
+
+ OutputType() = default;
+ constexpr OutputType(Value a) : v(a) {}
+ template<typename T> constexpr OutputType(T a) : v((Value)a) {}
+
+ constexpr operator Value() const { return v; }
+ explicit operator bool() const = delete;
+
+ const char* c_str() const
+ {
+ switch (v)
+ {
+ case STD:
+ return "stdout";
+ case MAX: // fallthrough
+ default:
+ return "(not set)";
+ }
+ }
+
+private:
+ Value v = STD;
+};
+
+class ExtractorWriter
+{
+public:
+ static ExtractorWriter* make_writer(OutputType);
+
+ ExtractorWriter(const ExtractorWriter&) = delete;
+ ExtractorWriter& operator=(const ExtractorWriter&) = delete;
+ ExtractorWriter(ExtractorWriter&&) = delete;
+
+ virtual ~ExtractorWriter() = default;
+
+ virtual void write(const char*) = 0;
+ virtual void lock() { }
+ virtual void unlock() { }
+
+protected:
+ ExtractorWriter() = default;
+};
+
+class StdExtractorWriter : public ExtractorWriter
+{
+public:
+ StdExtractorWriter() = default;
+
+ void write(const char* ss) override
+ { fprintf(stdout, "%s", ss); }
+
+ void lock() override
+ { write_mutex.lock(); }
+
+ void unlock() override
+ { write_mutex.unlock(); }
+
+private:
+ std::mutex write_mutex;
+};
+
+#endif
return;
int32_t body_len = 0;
-
const char* body = (const char*)he->get_client_body(body_len);
- body_len = std::min(config.client_body_depth, body_len);
-
if (!body || body_len <= 0)
return;
+ const size_t len = std::min((size_t)config.client_body_depth, (size_t)body_len);
+
assert(classifier);
float output = 0.0;
kaizen_stats.libml_calls++;
- if (!classifier->run(body, (size_t)body_len, output))
+ if (!classifier->run(body, len, output))
return;
- kaizen_stats.client_body_bytes += body_len;
+ kaizen_stats.client_body_bytes += len;
- debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "input (body): %.*s\n", body_len, body);
+ debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "input (body): %.*s\n", (int)len, body);
debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "output: %f\n", static_cast<double>(output));
if ((double)output > config.http_param_threshold)
int32_t query_len = 0;
const char* query = (const char*)he->get_uri_query(query_len);
- query_len = std::min(config.uri_depth, query_len);
-
if (!query || query_len <= 0)
return;
+ const size_t len = std::min((size_t)config.uri_depth, (size_t)query_len);
+
assert(classifier);
float output = 0.0;
kaizen_stats.libml_calls++;
- if (!classifier->run(query, (size_t)query_len, output))
+ if (!classifier->run(query, (size_t)len, output))
return;
- kaizen_stats.uri_bytes += query_len;
+ kaizen_stats.uri_bytes += len;
- debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "input (query): %.*s\n", query_len, query);
+ debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "input (query): %.*s\n", (int)len, query);
debug_logf(kaizen_trace, TRACE_CLASSIFIER, nullptr, "output: %f\n", static_cast<double>(output));
if ((double)output > config.http_param_threshold)
void Kaizen::show(const SnortConfig*) const
{
- ConfigLogger::log_value("uri_depth", config.uri_depth);
- ConfigLogger::log_value("client_body_depth", config.client_body_depth);
+ ConfigLogger::log_limit("uri_depth", config.uri_depth, -1);
+ ConfigLogger::log_limit("client_body_depth", config.client_body_depth, -1);
ConfigLogger::log_value("http_param_threshold", config.http_param_threshold);
}
bool Kaizen::configure(SnortConfig* sc)
{
- if (config.uri_depth > 0)
+ if (config.uri_depth != 0)
DataBus::subscribe(http_pub_key, HttpEventIds::REQUEST_HEADER, new HttpUriHandler(*this));
- if (config.client_body_depth > 0)
+ if (config.client_body_depth != 0)
DataBus::subscribe(http_pub_key, HttpEventIds::REQUEST_BODY, new HttpBodyHandler(*this));
if(!InspectorManager::get_inspector(KZ_ENGINE_NAME, true, sc))
"Field::length maximum value should not exceed client_body_depth type range");
if (v.is("uri_depth"))
- {
conf.uri_depth = v.get_int32();
- if (conf.uri_depth == -1)
- conf.uri_depth = INT32_MAX;
- }
else if (v.is("client_body_depth"))
- {
conf.client_body_depth = v.get_int32();
- if (conf.client_body_depth == -1)
- conf.client_body_depth = INT32_MAX;
- }
else if (v.is("http_param_threshold"))
conf.http_param_threshold = v.get_real();
extern const BaseApi* nin_reputation;
extern const BaseApi* nin_appid[];
+extern const BaseApi* nin_extractor[];
extern const BaseApi* nin_kaizen_engine[];
extern const BaseApi* nin_kaizen[];
extern const BaseApi* nin_port_scan[];
{
PluginManager::load_plugins(network_inspectors);
PluginManager::load_plugins(nin_appid);
+ PluginManager::load_plugins(nin_extractor);
PluginManager::load_plugins(nin_kaizen_engine);
PluginManager::load_plugins(nin_kaizen);
PluginManager::load_plugins(nin_port_scan);
return dlt;
}
+static int _pcap_compile_nopcap(int snaplen_arg, int linktype_arg,
+ struct bpf_program *program,
+ const char *buf, int optimize, bpf_u_int32 mask)
+{
+ pcap_t *p;
+ int ret;
+
+ p = pcap_open_dead(linktype_arg, snaplen_arg);
+ if (p == NULL)
+ return (PCAP_ERROR);
+ ret = pcap_compile(p, program, buf, optimize, mask);
+ pcap_close(p);
+ return (ret);
+}
+
static bool bpf_compile_and_validate()
{
// FIXIT-M This BPF compilation is not thread-safe and should be handled by the main thread
- if ( pcap_compile_nopcap(SNAP_LEN, get_dlt(), &bpf,
+ if ( _pcap_compile_nopcap(SNAP_LEN, get_dlt(), &bpf,
config.filter.c_str(), 1, 0) >= 0 )
{
if (bpf_validate(bpf.bf_insns, bpf.bf_len))
mod_ctor,
mod_dtor
},
- IT_PROBE,
+ IT_PROBE_FIRST,
PROTO_BIT__ANY_IP | PROTO_BIT__ETH,
nullptr, // buffers
nullptr, // service
static bool bpf_compile_and_validate_test()
{
- if (pcap_compile_nopcap(SNAP_LEN, DLT_EN10MB, &bpf,
+ if (_pcap_compile_nopcap(SNAP_LEN, DLT_EN10MB, &bpf,
config.filter.c_str(), 1, 0) >= 0)
{
if (bpf_validate(bpf.bf_insns, bpf.bf_len))
3. JSON
-===== File Layout
+==== File Layout
[options="header"]
|============================================================================
|Records |This is a stream of records. There may be an unlimited number.
|============================================================================
-===== File Header
+==== File Header
[options="header"]
|==========================================================================
|Schema |(schema size) bytes |Schema for parsing records in this file.
|==========================================================================
-===== Record
+==== Record
[options="header"]
|===========================================================================
#include "flow_ip_tracker.h"
+#include <appid/appid_api.h>
+#include "flow/stream_flow.h"
+#include "framework/pig_pen.h"
#include "hash/hash_defs.h"
#include "log/messages.h"
#include "protocols/packet.h"
+#include "perf_monitor.h"
#include "perf_pegs.h"
using namespace snort;
};
FlowStateValue* FlowIPTracker::find_stats(const SfIp* src_addr, const SfIp* dst_addr,
- int* swapped)
+ int* swapped, const char* appid_name, uint16_t src_port, uint16_t dst_port,
+ uint8_t ip_protocol, uint64_t flow_latency, uint64_t rule_latency)
{
FlowStateKey key;
FlowStateValue* value = nullptr;
}
value = (FlowStateValue*)ip_map->get_user_data(&key);
- if ( !value )
+ if ( value )
+ {
+ strncpy(value->appid_name, appid_name, sizeof(value->appid_name) - 1);
+ value->appid_name[sizeof(value->appid_name) - 1] = '\0';
+ if ( *swapped )
+ {
+ value->port_a = dst_port;
+ value->port_b = src_port;
+ }
+ else
+ {
+ value->port_a = src_port;
+ value->port_b = dst_port;
+ }
+ value->protocol = ip_protocol;
+ value->total_flow_latency = flow_latency;
+ value->total_rule_latency = rule_latency;
+ }
+ else
{
if ( ip_map->insert(&key, nullptr) != HASH_OK )
return nullptr;
&stats.state_changes[SFS_STATE_TCP_CLOSED]);
formatter->register_field("udp_created", (PegCount*)
&stats.state_changes[SFS_STATE_UDP_CREATED]);
+ formatter->register_field("app_id", appid_name);
+ formatter->register_field("port_a", port_a);
+ formatter->register_field("port_b", port_b);
+ formatter->register_field("protocol", protocol);
+ formatter->register_field("flow_latency", flow_latency);
+ formatter->register_field("rule_latency", rule_latency);
formatter->finalize_fields();
stats.total_packets = stats.total_bytes = 0;
const SfIp* src_addr = p->ptrs.ip_api.get_src();
const SfIp* dst_addr = p->ptrs.ip_api.get_dst();
+ char curr_appid_name[40] = {};
+ uint16_t src_port = 0;
+ uint16_t dst_port = 0;
+ uint8_t ip_protocol = 0;
+ uint64_t curr_flow_latency = 0;
+ uint64_t curr_rule_latency = 0;
+
+ PerfMonitor* perf_monitor = (PerfMonitor*)PigPen::get_inspector(PERF_NAME, true);
+ if ( perf_monitor->get_constraints()->flow_ip_all == true )
+ {
+ if ( p->flow )
+ {
+ src_port = p->ptrs.sp;
+ dst_port = p->ptrs.dp;
+
+ const AppIdSessionApi* appid_session_api = appid_api.get_appid_session_api(*p->flow);
+ if ( appid_session_api )
+ {
+ AppId service_id = APP_ID_NONE;
+ appid_session_api->get_app_id(&service_id, nullptr, nullptr, nullptr, nullptr);
+ const char* app_name = appid_api.get_application_name(service_id, *p->flow);
+ if ( app_name )
+ {
+ strncpy(curr_appid_name, app_name, sizeof(curr_appid_name) - 1);
+ curr_appid_name[sizeof(curr_appid_name) - 1] = '\0';
+ }
+ }
+ ip_protocol = p->flow->ip_proto;
+ curr_flow_latency = p->flow->flowstats.total_flow_latency;
+ curr_rule_latency = p->flow->flowstats.total_rule_latency;
+ }
+ }
int len = p->pktlen;
if (p->ptrs.tcph)
else if (p->ptrs.udph)
type = SFS_TYPE_UDP;
- FlowStateValue* value = find_stats(src_addr, dst_addr, &swapped);
+ FlowStateValue* value = find_stats(src_addr, dst_addr, &swapped, curr_appid_name,
+ src_port, dst_port, ip_protocol, curr_flow_latency, curr_rule_latency);
if ( !value )
return;
key->ipA.ntop(ip_a, sizeof(ip_a));
key->ipB.ntop(ip_b, sizeof(ip_b));
+
+ if (cur_stats->appid_name[0] != '\0')
+ strncpy(appid_name, cur_stats->appid_name, sizeof(appid_name) - 1);
+ else
+ strncpy(appid_name, "APPID_NONE", sizeof(appid_name) - 1);
+ appid_name[sizeof(appid_name) - 1] = '\0';
+
+ std::snprintf(port_a, sizeof(port_a), "%d", cur_stats->port_a);
+ std::snprintf(port_b, sizeof(port_b), "%d", cur_stats->port_b);
+ std::snprintf(protocol, sizeof(protocol), "%d", cur_stats->protocol);
+ std::snprintf(flow_latency, sizeof(flow_latency), "%lu", cur_stats->total_flow_latency);
+ std::snprintf(rule_latency, sizeof(rule_latency), "%lu", cur_stats->total_rule_latency);
+
memcpy(&stats, cur_stats, sizeof(stats));
write();
reset();
}
-int FlowIPTracker::update_state(const SfIp* src_addr, const SfIp* dst_addr, FlowState state)
+int FlowIPTracker::update_state(const SfIp* src_addr, const SfIp* dst_addr,
+ FlowState state, const char* appid_name, uint16_t src_port, uint16_t dst_port,
+ uint8_t ip_protocol, uint64_t flow_latency, uint64_t rule_latency)
{
int swapped;
- FlowStateValue* value = find_stats(src_addr, dst_addr, &swapped);
+ FlowStateValue* value = find_stats(src_addr, dst_addr, &swapped, appid_name, src_port, dst_port,
+ ip_protocol, flow_latency, rule_latency);
if ( !value )
return 1;
#include "hash/xhash.h"
+#include "network_inspectors/appid/application_ids.h"
#include "perf_tracker.h"
enum FlowState
struct FlowStateValue
{
- TrafficStats traffic_stats[SFS_TYPE_MAX];
- PegCount total_packets;
- PegCount total_bytes;
- PegCount state_changes[SFS_STATE_MAX];
+ char appid_name[40] = "APPID_NONE";
+ uint16_t port_a = 0;
+ uint16_t port_b = 0;
+ uint8_t protocol = 0;
+ TrafficStats traffic_stats[SFS_TYPE_MAX] = {};
+ PegCount total_packets = 0;
+ PegCount total_bytes = 0;
+ PegCount total_flow_latency = 0;
+ PegCount total_rule_latency = 0;
+ PegCount state_changes[SFS_STATE_MAX] = {};
};
class FlowIPTracker : public PerfTracker
void reset() override;
void update(snort::Packet*) override;
void process(bool) override;
- int update_state(const snort::SfIp* src_addr, const snort::SfIp* dst_addr, FlowState);
+ int update_state(const snort::SfIp* src_addr, const snort::SfIp* dst_addr, FlowState,
+ const char* appid_name, uint16_t src_port, uint16_t dst_port, uint8_t ip_protocol,
+ uint64_t flow_latency, uint64_t rule_latency);
snort::XHash* get_ip_map()
{ return ip_map; }
private:
FlowStateValue stats;
snort::XHash* ip_map;
- char ip_a[41], ip_b[41];
+ char ip_a[41], ip_b[41], port_a[8], port_b[8], protocol[8];
+ char appid_name[40] = "APPID_NONE", flow_latency[20] = {}, rule_latency[20] = {};
int perf_flags;
PerfConfig* perf_conf;
size_t memcap;
- FlowStateValue* find_stats(const snort::SfIp* src_addr, const snort::SfIp* dst_addr, int* swapped);
+ FlowStateValue* find_stats(const snort::SfIp* src_addr, const snort::SfIp* dst_addr,
+ int* swapped, const char* appid_name, uint16_t src_port, uint16_t dst_port,
+ uint8_t ip_protocol, uint64_t flow_latency, uint64_t rule_latency);
void write_stats();
void display_stats();
{ "flow_ip", Parameter::PT_BOOL, nullptr, "false",
"enable statistics on host pairs" },
+ { "flow_ip_all", Parameter::PT_BOOL, nullptr, "false",
+ "enable every stat of flow_ip profiling on host pairs" },
+
{ "packets", Parameter::PT_INT, "0:max32", "10000",
"minimum packets to report" },
PerfMonitor* perf_monitor;
};
+static bool current_packet_latency, current_rule_latency = false;
+
static const Parameter flow_ip_profiling_params[] =
{
{ "seconds", Parameter::PT_INT, "1:max32", nullptr,
{ "packets", Parameter::PT_INT, "0:max32", nullptr,
"minimum packets to report" },
+ { "flow_ip_all", Parameter::PT_BOOL, nullptr, nullptr,
+ "enable all flow ip statistics" },
+
{ nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
};
}
auto* new_constraints = new PerfConstraints(true, luaL_optint(L, 1, 0),
- luaL_optint(L, 2, 0));
+ luaL_optint(L, 2, 0), luaL_opt(L,lua_toboolean, 3, false));
ControlConn* ctrlcon = ControlConn::query_from_lua(L);
main_broadcast_command(new PerfMonFlowIPDebug(new_constraints, true, perf_monitor), ctrlcon);
- LogMessage("Enabling flow ip profiling with sample interval %d packet count %d\n",
- new_constraints->sample_interval, new_constraints->pkt_cnt);
+ if ( new_constraints->flow_ip_all )
+ {
+ const SnortConfig* sc = SnortConfig::get_conf();
+ if ( sc->get_packet_latency() )
+ current_packet_latency = true;
+ if ( sc->get_rule_latency() )
+ current_rule_latency = true;
+ sc->set_packet_latency(true);
+ sc->set_rule_latency(true);
+ }
+
+ LogMessage("Enabling flow ip profiling with sample interval %d packet count %d all stats tracking %s\n",
+ new_constraints->sample_interval, new_constraints->pkt_cnt,
+ ( new_constraints->flow_ip_all ) ? "enabled" : "disabled" );
return 0;
}
ControlConn* ctrlcon = ControlConn::query_from_lua(L);
main_broadcast_command(new PerfMonFlowIPDebug(new_constraints, false, perf_monitor), ctrlcon);
+ const SnortConfig* sc = SnortConfig::get_conf();
+
+ if ( !current_packet_latency )
+ sc->set_packet_latency(false);
+
+ if ( !current_rule_latency )
+ sc->set_rule_latency(false);
+
LogMessage("Disabling flow ip profiling\n");
return 0;
static const Command perf_module_cmds[] =
{
{ "enable_flow_ip_profiling", enable_flow_ip_profiling,
- flow_ip_profiling_params, "enable statistics on host pairs" },
+ flow_ip_profiling_params, "enable all statistics on host pairs" },
{ "disable_flow_ip_profiling", disable_flow_ip_profiling,
nullptr, "disable statistics on host pairs" },
{
config->sample_interval = v.get_uint32();
}
+ else if ( v.is("flow_ip_all") )
+ {
+ if ( v.get_bool() )
+ config->flow_ip_all = true;
+ }
else if ( v.is("flow_ip_memcap") )
{
config->flowip_memcap = v.get_size();
if ( idx != 0 && strcmp(fqn, "perf_monitor.modules") == 0 )
return config->modules.back().confirm_parse();
+ if ( config->flow_ip_all )
+ {
+ sc->set_packet_latency(true);
+ sc->set_rule_latency(true);
+ }
+
return true;
}
tmp->constraints->flow_ip_enabled = config->perf_flags & PERF_FLOWIP;
tmp->constraints->sample_interval = config->sample_interval;
tmp->constraints->pkt_cnt = config->pkt_cnt;
+ tmp->constraints->flow_ip_all = config->flow_ip_all;
config = nullptr;
return tmp;
bool flow_ip_enabled = false;
unsigned sample_interval = 0;
uint32_t pkt_cnt = 0;
+ bool flow_ip_all = false;
PerfConstraints() = default;
- PerfConstraints(bool en, unsigned interval, uint32_t cnt) :
- flow_ip_enabled(en), sample_interval(interval), pkt_cnt(cnt) { }
+ PerfConstraints(bool en, unsigned interval, uint32_t cnt, bool lat) :
+ flow_ip_enabled(en), sample_interval(interval), pkt_cnt(cnt), flow_ip_all(lat) { }
};
struct PerfConfig
uint64_t max_file_size = 0;
int flow_max_port_to_track = 0;
size_t flowip_memcap = 0;
+ bool flow_ip_all = false;
PerfFormat format = PerfFormat::CSV;
PerfOutput output = PerfOutput::TO_FILE;
std::vector<ModuleConfig> modules;
#include "perf_monitor.h"
+#include <appid/appid_api.h>
+#include "flow/stream_flow.h"
#include "framework/data_bus.h"
#include "framework/pig_pen.h"
#include "hash/hash_defs.h"
{
FlowIPTracker* tracker = perf_monitor.get_flow_ip();
- if (!tracker)
+ if (!tracker or !flow)
return;
FlowState state = SFS_STATE_MAX;
if ( state == SFS_STATE_MAX )
return;
- tracker->update_state(&flow->client_ip, &flow->server_ip, state);
+ char appid_name[40] = {};
+ uint16_t src_port = 0;
+ uint16_t dst_port = 0;
+ uint8_t ip_protocol = 0;
+ uint64_t flow_latency = 0;
+ uint64_t rule_latency = 0;
+
+ if ( perf_monitor.get_constraints()->flow_ip_all )
+ {
+ const AppIdSessionApi* appid_session_api = appid_api.get_appid_session_api(*flow);
+ if ( appid_session_api )
+ {
+ AppId service_id = APP_ID_NONE;
+ appid_session_api->get_app_id(&service_id, nullptr, nullptr, nullptr, nullptr);
+ const char* app_name = appid_api.get_application_name(service_id, *flow);
+ if ( app_name )
+ {
+ strncpy(appid_name, app_name, sizeof(appid_name) - 1);
+ appid_name[sizeof(appid_name) - 1] = '\0';
+ }
+ }
+ src_port = flow->client_port;
+ dst_port = flow->server_port;
+ ip_protocol = flow->ip_proto;
+ flow_latency = flow->flowstats.total_flow_latency;
+ rule_latency = flow->flowstats.total_rule_latency;
+ }
+
+ tracker->update_state(&flow->client_ip, &flow->server_ip, state, appid_name,
+ src_port, dst_port, ip_protocol, flow_latency, rule_latency);
}
private:
{
switch (po)
{
- case PerfOutput::TO_CONSOLE:
- return "console";
- case PerfOutput::TO_FILE:
- return "file";
+ case PerfOutput::TO_CONSOLE:
+ return "console";
+ case PerfOutput::TO_FILE:
+ return "file";
}
return "";
{
switch (pf)
{
- case PerfFormat::TEXT:
- return "text";
- case PerfFormat::CSV:
- return "csv";
- case PerfFormat::JSON:
- return "json";
- case PerfFormat::MOCK:
- return "mock";
+ case PerfFormat::TEXT:
+ return "text";
+ case PerfFormat::CSV:
+ return "csv";
+ case PerfFormat::JSON:
+ return "json";
+ case PerfFormat::MOCK:
+ return "mock";
}
return "";
ConfigLogger::log_value("flow_ports", config->flow_max_port_to_track);
if ( ConfigLogger::log_flag("flow_ip", config->perf_flags & PERF_FLOWIP) )
+ {
ConfigLogger::log_value("flow_ip_memcap", config->flowip_memcap);
+ ConfigLogger::log_value("flow_ip_all", config->flow_ip_all);
+ }
ConfigLogger::log_value("packets", config->pkt_cnt);
ConfigLogger::log_value("seconds", config->sample_interval);
PerfConstraints* PerfMonitor::get_original_constraints()
{
auto* new_constraints = new PerfConstraints(false, config->sample_interval,
- config->pkt_cnt);
+ config->pkt_cnt, config->flow_ip_all);
return new_constraints;
}
${TEST_FILES}
)
+add_subdirectory ( test )
+
install (FILES ${PACKET_IO_INCLUDES}
DESTINATION "${INCLUDE_INSTALL_PATH}/packet_io"
)
#include <algorithm>
#include <cstring>
+#include "flow/flow_key.h"
#include "protocols/packet.h"
namespace {
return false;
return match_constraints(*this, f.client_ip, f.server_ip, f.client_port,
- f.server_port, f.tenant);
+ f.server_port,
+#ifndef DISABLE_TENANT_ID
+ f.key->tenant_id);
+#else
+ 0);
+#endif
}
#ifdef UNIT_TEST
#include "protocols/ip.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
+#include "protocols/tcp_options.h"
#include "utils/util.h"
#include "active.h"
{ return s_pkt_trace ? s_pkt_trace->daq_activated : false; }
#endif
+static std::string stringify_tcp_options(const Packet* const pkt)
+{
+ std::ostringstream oss;
+ tcp::TcpOptIterator iter(pkt->ptrs.tcph, pkt);
+
+ for (const tcp::TcpOption& opt : iter)
+ {
+ switch (opt.code)
+ {
+ case tcp::TcpOptCode::WSCALE:
+ oss << "ws " << (uint16_t)opt.data[0] << ", ";
+ break;
+ case tcp::TcpOptCode::MAXSEG:
+ oss << "mss " << ntohs(*((const uint16_t*)(opt.data)) ) << ", ";
+ break;
+ case tcp::TcpOptCode::SACKOK:
+ oss << "sack OK, ";
+ break;
+ default:
+ break;
+ }
+ }
+ std::string opts = oss.str();
+ if (!opts.empty())
+ {
+ opts.insert(0, "options [");
+ opts.replace(opts.size() - 2, 2, "] ");
+ }
+ return opts;
+}
+
void PacketTracer::set_log_file(const std::string& file)
{ log_file = file; }
char tcpFlags[10];
p.ptrs.tcph->stringify_flags(tcpFlags);
+ std::string opts;
+ if (p.ptrs.tcph->th_flags & TH_SYN)
+ opts = stringify_tcp_options(&p);
+
if (p.ptrs.tcph->th_flags & TH_ACK)
- PacketTracer::log("Packet %" PRIu64 ": TCP %s, %s, seq %u, ack %u, dsize %u%s\n",
+ PacketTracer::log("Packet %" PRIu64 ": TCP %s, %s, seq %u, ack %u, win %u, %sdsize %u%s\n",
p.context->packet_number, tcpFlags, timestamp,
- p.ptrs.tcph->seq(), p.ptrs.tcph->ack(), p.dsize,
+ p.ptrs.tcph->seq(), p.ptrs.tcph->ack(), p.ptrs.tcph->win(), opts.c_str(), p.dsize,
p.is_retry() ? ", retry pkt" : "");
else
- PacketTracer::log("Packet %" PRIu64 ": TCP %s, %s, seq %u, dsize %u%s\n",
+ PacketTracer::log("Packet %" PRIu64 ": TCP %s, %s, seq %u, win %u, %sdsize %u%s\n",
p.context->packet_number, tcpFlags, timestamp, p.ptrs.tcph->seq(),
- p.dsize,
+ p.ptrs.tcph->win(), opts.c_str(), p.dsize,
p.is_retry() ? ", retry pkt" : "");
+ DAQ_PktTcpAckData_t* tcp_mack = (DAQ_PktTcpAckData_t*)p.daq_msg->meta[DAQ_PKT_META_TCP_ACK_DATA];
+ if ( tcp_mack )
+ PacketTracer::log("Meta_ack: ack %u, win %u\n",
+ ntohl(tcp_mack->tcp_ack_seq_num), ntohs(tcp_mack->tcp_window_size));
break;
}
ret.packets_received = left.packets_received - right.packets_received;
ret.packets_filtered = left.packets_filtered - right.packets_filtered;
ret.packets_injected = left.packets_injected - right.packets_injected;
+ ret.packets_outstanding = left.packets_outstanding - right.packets_outstanding;
for ( unsigned i = 0; i < MAX_DAQ_VERDICT; i++ )
ret.verdicts[i] = left.verdicts[i] - right.verdicts[i];
for ( unsigned i = 0; i < MAX_DAQ_VERDICT; i++ )
daq_stats.verdicts[i] = daq_stats_delta.verdicts[i];
- daq_stats.outstanding = new_daq_stats.hw_packets_received -
- new_daq_stats.packets_filtered - new_daq_stats.packets_received;
+ daq_stats.outstanding = new_daq_stats.packets_outstanding;
+
if ( daq_stats.outstanding > daq_stats.outstanding_max )
daq_stats.outstanding_max = daq_stats.outstanding;
--- /dev/null
+add_cpputest(sfdaq_counters_test
+ SOURCES
+ ../sfdaq_module.cc
+)
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// sfdaq_counters_test.cc author Arunkumar Kayambu <akayambu@cisco.com>
+
+// -----------------------------------------------------------------------------
+// unit tests
+// -----------------------------------------------------------------------------
+
+#include "packet_io/sfdaq.h"
+#include "packet_io/sfdaq_module.h"
+#include "sfdaq_module_stubs.h"
+
+#include <CppUTest/CommandLineTestRunner.h>
+#include <CppUTest/TestHarness.h>
+#include <CppUTestExt/MockSupport.h>
+
+using namespace snort;
+
+static DAQ_Stats_t* mock_stats_ptr = nullptr;
+SFDAQInstance *local_instance = nullptr;
+
+const DAQ_Stats_t* SFDAQ::get_stats() {
+ mock().actualCall("get_stats");
+ mock_stats_ptr->packets_outstanding = 20;
+ return mock_stats_ptr;
+}
+
+SFDAQInstance* SFDAQ::get_local_instance() {
+ mock().actualCall("get_local_instance");
+ return local_instance;
+}
+
+TEST_GROUP(sfdaq_module_counters)
+{
+ void setup() {
+ mock_stats_ptr = new DAQ_Stats_t();
+ local_instance = new SFDAQInstance(nullptr, 0, nullptr);
+ }
+
+ void teardown() {
+ mock().clear();
+ delete mock_stats_ptr;
+ delete local_instance;
+ }
+};
+
+TEST(sfdaq_module_counters, check_outstanding_counter)
+{
+ SFDAQModule sfdm;
+ const PegInfo* infos = sfdm.get_pegs();
+
+ // Set up the expectation
+ mock().expectOneCall("get_stats");
+ mock().expectOneCall("get_local_instance");
+
+ sfdm.prep_counts(false);
+ PegCount* p = sfdm.get_counts();
+ mock().checkExpectations();
+ for ( unsigned i = 0; infos[i].name; i++ )
+ {
+ if ( strcmp(infos[i].name, "packets_outstanding") == 0 )
+ CHECK(20 == p[i]);
+ }
+}
+
+int main(int argc, char** argv)
+{
+ return CommandLineTestRunner::RunAllTests(argc, argv);
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// sfdaq_module_stubs.h author Arunkumar Kayambu <akayambu@cisco.com>
+
+#include "main/snort.h"
+#include "packet_io/sfdaq_instance.h"
+#include "packet_io/sfdaq_config.h"
+#include "../trough.h"
+
+#ifndef SFDAQ_MODULE_STUBS_H
+#define SFDAQ_MODULE_STUBS_H
+
+namespace snort
+{
+Module::Module(char const*, char const*, snort::Parameter const*, bool)
+{
+ help = nullptr;
+ name = nullptr;
+ params = nullptr;
+ list = false;
+}
+PegCount Module::get_global_count(const char*) const { return 0; }
+void Module::sum_stats(bool) { }
+void Module::init_stats(bool) { }
+void Module::main_accumulate_stats() { }
+void Module::show_interval_stats(std::vector<unsigned>&, FILE*) { }
+void Module::show_stats() { }
+void Module::reset_stats() { }
+void ParseError(char const*, ...) {}
+SFDAQInstance::SFDAQInstance(char const*, unsigned int, SFDAQConfig const*)
+{
+ batch_size = 0;
+ instance_id = 1;
+ daq_msgs = nullptr;
+}
+SFDAQInstance::~SFDAQInstance() { }
+}
+
+SFDAQConfig::SFDAQConfig()
+{
+ batch_size = 0;
+ mru_size = 0;
+ timeout = 0;
+}
+SFDAQConfig::~SFDAQConfig() = default;
+void SFDAQModuleConfig::set_variable(char const*){}
+void SFDAQConfig::add_module_dir(char const*){}
+void SFDAQConfig::add_input(char const*){}
+void SFDAQConfig::set_mru_size(int){}
+void SFDAQConfig::set_batch_size(unsigned int){}
+void SFDAQConfig::overlay(SFDAQConfig const*){}
+std::atomic<unsigned> Trough::file_count{0};
+#endif
static std::string s_special_rules;
static std::string s_special_includer;
+void show_pcre_counts();
+
class RuleTreeHashKeyOps : public HashKeyOperations
{
public:
void ParseRulesFinish(SnortConfig* sc)
{
+ show_pcre_counts();
ShowPolicyStats(sc);
if ( !sc->dump_rule_info() )
if (input_str.find('$') == std::string::npos)
return(input_str);
- for (auto i = input_str.begin(); i < input_str.end(); i++)
+ auto i = input_str.begin();
+ while (i < input_str.end())
{
const char c = *i;
if (c == '"')
quote_toggle = !quote_toggle;
}
- if (c == '$' && !quote_toggle)
+ if (c != '$' or quote_toggle)
{
- auto begin = (i+1);
- auto end = begin;
- bool name_only = *begin != '(';
- if (!name_only)
- begin++;
-
- while (*end != '\0' && (
- ( name_only && (isalnum(*end) || *end == '_') ) ||
- ( !name_only && *end != ')' ) ) ) {
- end++;
- }
+ output << c;
+ i++;
+ continue;
+ }
- std::string var_name(begin, end);
- std::string var_aux;
+ auto begin = (i+1);
+ auto end = begin;
+ bool name_only = *begin != '(';
+ if (!name_only)
+ begin++;
- i = end;
+ while (*end != '\0' and (
+ ( name_only and (isalnum(*end) or *end == '_') ) or
+ ( !name_only and *end != ')' ) ) ) {
+ end++;
+ }
- char var_modifier = ' ';
+ std::string var_name(begin, end);
+ std::string var_aux;
- size_t p = var_name.find(':');
+ i = end;
- if (p != std::string::npos)
- {
- if (var_name.size() - p >= 2)
- {
- var_modifier = var_name[p+1];
- var_aux = var_name.substr(p+2);
- }
- var_name.resize(p);
- }
+ char var_modifier = ' ';
- std::string var_contents = VarSearch(var_name);
+ size_t p = var_name.find(':');
- switch (var_modifier)
+ if (p != std::string::npos)
+ {
+ if (var_name.size() - p >= 2)
{
- case '-':
- if (var_contents.empty())
- var_contents = var_aux;
- break;
-
- case '?':
- if (var_contents.empty())
- {
- if (!var_aux.empty())
- ParseAbort("%s", var_aux.c_str());
- else
- ParseAbort("undefined variable '%s'.", var_name.c_str());
- }
- break;
+ var_modifier = var_name[p+1];
+ var_aux = var_name.substr(p+2);
}
+ var_name.resize(p);
+ }
- // If variable not defined now, we're toast
- if (var_contents.empty())
- ParseAbort("undefined variable name: %s.", var_name.c_str());
+ std::string var_contents = VarSearch(var_name);
- output << var_contents;
- }
- else
+ switch (var_modifier)
{
- output << c;
+ case '-':
+ if (var_contents.empty())
+ var_contents = var_aux;
+ break;
+
+ case '?':
+ if (var_contents.empty())
+ {
+ if (!var_aux.empty())
+ ParseAbort("%s", var_aux.c_str());
+ else
+ ParseAbort("undefined variable '%s'.", var_name.c_str());
+ }
+ break;
}
+
+ // If variable not defined now, we're toast
+ if (var_contents.empty())
+ ParseAbort("undefined variable name: %s.", var_name.c_str());
+
+ output << var_contents;
+
+ if (!name_only)
+ i++;
}
return output.str();
#define ICMPv6_NS_MIN_LEN 24
#define ICMPv6_NA_MIN_LEN 24
-#define ICMPv6_RS_MIN_LEN 24
+#define ICMPv6_RS_MIN_LEN 8
#define ICMPv6_RA_MIN_LEN 16
+#define ICMPv6_RD_MIN_LEN 40
#define ICMPV6_OPTION_SOURCE_LINKLAYER_ADDRESS 1
#define ICMPV6_OPTION_TARGET_LINKLAYER_ADDRESS 2
#define ICMPV6_OPTION_REDIRECT_HEADER 4
#define ICMPV6_OPTION_MTU 5
-//enum class Icmp6Types : std::uint8_t
enum Icmp6Types : std::uint8_t
{
DESTINATION_UNREACHABLE = 1,
uint32_t mtu;
};
+struct NDPOptionFormatBasic
+{
+ uint8_t type;
+ uint8_t length;
+ // everything from this point depends on protocol,
+ // so should be implemented in a different structure
+};
+
struct ICMP6RouterAdvertisement
{
uint8_t type;
uint16_t lifetime;
uint32_t reachable_time;
uint32_t retrans_time;
+ NDPOptionFormatBasic* options_start;
};
struct ICMP6RouterSolicitation
uint8_t code;
uint16_t csum;
uint32_t reserved;
+ NDPOptionFormatBasic* options_start;
};
struct ICMP6NodeInfo
uint16_t flags;
uint64_t nonce;
};
+
+struct ICMP6NeighborSolicitation
+{
+ uint8_t type;
+ uint8_t code;
+ uint16_t csum;
+ uint32_t reserved;
+ uint64_t target_address_1;
+ uint64_t target_address_2;
+ NDPOptionFormatBasic* options_start;
+};
+
+struct ICMP6NeighborAdvertisement
+{
+ uint8_t type;
+ uint8_t code;
+ uint16_t csum;
+ uint32_t flags; // flags (3 bit) + reserved bytes
+ uint64_t target_address_1;
+ uint64_t target_address_2;
+ NDPOptionFormatBasic* options_start;
+};
+
+struct ICMP6Redirect
+{
+ uint8_t type;
+ uint8_t code;
+ uint16_t csum;
+ uint32_t reserved;
+ uint64_t target_address_1;
+ uint64_t target_address_2;
+ uint64_t dst_address_1;
+ uint64_t dst_address_2;
+ NDPOptionFormatBasic* options_start;
+};
+
} // namespace icmp
} // namespace snort
ActiveAction** action = nullptr;
ActiveAction* action_inst = nullptr;
- DAQ_Msg_h daq_msg; // DAQ message this packet came from
+ DAQ_Msg_h daq_msg = nullptr; // DAQ message this packet came from
SFDAQInstance* daq_instance = nullptr; // DAQ instance the message came from
// Everything beyond this point is set by PacketManager::decode()
http_event_ids.h
http_events.h
http_request_body_event.h
+ http_transaction_end_event.h
intrinsic_event_ids.h
netflow_event.h
opportunistic_tls_event.h
http_events.cc
dns_events.cc
http_request_body_event.cc
+ http_transaction_end_event.cc
sip_events.cc
)
REQUEST_HEADER,
RESPONSE_HEADER,
REQUEST_BODY,
+ END_OF_TRANSACTION,
num_ids
}; };
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// http_transaction_end_event.cc author Maya Dagon <mdagon@cisco.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "http_transaction_end_event.h"
+
+#include "service_inspectors/http_inspect/http_enum.h"
+#include "service_inspectors/http_inspect/http_msg_header.h"
+#include "service_inspectors/http_inspect/http_msg_request.h"
+#include "service_inspectors/http_inspect/http_msg_section.h"
+#include "service_inspectors/http_inspect/http_msg_status.h"
+#include "service_inspectors/http_inspect/http_transaction.h"
+
+using namespace snort;
+
+HttpTransactionEndEvent::HttpTransactionEndEvent(const HttpTransaction* const trans)
+ : transaction(trans) { }
+
+const Field& HttpTransactionEndEvent::get_client_header(uint64_t sub_id) const
+{
+ HttpMsgHeader* headers = transaction->get_header(HttpCommon::SRC_CLIENT);
+ if (headers == nullptr)
+ return Field::FIELD_NULL;
+
+ return headers->get_classic_buffer(HttpEnums::HTTP_BUFFER_HEADER, sub_id, 0);
+}
+
+const Field& HttpTransactionEndEvent::get_host_hdr() const
+{
+ return get_client_header(HttpEnums::HEAD_HOST);
+}
+
+const Field& HttpTransactionEndEvent::get_user_agent() const
+{
+ return get_client_header(HttpEnums::HEAD_USER_AGENT);
+}
+
+const Field& HttpTransactionEndEvent::get_referer_hdr() const
+{
+ return get_client_header(HttpEnums::HEAD_REFERER);
+}
+
+const Field& HttpTransactionEndEvent::get_origin_hdr() const
+{
+ return get_client_header(HttpEnums::HEAD_ORIGIN);
+}
+
+const Field& HttpTransactionEndEvent::get_uri() const
+{
+ if (transaction->get_request() == nullptr)
+ return Field::FIELD_NULL;
+
+ return transaction->get_request()->get_classic_buffer(HttpEnums::HTTP_BUFFER_URI, 0, 0);
+}
+
+const Field& HttpTransactionEndEvent::get_method() const
+{
+ if (transaction->get_request() == nullptr)
+ return Field::FIELD_NULL;
+
+ return transaction->get_request()->get_method();
+}
+
+const Field& HttpTransactionEndEvent::get_stat_code() const
+{
+ if (transaction->get_status() == nullptr)
+ return Field::FIELD_NULL;
+
+ return transaction->get_status()->get_status_code();
+}
+
+const Field& HttpTransactionEndEvent::get_stat_msg() const
+{
+ if (transaction->get_status() == nullptr)
+ return Field::FIELD_NULL;
+
+ return transaction->get_status()->get_reason_phrase();
+}
+
+HttpEnums::VersionId HttpTransactionEndEvent::get_version() const
+{
+ auto status = transaction->get_status();
+ if (!status and !transaction->get_request())
+ return HttpEnums::VERS__NOT_PRESENT;
+ return status ? status->get_version_id() : transaction->get_request()->get_version_id();
+}
+
+uint64_t HttpTransactionEndEvent::get_trans_depth() const
+{
+ if (transaction->get_request() != nullptr)
+ return transaction->get_request()->get_transaction_id();
+ if (transaction->get_status() != nullptr)
+ return transaction->get_status()->get_transaction_id();
+
+ return 0;
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// http_transaction_end_event.h author Maya Dagon <mdagon@cisco.com>
+
+#ifndef HTTP_TRANSACTION_END_EVENT_H
+#define HTTP_TRANSACTION_END_EVENT_H
+
+#include "framework/data_bus.h"
+#include "service_inspectors/http_inspect/http_enum.h"
+#include "service_inspectors/http_inspect/http_field.h"
+
+#include "http_event_ids.h"
+
+class HttpFlowData;
+class HttpMsgRequest;
+class HttpMsgStatus;
+class HttpTransaction;
+
+namespace snort
+{
+// This event is published each time a transaction is ending
+class SO_PUBLIC HttpTransactionEndEvent : public snort::DataEvent
+{
+public:
+ HttpTransactionEndEvent(const HttpTransaction* const);
+
+ const Field& get_host_hdr() const;
+ const Field& get_uri() const;
+ const Field& get_method() const;
+ const Field& get_stat_code() const;
+ const Field& get_stat_msg() const;
+ const Field& get_user_agent() const;
+ const Field& get_referer_hdr() const;
+ const Field& get_origin_hdr() const;
+ HttpEnums::VersionId get_version() const;
+ uint64_t get_trans_depth() const;
+
+private:
+ const Field& get_client_header(uint64_t sub_id) const;
+
+ const HttpTransaction* const transaction;
+};
+}
+#endif
SOURCES
../eve_process_event.h
)
+add_cpputest( pub_sub_http_transaction_end_event_test
+ SOURCES
+ ../http_transaction_end_event.cc
+ ../../service_inspectors/http_inspect/http_transaction.cc
+ ../../service_inspectors/http_inspect/http_flow_data.cc
+ ../../service_inspectors/http_inspect/http_test_manager.cc
+ ../../service_inspectors/http_inspect/http_test_input.cc
+ LIBS ${ZLIB_LIBRARIES}
+)
: flow(nullptr), packet_flags(0), xtradata_mask(0), proto_bits(0), alt_dsize(0), num_layers(0),
disable_inspect(true), sect(PS_NONE), active_inst(nullptr), pkth(nullptr), pkt(nullptr), layers(nullptr),
user_inspection_policy_id(0), user_ips_policy_id(0), user_network_policy_id(0), inspection_started_timestamp(0), vlan_idx(0),
- ts_packet_flags(0), allocated(false)
+ ts_packet_flags(0), allocated(false), daq_msg(nullptr)
{ }
Packet::~Packet() = default;
session_data(session_data_),
flow(flow_),
params(params_),
- transaction(HttpTransaction::attach_my_transaction(session_data, source_id_)),
+ transaction(HttpTransaction::attach_my_transaction(session_data, source_id_, flow)),
trans_num(STAT_NOT_PRESENT),
status_code_num(STAT_NOT_PRESENT),
source_id(source_id_),
void HttpMsgSection::update_depth() const{}
bool HttpMsgSection::run_detection(snort::Packet*) { return true; }
-HttpTransaction*HttpTransaction::attach_my_transaction(HttpFlowData*, HttpCommon::SourceId)
+HttpTransaction*HttpTransaction::attach_my_transaction(HttpFlowData*, HttpCommon::SourceId, snort::Flow*)
{ return nullptr; }
Field::Field(int32_t length, const uint8_t* start, bool own_the_buffer_) :
strt(start), len(length), own_the_buffer(own_the_buffer_)
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// pub_sub_http_transaction_end_event_test.cc author Maya Dagon <mdagon@cisco.com>
+// Unit test for the HttpTransactionEndEvent
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "pub_sub/http_transaction_end_event.h"
+#include "service_inspectors/http_inspect/http_common.h"
+#include "service_inspectors/http_inspect/http_enum.h"
+#include "service_inspectors/http_inspect/http_flow_data.h"
+#include "service_inspectors/http_inspect/http_inspect.h"
+#include "service_inspectors/http_inspect/http_module.h"
+#include "service_inspectors/http_inspect/http_msg_section.h"
+#include "service_inspectors/http_inspect/http_transaction.h"
+#include "service_inspectors/http_inspect/test/http_unit_test_helpers.h"
+
+#include <CppUTest/CommandLineTestRunner.h>
+#include <CppUTest/TestHarness.h>
+#include <CppUTestExt/MockSupport.h>
+
+using namespace snort;
+using namespace HttpCommon;
+using namespace HttpEnums;
+
+namespace snort
+{
+unsigned FlowData::flow_data_id = 0;
+FlowData::FlowData(unsigned, Inspector*) : next(nullptr), prev(nullptr), handler(nullptr), id(0)
+{ }
+FlowData::~FlowData() = default;
+FlowData* Flow::get_flow_data(uint32_t) const { return nullptr; }
+int Flow::set_flow_data(FlowData*) { return 0; }
+Flow::~Flow() = default;
+unsigned DataBus::get_id(PubKey const&) { return 0; }
+void DataBus::publish(unsigned int, unsigned int, DataEvent&, Flow*) { }
+int DetectionEngine::queue_event(unsigned int, unsigned int) { return 0; }
+fd_status_t File_Decomp_StopFree(fd_session_t*) { return File_Decomp_OK; }
+Inspector::Inspector() { }
+Inspector::~Inspector() = default;
+bool Inspector::likes(Packet*) { return true; }
+bool Inspector::get_buf(const char*, Packet*, InspectionBuffer&) { return false; }
+class StreamSplitter* Inspector::get_splitter(bool) { return nullptr; }
+const StreamBuffer StreamSplitter::reassemble(snort::Flow*, unsigned int, unsigned int, unsigned char const*, unsigned
+ int, unsigned int, unsigned int&)
+{
+ StreamBuffer buf { nullptr, 0 };
+ return buf;
+}
+unsigned StreamSplitter::max(snort::Flow*) { return 0; }
+}
+
+HttpParaList::UriParam::UriParam() { }
+HttpParaList::JsNormParam::~JsNormParam() { }
+HttpParaList::~HttpParaList() { }
+const Field Field::FIELD_NULL { STAT_NO_SOURCE };
+const Field& HttpMsgSection::get_classic_buffer(unsigned, uint64_t, uint64_t)
+{ return Field::FIELD_NULL; }
+HttpInspect::HttpInspect(const HttpParaList* para) :
+ params(para), xtra_trueip_id(0), xtra_uri_id(0),
+ xtra_host_id(0), xtra_jsnorm_id(0)
+{ }
+HttpInspect::~HttpInspect() = default;
+bool HttpInspect::configure(SnortConfig*) { return true; }
+void HttpInspect::show(const SnortConfig*) const { }
+bool HttpInspect::get_buf(unsigned, snort::Packet*, snort::InspectionBuffer&) { return true; }
+HttpCommon::SectionType HttpInspect::get_type_expected(snort::Flow*, HttpCommon::SourceId) const
+{ return SEC_DISCARD; }
+void HttpInspect::finish_hx_body(snort::Flow*, HttpCommon::SourceId, HttpCommon::HXBodyState,
+ bool) const { }
+void HttpInspect::set_hx_body_state(snort::Flow*, HttpCommon::SourceId, HttpCommon::HXBodyState) const { }
+bool HttpInspect::get_fp_buf(snort::InspectionBuffer::Type, snort::Packet*,
+ snort::InspectionBuffer&) { return false; }
+void HttpInspect::eval(snort::Packet*) { }
+void HttpInspect::eval(snort::Packet*, HttpCommon::SourceId, const uint8_t*, uint16_t) { }
+void HttpInspect::clear(snort::Packet*) { }
+bool HttpInspect::get_buf(snort::InspectionBuffer::Type, snort::Packet*, snort::InspectionBuffer&) { return false; }
+const uint8_t* HttpInspect::adjust_log_packet(snort::Packet*, uint16_t&) { return nullptr; }
+StreamSplitter::Status HttpStreamSplitter::scan(snort::Packet*, const uint8_t*, uint32_t, uint32_t, uint32_t*)
+{ return StreamSplitter::FLUSH; }
+StreamSplitter::Status HttpStreamSplitter::scan(snort::Flow*, const uint8_t*, uint32_t, uint32_t*)
+{ return StreamSplitter::FLUSH; }
+const snort::StreamBuffer HttpStreamSplitter::reassemble(snort::Flow*, unsigned, unsigned, const
+ uint8_t*, unsigned, uint32_t, unsigned&)
+{
+ StreamBuffer buf { nullptr, 0 };
+ return buf;
+}
+bool HttpStreamSplitter::finish(snort::Flow*) { return false; }
+void HttpStreamSplitter::prep_partial_flush(snort::Flow*, uint32_t) { }
+
+THREAD_LOCAL PegCount HttpModule::peg_counts[PEG_COUNT_MAX] = { };
+
+TEST_GROUP(pub_sub_http_transaction_end_event_test)
+{
+ Flow* const flow = new Flow;
+ HttpParaList params;
+ HttpFlowData* flow_data = new HttpFlowData(flow, ¶ms);
+ SectionType* const section_type = HttpUnitTestSetup::get_section_type(flow_data);
+ void setup() override
+ {
+ flow->gadget = new HttpInspect(¶ms);
+ }
+
+ void teardown() override
+ {
+ delete flow_data;
+ delete flow->gadget;
+ delete flow;
+ }
+};
+
+TEST(pub_sub_http_transaction_end_event_test, version_no_req_no_status)
+{
+ section_type[SRC_CLIENT] = SEC_REQUEST;
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
+ HttpTransactionEndEvent event(trans);
+ HttpEnums::VersionId version = event.get_version();
+ CHECK(version == HttpEnums::VERS__NOT_PRESENT);
+}
+
+int main(int argc, char** argv)
+{
+ return CommandLineTestRunner::RunAllTests(argc, argv);
+}
+
bnfa_state_t* pi; /* state indexes into ps */
bnfa_state_t ps_index=0;
unsigned nps;
- bnfa_state_t full[BNFA_MAX_ALPHABET_SIZE];
+ bnfa_state_t full[BNFA_MAX_ALPHABET_SIZE] = {};
/* count total state transitions, account for state and control words */
nps = 0;
* Context ids and associated uuids are stored in a queue and
* dequeued upon server response. Server response doesn't
* indicate by context id which bindings were accepted or
- * rejected, but the index or order they were in in the client
+ * rejected, but the index or order they were in the client
* bind or alter context, hence the queue.
*
********************************************************************/
#include "dce_http_proxy_module.h"
-#include "stream/tcp/tcp_stream_session.h"
+#include "managers/inspector_manager.h"
+#include "stream/tcp/tcp_session.h"
#include "dce_http_proxy_splitter.h"
{
if ( !p->test_session_flags(SSNFLAG_ABORT_CLIENT | SSNFLAG_ABORT_SERVER) )
{
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
DceHttpProxySplitter* c2s_splitter =
(DceHttpProxySplitter*)(tcp_session->get_splitter(true));
#include "dce_http_server_module.h"
-#include "stream/tcp/tcp_stream_session.h"
+#include "managers/inspector_manager.h"
+#include "stream/tcp/tcp_session.h"
#include "dce_http_server_splitter.h"
{
if ( !p->test_session_flags(SSNFLAG_ABORT_SERVER) )
{
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
DceHttpServerSplitter* splitter =
(DceHttpServerSplitter*)(tcp_session->get_splitter(false));
const uint32_t tid, const uint32_t mid, DCE2_Smb2SessionTracker** str, DCE2_Smb2TreeTracker** ttr, bool
lookup_cache = false)
{
- if(lookup_cache)
- {
- auto key = get_key(sid);
- *str = smb2_session_cache->find(key).get();
- }
- else
- {
- *str = DCE2_Smb2FindSidInSsd(ssd, sid).get();
- }
+ *str = DCE2_Smb2FindSidInSsd(ssd, sid).get();
if (!*str)
{
if (lookup_cache)
#ifndef DCE_SMB2_H
#define DCE_SMB2_H
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
#include "dce_db.h"
#include "dce_smb.h"
#include "hash/lru_cache_shared.h"
int16_t sgroup = 0;
uint32_t addressSpaceId = 0;
uint16_t vlan_tag = 0;
- uint16_t padding = 0;
+ uint16_t dport = 0;
uint64_t sid = 0;
+#ifndef DISABLE_TENANT_ID
uint32_t tenant_id = 0;
uint32_t padding2 = 0; // NOTE: If this changes, change do_hash too
+#endif
bool operator==(const Smb2SidHashKey& other) const
{
addressSpaceId == other.addressSpaceId and
vlan_tag == other.vlan_tag and
sid == other.sid and
- tenant_id == other.tenant_id );
+ dport == other.dport
+#ifndef DISABLE_TENANT_ID
+ and tenant_id == other.tenant_id
+#endif
+ );
}
};
PADDING_GUARD_END
a += d[12]; // sid[0]
b += d[13]; // sid[1]
+#ifndef DISABLE_TENANT_ID
c += d[14]; // tenant_id
-
+#endif
// padding2 is ignored.
finalize(a, b, c);
mix(a, b, c);
- a += d[12]; // vlan & pad
+ a += d[12]; // vlan & dport
b += d[13]; // ip_proto, pkt_type, version, flags
finalize(a, b, c);
memcpy(key.cip, flow->client_ip.get_ip6_ptr(), 4 * sizeof(uint32_t));
memcpy(key.sip, flow->server_ip.get_ip6_ptr(), 4 * sizeof(uint32_t));
key.mplsLabel = flow->key->mplsLabel;
+ key.dport = flow->server_port;
key.cgroup = flow->client_group;
key.sgroup = flow->server_group;
key.addressSpaceId = flow->key->addressSpaceId;
key.vlan_tag = flow->key->vlan_tag;
key.sid = sid;
+#ifndef DISABLE_TENANT_ID
key.tenant_id = flow->key->tenant_id;
+#endif
}
return key;
}
current_size += size;
if ( size > 0)
{
- // Checking 1+ size prevents crash if max_size is too low to hold even a single entry
- if ( current_size > max_size and list.size() > 1 )
- LruLocal::prune();
if ( stats.cache_max < current_size )
stats.cache_max = current_size;
}
{ CountType::SUM, "binds", "total connection-oriented binds" },
{ CountType::SUM, "bind_acks", "total connection-oriented binds acks" },
{ CountType::SUM, "alter_contexts", "total connection-oriented alter contexts" },
- { CountType::SUM, "alter_context_responses",
- "total connection-oriented alter context responses" },
+ { CountType::SUM, "alter_context_responses", "total connection-oriented alter context responses" },
{ CountType::SUM, "bind_naks", "total connection-oriented bind naks" },
{ CountType::SUM, "requests", "total connection-oriented requests" },
{ CountType::SUM, "responses", "total connection-oriented responses" },
{ CountType::SUM, "auth3s", "total connection-oriented auth3s" },
{ CountType::SUM, "shutdowns", "total connection-oriented shutdowns" },
{ CountType::SUM, "rejects", "total connection-oriented rejects" },
- { CountType::SUM, "ms_rpc_http_pdus",
- "total connection-oriented MS requests to send RPC over HTTP" },
+ { CountType::SUM, "ms_rpc_http_pdus", "total connection-oriented MS requests to send RPC over HTTP" },
{ CountType::SUM, "other_requests", "total connection-oriented other requests" },
{ CountType::SUM, "other_responses", "total connection-oriented other responses" },
{ CountType::SUM, "request_fragments", "total connection-oriented request fragments" },
{ CountType::SUM, "response_fragments", "total connection-oriented response fragments" },
- { CountType::SUM, "client_max_fragment_size",
- "connection-oriented client maximum fragment size" },
- { CountType::SUM, "client_min_fragment_size",
- "connection-oriented client minimum fragment size" },
- { CountType::SUM, "client_segs_reassembled",
- "total connection-oriented client segments reassembled" },
- { CountType::SUM, "client_frags_reassembled",
- "total connection-oriented client fragments reassembled" },
- { CountType::SUM, "server_max_fragment_size",
- "connection-oriented server maximum fragment size" },
- { CountType::SUM, "server_min_fragment_size",
- "connection-oriented server minimum fragment size" },
- { CountType::SUM, "server_segs_reassembled",
- "total connection-oriented server segments reassembled" },
- { CountType::SUM, "server_frags_reassembled",
- "total connection-oriented server fragments reassembled" },
+ { CountType::MAX, "client_max_fragment_size", "connection-oriented client maximum fragment size" },
+ { CountType::SUM, "client_min_fragment_size", "connection-oriented client minimum fragment size" },
+ { CountType::SUM, "client_segs_reassembled", "total connection-oriented client segments reassembled" },
+ { CountType::SUM, "client_frags_reassembled", "total connection-oriented client fragments reassembled" },
+ { CountType::MAX, "server_max_fragment_size", "connection-oriented server maximum fragment size" },
+ { CountType::SUM, "server_min_fragment_size", "connection-oriented server minimum fragment size" },
+ { CountType::SUM, "server_segs_reassembled", "total connection-oriented server segments reassembled" },
+ { CountType::SUM, "server_frags_reassembled", "total connection-oriented server fragments reassembled" },
{ CountType::SUM, "sessions", "total smb sessions" },
{ CountType::SUM, "packets", "total smb packets" },
{ CountType::SUM, "ignored_bytes", "total ignored bytes" },
{ CountType::SUM, "binds", "total connection-oriented binds" },
{ CountType::SUM, "bind_acks", "total connection-oriented binds acks" },
{ CountType::SUM, "alter_contexts", "total connection-oriented alter contexts" },
- { CountType::SUM, "alter_context_responses",
- "total connection-oriented alter context responses" },
+ { CountType::SUM, "alter_context_responses", "total connection-oriented alter context responses" },
{ CountType::SUM, "bind_naks", "total connection-oriented bind naks" },
{ CountType::SUM, "requests", "total connection-oriented requests" },
{ CountType::SUM, "responses", "total connection-oriented responses" },
{ CountType::SUM, "auth3s", "total connection-oriented auth3s" },
{ CountType::SUM, "shutdowns", "total connection-oriented shutdowns" },
{ CountType::SUM, "rejects", "total connection-oriented rejects" },
- { CountType::SUM, "ms_rpc_http_pdus",
- "total connection-oriented MS requests to send RPC over HTTP" },
+ { CountType::SUM, "ms_rpc_http_pdus", "total connection-oriented MS requests to send RPC over HTTP" },
{ CountType::SUM, "other_requests", "total connection-oriented other requests" },
{ CountType::SUM, "other_responses", "total connection-oriented other responses" },
{ CountType::SUM, "request_fragments", "total connection-oriented request fragments" },
{ CountType::SUM, "response_fragments", "total connection-oriented response fragments" },
- { CountType::SUM, "client_max_fragment_size",
- "connection-oriented client maximum fragment size" },
- { CountType::SUM, "client_min_fragment_size",
- "connection-oriented client minimum fragment size" },
- { CountType::SUM, "client_segs_reassembled",
- "total connection-oriented client segments reassembled" },
- { CountType::SUM, "client_frags_reassembled",
- "total connection-oriented client fragments reassembled" },
- { CountType::SUM, "server_max_fragment_size",
- "connection-oriented server maximum fragment size" },
- { CountType::SUM, "server_min_fragment_size",
- "connection-oriented server minimum fragment size" },
- { CountType::SUM, "server_segs_reassembled",
- "total connection-oriented server segments reassembled" },
- { CountType::SUM, "server_frags_reassembled",
- "total connection-oriented server fragments reassembled" },
+ { CountType::MAX, "client_max_fragment_size", "connection-oriented client maximum fragment size" },
+ { CountType::SUM, "client_min_fragment_size", "connection-oriented client minimum fragment size" },
+ { CountType::SUM, "client_segs_reassembled", "total connection-oriented client segments reassembled" },
+ { CountType::SUM, "client_frags_reassembled", "total connection-oriented client fragments reassembled" },
+ { CountType::MAX, "server_max_fragment_size", "connection-oriented server maximum fragment size" },
+ { CountType::SUM, "server_min_fragment_size", "connection-oriented server minimum fragment size" },
+ { CountType::SUM, "server_segs_reassembled", "total connection-oriented server segments reassembled" },
+ { CountType::SUM, "server_frags_reassembled", "total connection-oriented server fragments reassembled" },
{ CountType::SUM, "tcp_sessions", "total tcp sessions" },
{ CountType::SUM, "tcp_expected_sessions", "total tcp dynamic endpoint expected sessions" },
{ CountType::SUM, "tcp_expected_realized", "total tcp dynamic endpoint expected realized sessions" },
{ CountType::END, nullptr, nullptr }
};
-Dce2TcpModule::Dce2TcpModule() : Module(DCE2_TCP_NAME, DCE2_TCP_HELP, s_params)
-{
-}
+Dce2TcpModule::Dce2TcpModule() : Module(DCE2_TCP_NAME, DCE2_TCP_HELP, s_params)
+{ }
const RuleMap* Dce2TcpModule::get_rules() const
{
{ CountType::SUM, "other_requests", "total connection-less other requests" },
{ CountType::SUM, "other_responses", "total connection-less other responses" },
{ CountType::SUM, "fragments", "total connection-less fragments" },
- { CountType::SUM, "max_fragment_size", "connection-less maximum fragment size" },
+ { CountType::MAX, "max_fragment_size", "connection-less maximum fragment size" },
{ CountType::SUM, "frags_reassembled", "total connection-less fragments reassembled" },
{ CountType::SUM, "max_seqnum", "max connection-less seqnum" },
{ CountType::NOW, "concurrent_sessions", "total concurrent sessions" },
{ CountType::END, nullptr, nullptr }
};
-Dce2UdpModule::Dce2UdpModule() : Module(DCE2_UDP_NAME, DCE2_UDP_HELP, s_params), config {}
+Dce2UdpModule::Dce2UdpModule() : Module(DCE2_UDP_NAME, DCE2_UDP_HELP, s_params), config {}
{ }
void Dce2UdpModule::set_trace(const Trace* trace) const
Dce2SmbFlowData* fd = (Dce2SmbFlowData*)p->flow->get_flow_data(
Dce2SmbFlowData::inspector_id);
p->flow->free_flow_data(fd);
- DCE2_Smb2SsnData* dce2_smb2_sess = dce2_create_new_smb2_session(p, config);
+ DCE2_Smb2SsnData* dce2_smb2_sess = dce2_create_new_smb2_session(p, config, true);
DCE2_Smb2Process(dce2_smb2_sess);
if (!dce2_detected)
DCE2_Detect(&dce2_smb2_sess->sd);
return((DCE2_Smb2SsnData*)fd->dce2_smb_session_data);
}
-DCE2_Smb2SsnData* dce2_create_new_smb2_session(Packet* p, dce2SmbProtoConf* config)
+DCE2_Smb2SsnData* dce2_create_new_smb2_session(Packet* p, dce2SmbProtoConf* config, bool upgrade)
{
DCE2_Smb2SsnData* dce2_smb2_sess = set_new_dce2_smb2_session(p);
if ( dce2_smb2_sess )
DCE2_ResetRopts(&dce2_smb2_sess->sd, p);
- dce2_smb_stats.smb_sessions++;
+ if (upgrade)
+ dce2_smb_stats.total_smb1_sessions--;
+ else
+ dce2_smb_stats.smb_sessions++;
dce2_smb_stats.total_smb2_sessions++;
dce2_smb2_sess->sd.trans = DCE2_TRANS_TYPE__SMB;
void DCE2_SmbInitGlobals();
void DCE2_Smb1Process(struct DCE2_SmbSsnData*);
struct DCE2_SmbSsnData* dce2_create_new_smb_session(snort::Packet*, struct dce2SmbProtoConf*);
-struct DCE2_Smb2SsnData* dce2_create_new_smb2_session(snort::Packet*, struct dce2SmbProtoConf*);
+struct DCE2_Smb2SsnData* dce2_create_new_smb2_session(snort::Packet*, struct dce2SmbProtoConf*, bool upgrade = false);
void DCE2_SmbDataFree(DCE2_SmbSsnData*);
void set_smb_reassembled_data(uint8_t* nb_ptr, uint16_t co_len);
void Dns::eval(Packet* p)
{
// precondition - what we registered for
- assert((p->is_udp() and p->dsize and p->data) or p->has_tcp_data());
+ assert((p->is_udp() and p->dsize and p->data) or p->has_tcp_data() or p->has_udp_quic_data());
assert(p->flow);
++dnsstats.packets;
if (!needNextPacket and dnsSessionData->has_events())
DataBus::publish(Dns::get_pub_id(), DnsEventIds::DNS_RESPONSE_DATA, dnsSessionData->dns_events);
+
+ if (p->type() == PktType::UDP)
+ p->flow->session_state |= STREAM_STATE_CLOSED;
}
else
{
+set(FTP_INCLUDES
+ ftp_data.h
+ ftpdata_splitter.h
+ ftp_module.h
+ ftpp_ui_config.h
+ kmap.h
+)
set (FILE_LIST
ft_main.cc
endif (STATIC_INSPECTORS)
+install(FILES ${FTP_INCLUDES}
+ DESTINATION "${INCLUDE_INSTALL_PATH}/service_inspectors/ftp_telnet"
+)
#include "utils/util.h"
#include "ft_main.h"
-#include "ftp_module.h"
#include "ftpp_si.h"
#include "ftpdata_splitter.h"
ftstats.total_sessions_mss_changed++;
}
-//-------------------------------------------------------------------------
-// class stuff
-//-------------------------------------------------------------------------
-
-class FtpData : public Inspector
-{
-public:
- FtpData() = default;
-
- void eval(Packet*) override;
- StreamSplitter* get_splitter(bool to_server) override;
-
- bool can_carve_files() const override
- { return true; }
-
- bool can_start_tls() const override
- { return true; }
-};
-
-class FtpDataModule : public Module
-{
-public:
- FtpDataModule() : Module(FTP_DATA_NAME, s_help) { }
-
- const PegInfo* get_pegs() const override;
- PegCount* get_counts() const override;
- ProfileStats* get_profile() const override;
-
- bool set(const char*, Value&, SnortConfig*) override
- { return false; }
-
- Usage get_usage() const override
- { return INSPECT; }
-
- bool is_bindable() const override
- { return true; }
-};
-
const PegInfo* FtpDataModule::get_pegs() const
{ return simple_pegs; }
#define FTP_DATA_H
#include "framework/inspector.h"
+#include "ftp_module.h"
extern const snort::InspectApi fd_api;
+#define FTP_DATA_NAME "ftp_data"
+#define s_help \
+ "FTP data channel handler"
+
+class SO_PUBLIC FtpData : public snort::Inspector
+{
+public:
+ FtpData() = default;
+
+ void eval(snort::Packet*) override;
+ snort::StreamSplitter* get_splitter(bool to_server) override;
+
+ bool can_carve_files() const override
+ { return true; }
+
+ bool can_start_tls() const override
+ { return true; }
+};
+
+class FtpDataModule : public snort::Module
+{
+public:
+ FtpDataModule() : snort::Module(FTP_DATA_NAME, s_help) { }
+
+ const PegInfo* get_pegs() const override;
+ PegCount* get_counts() const override;
+ snort::ProfileStats* get_profile() const override;
+
+ bool set(const char*, snort::Value&, snort::SnortConfig*) override
+ { return false; }
+
+ Usage get_usage() const override
+ { return INSPECT; }
+
+ bool is_bindable() const override
+ { return true; }
+};
#endif
}
StreamSplitter::Status FtpDataSplitter::scan(Packet* pkt, const uint8_t*, uint32_t len,
- uint32_t, uint32_t* fp)
+ uint32_t flags, uint32_t* fp)
{
Flow* flow = pkt->flow;
assert(flow);
fdfd->session.mss_changed = true;
expected_seg_size = len;
- if (pkt->ptrs.tcph and !pkt->ptrs.tcph->is_fin())
+ if (!flow->assistant_gadget && pkt->ptrs.tcph and !pkt->ptrs.tcph->is_fin())
{
// set flag for signature calculation in case this is the last packet
fdfd->session.packet_flags |= FTPDATA_FLG_FLUSH;
return SEARCH;
}
}
+
+ if (flow->assistant_gadget && (flags & FTPDATA_FLG_FLUSH))
+ {
+ fdfd = (FtpDataFlowData*)flow->get_flow_data(FtpDataFlowData::inspector_id);
+ if (!fdfd)
+ return SEARCH;
+
+ fdfd->session.packet_flags |= FTPDATA_FLG_FLUSH;
+ pkt->active->hold_packet(pkt);
+ return SEARCH;
+ }
}
if ((segs >= 2 and bytes >= min) or (pkt->ptrs.tcph and pkt->ptrs.tcph->is_fin()))
//---------------------------------------------------------------------------------
// FtpDataSplitter - flush when current seg size is different from previous segment
//---------------------------------------------------------------------------------
-class FtpDataSplitter : public snort::StreamSplitter
+class SO_PUBLIC FtpDataSplitter : public snort::StreamSplitter
{
public:
FtpDataSplitter(bool b, uint16_t sz = 0) : snort::StreamSplitter(b)
ptr = lf;
else if ( cr && lf )
ptr = ( cr > lf ) ? cr : lf;
- if ( ptr )
+
+ const uint8_t* iac_ptr = static_cast<const uint8_t*>(memchr( read_ptr, TNC_IAC, end - read_ptr));
+ if ( (ptr && iac_ptr && ptr < iac_ptr) || (ptr && !iac_ptr) )
{
fp_ptr = ptr;
read_ptr = fp_ptr;
}
- const uint8_t* iac_ptr = static_cast<const uint8_t*>(memchr( read_ptr, TNC_IAC, end - read_ptr));
if ( iac_ptr )
{
state = TELNET_IAC;
could easily manipulate, such as header length, chunking, compression, and encodings. The maximum
depth is computed against normalized message body data.
-===== Partial inspection
+==== Partial inspection
include::dev_notes_partial_inspection.txt[]
-===== Message section
+==== Message section
include::dev_notes_message_section.txt[]
-===== Field class
+==== Field class
include::dev_notes_field_class.txt[]
-===== Support for custom xff type headers
+==== Support for custom xff type headers
include::dev_notes_hff_headers.txt[]
-===== URI normalization
+==== URI normalization
include::dev_notes_uri_norm.txt[]
-===== JS normalization
+==== JS normalization
include::dev_notes_js_norm.txt[]
-===== Reassembling chunked message
+==== Reassembling chunked message
include::dev_notes_chunked_processing.txt[]
-===== MIME inspection
+==== MIME inspection
include::dev_notes_mime_inspection.txt[]
-===== HI test tool usage
+==== HI test tool usage
include::dev_notes_test_tool.txt[]
return SCAN_DISCARD_PIECE;
}
- if (data_seen >= flow_target)
+ if (data_seen >= flow_target || octets_seen + length == MAX_OCTETS)
{
// We passed the flow_target and stretched to the end of the segment
+ // Or we reached the max we can reassemble in current section
data_seen = 0;
num_flush = length;
return SCAN_FOUND_PIECE;
#define HTTP_ENUM_H
#include <cstdint>
+#include <map>
+#include <string>
namespace HttpEnums
{
PEG_CONCURRENT_SESSIONS, PEG_MAX_CONCURRENT_SESSIONS, PEG_SCRIPT_DETECTION,
PEG_PARTIAL_INSPECT, PEG_EXCESS_PARAMS, PEG_PARAMS, PEG_CUTOVERS, PEG_SSL_SEARCH_ABND_EARLY,
PEG_PIPELINED_FLOWS, PEG_PIPELINED_REQUESTS, PEG_TOTAL_BYTES, PEG_JS_INLINE, PEG_JS_EXTERNAL,
- PEG_JS_PDF, PEG_SKIP_MIME_ATTACH, PEG_COUNT_MAX };
+ PEG_JS_PDF, PEG_SKIP_MIME_ATTACH, PEG_COMPRESSED_GZIP, PEG_COMPRESSED_NOT_SUPPORTED,
+ PEG_COMPRESSED_UNKNOWN, PEG_COUNT_MAX};
// Result of scanning by splitter
enum ScanResult { SCAN_NOT_FOUND, SCAN_NOT_FOUND_ACCELERATE, SCAN_FOUND, SCAN_FOUND_PIECE,
CHUNK_OPTIONS, CHUNK_HCRLF, CHUNK_DATA, CHUNK_DCRLF1, CHUNK_DCRLF2, CHUNK_BAD };
// List of possible HTTP versions.
+// When making changes to this enum, also update VersionStrToEnum and VersionEnumToStr
enum VersionId { VERS__PROBLEMATIC=-12, VERS__NOT_PRESENT=-11, VERS__OTHER=1,
VERS_1_0, VERS_1_1, VERS_2_0, VERS_3_0, VERS_0_9, VERS__MIN = VERS__PROBLEMATIC,
VERS__MAX = VERS_0_9};
HEAD_CONTENT_TYPE, HEAD_EXPIRES, HEAD_LAST_MODIFIED, HEAD_X_FORWARDED_FOR, HEAD_TRUE_CLIENT_IP,
HEAD_X_WORKING_WITH, HEAD_CONTENT_TRANSFER_ENCODING, HEAD_MIME_VERSION, HEAD_PROXY_AGENT,
HEAD_CONTENT_DISPOSITION, HEAD_HTTP2_SETTINGS, HEAD_RESTRICT_ACCESS_TO_TENANTS,
- HEAD_RESTRICT_ACCESS_CONTEXT, HEAD__MAX_VALUE };
+ HEAD_RESTRICT_ACCESS_CONTEXT, HEAD_ORIGIN, HEAD__MAX_VALUE };
// All the infractions we might find while parsing and analyzing a message
enum Infraction
extern const bool is_sp_tab_quote_dquote[256];
extern const bool is_print_char[256]; // printable includes SP, tab, CR, LF
extern const bool is_sp_comma[256];
+extern const std::map <std::string, VersionId> VersionStrToEnum;
+extern const std::map <HttpEnums::VersionId, const char*> VersionEnumToStr;
} // end namespace HttpEnums
delete script_finder;
}
-bool HttpInspect::configure(SnortConfig*)
+bool HttpInspect::configure(SnortConfig* sc)
{
params->js_norm_param.configure();
- params->mime_decode_conf->sync_all_depths();
+ params->mime_decode_conf->sync_all_depths(sc);
pub_id = DataBus::get_id(http_pub_key);
return true;
HttpMsgRequest* const request = current_section->get_request();
if (request == nullptr)
return 0;
- const Field& uri = request->get_uri();
+ const Field& uri = request->get_uri_norm_classic();
if (uri.length() <= 0)
return 0;
#include "decompress/file_olefile.h"
#include "file_api/file_flows.h"
#include "file_api/file_service.h"
+#include "hash/hash_key_operations.h"
#include "helpers/buffer_data.h"
#include "js_norm/js_enum.h"
#include "pub_sub/http_request_body_event.h"
const FileDirection dir = source_id == SRC_SERVER ? FILE_DOWNLOAD : FILE_UPLOAD;
- const uint64_t file_index = get_header(source_id)->get_file_cache_index();
+ uint64_t file_index = get_header(source_id)->get_file_cache_index();
+
+ const uint8_t* filename_buffer = nullptr;
+ uint32_t filename_length = 0;
+ const uint8_t* uri_buffer = nullptr;
+ uint32_t uri_length = 0;
+ if (request != nullptr)
+ get_file_info(dir, filename_buffer, filename_length, uri_buffer, uri_length);
bool continue_processing_file = file_flows->file_process(p, file_index, file_data.start(),
fp_length, session_data->file_octets[source_id], dir,
- get_header(source_id)->get_multi_file_processing_id(), file_position);
+ get_header(source_id)->get_multi_file_processing_id(), file_position,
+ filename_buffer, filename_length);
if (continue_processing_file)
{
session_data->file_depth_remaining[source_id] -= fp_length;
{
if (request != nullptr)
{
- const uint8_t* filename_buffer;
- const uint8_t* uri_buffer;
- uint32_t filename_length;
- uint32_t uri_length;
- get_file_info(dir, filename_buffer, filename_length, uri_buffer, uri_length);
-
continue_processing_file = file_flows->set_file_name(filename_buffer,
filename_length, 0,
get_header(source_id)->get_multi_file_processing_id(), uri_buffer,
{
case CONTENTCODE_GZIP:
case CONTENTCODE_X_GZIP:
+ HttpModule::increment_peg_counts(PEG_COMPRESSED_GZIP);
compression = CMP_GZIP;
break;
case CONTENTCODE_DEFLATE:
break;
case CONTENTCODE__OTHER:
// The ones we never heard of
+ HttpModule::increment_peg_counts(PEG_COMPRESSED_UNKNOWN);
add_infraction(INF_UNKNOWN_ENCODING);
create_event(EVENT_UNKNOWN_ENCODING);
break;
default:
// The ones we know by name but don't support
+ HttpModule::increment_peg_counts(PEG_COMPRESSED_NOT_SUPPORTED);
add_infraction(INF_UNSUPPORTED_ENCODING);
create_event(EVENT_UNSUPPORTED_ENCODING);
break;
session_data(session_data_),
flow(flow_),
params(params_),
- transaction(HttpTransaction::attach_my_transaction(session_data, source_id_)),
+ transaction(HttpTransaction::attach_my_transaction(session_data, source_id_, flow)),
trans_num(session_data->expected_trans_num[source_id_]),
status_code_num((source_id_ == SRC_SERVER) ? session_data->status_code_num : STAT_NOT_PRESENT),
source_id(source_id_),
&NORMALIZER_TOKEN_LIST, // HEAD_HTTP2_SETTINGS
&NORMALIZER_BASIC, // HEAD_RESTRICT_ACCESS_TO_TENANTS
&NORMALIZER_BASIC, // HEAD_RESTRICT_ACCESS_CONTEXT
+ &NORMALIZER_URI, // HEAD_ORIGIN
&NORMALIZER_BASIC, // HEAD__MAX_VALUE
&NORMALIZER_BASIC, // HEAD_CUSTOM_XFF_HEADER
&NORMALIZER_BASIC, // HEAD_CUSTOM_XFF_HEADER
{ HEAD_HTTP2_SETTINGS, "http2-settings" },
{ HEAD_RESTRICT_ACCESS_TO_TENANTS, "restrict-access-to-tenants" },
{ HEAD_RESTRICT_ACCESS_CONTEXT, "restrict-access-context" },
+ { HEAD_ORIGIN, "origin" },
{ 0, nullptr }
};
{ CountType::SUM, "js_external_scripts", "total number of external JavaScripts processed" },
{ CountType::SUM, "js_pdf_scripts", "total number of PDF files processed" },
{ CountType::SUM, "skip_mime_attach", "total number of HTTP requests with too many MIME attachments to inspect" },
+ { CountType::SUM, "compressed_gzip", "total number of HTTP bodies compressed with GZIP" },
+ { CountType::SUM, "compressed_not_supported", "total number of HTTP bodies compressed with known but not supported methods" },
+ { CountType::SUM, "compressed_unknown", "total number of HTTP bodies compressed with unknown methods" },
{ CountType::END, nullptr, nullptr }
};
false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false
};
+const std::map <std::string, VersionId> HttpEnums::VersionStrToEnum =
+{
+ { "malformed", VERS__PROBLEMATIC },
+ { "other", VERS__OTHER },
+ { "1.0", VERS_1_0 },
+ { "1.1", VERS_1_1 },
+ { "2.0", VERS_2_0 },
+ { "3.0", VERS_3_0 },
+ { "0.9", VERS_0_9 }
+};
+
+const std::map <VersionId, const char*> HttpEnums::VersionEnumToStr =
+{
+ { VERS_1_0, "1.0" },
+ { VERS_1_1, "1.1" },
+ { VERS_2_0, "2.0" },
+ { VERS_3_0, "3.0" },
+ { VERS_0_9, "0.9" }
+};
#include "http_transaction.h"
+#include "pub_sub/http_transaction_end_event.h"
+
#include "http_common.h"
#include "http_enum.h"
#include "http_event.h"
+#include "http_inspect.h"
#include "http_msg_body.h"
#include "http_msg_header.h"
#include "http_msg_request.h"
}
}
-HttpTransaction::HttpTransaction(HttpFlowData* session_data_): session_data(session_data_)
+HttpTransaction::HttpTransaction(HttpFlowData* session_data_, snort::Flow* const f)
+ : session_data(session_data_), flow(f)
{
infractions[0] = nullptr;
infractions[1] = nullptr;
+ HttpInspect* const hi = (session_data->is_for_httpx()) ?
+ (HttpInspect*)(flow->assistant_gadget) : (HttpInspect*)(flow->gadget);
+ pub_id = hi->get_pub_id();
+}
+
+void HttpTransaction::publish_end_of_transaction()
+{
+ HttpTransactionEndEvent http_event(this);
+ DataBus::publish(pub_id, HttpEventIds::END_OF_TRANSACTION, http_event, flow);
}
HttpTransaction::~HttpTransaction()
{
+ publish_end_of_transaction();
delete request;
delete status;
for (int k = 0; k <= 1; k++)
}
HttpTransaction* HttpTransaction::attach_my_transaction(HttpFlowData* session_data, SourceId
- source_id)
+ source_id, Flow* const f)
{
// This factory method:
// 1. creates new transactions for all request messages and orphaned response messages
}
}
}
- session_data->transaction[SRC_CLIENT] = new HttpTransaction(session_data);
+ session_data->transaction[SRC_CLIENT] = new HttpTransaction(session_data, f);
// The StreamSplitter generates infractions related to this transaction while splitting the
// request line and keeps them in temporary storage in the FlowData. Now we move them here.
if (session_data->pipeline_underflow)
{
// A previous underflow separated the two sides forever
- session_data->transaction[SRC_SERVER] = new HttpTransaction(session_data);
+ session_data->transaction[SRC_SERVER] = new HttpTransaction(session_data, f);
}
else if ((session_data->transaction[SRC_SERVER] = session_data->take_from_pipeline()) ==
nullptr)
// Either there is no request at all or there is a request but a previous response
// already took it. Either way we have more responses than requests.
session_data->pipeline_underflow = true;
- session_data->transaction[SRC_SERVER] = new HttpTransaction(session_data);
+ session_data->transaction[SRC_SERVER] = new HttpTransaction(session_data, f);
}
else if (session_data->type_expected[SRC_CLIENT] == SEC_REQUEST)
{
public:
~HttpTransaction();
- static HttpTransaction* attach_my_transaction(HttpFlowData* session_data,
- HttpCommon::SourceId source_id);
- static void delete_transaction(HttpTransaction* transaction, HttpFlowData* session_data);
+ static HttpTransaction* attach_my_transaction(HttpFlowData*,
+ HttpCommon::SourceId, snort::Flow* const);
+ static void delete_transaction(HttpTransaction*, HttpFlowData*);
HttpMsgRequest* get_request() const { return request; }
void set_request(HttpMsgRequest* request_) { request = request_; }
{ trailer[source_id] = trailer_; }
void set_body(HttpMsgBody* latest_body);
- HttpInfractions* get_infractions(HttpCommon::SourceId source_id);
+ HttpInfractions* get_infractions(HttpCommon::SourceId);
void set_one_hundred_response();
bool final_response() const { return !second_response_expected; }
HttpTransaction* next = nullptr;
private:
- HttpTransaction(HttpFlowData* session_data_);
- void discard_section(HttpMsgSection* section);
+ HttpTransaction(HttpFlowData*, snort::Flow* const);
+ void discard_section(HttpMsgSection*);
+ void publish_end_of_transaction();
HttpFlowData* const session_data;
// parallel.
bool shared_ownership = false;
+ unsigned pub_id;
+ snort::Flow* const flow;
+
// Estimates of how much memory http_inspect uses to process a transaction
static const uint16_t small_things = 400; // minor memory costs not otherwise accounted for
static const uint16_t transaction_memory_usage_estimate;
HttpIpsOption(const HttpRuleOptModule* cm) :
snort::IpsOption(cm->key),
buffer_info(cm->rule_opt_index, cm->sub_id, cm->form),
- cat(cm->cat), pdu_section(cm->pdu_section) {}
+ cat(cm->sub_id and cm->cat == snort::CAT_SET_FAST_PATTERN ? snort::CAT_SET_SUB_SECTION : cm->cat),
+ pdu_section(cm->pdu_section) {}
snort::CursorActionType get_cursor_type() const override { return cat; }
EvalStatus eval(Cursor&, snort::Packet*) override = 0;
uint32_t hash() const override;
return true;
}
-static const std::map <std::string, VersionId> VersionStrToEnum =
-{
- { "malformed", VERS__PROBLEMATIC },
- { "other", VERS__OTHER },
- { "1.0", VERS_1_0 },
- { "1.1", VERS_1_1 },
- { "2.0", VERS_2_0 },
- { "3.0", VERS_3_0 },
- { "0.9", VERS_0_9 }
-};
-
bool HttpVersionRuleOptModule::parse_version_list(Value& v)
{
v.set_first_token();
#include "config.h"
#endif
+#include "pub_sub/http_transaction_end_event.h"
#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_enum.h"
#include "service_inspectors/http_inspect/http_flow_data.h"
+#include "service_inspectors/http_inspect/http_inspect.h"
#include "service_inspectors/http_inspect/http_module.h"
#include "service_inspectors/http_inspect/http_transaction.h"
#include "service_inspectors/http2_inspect/http2_flow_data.h"
+#include "http_unit_test_helpers.h"
+
#include <CppUTest/CommandLineTestRunner.h>
#include <CppUTest/TestHarness.h>
#include <CppUTestExt/MockSupport.h>
FlowData* Flow::get_flow_data(uint32_t) const { return nullptr; }
int Flow::set_flow_data(FlowData*) { return 0;}
Flow::~Flow() = default;
+unsigned DataBus::get_id(PubKey const&) { return 0; }
+void DataBus::publish(unsigned int, unsigned int, DataEvent&, Flow*) {}
+HttpTransactionEndEvent::HttpTransactionEndEvent(const HttpTransaction* const trans):
+ transaction(trans) {}
+Inspector::Inspector() { }
+Inspector::~Inspector() = default;
+bool Inspector::likes(Packet*) { return true; }
+bool Inspector::get_buf(const char*, Packet*, InspectionBuffer&) { return false; }
+class StreamSplitter* Inspector::get_splitter(bool) { return nullptr; }
+const StreamBuffer StreamSplitter::reassemble(snort::Flow*, unsigned int, unsigned int, unsigned char const*, unsigned
+ int, unsigned int, unsigned int&)
+{
+ StreamBuffer buf { nullptr, 0 };
+ return buf;
+}
+unsigned StreamSplitter::max(snort::Flow*) { return 0; }
}
HttpParaList::UriParam::UriParam() {}
unsigned Http2FlowData::inspector_id = 0;
uint32_t Http2FlowData::get_processing_stream_id() const { return 0; }
+HttpInspect::HttpInspect(const HttpParaList* para) :
+ params(para), xtra_trueip_id(0), xtra_uri_id(0),
+ xtra_host_id(0), xtra_jsnorm_id(0)
+{ }
+HttpInspect::~HttpInspect() = default;
+bool HttpInspect::configure(SnortConfig*) { return true; }
+void HttpInspect::show(const SnortConfig*) const { }
+bool HttpInspect::get_buf(unsigned, snort::Packet*, snort::InspectionBuffer&) { return true; }
+HttpCommon::SectionType HttpInspect::get_type_expected(snort::Flow*, HttpCommon::SourceId) const
+{ return SEC_DISCARD; }
+void HttpInspect::finish_hx_body(snort::Flow*, HttpCommon::SourceId, HttpCommon::HXBodyState,
+ bool) const { }
+void HttpInspect::set_hx_body_state(snort::Flow*, HttpCommon::SourceId, HttpCommon::HXBodyState) const { }
+bool HttpInspect::get_fp_buf(snort::InspectionBuffer::Type, snort::Packet*,
+ snort::InspectionBuffer&) { return false; }
+void HttpInspect::eval(snort::Packet*) { }
+void HttpInspect::eval(snort::Packet*, HttpCommon::SourceId, const uint8_t*, uint16_t) { }
+void HttpInspect::clear(snort::Packet*) { }
+bool HttpInspect::get_buf(snort::InspectionBuffer::Type, snort::Packet*, snort::InspectionBuffer&) { return false; }
+const uint8_t* HttpInspect::adjust_log_packet(snort::Packet*, uint16_t&) { return nullptr; }
+StreamSplitter::Status HttpStreamSplitter::scan(snort::Packet*, const uint8_t*, uint32_t, uint32_t, uint32_t*)
+{ return StreamSplitter::FLUSH; }
+StreamSplitter::Status HttpStreamSplitter::scan(snort::Flow*, const uint8_t*, uint32_t, uint32_t*)
+{ return StreamSplitter::FLUSH; }
+const snort::StreamBuffer HttpStreamSplitter::reassemble(snort::Flow*, unsigned, unsigned, const
+ uint8_t*, unsigned, uint32_t, unsigned&)
+{
+ StreamBuffer buf { nullptr, 0 };
+ return buf;
+}
+bool HttpStreamSplitter::finish(snort::Flow*) { return false; }
+void HttpStreamSplitter::prep_partial_flush(snort::Flow*, uint32_t) { }
THREAD_LOCAL PegCount HttpModule::peg_counts[PEG_COUNT_MAX] = { };
-class HttpUnitTestSetup
-{
-public:
- static SectionType* get_section_type(HttpFlowData* flow_data)
- { assert(flow_data!=nullptr); return flow_data->section_type; }
- static SectionType* get_type_expected(HttpFlowData* flow_data)
- { assert(flow_data!=nullptr); return flow_data->type_expected; }
-};
-
TEST_GROUP(http_transaction_test)
{
Flow* const flow = new Flow;
SectionType* const section_type = HttpUnitTestSetup::get_section_type(flow_data);
SectionType* const type_expected = HttpUnitTestSetup::get_type_expected(flow_data);
+ void setup() override
+ {
+ flow->gadget = new HttpInspect(¶ms);
+ }
void teardown() override
{
delete flow_data;
+ delete flow->gadget;
delete flow;
}
};
// Request
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_BODY_CHUNK;
section_type[SRC_CLIENT] = SEC_BODY_CHUNK;
for (unsigned k=0; k<100; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
}
type_expected[SRC_CLIENT] = SEC_TRAILER;
section_type[SRC_CLIENT] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
// Response
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<100; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, orphan_response)
// Response message without a request
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_SERVER] = SEC_STATUS;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow);
CHECK(trans != nullptr);
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<10; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, simple_pipeline)
{
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans[k] != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
for (unsigned j=0; j < k; j++)
{
CHECK(trans[k] != trans[j]);
for (unsigned k=0; k < 4; k++)
{
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CL;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
}
// Response starts before request completes, request completes first
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_BODY_CHUNK;
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_CLIENT] = SEC_BODY_CHUNK;
for (unsigned k=0; k<4; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
}
type_expected[SRC_CLIENT] = SEC_TRAILER;
section_type[SRC_CLIENT] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<6; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, pipeline_underflow)
// Underflow scenario with request, two responses, request, response
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_STATUS;
- trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER);
+ trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow);
CHECK(trans != nullptr);
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans2 = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans2 = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK((trans2 != nullptr) && (trans2 != trans));
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_SERVER] = SEC_STATUS;
- trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER);
+ trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow);
CHECK((trans != nullptr) && (trans != trans2));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, concurrent_request_response_underflow)
// Response starts before request completes, response completes first, second response
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_BODY_CHUNK;
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<6; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_CLIENT] = SEC_BODY_CHUNK;
for (unsigned k=0; k<4; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
}
type_expected[SRC_CLIENT] = SEC_TRAILER;
section_type[SRC_CLIENT] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_SERVER] = SEC_STATUS;
- HttpTransaction* trans2 = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER);
+ HttpTransaction* trans2 = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow);
CHECK((trans2 != nullptr) && (trans2 != trans));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<6; k++)
{
- CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans2 == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, basic_continue)
// Request headers
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_BODY_CHUNK;
// Interim response
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
trans->set_one_hundred_response();
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
// Request body
section_type[SRC_CLIENT] = SEC_BODY_CHUNK;
for (unsigned k=0; k<4; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
}
type_expected[SRC_CLIENT] = SEC_TRAILER;
section_type[SRC_CLIENT] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
// Second response
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<6; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, multiple_continue)
// Request headers
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_BODY_CHUNK;
// Interim responses
for (unsigned k=0; k < 10; k++)
{
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
trans->set_one_hundred_response();
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
// Request body
section_type[SRC_CLIENT] = SEC_BODY_CHUNK;
for (unsigned k=0; k<4; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
}
type_expected[SRC_CLIENT] = SEC_TRAILER;
section_type[SRC_CLIENT] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
type_expected[SRC_CLIENT] = SEC_REQUEST;
// Final response
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
for (unsigned k=0; k<6; k++)
{
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
TEST(http_transaction_test, multiple_orphan_continue)
{
// Interim response
section_type[SRC_SERVER] = SEC_STATUS;
- HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER);
+ HttpTransaction* trans = HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow);
CHECK(trans != nullptr);
trans->set_one_hundred_response();
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
// Final response
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CHUNK;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_TRAILER;
- CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
}
{
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans[k] != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
for (unsigned j=0; j < k; j++)
{
CHECK(trans[k] != trans[j]);
for (unsigned k=0; k < 3; k++)
{
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CL;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
// Interim response to fourth request
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
trans[3]->set_one_hundred_response();
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
// Finish the fourth request
section_type[SRC_CLIENT] = SEC_BODY_CL;
- CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans[3] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
// Requests 5-7 in pipeline
for (unsigned k=4; k < 7; k++)
{
type_expected[SRC_CLIENT] = SEC_REQUEST;
section_type[SRC_CLIENT] = SEC_REQUEST;
- trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT);
+ trans[k] = HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow);
CHECK(trans[k] != nullptr);
type_expected[SRC_CLIENT] = SEC_HEADER;
section_type[SRC_CLIENT] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_CLIENT, flow));
for (unsigned j=5; j < k; j++)
{
CHECK(trans[k] != trans[j]);
for (unsigned k=3; k < 7; k++)
{
section_type[SRC_SERVER] = SEC_STATUS;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_HEADER;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
section_type[SRC_SERVER] = SEC_BODY_CL;
- CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER));
+ CHECK(trans[k] == HttpTransaction::attach_my_transaction(flow_data, SRC_SERVER, flow));
}
}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+// http_unit_test_helpers.h author Maya Dagon <mdagon@cisco.com>
+// Code moved from http_transaction_test.cc, author Tom Peters <thopeter@cisco.com>
+
+#ifndef HTTP_UNIT_TEST_HELPERS_H
+#define HTTP_UNIT_TEST_HELPERS_H
+
+#include "service_inspectors/http_inspect/http_common.h"
+#include "service_inspectors/http_inspect/http_flow_data.h"
+
+class HttpUnitTestSetup
+{
+public:
+ static HttpCommon::SectionType* get_section_type(HttpFlowData* flow_data)
+ { assert(flow_data!=nullptr); return flow_data->section_type; }
+ static HttpCommon::SectionType* get_type_expected(HttpFlowData* flow_data)
+ { assert(flow_data!=nullptr); return flow_data->type_expected; }
+};
+
+#endif
delete config;
}
-bool Imap::configure(SnortConfig*)
+bool Imap::configure(SnortConfig* sc)
{
- config->decode_conf.sync_all_depths();
+ config->decode_conf.sync_all_depths(sc);
if (config->decode_conf.get_file_depth() > -1)
config->log_config.log_filename = true;
delete config;
}
-bool Pop::configure(SnortConfig* )
+bool Pop::configure(SnortConfig* sc)
{
- config->decode_conf.sync_all_depths();
+ config->decode_conf.sync_all_depths(sc);
if (config->decode_conf.get_file_depth() > -1)
config->log_config.log_filename = true;
if (sessp->state_flags & SIP_FLG_MISSED_PACKETS)
return;
- SIP_Process(p,sessp, config);
+ if (!SIP_Process(p,sessp, config))
+ sessp->sip_aborted = true;
}
//-------------------------------------------------------------------------
ConfigLogger::log_value("max_uri_len", config->maxUriLen);
ConfigLogger::log_value("max_via_len", config->maxViaLen);
ConfigLogger::log_list("methods", methods.c_str());
+ ConfigLogger::log_value("sip_timeout", config->sip_timeout);
+ ConfigLogger::log_value("sip_media_timeout", config->sip_media_timeout);
+ ConfigLogger::log_value("sip_invite_timeout", config->sip_invite_timeout);
+ ConfigLogger::log_value("sip_disconnect_timeout", config->sip_disconnect_timeout);
}
void Sip::eval(Packet* p)
SIP_DialogList dialogs;
SIP_Roptions ropts;
SIP_PROTO_CONF *sip_config;
+ bool sip_aborted;
static unsigned pub_id;
};
PegCount sessions;
PegCount concurrent_sessions;
PegCount max_concurrent_sessions;
+ PegCount aborted_sessions;
PegCount events;
PegCount dialogs;
PegCount ignoreChannels;
uint16_t maxViaLen; // Maximum Via field size
uint16_t maxContactLen; // Maximum Contact field size
uint16_t maxContentLen; // Maximum Content length
+ uint64_t sip_timeout;
+ uint64_t sip_media_timeout;
+ uint64_t sip_invite_timeout;
+ uint64_t sip_disconnect_timeout;
bool ignoreChannel; // Whether to ignore media channels found by SIP PP
};
switch (methodFlag)
{
case SIP_METHOD_INVITE:
-
+ {
ret = SIP_processInvite(sipMsg, dialog, dList);
+ if (ret and (config->sip_invite_timeout))
+ {
+ p->flow->set_idle_timeout(config->sip_invite_timeout);
+ }
break;
-
+ }
case SIP_METHOD_CANCEL:
-
+ {
if (nullptr == dialog)
return false;
/*dialog can be deleted in the early state*/
if ((SIP_DLG_EARLY == dialog->state)||(SIP_DLG_INVITING == dialog->state)
|| (SIP_DLG_CREATE == dialog->state))
SIP_deleteDialog(dialog, dList);
-
+ if (config->sip_disconnect_timeout)
+ p->flow->set_idle_timeout(config->sip_disconnect_timeout);
break;
+ }
case SIP_METHOD_ACK:
-
+ {
SIP_processACK(sipMsg, dialog, dList, p, config);
+ if (config->sip_timeout)
+ p->flow->set_idle_timeout(config->sip_timeout);
break;
+ }
case SIP_METHOD_BYE:
-
+ {
if (SIP_DLG_ESTABLISHED == dialog->state)
dialog->state = SIP_DLG_TERMINATING;
+
+ if (config->sip_disconnect_timeout)
+ p->flow->session_state |= STREAM_STATE_CLOSED;
break;
+ }
default:
// media session
if ( !SIP_checkMediaChange(sipMsg, dialog) )
{
+ if (config->sip_media_timeout)
+ p->flow->set_idle_timeout(config->sip_media_timeout);
SIP_updateMedias(sipMsg.mediaSession, dialog->mediaSessions);
SIP_ignoreChannels(*currDialog, p,config);
sipMsg.mediaUpdated = true;
#include "config.h"
#endif
+#include "log/messages.h"
#include "sip_module.h"
#include <cassert>
{ "methods", Parameter::PT_STRING, nullptr, default_methods,
"list of methods to check in SIP messages" },
+ { "sip_timeout", Parameter::PT_INT, "0:", "0",
+ "SIP Timeout value in milliseconds" },
+
+ { "sip_media_timeout", Parameter::PT_INT, "0:", "0",
+ "SIP Media timeout milliseconds" },
+
+ { "sip_invite_timeout", Parameter::PT_INT, "0:", "0",
+ "SIP Invite timeout milliseconds" },
+
+ { "sip_disconnect_timeout", Parameter::PT_INT, "0:", "0",
+ "SIP Disconnect timeout milliseconds" },
+
{ nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr }
};
{ CountType::SUM, "sessions", "total sessions" },
{ CountType::NOW, "concurrent_sessions", "total concurrent SIP sessions" },
{ CountType::MAX, "max_concurrent_sessions", "maximum concurrent SIP sessions" },
+ { CountType::SUM, "aborted_sessions", "total session aborted" },
{ CountType::SUM, "events", "events generated" },
{ CountType::SUM, "dialogs", "total dialogs" },
{ CountType::SUM, "ignored_channels", "total channels ignored" },
else if ( v.is("methods") )
sip_methods = v.get_string();
+ else if ( v.is("sip_timeout") )
+ conf->sip_timeout = v.get_uint64()/1000;
+
+ else if ( v.is("sip_invite_timeout") )
+ conf->sip_invite_timeout = v.get_uint64()/1000;
+
+ else if ( v.is("sip_media_timeout") )
+ conf->sip_media_timeout = v.get_uint64()/1000;
+
+ else if ( v.is("sip_disconnect_timeout") )
+ conf->sip_disconnect_timeout = v.get_uint64()/1000;
+
return true;
}
#include <cctype>
#include <cstring>
+#include "protocols/packet.h"
+
+#include "sip.h"
+
using namespace snort;
const char SipSplitter::content_len_key[] = "Content-Length";
}
StreamSplitter::Status SipSplitter::scan(
- Packet*, const uint8_t* data, uint32_t len,
+ Packet* pkt, const uint8_t* data, uint32_t len,
uint32_t, uint32_t* fp)
{
+ SIPData* sip_sess;
+ Flow* flow = pkt->flow;
+
+ sip_sess = get_sip_session_data(flow);
+ if (sip_sess && sip_sess->sip_aborted) {
+ sip_stats.aborted_sessions++;
+ return StreamSplitter::ABORT;
+ }
+
for (uint32_t i = 0; i < len; i++)
{
uint8_t ch = data[i];
#include "sip_splitter_test.h"
#include "log/messages.h"
+#include "protocols/packet.h"
#include "service_inspectors/sip/sip_splitter.h"
#include "stream/stream_splitter.h"
using namespace snort;
+Packet::Packet(bool)
+{
+ memset((char*) this , 0, sizeof(*this));
+}
+
+Packet::~Packet() = default;
+
TEST_GROUP(sip_splitter_scan_test)
{
SipSplitterUT ssut = SipSplitterUT(SipSplitter(true));
TEST(sip_splitter_scan_test, scan_start_content_len_test)
{
uint32_t fp = 0;
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ Packet p(true);
+
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"0xBEEF0xBEEF\n", 13, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_CONTENT_LEN_CMD);
TEST(sip_splitter_scan_test, scan_start_content_len_negative_test)
{
uint32_t fp = 0;
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ Packet p(true);
+
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"0xBEEF0xBEEF", 12, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_START_STATE);
TEST(sip_splitter_scan_test, scan_process_cmd_test)
{
uint32_t fp = 0;
+ Packet p(true);
+
ssut.splitter_set_paf_state(SIP_PAF_CONTENT_LEN_CMD);
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"C", 1, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_CONTENT_LEN_CMD);
TEST(sip_splitter_scan_test, scan_content_len_convert_body_search_test)
{
uint32_t fp = 0;
+ Packet p(true);
+
ssut.splitter_set_paf_state(SIP_PAF_CONTENT_LEN_CONVERT);
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"144 ", 4, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_BODY_SEARCH);
TEST(sip_splitter_scan_test, scan_content_len_invalid_test)
{
uint32_t fp = 0;
+ Packet p(true);
+
ssut.splitter_set_paf_state(SIP_PAF_CONTENT_LEN_CONVERT);
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"144i", 4, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_TRUE(ssut.is_init());
TEST(sip_splitter_scan_test, scan_search_body_test)
{
uint32_t fp = 0;
+ Packet p(true);
+
ssut.splitter_set_paf_state(SIP_PAF_BODY_SEARCH);
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"\r\n\r\n", 4, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_FLUSH_STATE);
ssut.splitter_reset_states();
ssut.splitter_set_paf_state(SIP_PAF_BODY_SEARCH);
- ret = ssut.splitter_scan(nullptr, (const uint8_t *)"\n\n", 2, 0, &fp);
+ ret = ssut.splitter_scan(&p, (const uint8_t *)"\n\n", 2, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_FLUSH_STATE);
CHECK_EQUAL(fp, 0);
TEST(sip_splitter_scan_test, scan_flush_test)
{
uint32_t fp = 0;
+ Packet p(true);
+
ssut.splitter_set_paf_state(SIP_PAF_FLUSH_STATE);
ssut.splitter_set_content_length(6);
// Sip splitter starts searching body from one character behind the actual body.
- StreamSplitter::Status ret = ssut.splitter_scan(nullptr,
+ StreamSplitter::Status ret = ssut.splitter_scan(&p,
(const uint8_t *)"\nfoobar", 7, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::FLUSH);
CHECK_TRUE(ssut.is_init());
ssut.splitter_set_paf_state(SIP_PAF_FLUSH_STATE);
ssut.splitter_set_content_length(12);
- ret = ssut.splitter_scan(nullptr, (const uint8_t *)"\nfoobar", 7, 0, &fp);
+ ret = ssut.splitter_scan(&p, (const uint8_t *)"\nfoobar", 7, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::SEARCH);
CHECK_EQUAL(ssut.splitter_get_paf_state(), SIP_PAF_FLUSH_STATE);
CHECK_EQUAL(ssut.splitter_get_content_length(), 5);
CHECK_EQUAL(fp, 0);
//Continue scanning the remaining buffer
- ret = ssut.splitter_scan(nullptr, (const uint8_t *)"foobar", 6, 0, &fp);
+ ret = ssut.splitter_scan(&p, (const uint8_t *)"foobar", 6, 0, &fp);
CHECK_EQUAL(ret, StreamSplitter::FLUSH);
CHECK_TRUE(ssut.is_init());
CHECK_EQUAL(fp, 6);
#include "stream/stream_splitter.h"
#include "service_inspectors/sip/sip_splitter.h"
+#include "service_inspectors/sip/sip.h"
//stubs to avoid link errors
const snort::StreamBuffer snort::StreamSplitter::reassemble(snort::Flow*, unsigned int, unsigned int,
unsigned char const*, unsigned int, unsigned int, unsigned int &) { return {}; }
unsigned snort::StreamSplitter::max(snort::Flow *) { return 0; }
+SIPData* get_sip_session_data(const snort::Flow*) { return nullptr; }
const uint8_t line_feed = '\n';
const uint8_t carriage_return = '\r';
const uint8_t no_lf_cr = '\t';
+THREAD_LOCAL SipStats sip_stats;
//characters recognized by isspace() as spaces
const uint8_t spaces[] = {' ', '\t', '\n', '\v', '\f', '\r'};
delete config;
}
-bool Smtp::configure(SnortConfig*)
+bool Smtp::configure(SnortConfig* sc)
{
SMTP_RegXtraDataFuncs(config);
- config->decode_conf.sync_all_depths();
+ config->decode_conf.sync_all_depths(sc);
if (config->decode_conf.get_file_depth() > -1)
config->log_config.log_filename = true;
-The wizard uses hexes, spells and curses to determine the most likely service
+The wizard uses hexes, spells and curses to determine the most likely service
on a flow. It does not determine the service with certainty; that is the job of
the service inspector or appId. The goal is to get the most likely service
inspector engaged as quickly as possible.
Hexes and spells differ in the following aspects:
- * spells allow wildcards matching any number of consecutive characters
- whereas hexes allow a single wild char.
- * spells are case insensitive whereas hexes are case sensitive.
- * spells automatically skip leading whitespace (at very start of flow).
+* spells allow wildcards matching any number of consecutive characters
+ whereas hexes allow a single wild char.
+* spells are case insensitive whereas hexes are case sensitive.
+* spells automatically skip leading whitespace (at very start of flow).
To match "`*`" symbol in traffic, put "`**`" in a spell.
+For example:
- For example: traffic "* OK" will match the pattern "** OK".
+ traffic "* OK" will match the pattern "** OK".
A series of asterisks is matched from left to right. '***' is seen as "*<glob>".
==== Concepts
- * `MagicPage` - leaf of a trie. Represents a symbol of a pattern.
- * `MagicBook` - trie itself. Represents a set of patterns for the wizard instance.
- ** `SpellBook` - `MagicBook` implementation for spells.
- ** `HexBook` - `MagicBook` implementation for hexes.
- * `MagicSplitter` - object related to a stream. Applies wizard logic to a stream.
- * `Wand` - contains state of wizard patterns for a stream.
- * `CurseDetails` - settings of a curse. Contains identifiers and algorithm.
- * `CurseTracker` - state of a curse.
- * `CurseBook` - contains all configured curses.
- * `CurseServiceTracker` - instance of a curse. Contains settings and state.
+* `MagicPage` - leaf of a trie. Represents a symbol of a pattern.
+* `MagicBook` - trie itself. Represents a set of patterns for the wizard instance.
+ ** `SpellBook` - `MagicBook` implementation for spells.
+ ** `HexBook` - `MagicBook` implementation for hexes.
+* `MagicSplitter` - object related to a stream. Applies wizard logic to a stream.
+* `Wand` - contains state of wizard patterns for a stream.
+* `CurseDetails` - settings of a curse. Contains identifiers and algorithm.
+* `CurseTracker` - state of a curse.
+* `CurseBook` - contains all configured curses.
+* `CurseServiceTracker` - instance of a curse. Contains settings and state.
==== MagicSplitter
Each flow contains two `MagicSplitter` objects: client-to-server and server-to-client.
Each `MagicSplitter` contains `Wand` that stores pointers unique for the flow:
- 1. MagicPage of Hex
- 2. MagicPage of Spell
- 3. Vector of all curses
+1. MagicPage of Hex
+2. MagicPage of Spell
+3. Vector of all curses
Where 1 and 2 - point to the current page in pattern.
==== Spell matching algorithm
-
-The spell matching algorithm is defined in `SpellBook::find_spell()` method.
+
+The spell matching algorithm is defined in `SpellBook::find_spell()` method.
In general, `MagicPage::next` array is an alphabet (ASCII table),
each element of which can exist or be absent. Thus, if an element exists
in the position of a certain symbol, it means that there is a pattern with
-such a sequence of symbols.
+such a sequence of symbols.
+
+Example:
- Example:
- User configured only one pattern: "ABC"
- MagicPage(root)::next - all elements beside (int)A is nullptr.
- MagicPage(A)::next - all elements beside (int)B is nullptr.
- MagicPage(B)::next - all elements beside (int)C is nullptr.
+ User configured only one pattern: "ABC"
+ MagicPage(root)::next - all elements beside (int)A is nullptr.
+ MagicPage(A)::next - all elements beside (int)B is nullptr.
+ MagicPage(B)::next - all elements beside (int)C is nullptr.
-Wizard iterates over the data from begin to end, checking at each iteration
-if there is a transition from the current character of the pattern to the
+Wizard iterates over the data from begin to end, checking at each iteration
+if there is a transition from the current character of the pattern to the
next character of the data.
`MagicPage::any` reflects a glob (wildcard). If wizard transitioned to a glob of the pattern,
a loop is started, in which wizard is trying to match the pattern from the current symbol of the
-data. If it failed to match the pattern from the current symbol of the data, it moves
-to the next symbol and tries again, and so either until it matches the pattern or the
+data. If it failed to match the pattern from the current symbol of the data, it moves
+to the next symbol and tries again, and so either until it matches the pattern or the
data runs out.
-`MagicPage::value` is not empty only in those positions that are the ends of some pattern.
-Thus, if, after a complete pass through the data, the wizard have reached a position in which this
+`MagicPage::value` is not empty only in those positions that are the ends of some pattern.
+Thus, if, after a complete pass through the data, the wizard have reached a position in which this
field is not empty, means that it has matched the pattern.
It should be mentioned that the matching of spells is case-independent, this is
==== Hex matching algorithm
-The algorithm for matching hexes is defined in `HexBook::find_spell()` and
+The algorithm for matching hexes is defined in `HexBook::find_spell()` and
identical to the algorithm for spells, but lacks:
- 1) converting to an uppercase, since hexes work with raw data;
- 2) loops for working with the glob, since glob in hexes replaces exactly one symbol;
- 3) saving the position of the glob between packets.
+* converting to an uppercase, since hexes work with raw data;
+* loops for working with the glob, since glob in hexes replaces exactly one symbol;
+* saving the position of the glob between packets.
==== TCP traffic processing
-Execution starts from the `MagicSplitter::scan()`.
+Execution starts from the `MagicSplitter::scan()`.
Since we want to be able to match patterns between packets in a stream, wizard need to
-save the state of the pattern at the end of the processing of a particular packet.
+save the state of the pattern at the end of the processing of a particular packet.
The state of the pattern is saved in the `MagicSplitter::wand`. However, for spells,
it needs to keep the presence of a glob between packages. This is implemented with
`MagicSplitter::bookmark`.
-Spells, hexes and curses are called inside the `Wizard::cast_spell()`.
+Spells, hexes and curses are called inside the `Wizard::cast_spell()`.
There wizard determines the search depth and sequentially calls the processing methods.
If wizard matched the pattern in the `Wizard::cast_spell()`, it increments `tcp_hits`.
If it didn't, then it checks whether it reached the limit of `max_search_depth`.
-If wizard has reached the limit of `max_search_depth` and has't matched a pattern,
+If wizard has reached the limit of `max_search_depth` and has't matched a pattern,
then it nullifies `Wand::spell` and `Wand::hex`, thus further in `Wizard::finished()` it'll
know that this flow can be abandoned and raise `tcp_misses` by 1.
Way of processing UDP is similar to TCP but has some differences:
- 1. Instead `MagicSplitter::scan()`, processing starts from `Wizard::eval()`;
- 2. Wizard processes only the first packet of UDP "meta-flow", so for
- every packet amount of previously processed bytes sets at 0;
- 3. There isn't any bookmark - UDP doesn't support wildcard over several packets.
- 4. The wizard don't need to check `Wizard::finished()`, because it processes only the
- first packet of UDP "meta-flow". So, if it hasn't matched anything in
- `Wizard::cast_spell()`, it increments `udp_misses` and unbinds itself from the flow.
+1. Instead `MagicSplitter::scan()`, processing starts from `Wizard::eval()`;
+2. Wizard processes only the first packet of UDP "meta-flow", so for
+ every packet amount of previously processed bytes sets at 0;
+3. There isn't any bookmark - UDP doesn't support wildcard over several packets.
+4. The wizard don't need to check `Wizard::finished()`, because it processes only the
+ first packet of UDP "meta-flow". So, if it hasn't matched anything in
+ `Wizard::cast_spell()`, it increments `udp_misses` and unbinds itself from the flow.
==== Additional info
Every flow gets a context (in `MagicSplitter`), where wizard stores flow's processing state.
Each flow is processed independently from others.
-Currently wizard cannot roll back on the pattern, so if it reaches a certain symbol
-of the pattern, it cannot go back. In some cases this will lead to the fact that
+Currently wizard cannot roll back on the pattern, so if it reaches a certain symbol
+of the pattern, it cannot go back. In some cases this will lead to the fact that
the pattern that could be matched will not be matched.
- For example:
- Patterns: "foobar", "foo*"
- Content: "foobaz"
+For example:
+
+ Patterns: "foobar", "foo*"
+ Content: "foobaz"
Unfortunately, none of the available patterns will match in such case.
- This is due to the fact that the symbols have a higher priority than
+ This is due to the fact that the symbols have a higher priority than
the glob. So from the MagicPage(O) wizard will transit to the MagicPage(B)
of the "foobar" pattern and will not process glob. Further, in MagicPage(А)::next[]
it will not find MagicPage by the symbol "z" and will consider the pattern unmatched.
#include <assert.h>
-bool CurseBook::mms_curse(const uint8_t* data, unsigned len, CurseTracker* tracker)
+enum PresCtx
{
- // peg the tracker to MMS
- MmsTracker& mms = tracker->mms;
+ PRES_CTX_ACSE = 1,
+ PRES_CTX_MMS = 3,
+};
- // if the state is set to MMS_STATE__SEARCH it means we most likely
- // have a split pipelined message coming through and will need to
- // reset the state
- if ( mms.state == MMS_STATE__SEARCH )
+static bool verify_search_depth_idx( unsigned len, uint32_t idx, uint32_t max_depth_idx )
+{
+ return idx + max_depth_idx < len;
+}
+
+static uint32_t search_for_osi_session_spdu( MmsTracker& mms, const uint8_t* data, uint32_t idx )
+{
+ switch ( data[idx] )
{
- mms.state = mms.last_state;
+ // check for a known MMS message tag in the event Session/Pres/ACSE aren't used
+ case MMS_CONFIRMED_REQUEST_TAG: // fallthrough intentional
+ case MMS_CONFIRMED_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CONFIRMED_ERROR_TAG: // fallthrough intentional
+ case MMS_UNCONFIRMED_TAG: // fallthrough intentional
+ case MMS_REJECT_TAG: // fallthrough intentional
+ case MMS_CANCEL_REQUEST_TAG: // fallthrough intentional
+ case MMS_CANCEL_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CANCEL_ERROR_TAG: // fallthrough intentional
+ case MMS_INITIATE_REQUEST_TAG: // fallthrough intentional
+ case MMS_INITIATE_RESPONSE_TAG: // fallthrough intentional
+ case MMS_INITIATE_ERROR_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_REQUEST_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_ERROR_TAG:
+ {
+ // when it looks like MMS this early, drop the index back one and push processing down to the full MMS parsing
+ // taking this approach as there are other paths into the MMS determination
+ if ( idx > 0 )
+ {
+ idx--;
+ mms.state = MMS_STATE__MMS;
+ break;
+ }
+
+ // default to MMS not found
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
}
- // define all known MMS tags to check for
+ // if mms isn't found, search for a supported OSI Session layer
+
+ // check for the Confirmed Request/Response path
+ case MMS_OSI_SESSION_SPDU_GT_DT:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_GT_LEN;
+ break;
+ }
+
+ // check for the Initiate Request path
+ case MMS_OSI_SESSION_SPDU_CN:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_CN_LEN;
+ break;
+ }
+
+ // check for the Initiate Response path
+ case MMS_OSI_SESSION_SPDU_AC:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_AC_LEN;
+ break;
+ }
+
+ // if neither are found, it is most likely not MMS
+ default:
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
+ }
+ }
+ return idx;
+}
+
+static uint32_t search_for_pres_ctx( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
+ // define some constants for search windows
+ constexpr uint32_t fully_encoded_data_window = 3;
+ constexpr uint32_t pres_ctx_window = 3;
+ constexpr uint32_t max_depth_idx = fully_encoded_data_window + pres_ctx_window;
+
+ // try to determine if the ACSE layer exists
+ // len reduction is to allow space for the forward looking checks
+ bool ctx_likely = false;
+ for ( uint32_t init_byte = idx; ( init_byte < len - max_depth_idx ) && !ctx_likely; init_byte++)
+ {
+ // require the user-data fully encoded data tag ( | 61 | )
+ if ( data[init_byte] != 0x61 )
+ {
+ continue;
+ }
+
+ // make sure there is still enough space left in the buffer for the forward search
+ // allow up to two bytes for the fully encoded data field
+ if ( !verify_search_depth_idx( len, init_byte, max_depth_idx ) )
+ {
+ // not enough data to process
+ assert( !ctx_likely );
+ break;
+ }
+
+ for ( uint32_t encode_tag_shift = 1; ( encode_tag_shift < fully_encoded_data_window ) && !ctx_likely; encode_tag_shift++ )
+ {
+ uint32_t encode_tag_byte = init_byte + encode_tag_shift;
+ // look for the ' fully encoded data ' user data tag ( | 30 | );
+ if ( data[encode_tag_byte] != 0x30 )
+ {
+ continue;
+ }
+
+ for ( uint32_t pres_ctx_shift = 1; ( pres_ctx_shift < + pres_ctx_window ) && !ctx_likely; pres_ctx_shift++ )
+ {
+ // look for the presentation context tag and length ( | 02 01 | )
+ bool ctx_b1 = data[encode_tag_byte + pres_ctx_shift] == 0x02;
+ bool ctx_b2 = data[encode_tag_byte + pres_ctx_shift + 1] == 0x01;
+ if ( ctx_b1 && ctx_b2 )
+ {
+ switch ( data[encode_tag_byte + pres_ctx_shift + 2] )
+ {
+ // set the state accordingly when the OSI ACSE presentation context ( | 01 | ) has been found
+ case PresCtx::PRES_CTX_ACSE:
+ {
+ // place the index at the last byte of the presentation context
+ idx = init_byte + encode_tag_shift + pres_ctx_shift + 2;
+ mms.state = MMS_STATE__OSI_ACSE;
+ ctx_likely = true;
+ break;
+ }
+ // set the state accordingly when the MMS presentation context ( | 03 | ) has been found
+ case PresCtx::PRES_CTX_MMS:
+ {
+ // place the index at the last byte of the presentation context
+ idx = init_byte + encode_tag_shift + pres_ctx_shift + 2;
+ mms.state = MMS_STATE__MMS;
+ ctx_likely = true;
+ break;
+ }
+ // no default as we want to keep looking if an acceptable context is not found
+ }
+ }
+ }
+ }
+ }
+ return idx;
+}
+
+static uint32_t search_for_osi_session_spdu_params( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
enum
{
- MMS_CONFIRMED_REQUEST_TAG = 0xA0,
- MMS_CONFIRMED_RESPONSE_TAG = 0xA1,
- MMS_CONFIRMED_ERROR_TAG = 0xA2,
- MMS_UNCONFIRMED_TAG = 0xA3,
- MMS_REJECT_TAG = 0xA4,
- MMS_CANCEL_REQUEST_TAG = 0x85,
- MMS_CANCEL_RESPONSE_TAG = 0x86,
- MMS_CANCEL_ERROR_TAG = 0xA7,
- MMS_INITIATE_REQUEST_TAG = 0xA8,
- MMS_INITIATE_RESPONSE_TAG = 0xA9,
- MMS_INITIATE_ERROR_TAG = 0xAA,
- MMS_CONCLUDE_REQUEST_TAG = 0x8B,
- MMS_CONCLUDE_RESPONSE_TAG = 0x8C,
- MMS_CONCLUDE_ERROR_TAG = 0xAD,
+ CN_SPDU_PARAM__CONNECT_ACCEPT_ITEM = 0x05,
+ CN_SPDU_PARAM__SESSION_REQUIREMENT = 0x14,
+ CN_SPDU_PARAM__SESSION_USER_DATA = 0xC1,
};
- uint32_t idx = 0;
- while ( idx < len )
+ // len reduction is to allow space for the forward looking checks
+ while ( idx < len - 3 )
{
- switch ( mms.state )
+ // check for the possibility of a connect accept item
+ if ( data[idx] == CN_SPDU_PARAM__CONNECT_ACCEPT_ITEM )
{
- case MMS_STATE__TPKT_VER:
+ // track that the item was found
+ mms.connect_accept_item_likely = true;
+ }
+ // check for the possibility of a session requirement
+ else if ( data[idx] == CN_SPDU_PARAM__SESSION_REQUIREMENT )
+ {
+ // track that the item was found
+ mms.session_requirement_likely = true;
+ }
+ // check for the possibility of a session user data item
+ else if ( data[idx] == CN_SPDU_PARAM__SESSION_USER_DATA )
+ {
+ // when this has been found and there is a good chance all of the other required items exist, move on to look for mms
+ if ( mms.connect_accept_item_likely &&
+ mms.session_requirement_likely )
{
- mms.state = MMS_STATE__TPKT_RES;
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_USER_DATA_LEN;
break;
}
-
- case MMS_STATE__TPKT_RES:
+ // otherwise it is unlikely that this is mms
+ else
{
- mms.state = MMS_STATE__TPKT_LEN1;
+ mms.state = MMS_STATE__NOT_FOUND;
break;
}
+ }
+ // no else case as we want to allow for the possibility of non-standard items
+
+ // increment the index to look at the item length field
+ idx++;
+ // increment the index to the end of the item data
+ idx += data[idx];
+ // increment the index to the start of the next item
+ idx++;
+
+ // if the index has gotten larger than the available buffer, bail
+ if ( idx >= len )
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
+ }
+ }
+ return idx;
+}
+
+static uint32_t search_for_osi_acse_type( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
+ enum
+ {
+ OSI_ACSE_AARQ = 0x60,
+ OSI_ACSE_AARE = 0x61,
+ };
- case MMS_STATE__TPKT_LEN1:
+ constexpr uint32_t max_search_depth_idx = 3;
+ if ( verify_search_depth_idx( len, idx, max_search_depth_idx ) )
+ {
+ // when ACSE is likely found, do processing at that layer to determine if MMS is being transported
+ // length field could be 1-2 bytes depending on encoding
+ for ( uint32_t j = 0; j < max_search_depth_idx; j++ )
+ {
+ // look for either the AARQ ( | 60 | ) or AARE ( | 61 | ) tag
+ if ( data[idx+j] == OSI_ACSE_AARQ || data[idx+j] == OSI_ACSE_AARE )
{
- mms.state = MMS_STATE__TPKT_LEN2;
+ mms.state = MMS_STATE__OSI_ACSE_DATA;
+ idx += j;
break;
}
+ }
+ }
+ return idx;
+}
- case MMS_STATE__TPKT_LEN2:
+static uint32_t search_for_osi_acse_data( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
+ // this will likely be a good distance from the current position
+ // minus three is to give space for the forward checks for the direct and indirect reference
+ for ( uint32_t k = 0; idx+k < len-3; k++ )
+ {
+ // look for the MMS presentation context ( | 02 01 03 | )
+ if ( data[idx+k] == 0x02 && data[idx+k+1] == 0x01 && data[idx+k+2] == 0x03 )
+ {
+ mms.state = MMS_STATE__MMS;
+ // increment the index to the end of the mms context id reference
+ idx += k+2;
+ break;
+ }
+ }
+ return idx;
+}
+
+static uint32_t search_for_mms( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
+ constexpr uint32_t max_search_depth_idx = 2;
+ if ( verify_search_depth_idx( len, idx, max_search_depth_idx ) )
+ {
+ // search within the next two bytes to determine if mms is likely
+ for ( uint32_t j = 0; j < max_search_depth_idx; j++ )
+ {
+ // for each remaining byte check to see if it is in the known tag map
+ switch ( data[idx+j] )
{
- mms.state = MMS_STATE__COTP_LEN;
+ case MMS_CONFIRMED_REQUEST_TAG:
+ {
+ mms.state = MMS_STATE__MMS_CONFIRMED_REQUEST;
+ idx += j;
break;
}
-
- case MMS_STATE__COTP_LEN:
+ case MMS_CONFIRMED_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CONFIRMED_ERROR_TAG: // fallthrough intentional
+ case MMS_UNCONFIRMED_TAG: // fallthrough intentional
+ case MMS_REJECT_TAG: // fallthrough intentional
+ case MMS_CANCEL_REQUEST_TAG: // fallthrough intentional
+ case MMS_CANCEL_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CANCEL_ERROR_TAG: // fallthrough intentional
+ case MMS_INITIATE_REQUEST_TAG: // fallthrough intentional
+ case MMS_INITIATE_RESPONSE_TAG: // fallthrough intentional
+ case MMS_INITIATE_ERROR_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_REQUEST_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_RESPONSE_TAG: // fallthrough intentional
+ case MMS_CONCLUDE_ERROR_TAG:
{
- mms.state = MMS_STATE__COTP_PDU;
+ // if an MMS tag exists in the remaining data,
+ // hand off to the MMS service inspector
+ mms.state = MMS_STATE__FOUND;
+ idx += j;
break;
}
+ // no default as we want to keep searching if a result wasn't found in this loop
+ }
- case MMS_STATE__COTP_PDU:
+ // move on to the next case when a state has been set
+ if ( mms.state != MMS_STATE__NOT_FOUND )
{
- // 7 6 5 4 3 2 1 0
- // ---------------
- // . . . . x x x x Destination Reference
- // x x x x . . . . PDU Type
- const uint32_t MMS_COTP_PDU_DT_DATA = 0x0F;
+ break;
+ }
+ }
+ }
+ return idx;
+}
- if ( data[idx] >> 0x04 != MMS_COTP_PDU_DT_DATA )
- {
- mms.state = MMS_STATE__NOT_FOUND;
- break;
- }
+static uint32_t search_for_mms_confirmed_request( MmsTracker& mms, const uint8_t* data, unsigned len, uint32_t idx )
+{
+ // default to MMS not found
+ mms.state = MMS_STATE__NOT_FOUND;
+
+ // confirmed service request types
+ enum
+ {
+ DEFINE_NAMED_TYPE_MESSAGE = 0xAE,
+ DEFINE_NAMED_VARIABLE_MESSAGE = 0xA7,
+ DEFINE_NAMED_VARIABLE_LIST_MESSAGE = 0xAB,
+ DEFINE_SCATTERED_ACCESS_MESSAGE = 0xA8,
+ DEFINE_SEMAPHORE_MESSAGE = 0xB5,
+ DELETE_NAMED_TYPE_MESSAGE = 0xB0,
+ DELETE_NAMED_VARIABLE_LIST_MESSAGE = 0xAD,
+ DELETE_SEMAPHORE_MESSAGE = 0xB6,
+ DELETE_VARIABLE_ACCESS_MESSAGE = 0xAA,
+ DOWNLOAD_SEGMENT_MESSAGE = 0x9B,
+ GET_NAME_LIST_MESSAGE = 0xA1,
+ GET_NAMED_TYPE_ATTRIBUTES_MESSAGE = 0xAF,
+ GET_NAMED_VARIABLE_LIST_ATTRIBUTES_MESSAGE = 0xAC,
+ GET_SCATTERED_ACCESS_ATTRIBUTES_MESSAGE = 0xA9,
+ GET_VARIABLE_ACCESS_ATTRIBUTES_MESSAGE = 0xA6,
+ IDENTIFY_MESSAGE = 0x82,
+ INITIATE_DOWNLOAD_SEQUENCE_MESSAGE = 0xBA,
+ INITIATE_UPLOAD_SEQUENCE_MESSAGE = 0x9D,
+ INPUT_MESSAGE = 0xB1,
+ OUTPUT_MESSAGE = 0xB2,
+ READ_MESSAGE = 0xA4,
+ RELINQUISH_CONTROL_MESSAGE = 0xB4,
+ RENAME_MESSAGE = 0xA3,
+ REPORT_POOL_SEMAPHORE_STATUS_MESSAGE = 0xB8,
+ REPORT_SEMAPHORE_ENTRY_STATUS_MESSAGE = 0xB9,
+ REPORT_SEMAPHORE_STATUS_MESSAGE = 0xB7,
+ STATUS_MESSAGE = 0x80,
+ TAKE_CONTROL_MESSAGE = 0xB3,
+ TERMINATE_DOWNLOAD_SEQUENCE_MESSAGE = 0xBC,
+ UPLOAD_SEGMENT_MESSAGE = 0x9E,
+ WRITE_MESSAGE = 0xA5,
+ EXPANSION_9F = 0x9F,
+ EXPANSION_BF = 0xBF,
+ };
- mms.state = MMS_STATE__COTP_TPDU_NUM;
+ // look for a known Confirmed Service Request 4-6 bytes away
+ constexpr uint32_t max_search_depth_idx = 6;
+ if ( verify_search_depth_idx( len, idx, max_search_depth_idx ) )
+ {
+ for ( uint32_t j = 3; j <= max_search_depth_idx; j++ )
+ {
+ // check for any of the single byte service tags
+ switch ( data[idx+j] )
+ {
+ case DEFINE_NAMED_TYPE_MESSAGE: // fallthrough intentional
+ case DEFINE_NAMED_VARIABLE_MESSAGE: // fallthrough intentional
+ case DEFINE_NAMED_VARIABLE_LIST_MESSAGE: // fallthrough intentional
+ case DEFINE_SCATTERED_ACCESS_MESSAGE: // fallthrough intentional
+ case DEFINE_SEMAPHORE_MESSAGE: // fallthrough intentional
+ case DELETE_NAMED_TYPE_MESSAGE: // fallthrough intentional
+ case DELETE_NAMED_VARIABLE_LIST_MESSAGE: // fallthrough intentional
+ case DELETE_SEMAPHORE_MESSAGE: // fallthrough intentional
+ case DELETE_VARIABLE_ACCESS_MESSAGE: // fallthrough intentional
+ case DOWNLOAD_SEGMENT_MESSAGE: // fallthrough intentional
+ case GET_NAME_LIST_MESSAGE: // fallthrough intentional
+ case GET_NAMED_TYPE_ATTRIBUTES_MESSAGE: // fallthrough intentional
+ case GET_NAMED_VARIABLE_LIST_ATTRIBUTES_MESSAGE: // fallthrough intentional
+ case GET_SCATTERED_ACCESS_ATTRIBUTES_MESSAGE: // fallthrough intentional
+ case GET_VARIABLE_ACCESS_ATTRIBUTES_MESSAGE: // fallthrough intentional
+ case IDENTIFY_MESSAGE: // fallthrough intentional
+ case INITIATE_DOWNLOAD_SEQUENCE_MESSAGE: // fallthrough intentional
+ case INITIATE_UPLOAD_SEQUENCE_MESSAGE: // fallthrough intentional
+ case INPUT_MESSAGE: // fallthrough intentional
+ case OUTPUT_MESSAGE: // fallthrough intentional
+ case READ_MESSAGE: // fallthrough intentional
+ case RELINQUISH_CONTROL_MESSAGE: // fallthrough intentional
+ case RENAME_MESSAGE: // fallthrough intentional
+ case REPORT_POOL_SEMAPHORE_STATUS_MESSAGE: // fallthrough intentional
+ case REPORT_SEMAPHORE_ENTRY_STATUS_MESSAGE: // fallthrough intentional
+ case REPORT_SEMAPHORE_STATUS_MESSAGE: // fallthrough intentional
+ case STATUS_MESSAGE: // fallthrough intentional
+ case TAKE_CONTROL_MESSAGE: // fallthrough intentional
+ case TERMINATE_DOWNLOAD_SEQUENCE_MESSAGE: // fallthrough intentional
+ case UPLOAD_SEGMENT_MESSAGE: // fallthrough intentional
+ case WRITE_MESSAGE: // fallthrough intentional
+ case EXPANSION_9F: // fallthrough intentional
+ case EXPANSION_BF:
+ {
+ idx += j;
+ mms.state = MMS_STATE__FOUND;
break;
}
+ // no default as we want to keep searching if a result wasn't found in this loop
+ }
- case MMS_STATE__COTP_TPDU_NUM:
+ // move on to the next case when a state has been set
+ if ( mms.state != MMS_STATE__NOT_FOUND )
{
- mms.state = MMS_STATE__OSI_SESSION_SPDU;
break;
}
+ }
+ }
- case MMS_STATE__OSI_SESSION_SPDU:
+ return idx;
+}
+
+bool CurseBook::mms_curse( const uint8_t* data, unsigned len, CurseTracker* tracker )
+{
+ // peg the tracker to MMS
+ MmsTracker& mms = tracker->mms;
+
+ // if the state is set to MMS_STATE__SEARCH it means we most likely
+ // have a split pipelined message coming through and will need to
+ // reset the state
+ if ( mms.state == MMS_STATE__SEARCH )
+ {
+ mms.state = mms.last_state;
+ }
+
+ uint32_t idx = 0;
+ while ( idx < len )
+ {
+ switch ( mms.state )
+ {
+ case MMS_STATE__TPKT_VER:
+ {
+ // MMS is only known to run over version | 03 |
+ if ( data[idx] != 0x03 )
{
- // define all known OSI Session layer SPDU tags to check
- enum
- {
- MMS_OSI_SESSION_SPDU_GT_DT = 0x01,
- MMS_OSI_SESSION_SPDU_CN = 0x0D,
- MMS_OSI_SESSION_SPDU_AC = 0x0E,
- };
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
+ }
- switch ( data[idx] )
- {
- // check for a known MMS message tag in the event Session/Pres/ACSE aren't used
- case MMS_CONFIRMED_REQUEST_TAG: // fallthrough intentional
- case MMS_CONFIRMED_RESPONSE_TAG: // fallthrough intentional
- case MMS_CONFIRMED_ERROR_TAG: // fallthrough intentional
- case MMS_UNCONFIRMED_TAG: // fallthrough intentional
- case MMS_REJECT_TAG: // fallthrough intentional
- case MMS_CANCEL_REQUEST_TAG: // fallthrough intentional
- case MMS_CANCEL_RESPONSE_TAG: // fallthrough intentional
- case MMS_CANCEL_ERROR_TAG: // fallthrough intentional
- case MMS_INITIATE_REQUEST_TAG: // fallthrough intentional
- case MMS_INITIATE_RESPONSE_TAG: // fallthrough intentional
- case MMS_INITIATE_ERROR_TAG: // fallthrough intentional
- case MMS_CONCLUDE_REQUEST_TAG: // fallthrough intentional
- case MMS_CONCLUDE_RESPONSE_TAG: // fallthrough intentional
- case MMS_CONCLUDE_ERROR_TAG:
- {
- // if an MMS tag exists in the remaining data,
- // hand off to the MMS service inspector
- mms.state = MMS_STATE__FOUND;
- break;
- }
+ mms.state = MMS_STATE__TPKT_RES;
+ break;
+ }
- // if mms isn't found, search for an OSI Session layer
- case MMS_OSI_SESSION_SPDU_GT_DT: // fallthrough intentional
- case MMS_OSI_SESSION_SPDU_CN: // fallthrough intentional
- case MMS_OSI_SESSION_SPDU_AC:
- {
- mms.state = MMS_STATE__MMS;
- break;
- }
+ case MMS_STATE__TPKT_RES:
+ {
+ mms.state = MMS_STATE__TPKT_LEN1;
+ break;
+ }
- // if neither are found, it is most likely not MMS
- default:
- {
- mms.state = MMS_STATE__NOT_FOUND;
- }
- }
+ case MMS_STATE__TPKT_LEN1:
+ {
+ mms.state = MMS_STATE__TPKT_LEN2;
+ break;
+ }
+ case MMS_STATE__TPKT_LEN2:
+ {
+ mms.state = MMS_STATE__COTP_LEN;
+ break;
+ }
+
+ case MMS_STATE__COTP_LEN:
+ {
+ mms.state = MMS_STATE__COTP_PDU;
+ break;
+ }
+
+ case MMS_STATE__COTP_PDU:
+ {
+ // 7 6 5 4 3 2 1 0
+ // ---------------
+ // . . . . x x x x Destination Reference
+ // x x x x . . . . PDU Type
+ constexpr uint32_t MMS_COTP_PDU_DT_DATA = 0x0F;
+
+ if ( data[idx] >> 0x04 != MMS_COTP_PDU_DT_DATA )
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
break;
}
- case MMS_STATE__MMS:
- {
- // loop through the remaining bytes in the buffer checking for known MMS tags
- for ( uint32_t i=idx; i < len; i++ )
- {
- // for each remaining byte check to see if it is in the known tag map
- switch ( data[i] )
- {
- case MMS_CONFIRMED_REQUEST_TAG: // fallthrough intentional
- case MMS_CONFIRMED_RESPONSE_TAG: // fallthrough intentional
- case MMS_CONFIRMED_ERROR_TAG: // fallthrough intentional
- case MMS_UNCONFIRMED_TAG: // fallthrough intentional
- case MMS_REJECT_TAG: // fallthrough intentional
- case MMS_CANCEL_REQUEST_TAG: // fallthrough intentional
- case MMS_CANCEL_RESPONSE_TAG: // fallthrough intentional
- case MMS_CANCEL_ERROR_TAG: // fallthrough intentional
- case MMS_INITIATE_REQUEST_TAG: // fallthrough intentional
- case MMS_INITIATE_RESPONSE_TAG: // fallthrough intentional
- case MMS_INITIATE_ERROR_TAG: // fallthrough intentional
- case MMS_CONCLUDE_REQUEST_TAG: // fallthrough intentional
- case MMS_CONCLUDE_RESPONSE_TAG: // fallthrough intentional
- case MMS_CONCLUDE_ERROR_TAG:
- {
- // if an MMS tag exists in the remaining data,
- // hand off to the MMS service inspector
- mms.state = MMS_STATE__FOUND;
- break;
- }
- // no default here as it we don't know when we would hit
- // the first MMS tag without doing full parsing
- }
+ mms.state = MMS_STATE__COTP_TPDU_NUM;
+ break;
+ }
- // exit the loop when a state has been determined
- if ( mms.state == MMS_STATE__NOT_FOUND
- or mms.state == MMS_STATE__SEARCH
- or mms.state == MMS_STATE__FOUND )
- {
- break;
- }
- }
+ case MMS_STATE__COTP_TPDU_NUM:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU;
+ break;
+ }
+
+ case MMS_STATE__OSI_SESSION_SPDU:
+ {
+ idx = search_for_osi_session_spdu(mms, data, idx);
+ break;
+ }
+
+ //
+ // State path for most MMS messages
+ //
+
+ // check the length field of a GT SPDU
+ case MMS_STATE__OSI_SESSION_SPDU_GT_LEN:
+ {
+ // length should always be zero for MMS
+ if ( data[idx] != 0x00 )
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
break;
}
- case MMS_STATE__FOUND:
- {
- mms.state = MMS_STATE__TPKT_VER;
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_DT;
+ break;
+ }
- return true;
+ // check for the tag of a DT SPDU
+ case MMS_STATE__OSI_SESSION_SPDU_DT:
+ {
+ // tag should always be | 01 | for DT
+ if ( data[idx] != MMS_OSI_SESSION_SPDU_GT_DT )
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
}
- case MMS_STATE__NOT_FOUND:
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_DT_LEN;
+ break;
+ }
+
+ // check the length field of a DT SPDU
+ case MMS_STATE__OSI_SESSION_SPDU_DT_LEN:
+ {
+ // length should always be zero for MMS
+ if ( data[idx] != 0x00 )
{
- mms.state = MMS_STATE__TPKT_VER;
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
+ }
+
+ mms.state = MMS_STATE__OSI_PRES_USER_DATA;
+ break;
+ }
+
+ // process the User Data Presentation type
+ case MMS_STATE__OSI_PRES_USER_DATA:
+ {
+ idx = search_for_pres_ctx(mms, data, len, idx);
+ break;
+ }
+
+
+ //
+ // State path for Initiate-Request and Initiate-Response
+ //
+
+ // skip the CN SPDU length field
+ case MMS_STATE__OSI_SESSION_SPDU_CN_LEN:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_PARAMS;
+ break;
+ }
+
+ // skip the AC SPDU length field
+ case MMS_STATE__OSI_SESSION_SPDU_AC_LEN:
+ {
+ mms.state = MMS_STATE__OSI_SESSION_SPDU_PARAMS;
+ break;
+ }
+
+ // check the parameters
+ case MMS_STATE__OSI_SESSION_SPDU_PARAMS:
+ {
+ idx = search_for_osi_session_spdu_params(mms, data, len, idx);
+ break;
+ }
- return false;
+ // jump over the length field
+ case MMS_STATE__OSI_SESSION_SPDU_USER_DATA_LEN:
+ {
+ mms.state = MMS_STATE__OSI_PRES_CP_CPA;
+ break;
+ }
+
+ // process a CP or CPA presentation type
+ case MMS_STATE__OSI_PRES_CP_CPA:
+ {
+ // look for the ' CP ' or ' CPA ' presentation type tag ( | 31 | );
+ if ( data[idx] != 0x31 )
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
+ break;
}
- default:
+ mms.state = MMS_STATE__OSI_PRES_CP_CPA_USER_DATA_ACSE_LOCATE;
+ break;
+ }
+
+ //
+ case MMS_STATE__OSI_PRES_CP_CPA_USER_DATA_ACSE_LOCATE:
+ {
+ idx = search_for_pres_ctx(mms, data, len, idx);
+ break;
+ }
+
+ // check for ACSE
+ case MMS_STATE__OSI_ACSE:
+ {
+ // look for the presentation data values single ASN1 type ( | A0 | );
+ if ( data[idx] != 0xA0 )
{
mms.state = MMS_STATE__NOT_FOUND;
- assert(false);
break;
}
+
+ mms.state = MMS_STATE__OSI_ACSE_TYPE;
+ break;
+ }
+
+ // look for a supported ACSE type tag
+ case MMS_STATE__OSI_ACSE_TYPE:
+ {
+ idx = search_for_osi_acse_type(mms, data, len, idx);
+ break;
+ }
+
+ //
+ case MMS_STATE__OSI_ACSE_DATA:
+ {
+ idx = search_for_osi_acse_data(mms, data, len, idx);
+ break;
+ }
+
+
+ //
+ // State path for MMS convergence
+ //
+
+ // Look for a byte known to represent a MMS tag
+ case MMS_STATE__MMS:
+ {
+ idx = search_for_mms(mms, data, len, idx);
+ break;
+ }
+
+ case MMS_STATE__MMS_CONFIRMED_REQUEST:
+ {
+ idx = search_for_mms_confirmed_request(mms, data, len, idx);
+ break;
+ }
+
+ case MMS_STATE__FOUND:
+ {
+ mms.state = MMS_STATE__TPKT_VER;
+
+ return true;
+ }
+
+ case MMS_STATE__NOT_FOUND:
+ {
+ mms.state = MMS_STATE__TPKT_VER;
+
+ return false;
+ }
+
+ default:
+ {
+ mms.state = MMS_STATE__NOT_FOUND;
+ assert( false );
+ break;
+ }
}
idx++;
MMS_STATE__COTP_PDU,
MMS_STATE__COTP_TPDU_NUM,
MMS_STATE__OSI_SESSION_SPDU,
+ MMS_STATE__OSI_SESSION_SPDU_GT_LEN,
+ MMS_STATE__OSI_SESSION_SPDU_DT,
+ MMS_STATE__OSI_SESSION_SPDU_DT_LEN,
+ MMS_STATE__OSI_SESSION_SPDU_CN_LEN,
+ MMS_STATE__OSI_SESSION_SPDU_AC_LEN,
+ MMS_STATE__OSI_SESSION_SPDU_PARAMS,
+ MMS_STATE__OSI_SESSION_SPDU_USER_DATA_LEN,
+ MMS_STATE__OSI_PRES_CP_CPA,
+ MMS_STATE__OSI_PRES_CP_CPA_USER_DATA_ACSE_LOCATE,
+ MMS_STATE__OSI_PRES_USER_DATA,
+ MMS_STATE__OSI_ACSE,
+ MMS_STATE__OSI_ACSE_TYPE,
+ MMS_STATE__OSI_ACSE_DATA,
MMS_STATE__MMS,
+ MMS_STATE__MMS_CONFIRMED_REQUEST,
MMS_STATE__FOUND,
MMS_STATE__SEARCH,
MMS_STATE__NOT_FOUND,
};
+// define all known MMS tags to check for
+enum
+{
+ MMS_CONFIRMED_REQUEST_TAG = 0xA0,
+ MMS_CONFIRMED_RESPONSE_TAG = 0xA1,
+ MMS_CONFIRMED_ERROR_TAG = 0xA2,
+ MMS_UNCONFIRMED_TAG = 0xA3,
+ MMS_REJECT_TAG = 0xA4,
+ MMS_CANCEL_REQUEST_TAG = 0x85,
+ MMS_CANCEL_RESPONSE_TAG = 0x86,
+ MMS_CANCEL_ERROR_TAG = 0xA7,
+ MMS_INITIATE_REQUEST_TAG = 0xA8,
+ MMS_INITIATE_RESPONSE_TAG = 0xA9,
+ MMS_INITIATE_ERROR_TAG = 0xAA,
+ MMS_CONCLUDE_REQUEST_TAG = 0x8B,
+ MMS_CONCLUDE_RESPONSE_TAG = 0x8C,
+ MMS_CONCLUDE_ERROR_TAG = 0xAD,
+};
+
+// define all applicable OSI Session layer SPDU tags to check
+enum
+{
+ MMS_OSI_SESSION_SPDU_GT_DT = 0x01,
+ MMS_OSI_SESSION_SPDU_CN = 0x0D,
+ MMS_OSI_SESSION_SPDU_AC = 0x0E,
+};
+
class MmsTracker
{
public:
MMS_State state = MMS_State::MMS_STATE__TPKT_VER;
MMS_State last_state = MMS_State::MMS_STATE__TPKT_VER;
+ bool connect_accept_item_likely = false;
+ bool session_requirement_likely = false;
};
#endif
add_subdirectory(test)
set (STREAM_INCLUDES
+ flush_bucket.h
paf.h
+ pafng.h
stream.h
stream_splitter.h
)
flush_bucket.h
paf.cc
paf_stats.h
+ pafng.cc
)
install (FILES ${STREAM_INCLUDES}
skey.padding = skey.flags.padding_bits = 0;
skey.flags.group_used = p->is_inter_group_flow();
skey.init_groups(p->pkth->ingress_group, p->pkth->egress_group, reversed);
+#ifndef DISABLE_TENANT_ID
skey.tenant_id = p->pkth->tenant_id;
-
+#endif
switch (p->type())
{
case PktType::TCP:
struct IpStats
{
SESSION_STATS;
- PegCount total_bytes; // total_ip_bytes_processed
- PegCount total; // total_ipfragmented_packets
- PegCount current_frags; // iCurrentFrags
- PegCount max_frags; // iMaxFrags
- PegCount reassembles; // total_ipreassembled_packets / iFragFlushes
+ PegCount total_bytes;
+ PegCount total;
+ PegCount current_frags;
+ PegCount max_frags;
+ PegCount reassembles;
PegCount discards;
- PegCount frag_timeouts; // iFragTimeouts
+ PegCount frag_timeouts;
PegCount overlaps;
PegCount anomalies;
PegCount alerts;
PegCount drops;
- PegCount trackers_created; // iFragCreates
+ PegCount trackers_created;
PegCount trackers_released;
- PegCount trackers_cleared; // iFragDeletes - delete meant dump the frag list
- PegCount trackers_completed;// iFragComplete
- PegCount nodes_created; // iFragInserts tracked a similar stat (# calls to insert)
+ PegCount trackers_cleared;
+ PegCount trackers_completed;
+ PegCount nodes_created;
PegCount nodes_released;
- PegCount reassembled_bytes; // total_ipreassembled_bytes
- PegCount fragmented_bytes; // total_ipfragmented_bytes
+ PegCount reassembled_bytes;
+ PegCount fragmented_bytes;
};
extern const PegInfo ip_pegs[];
{ CountType::SUM, "total_bytes", "total number of bytes processed" },
{ CountType::SUM, "total_frags", "total fragments" },
{ CountType::NOW, "current_frags", "current fragments" },
- { CountType::SUM, "max_frags", "max fragments" },
+ { CountType::MAX, "max_frags", "max fragments" },
{ CountType::SUM, "reassembled", "reassembled datagrams" },
{ CountType::SUM, "discards", "fragments discarded" },
{ CountType::SUM, "frag_timeouts", "datagrams abandoned" },
// occurs at the paf_max byte. So, we manually set the data's length and
// total queued bytes (px.len) to guarantee that at most paf_max bytes will
// be analyzed and flushed since the last flush point. It should also be
- // noted that we perform the check here rather in in paf_flush() to
+ // noted that we perform the check here rather in paf_flush() to
// avoid scanning the same data twice. The first scan would analyze the
// entire segment and the second scan would analyze this segments
// unflushed data.
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// pafng.cc author davis mcpherson davmcphe@cisco.com
+// based on paf.cc author Russ Combs <rcombs@sourcefire.com>
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "pafng.h"
+
+#include "detection/detection_engine.h"
+#include "protocols/packet.h"
+#include "protocols/tcp.h"
+
+using namespace snort;
+
+//--------------------------------------------------------------------
+// private state
+//--------------------------------------------------------------------
+
+
+#define PAF_LIMIT_FUZZ 1500
+
+// 255 is max pseudo-random flush point; eth mtu ensures that maximum flushes
+// are not trimmed which throws off the tracking total in stream5_paf.c
+// max paf max = max datagram - eth mtu - 255 = 63780
+#define MAX_PAF_MAX (65535 - PAF_LIMIT_FUZZ - 255)
+
+extern THREAD_LOCAL snort::ProfileStats pafPerfStats;
+
+//--------------------------------------------------------------------
+
+uint32_t ProtocolAwareFlusher::paf_flush (const PafAux& px, uint32_t* flags)
+{
+ uint32_t at = 0;
+ *flags &= ~(PKT_PDU_HEAD | PKT_PDU_TAIL);
+
+ switch ( px.ft )
+ {
+ case FT_NOP:
+ return -1;
+
+ case FT_SFP:
+ *flags = 0;
+ return -1;
+
+ case FT_PAF:
+ at = fpt;
+ *flags |= PKT_PDU_TAIL;
+ break;
+
+ case FT_LIMIT:
+ if (fpt > px.len)
+ {
+ at = px.len;
+ fpt -= px.len;
+ }
+ else
+ {
+ at = fpt;
+ fpt = px.len - fpt; // number of characters scanned but not flushing
+ }
+ break;
+
+ // use of px.len is suboptimal here because the actual amount
+ // flushed is determined later and can differ in certain cases
+ // such as exceeding s5_pkt->max_dsize. the actual amount
+ // flushed would ideally be applied to fpt later. for
+ // now we try to circumvent such cases so we track correctly.
+ //
+ // FIXIT-L max_dsize should no longer be exceeded since it excludes headers.
+ case FT_MAX:
+ at = px.len;
+ if ( fpt > px.len )
+ fpt -= px.len;
+ else
+ fpt = 0;
+ break;
+ }
+
+ if ( !at || !px.len )
+ return -1;
+
+ // safety - prevent seq + at < seq
+ if ( at > 0x7FFFFFFF )
+ at = 0x7FFFFFFF;
+
+ if ( !tot )
+ *flags |= PKT_PDU_HEAD;
+
+ if ( *flags & PKT_PDU_TAIL )
+ tot = 0;
+ else
+ tot += at;
+
+ return at;
+}
+
+//--------------------------------------------------------------------
+
+bool ProtocolAwareFlusher::paf_callback (PafAux& px, Packet* pkt, const uint8_t* data,
+ uint32_t len, uint32_t flags)
+{
+ fpt = 0;
+ state = splitter->scan(pkt, data, len, flags, &fpt);
+
+ if ( state == StreamSplitter::ABORT || state == StreamSplitter::STOP )
+ return false;
+
+ if ( state != StreamSplitter::SEARCH )
+ {
+ fpt += px.idx;
+ if ( fpt <= px.len )
+ {
+ px.idx = fpt;
+ return true;
+ }
+ }
+ px.idx = px.len;
+ return false;
+}
+
+//--------------------------------------------------------------------
+
+bool ProtocolAwareFlusher::paf_eval(PafAux& px, Packet* pkt, uint32_t flags,
+ const uint8_t* data, uint32_t len)
+{
+ switch ( state )
+ {
+ case StreamSplitter::SEARCH:
+ if ( px.len > px.idx )
+ return paf_callback(px, pkt, data, len, flags);
+
+ return false;
+
+ case StreamSplitter::FLUSH:
+ if ( px.len >= fpt )
+ {
+ px.ft = FT_PAF;
+ state = StreamSplitter::SEARCH;
+ return true;
+ }
+ if ( px.len >= splitter->max(pkt->flow) )
+ {
+ px.ft = FT_MAX;
+ return false;
+ }
+ return false;
+
+ case StreamSplitter::LIMIT:
+ // if we are within PAF_LIMIT_FUZZ character of paf_max ...
+ if ( px.len + PAF_LIMIT_FUZZ >= splitter->max(pkt->flow))
+ {
+ px.ft = FT_LIMIT;
+ state = StreamSplitter::LIMITED;
+ return false;
+ }
+ state = StreamSplitter::SEARCH;
+ return false;
+
+ case StreamSplitter::SKIP:
+ if ( px.len > fpt )
+ {
+ if ( fpt > px.idx )
+ {
+ uint32_t delta = fpt - px.idx;
+ if ( delta > len )
+ return false;
+
+ data += delta;
+ len -= delta;
+ }
+ px.idx = fpt;
+ return paf_callback(px, pkt, data, len, flags);
+ }
+ return false;
+
+ case StreamSplitter::LIMITED:
+ // increment position by previously scanned bytes. set in paf_flush
+ state = StreamSplitter::SEARCH;
+ px.idx += fpt;
+ fpt = 0;
+ return true;
+
+ default:
+ // StreamSplitter::ABORT || StreamSplitter::START
+ break;
+ }
+
+ px.ft = FT_SFP;
+ return false;
+}
+
+//--------------------------------------------------------------------
+
+int32_t ProtocolAwareFlusher::paf_check (Packet* pkt, const uint8_t* data, uint32_t len,
+ uint32_t total, uint32_t seq, uint32_t* flags)
+{
+ Profile profile(pafPerfStats); // cppcheck-suppress unreadVariable
+ PafAux px;
+
+ if ( !paf_initialized() )
+ {
+ seq_num = pos = seq;
+ fpt = tot = 0;
+ state = StreamSplitter::SEARCH;
+ }
+ else if ( SEQ_GT(seq, seq_num) )
+ {
+ // if seq jumped we have a gap. Flush any queued data, then abort
+ px.len = total - len;
+
+ if ( px.len )
+ {
+ fpt = 0;
+ px.ft = FT_MAX;
+ state = StreamSplitter::ABORT;
+ return paf_flush(px, flags);
+ }
+ *flags = 0;
+ state = StreamSplitter::ABORT;
+ return -1;
+ }
+ else if ( SEQ_LEQ(seq + len, seq_num) )
+ {
+ return -1;
+ }
+ else if ( SEQ_LT(seq, seq_num) )
+ {
+ uint32_t shift = seq_num - seq;
+ data += shift;
+ len -= shift;
+ }
+
+ seq_num += len;
+ px.idx = total - len;
+
+ // if 'total' is greater than the maximum paf_max AND 'total' is greater
+ // than paf_max bytes (i.e. after we have finished analyzing the
+ // current segment, total bytes analyzed will be greater than the
+ // configured paf_max == splitter->max(), we must ensure a flush
+ // occurs at the paf_max byte. So, we manually set the data's length and
+ // total queued bytes (px.len) to guarantee that at most paf_max bytes will
+ // be analyzed and flushed since the last flush point. It should also be
+ // noted that we perform the check here rather in in paf_flush() to
+ // avoid scanning the same data twice. The first scan would analyze the
+ // entire segment and the second scan would analyze this segments
+ // unflushed data.
+ if ( total >= MAX_PAF_MAX && total > splitter->max(pkt->flow) )
+ {
+ px.len = MAX_PAF_MAX;
+ len = len + px.len - total;
+ }
+ else
+ px.len = total;
+
+ do
+ {
+ px.ft = FT_NOP;
+ uint32_t idx = px.idx;
+
+ const bool cont = paf_eval(px, pkt, *flags, data, len);
+
+ if ( px.ft != FT_NOP )
+ {
+ int32_t fp = paf_flush(px, flags);
+ paf_jump(fp);
+ return fp;
+ }
+
+ if ( !cont )
+ break;
+
+ if ( px.idx > idx )
+ {
+ uint32_t shift = px.idx - idx;
+ if ( shift > len )
+ shift = len;
+ data += shift;
+ len -= shift;
+ }
+ }
+ while ( true );
+
+ if ( state == StreamSplitter::ABORT )
+ *flags = 0;
+
+ else if ( (state != StreamSplitter::FLUSH) && (px.len > splitter->max(pkt->flow)) )
+ {
+ px.ft = FT_MAX;
+ uint32_t fp = paf_flush(px, flags);
+ paf_jump(fp);
+ return fp;
+ }
+
+ return -1;
+}
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+//--------------------------------------------------------------------
+// protocol aware flushing stuff
+// pafng.h author davis mcpherson davmcphe@cisco.com
+//--------------------------------------------------------------------
+
+#ifndef PAFNG_H
+#define PAFNG_H
+
+#include "main/snort_types.h"
+#include "main/thread.h"
+#include "profiler/profiler_defs.h"
+#include "stream/stream_splitter.h"
+
+namespace snort
+{
+struct Packet;
+}
+
+enum FlushType
+{
+ FT_NOP, // no flush
+ FT_SFP, // abort paf
+ FT_PAF, // flush to paf pt when len >= paf
+ FT_LIMIT, // flush to paf pt, don't update flags
+ FT_MAX // flush len when len >= max
+};
+
+struct PafAux
+{
+ FlushType ft;
+ uint32_t len; // total bytes queued
+ uint32_t idx; // offset from start of queued bytes
+};
+
+class ProtocolAwareFlusher
+{
+public:
+ ProtocolAwareFlusher() { }
+ ~ProtocolAwareFlusher() { }
+
+ SO_PUBLIC void paf_setup(snort::StreamSplitter* ss)
+ {
+ splitter = ss;
+ state = snort::StreamSplitter::START;
+ }
+
+ void paf_reset ()
+ { state = snort::StreamSplitter::START; }
+
+ void paf_clear ()
+ { state = snort::StreamSplitter::ABORT; }
+
+ uint32_t paf_position ()
+ { return seq_num; }
+
+ SO_PUBLIC uint32_t paf_initialized ()
+ { return ( state != snort::StreamSplitter::START ); }
+
+ SO_PUBLIC void paf_initialize(uint32_t seq)
+ {
+ seq_num = pos = seq;
+ fpt = tot = 0;
+ state = snort::StreamSplitter::SEARCH;
+ }
+
+ uint32_t paf_active ()
+ { return ( state != snort::StreamSplitter::ABORT ); }
+
+ void paf_jump(uint32_t n)
+ {
+ pos += n;
+ seq_num = pos;
+ }
+
+ // called on each in order segment
+ SO_PUBLIC int32_t paf_check(snort::Packet* p, const uint8_t* data, uint32_t len,
+ uint32_t total, uint32_t seqnum, uint32_t* flags);
+
+ uint32_t seq_num = 0; // stream cursor
+ uint32_t pos = 0; // last flush position
+ uint32_t fpt = 0; // current flush point
+ uint32_t tot = 0; // total bytes flushed
+ snort::StreamSplitter::Status state = snort::StreamSplitter::START; // current scan state
+
+private:
+ uint32_t paf_flush (const PafAux& px, uint32_t* flags);
+ bool paf_callback (PafAux&, snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags);
+ bool paf_eval (PafAux&, snort::Packet*, uint32_t flags, const uint8_t* data, uint32_t len);
+
+ snort::StreamSplitter* splitter = nullptr;
+};
+
+#endif
+
#include "utils/util.h"
#include "tcp/tcp_session.h"
-#include "tcp/tcp_stream_session.h"
#include "tcp/tcp_stream_tracker.h"
using namespace snort;
const SfIp* srcIP, uint16_t srcPort,
const SfIp* dstIP, uint16_t dstPort,
uint16_t vlan, uint32_t mplsId, uint32_t addressSpaceId,
+#ifndef DISABLE_TENANT_ID
uint32_t tenant_id,
+#endif
+ bool significant_groups,
int16_t ingress_group, int16_t egress_group)
{
FlowKey key;
const SnortConfig* sc = SnortConfig::get_conf();
- key.init(sc, type, proto, srcIP, srcPort, dstIP, dstPort, vlan, mplsId,
- addressSpaceId, tenant_id, ingress_group, egress_group);
+ key.init(sc, type, proto, srcIP, srcPort, dstIP, dstPort, vlan, mplsId, addressSpaceId,
+#ifndef DISABLE_TENANT_ID
+ tenant_id,
+#endif
+ significant_groups, ingress_group, egress_group);
return get_flow(&key);
}
{
assert(flow and flow->session and flow->pkt_type == PktType::TCP);
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
tcp_session->start_proxy();
}
return flow->session->disable_reassembly(flow);
}
-char Stream::get_reassembly_direction(Flow* flow)
-{
- assert(flow && flow->session);
- return flow->session->get_reassembly_direction();
-}
-
bool Stream::is_stream_sequenced(Flow* flow, uint8_t dir)
{
assert(flow && flow->session);
{
assert(flow and flow->session and flow->pkt_type == PktType::TCP);
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
return tcp_session->get_mss(to_server);
}
{
assert(flow and flow->session and flow->pkt_type == PktType::TCP);
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
return tcp_session->get_tcp_options_len(to_server);
}
{
assert(flow and flow->session and flow->pkt_type == PktType::TCP);
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
return tcp_session->can_set_no_ack();
}
{
assert(flow and flow->session and flow->pkt_type == PktType::TCP);
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
return tcp_session->set_no_ack(on_off);
}
if ( flow->pkt_type == PktType::TCP )
{
if ( to_server )
- ((TcpStreamSession*)flow->session)->server.perform_partial_flush();
+ ((TcpSession*)flow->session)->server.perform_partial_flush();
else
- ((TcpStreamSession*)flow->session)->client.perform_partial_flush();
+ ((TcpSession*)flow->session)->client.perform_partial_flush();
}
}
if (!flow or !flow->session or !(flow->pkt_type == PktType::TCP))
return false;
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
if (tcp_session->held_packet_dir == SSN_DIR_NONE)
return false;
TEST_CASE("Stream API", "[stream_api][stream]")
{
// initialization code here
+ TcpNormalizerFactory::initialize();
Flow* flow = new Flow;
SECTION("set/get ignore direction")
}
delete flow;
+ TcpNormalizerFactory::term();
}
#endif
Flow*, Packet* p, uint32_t gid, uint32_t sid,
uint32_t eventId, uint32_t eventSecond);
+
static void disable_reassembly(Flow*);
- static char get_reassembly_direction(Flow*);
// Returns true if stream data for the flow is in sequence, otherwise return false.
static bool is_stream_sequenced(Flow*, uint8_t dir);
PktType type, IpProtocol proto,
const snort::SfIp* a1, uint16_t p1, const snort::SfIp* a2, uint16_t p2,
uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId,
- uint32_t tenant_id, int16_t ingress_group = DAQ_PKTHDR_UNKNOWN,
+#ifndef DISABLE_TENANT_ID
+ uint32_t tenant_id,
+#endif
+ bool significant_groups, int16_t ingress_group = DAQ_PKTHDR_UNKNOWN,
int16_t egress_group = DAQ_PKTHDR_UNKNOWN);
static Flow* get_flow(
held_packet_queue.h
ips_stream_reassemble.cc
ips_stream_size.cc
- segment_overlap_editor.cc
- segment_overlap_editor.h
stream_tcp.cc
stream_tcp.h
+ tcp_alerts.cc
+ tcp_alerts.h
tcp_defs.h
tcp_event_logger.cc
tcp_event_logger.h
tcp_normalizer.h
tcp_normalizers.cc
tcp_normalizers.h
+ tcp_overlap_resolver.cc
+ tcp_overlap_resolver.h
+ tcp_reassembler_ids.cc
+ tcp_reassembler_ids.h
+ tcp_reassembler_ips.cc
+ tcp_reassembler_ips.h
tcp_reassembler.cc
tcp_reassembler.h
- tcp_reassemblers.cc
- tcp_reassemblers.h
+ tcp_reassembly_segments.cc
+ tcp_reassembly_segments.h
tcp_segment_descriptor.cc
tcp_segment_descriptor.h
tcp_segment_node.cc
tcp_state_time_wait.h
tcp_stream_config.cc
tcp_stream_config.h
- tcp_stream_session.cc
- tcp_stream_session.h
tcp_stream_tracker.cc
tcp_stream_tracker.h
tcp_trace.cc
if (!pkt->flow || !pkt->ptrs.tcph)
return NO_MATCH;
+ Flow* flow = (Flow*)pkt->flow;
+
+ if ( !srod.enable ) /* Turn it off */
+ {
+ if ( srod.direction & SSN_DIR_FROM_SERVER )
+ Stream::set_splitter(flow, true);
+
+ if ( srod.direction & SSN_DIR_FROM_CLIENT )
+ Stream::set_splitter(flow, false);
+ }
+ else
+ {
+ // FIXIT-M PAF need to instantiate service splitter?
+ // FIXIT-M PAF need to check for ips / on-data
+ if ( srod.direction & SSN_DIR_FROM_SERVER )
+ Stream::set_splitter(flow, true, new AtomSplitter(true));
+
+ if ( srod.direction & SSN_DIR_FROM_CLIENT )
+ Stream::set_splitter(flow, false, new AtomSplitter(false));
+ }
+
+ if (srod.fastpath)
{
- Flow* lwssn = (Flow*)pkt->flow;
- TcpSession* tcpssn = (TcpSession*)lwssn->session;
-
- if ( !srod.enable ) /* Turn it off */
- {
- if ( srod.direction & SSN_DIR_FROM_SERVER )
- {
- tcpssn->server.set_flush_policy(STREAM_FLPOLICY_IGNORE);
- Stream::set_splitter(lwssn, true);
- }
-
- if ( srod.direction & SSN_DIR_FROM_CLIENT )
- {
- tcpssn->client.set_flush_policy(STREAM_FLPOLICY_IGNORE);
- Stream::set_splitter(lwssn, false);
- }
- }
- else
- {
- // FIXIT-M PAF need to instantiate service splitter?
- // FIXIT-M PAF need to check for ips / on-data
- if ( srod.direction & SSN_DIR_FROM_SERVER )
- {
- tcpssn->server.set_flush_policy(STREAM_FLPOLICY_ON_ACK);
- Stream::set_splitter(lwssn, true, new AtomSplitter(true));
- }
-
- if ( srod.direction & SSN_DIR_FROM_CLIENT )
- {
- tcpssn->client.set_flush_policy(STREAM_FLPOLICY_ON_ACK);
- Stream::set_splitter(lwssn, false, new AtomSplitter(false));
- }
- }
-
- if (srod.fastpath)
- {
- /* Turn off inspection */
- lwssn->ssn_state.ignore_direction |= srod.direction;
- DetectionEngine::disable_all(pkt);
-
- /* TBD: Set TF_FORCE_FLUSH ? */
- }
+ /* Turn off inspection */
+ flow->ssn_state.ignore_direction |= srod.direction;
+ DetectionEngine::disable_all(pkt);
+
+ /* TBD: Set TF_FORCE_FLUSH ? */
}
if (srod.alert)
ReassembleModule* reassembler = ( ReassembleModule* )ips_stream_reassemble->mod_ctor();
REQUIRE( reassembler != nullptr );
+ TcpNormalizerFactory::initialize();
+
Flow* flow = new Flow;
Packet* pkt = get_syn_packet(flow);
Cursor cursor(pkt);
}
#endif
release_packet(pkt);
+ TcpNormalizerFactory::term();
delete flow;
ips_stream_reassemble->mod_dtor(reassembler);
}
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// segment_overlap_editor.cc author davis mcpherson <davmcphe@cisco.com>
-// Created on: Oct 11, 2015
-
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
-#include "segment_overlap_editor.h"
-
-#include "detection/detection_engine.h"
-#include "log/messages.h"
-
-#include "tcp_module.h"
-#include "tcp_normalizers.h"
-#include "tcp_session.h"
-
-using namespace snort;
-
-void SegmentOverlapState::init_sos(TcpSession* ssn, StreamPolicy pol)
-{
- session = ssn;
- reassembly_policy = pol;
-
- seglist.reset();
-
- seglist_base_seq = 0;
- seg_count = 0;
- seg_bytes_total = 0;
- seg_bytes_logical = 0;
- total_bytes_queued = 0;
- total_segs_queued = 0;
- overlap_count = 0;
-
- tsd = nullptr;
- left = nullptr;
- right = nullptr;
- rdata = nullptr;
-
- seq = 0;
- seq_end = 0;
- len = 0;
- overlap = 0;
- slide = 0;
- trunc_len = 0;
- rsize = 0;
- rseq = 0;
- keep_segment = true;
-
- tcp_ips_data = Normalize_GetMode(NORM_TCP_IPS);
-}
-
-void SegmentOverlapState::init_soe(
- TcpSegmentDescriptor& tsd, TcpSegmentNode* left, TcpSegmentNode* right)
-{
- this->tsd = &tsd;
- this->left = left;
- this->right = right;
-
- seq = tsd.get_seq();
- seq_end = tsd.get_end_seq();
- len = tsd.get_len();
-
- overlap = 0;
- slide = 0;
- trunc_len = 0;
-
- rdata = tsd.get_pkt()->data;
- rsize = tsd.get_len();
- rseq = tsd.get_seq();
-
- keep_segment = true;
-}
-
-bool SegmentOverlapEditor::is_segment_retransmit(
- TcpReassemblerState& trs, bool* full_retransmit)
-{
- // Don't want to count retransmits as overlaps or do anything
- // else with them. Account for retransmits of multiple PDUs
- // in one segment.
- bool* pb = (trs.sos.rseq == trs.sos.tsd->get_seq()) ? full_retransmit : nullptr;
-
- if ( trs.sos.right->is_retransmit(trs.sos.rdata, trs.sos.rsize,
- trs.sos.rseq, trs.sos.right->i_len, pb) )
- {
- trs.sos.tsd->set_retransmit_flag();
-
- if ( !(*full_retransmit) )
- {
- trs.sos.rdata += trs.sos.right->i_len;
- trs.sos.rsize -= trs.sos.right->i_len;
- trs.sos.rseq += trs.sos.right->i_len;
- trs.sos.seq += trs.sos.right->i_len;
- trs.sos.left = trs.sos.right;
- trs.sos.right = trs.sos.right->next;
- }
- else
- trs.sos.rsize = 0;
-
- if ( trs.sos.rsize == 0 )
- {
- // All data was retransmitted
- snort::DetectionEngine::disable_content(trs.sos.tsd->get_pkt());
- trs.sos.keep_segment = false;
- }
-
- return true;
- }
-
- return false;
-}
-
-void SegmentOverlapEditor::eval_left(TcpReassemblerState& trs)
-{
- if ( trs.sos.left )
- insert_left_overlap(trs);
-}
-
-void SegmentOverlapEditor::eval_right(TcpReassemblerState& trs)
-{
- while ( trs.sos.right && SEQ_LT(trs.sos.right->i_seq, trs.sos.seq_end) )
- {
- trs.sos.trunc_len = 0;
-
- assert(SEQ_LEQ(trs.sos.seq, trs.sos.right->i_seq));
- trs.sos.overlap = ( int )( trs.sos.seq_end - trs.sos.right->i_seq );
-
- // Treat sequence number overlap as a retransmission,
- // only check right side since left side happens rarely
- trs.sos.session->flow->call_handlers(trs.sos.tsd->get_pkt(), false);
- if ( trs.sos.overlap < trs.sos.right->i_len )
- {
- if ( trs.sos.right->is_retransmit(trs.sos.rdata, trs.sos.rsize,
- trs.sos.rseq, trs.sos.right->i_len, nullptr) )
- {
- // All data was retransmitted
- trs.sos.tsd->set_retransmit_flag();
- snort::DetectionEngine::disable_content(trs.sos.tsd->get_pkt());
- trs.sos.keep_segment = false;
- tcpStats.full_retransmits++;
- }
- else
- {
- tcpStats.overlaps++;
- trs.sos.overlap_count++;
- insert_right_overlap(trs);
- }
-
- break;
- }
- else // Full overlap
- {
- bool full_retransmit = false;
- // Don't want to count retransmits as overlaps or do anything
- // else with them. Account for retransmits of multiple PDUs
- // in one segment.
- if ( is_segment_retransmit(trs, &full_retransmit) )
- {
- if ( full_retransmit )
- {
- tcpStats.full_retransmits++;
- break;
- }
-
- continue;
- }
-
- tcpStats.overlaps++;
- trs.sos.overlap_count++;
- insert_full_overlap(trs);
-
- if ( trs.sos.keep_segment == false )
- return;
- }
- }
-}
-
-void SegmentOverlapEditor::drop_old_segment(TcpReassemblerState& trs)
-{
- TcpSegmentNode* drop_seg = trs.sos.right;
- trs.sos.right = trs.sos.right->next;
- delete_reassembly_segment(trs, drop_seg);
-}
-
-void SegmentOverlapEditor::left_overlap_keep_first(TcpReassemblerState& trs)
-{
- // NOTE that overlap will always be less than left->size since
- // seq is always greater than left->seq
- assert(SEQ_GT(trs.sos.seq, trs.sos.left->i_seq));
-
- trs.sos.overlap = trs.sos.left->i_seq + trs.sos.left->i_len - trs.sos.seq;
-
- if ( trs.sos.len < trs.sos.overlap )
- trs.sos.overlap = trs.sos.len;
-
- if ( trs.sos.overlap > 0 )
- {
- tcpStats.overlaps++;
- trs.sos.overlap_count++;
-
- if ( SEQ_GT(trs.sos.left->i_seq + trs.sos.left->i_len, trs.sos.seq_end) )
- {
- if (trs.sos.tcp_ips_data == NORM_MODE_ON)
- {
- unsigned offset = trs.sos.tsd->get_seq() - (trs.sos.left->i_seq - trs.sos.left->o_offset);
- trs.sos.tsd->rewrite_payload(0, trs.sos.left->data + offset);
- }
- norm_stats[PC_TCP_IPS_DATA][trs.sos.tcp_ips_data]++;
- }
- else
- {
- if ( trs.sos.tcp_ips_data == NORM_MODE_ON )
- {
- unsigned offset = trs.sos.tsd->get_seq() - (trs.sos.left->i_seq - trs.sos.left->o_offset);
- unsigned length = trs.sos.left->i_seq + trs.sos.left->i_len - trs.sos.tsd->get_seq();
- trs.sos.tsd->rewrite_payload(0, trs.sos.left->data + offset, length);
- }
-
- norm_stats[PC_TCP_IPS_DATA][trs.sos.tcp_ips_data]++;
- }
-
- trs.sos.seq += trs.sos.overlap;
- }
-}
-
-void SegmentOverlapEditor::left_overlap_trim_first(TcpReassemblerState& trs)
-{
- assert(SEQ_GT(trs.sos.seq, trs.sos.left->i_seq));
-
- trs.sos.overlap = trs.sos.left->i_seq + trs.sos.left->i_len - trs.sos.seq;
-
- if ( trs.sos.overlap > 0 )
- {
- tcpStats.overlaps++;
- trs.sos.overlap_count++;
-
- if ( SEQ_GEQ(trs.sos.left->i_seq + trs.sos.left->i_len, trs.sos.seq + trs.sos.len) )
- {
- // existing packet overlaps new on both sides. Drop the new data.
- trs.sos.seq += trs.sos.len;
- }
- else
- {
- /* Otherwise, trim the old data accordingly */
- trs.sos.left->c_len -= ( int16_t )trs.sos.overlap;
- trs.sos.left->i_len -= ( int16_t )trs.sos.overlap;
- trs.sos.seg_bytes_logical -= trs.sos.overlap;
- }
- }
-}
-
-void SegmentOverlapEditor::left_overlap_keep_last(TcpReassemblerState& trs)
-{
- assert(SEQ_GT(trs.sos.seq, trs.sos.left->i_seq));
-
- trs.sos.overlap = trs.sos.left->i_seq + trs.sos.left->i_len - trs.sos.seq;
-
- if ( trs.sos.overlap > 0 )
- {
- tcpStats.overlaps++;
- trs.sos.overlap_count++;
-
- /* True "Last" policy" */
- if ( SEQ_GT(trs.sos.left->i_seq + trs.sos.left->i_len, trs.sos.seq + trs.sos.len) )
- {
- /* New data is overlapped on both sides by existing data. Existing data needs to be
- * split and the new data inserted in the middle.
- * Need to duplicate left. Adjust that seq by + (seq + len) and
- * size by - (seq + len - left->i_seq).
- */
- dup_reassembly_segment(trs, trs.sos.left, &trs.sos.right);
-
- trs.sos.left->c_len -= (int16_t)trs.sos.overlap;
- trs.sos.left->i_len -= (int16_t)trs.sos.overlap;
-
- trs.sos.right->i_seq = trs.sos.seq + trs.sos.len;
- trs.sos.right->c_seq = trs.sos.right->i_seq;
- uint16_t delta = (int16_t)(trs.sos.right->i_seq - trs.sos.left->i_seq);
- trs.sos.right->c_len -= delta;
- trs.sos.right->i_len -= delta;
- trs.sos.right->offset += delta;
- trs.sos.right->o_offset += delta;
-
- trs.sos.seg_bytes_logical -= trs.sos.len;
- }
- else
- {
- trs.sos.left->c_len -= (int16_t)trs.sos.overlap;
- trs.sos.left->i_len -= (int16_t)trs.sos.overlap;
- trs.sos.seg_bytes_logical -= trs.sos.overlap;
- }
- }
-}
-
-void SegmentOverlapEditor::right_overlap_truncate_existing(TcpReassemblerState& trs)
-{
- if ( SEQ_EQ(trs.sos.right->i_seq, trs.sos.seq) &&
- ( trs.sos.reassembly_policy != StreamPolicy::OS_LAST ) )
- {
- trs.sos.seq += trs.sos.overlap;
- }
- else
- {
- /* partial overlap */
- trs.sos.right->i_seq += trs.sos.overlap;
- trs.sos.right->c_seq = trs.sos.right->i_seq;
- trs.sos.right->offset += trs.sos.overlap;
- trs.sos.right->o_offset += trs.sos.overlap;
- trs.sos.right->c_len -= (int16_t)trs.sos.overlap;
- trs.sos.right->i_len -= ( int16_t )trs.sos.overlap;
- trs.sos.seg_bytes_logical -= trs.sos.overlap;
- trs.sos.total_bytes_queued -= trs.sos.overlap;
- }
-}
-
-void SegmentOverlapEditor::right_overlap_truncate_new(TcpReassemblerState& trs)
-{
- if (trs.sos.tcp_ips_data == NORM_MODE_ON)
- {
- unsigned offset = trs.sos.right->i_seq - trs.sos.tsd->get_seq();
- unsigned length = trs.sos.tsd->get_seq() + trs.sos.tsd->get_len() - trs.sos.right->i_seq;
- trs.sos.tsd->rewrite_payload(offset, trs.sos.right->data + trs.sos.right->o_offset, length);
- }
-
- norm_stats[PC_TCP_IPS_DATA][trs.sos.tcp_ips_data]++;
- trs.sos.trunc_len = trs.sos.overlap;
-}
-
-// REASSEMBLY_POLICY_FIRST:
-// REASSEMBLY_POLICY_VISTA:
-void SegmentOverlapEditor::full_right_overlap_truncate_new(TcpReassemblerState& trs)
-{
-
- if ( trs.sos.tcp_ips_data == NORM_MODE_ON )
- {
- unsigned offset = trs.sos.right->i_seq - trs.sos.tsd->get_seq();
-
- if ( !offset && zwp_data_mismatch(trs, *trs.sos.tsd, trs.sos.right->i_len))
- {
- trs.tracker->normalizer.session_blocker(*trs.sos.tsd);
- trs.sos.keep_segment = false;
- return;
- }
-
- trs.sos.tsd->rewrite_payload(offset, trs.sos.right->data + trs.sos.right->o_offset, trs.sos.right->i_len);
- }
-
- norm_stats[PC_TCP_IPS_DATA][trs.sos.tcp_ips_data]++;
-
- if ( SEQ_EQ(trs.sos.right->i_seq, trs.sos.seq) )
- {
- /* Overlap is greater than or equal to right->size
- * slide gets set before insertion */
- trs.sos.seq += trs.sos.right->i_len;
- trs.sos.left = trs.sos.right;
- trs.sos.right = trs.sos.right->next;
- }
- else
- {
- // seq is less than right->i_seq, set trunc length and slide
- // and insert chunk before current right segment...
- trs.sos.trunc_len = trs.sos.overlap;
- trs.sos.slide = trs.sos.seq - trs.sos.tsd->get_seq();
- add_reassembly_segment(trs, *trs.sos.tsd, trs.sos.len, trs.sos.slide,
- trs.sos.trunc_len, trs.sos.seq, trs.sos.left);
-
- // Set seq to end of right since overlap was greater than or equal to right->size and
- // inserted seq has been truncated to beginning of right and reset trunc length to 0
- // since we may fall out of loop if next right is null
- trs.sos.seq = trs.sos.right->i_seq + trs.sos.right->i_len;
- trs.sos.left = trs.sos.right;
- trs.sos.right = trs.sos.right->next;
- trs.sos.trunc_len = 0;
- }
-}
-
-// REASSEMBLY_POLICY_WINDOWS:
-// REASSEMBLY_POLICY_WINDOWS2K3:
-// REASSEMBLY_POLICY_BSD:
-// REASSEMBLY_POLICY_MACOS:
-void SegmentOverlapEditor::full_right_overlap_os1(TcpReassemblerState& trs)
-{
- if ( SEQ_GEQ(trs.sos.seq_end, trs.sos.right->i_seq + trs.sos.right->i_len) and
- SEQ_LT(trs.sos.seq, trs.sos.right->i_seq) )
- {
- drop_old_segment(trs);
- }
- else
- full_right_overlap_truncate_new(trs);
-}
-
-// REASSEMBLY_POLICY_LINUX:
-// REASSEMBLY_POLICY_HPUX10:
-// REASSEMBLY_POLICY_IRIX:
-void SegmentOverlapEditor::full_right_overlap_os2(TcpReassemblerState& trs)
-{
- if ( SEQ_GEQ(trs.sos.seq_end, trs.sos.right->i_seq + trs.sos.right->i_len) and
- SEQ_LT(trs.sos.seq, trs.sos.right->i_seq) )
- {
- drop_old_segment(trs);
- }
- else if ( SEQ_GT(trs.sos.seq_end, trs.sos.right->i_seq + trs.sos.right->i_len) and
- SEQ_EQ(trs.sos.seq, trs.sos.right->i_seq) )
- {
- drop_old_segment(trs);
- }
- else
- full_right_overlap_truncate_new(trs);
-}
-
-// REASSEMBLY_POLICY_HPUX11:
-// REASSEMBLY_POLICY_SOLARIS:
-void SegmentOverlapEditor::full_right_overlap_os3(TcpReassemblerState& trs)
-{
- // If this packet is wholly overlapping and the same size as a previous one and we have not
- // received the one immediately preceding, we take the FIRST.
- if ( SEQ_EQ(trs.sos.right->i_seq, trs.sos.seq) && (trs.sos.right->i_len == trs.sos.len)
- && (trs.sos.left && !SEQ_EQ(trs.sos.left->i_seq + trs.sos.left->i_len, trs.sos.seq)) )
- {
- trs.sos.trunc_len = trs.sos.right->i_len;
- trs.sos.rdata += trs.sos.right->i_len;
- trs.sos.rsize -= trs.sos.right->i_len;
- trs.sos.rseq += trs.sos.right->i_len;
- trs.sos.seq += trs.sos.right->i_len;
- trs.sos.left = trs.sos.right;
- trs.sos.right = trs.sos.right->next;
- }
- else
- drop_old_segment(trs);
-}
-
-// REASSEMBLY_POLICY_OLD_LINUX:
-// REASSEMBLY_POLICY_LAST:
-void SegmentOverlapEditor::full_right_overlap_os4(TcpReassemblerState& trs)
-{ drop_old_segment(trs); }
-
-void SegmentOverlapEditor::full_right_overlap_os5(TcpReassemblerState& trs)
-{
- full_right_overlap_truncate_new(trs);
-}
-
-bool SegmentOverlapEditor::zwp_data_mismatch(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd, uint32_t overlap)
-{
- if ( overlap == MAX_ZERO_WIN_PROBE_LEN
- and trs.sos.right->i_seq == trs.tracker->normalizer.get_zwp_seq()
- and (trs.sos.right->data[0] != tsd.get_pkt()->data[0]) )
- {
- return tsd.is_nap_policy_inline();
- }
-
- return false;
-}
-
-void SegmentOverlapEditor::print(TcpReassemblerState& trs)
-{
- LogMessage(" seglist_base_seq: %X\n", trs.sos.seglist_base_seq);
- LogMessage(" seglist head: %p\n", (void*)trs.sos.seglist.head);
- LogMessage(" seglist tail: %p\n", (void*)trs.sos.seglist.tail);
- LogMessage(" seglist current: %p\n", (void*)trs.sos.seglist.cur_rseg);
- LogMessage(" seg_count: %d\n", trs.sos.seg_count);
- LogMessage(" seg_bytes_total: %d\n", trs.sos.seg_bytes_total);
- LogMessage(" seg_bytes_logical: %d\n", trs.sos.seg_bytes_logical);
-}
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// segment_overlap_editor.h author davis mcpherson <davmcphe@cisco.com>
-// Created on: Oct 11, 2015
-
-#ifndef SEGMENT_OVERLAP_EDITOR_H
-#define SEGMENT_OVERLAP_EDITOR_H
-
-#include <vector>
-
-#include "normalize/norm_stats.h"
-#include "stream/paf.h"
-#include "stream/stream.h"
-#include "tcp_segment_node.h"
-
-class TcpSession;
-class TcpStreamTracker;
-
-struct SegmentOverlapState
-{
- TcpSession* session;
- TcpSegmentDescriptor* tsd;
- TcpSegmentNode* left;
- TcpSegmentNode* right;
- const uint8_t* rdata;
-
- TcpSegmentList seglist;
- uint32_t seglist_base_seq; /* seq of first queued segment */
- uint32_t seg_count; /* number of current queued segments */
- uint32_t seg_bytes_total; /* total bytes currently queued */
- uint32_t seg_bytes_logical; /* logical bytes queued (total - overlaps) */
- uint32_t total_bytes_queued; /* total bytes queued (life of session) */
- uint32_t total_segs_queued; /* number of segments queued (life) */
- uint32_t overlap_count; /* overlaps encountered */
-
- uint32_t seq;
- uint32_t seq_end;
- uint32_t rseq;
-
- int32_t overlap;
- int32_t slide;
- int32_t trunc_len;
-
- uint16_t len;
- uint16_t rsize;
- int8_t tcp_ips_data;
- StreamPolicy reassembly_policy;
-
- bool keep_segment;
-
- ~SegmentOverlapState()
- { seglist.reset(); }
-
- void init_sos(TcpSession*, StreamPolicy);
- void init_soe(TcpSegmentDescriptor& tsd, TcpSegmentNode* left, TcpSegmentNode* right);
-};
-
-struct StreamAlertInfo : snort::AlertInfo
-{
- StreamAlertInfo(uint32_t gid_, uint32_t sid_, uint32_t seq_num_ = 0, uint32_t id_ = 0, uint32_t ts_ = 0)
- : snort::AlertInfo(gid_, sid_, id_, ts_), seq(seq_num_)
- {}
-
- uint32_t seq;
-};
-
-struct TcpReassemblerState
-{
- SegmentOverlapState sos;
- TcpStreamTracker* tracker;
- uint32_t flush_count; // number of flushed queued segments
- uint32_t xtradata_mask; // extra data available to log
- std::vector<StreamAlertInfo> alerts;
- uint8_t ignore_dir;
- uint8_t packet_dir;
- bool server_side;
- PAF_State paf_state;
-};
-
-class SegmentOverlapEditor
-{
-protected:
- SegmentOverlapEditor() = default;
- virtual ~SegmentOverlapEditor() = default;
-
- void eval_left(TcpReassemblerState&);
- void eval_right(TcpReassemblerState&);
-
- virtual bool is_segment_retransmit(TcpReassemblerState&, bool*);
- virtual void drop_old_segment(TcpReassemblerState&);
- virtual bool zwp_data_mismatch(TcpReassemblerState&, TcpSegmentDescriptor&, uint32_t);
-
- virtual void left_overlap_keep_first(TcpReassemblerState&);
- virtual void left_overlap_trim_first(TcpReassemblerState&);
- virtual void left_overlap_keep_last(TcpReassemblerState&);
- virtual void right_overlap_truncate_existing(TcpReassemblerState&);
- virtual void right_overlap_truncate_new(TcpReassemblerState&);
- virtual void full_right_overlap_truncate_new(TcpReassemblerState&);
- virtual void full_right_overlap_os1(TcpReassemblerState&);
- virtual void full_right_overlap_os2(TcpReassemblerState&);
- virtual void full_right_overlap_os3(TcpReassemblerState&);
- virtual void full_right_overlap_os4(TcpReassemblerState&);
- virtual void full_right_overlap_os5(TcpReassemblerState&);
-
- virtual void insert_left_overlap(TcpReassemblerState&) = 0;
- virtual void insert_right_overlap(TcpReassemblerState&) = 0;
- virtual void insert_full_overlap(TcpReassemblerState&) = 0;
-
- virtual void add_reassembly_segment(
- TcpReassemblerState&, TcpSegmentDescriptor&, uint16_t, uint32_t,
- uint32_t, uint32_t, TcpSegmentNode*) = 0;
-
- virtual void dup_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*, TcpSegmentNode**) = 0;
- virtual int delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*) = 0;
- virtual void print(TcpReassemblerState&);
-};
-
-#endif
#include "tcp_ha.h"
#include "tcp_module.h"
+#include "tcp_overlap_resolver.h"
#include "tcp_session.h"
-#include "tcp_reassemblers.h"
#include "tcp_state_machine.h"
using namespace snort;
static void stream_tcp_pinit()
{
TcpStateMachine::initialize();
- TcpReassemblerFactory::initialize();
+ TcpOverlapResolverFactory::initialize();
TcpNormalizerFactory::initialize();
}
static void stream_tcp_pterm()
{
TcpStateMachine::term();
- TcpReassemblerFactory::term();
+ TcpOverlapResolverFactory::term();
TcpNormalizerFactory::term();
}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_alerts.cc author davis mcpherson <davmcphe@cisco.com>
+// Created on: Nov 7, 2023
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include <cassert>
+
+#include "tcp_alerts.h"
+
+#include "detection/context_switcher.h"
+
+#include "tcp_session.h"
+
+using namespace snort;
+
+static void purge_alerts_callback_ackd(IpsContext *c)
+{
+ TcpSession *session = (TcpSession*) c->packet->flow->session;
+
+ if (c->packet->is_from_server())
+ session->client.tcp_alerts.purge_alerts(c->packet->flow);
+ else
+ session->server.tcp_alerts.purge_alerts(c->packet->flow);
+}
+
+static void purge_alerts_callback_ips(IpsContext *c)
+{
+ TcpSession *session = (TcpSession*) c->packet->flow->session;
+
+ if (c->packet->is_from_server())
+ session->server.tcp_alerts.purge_alerts(c->packet->flow);
+ else
+ session->client.tcp_alerts.purge_alerts(c->packet->flow);
+}
+
+bool TcpAlerts::add_alert(uint32_t gid, uint32_t sid)
+{
+ assert(
+ alerts.size()
+ <= (uint32_t )(get_ips_policy()->rules_loaded + get_ips_policy()->rules_shared));
+
+ if (!this->check_alerted(gid, sid))
+ alerts.emplace_back(gid, sid);
+
+ return true;
+}
+
+bool TcpAlerts::check_alerted(uint32_t gid, uint32_t sid)
+{
+ return std::any_of(alerts.cbegin(), alerts.cend(), [gid, sid](const StreamAlertInfo &alert)
+ { return alert.gid == gid && alert.sid == sid;});
+}
+
+int TcpAlerts::update_alert(uint32_t gid, uint32_t sid, uint32_t event_id, uint32_t event_second)
+{
+ // FIXIT-M comparison of seq_num is wrong, compare value is always 0, should be seq_num of wire packet
+ uint32_t seq_num = 0;
+
+ auto it = std::find_if(alerts.begin(), alerts.end(),
+ [gid, sid, seq_num](const StreamAlertInfo &alert)
+ { return alert.gid == gid && alert.sid == sid && SEQ_EQ(alert.seq, seq_num);});
+ if (it != alerts.end())
+ {
+ (*it).event_id = event_id;
+ (*it).event_second = event_second;
+ return 0;
+ }
+
+ return -1;
+}
+
+void TcpAlerts::purge_alerts(Flow* flow)
+{
+ for (auto &alert : alerts)
+ Stream::log_extra_data(flow, xtradata_mask, alert);
+
+ if (!flow->is_suspended())
+ alerts.clear();
+}
+
+void TcpAlerts::purge_alerts(Packet& last_pdu, bool ips_enabled)
+{
+ if ( ips_enabled )
+ last_pdu.context->register_post_callback(purge_alerts_callback_ips);
+ else
+ last_pdu.context->register_post_callback(purge_alerts_callback_ackd);
+}
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2014-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_alerts.h author davis mcpherson <davmcphe@cisco.com>
+// Created on: Jul 31, 2015
+
+#ifndef TCP_ALERTS_H
+#define TCP_ALERTS_H
+
+#include <cstdint>
+
+#include "protocols/packet.h"
+#include "stream/stream.h"
+
+struct StreamAlertInfo: snort::AlertInfo
+{
+ StreamAlertInfo(uint32_t gid_, uint32_t sid_, uint32_t seq_num_ = 0, uint32_t id_ = 0,
+ uint32_t ts_ = 0) : snort::AlertInfo(gid_, sid_, id_, ts_), seq(seq_num_)
+ { }
+
+ uint32_t seq;
+};
+
+class TcpAlerts
+{
+public:
+ TcpAlerts() = default;
+
+ void clear()
+ {
+ xtradata_mask = 0;
+ alerts.clear();
+ }
+
+ bool add_alert(uint32_t gid, uint32_t sid);
+ bool check_alerted(uint32_t gid, uint32_t sid);
+ int update_alert(uint32_t gid, uint32_t sid, uint32_t event_id, uint32_t event_second);
+ void purge_alerts(snort::Flow* flow);
+ void purge_alerts(snort::Packet& last_pdu, bool ips_enabled);
+
+ void set_xtradata_mask(uint32_t mask)
+ {
+ xtradata_mask = mask;
+ }
+
+ uint32_t get_xtradata_mask() const
+ {
+ return xtradata_mask;
+ }
+
+private:
+
+ uint32_t xtradata_mask = 0; // extra data available to log
+ std::vector<StreamAlertInfo> alerts;
+
+};
+
+#endif
#define PAWS_WINDOW 60
#define PAWS_24DAYS 2073600 /* 24 days in seconds */
-#define STREAM_UNALIGNED 0
-#define STREAM_ALIGNED 1
-
#define STREAM_DEFAULT_MAX_SMALL_SEG_SIZE 0 /* disabled */
#define STREAM_DEFAULT_CONSEC_SMALL_SEGS 0 /* disabled */
#define SLAM_MAX 4
#define MAX_ZERO_WIN_PROBE_LEN 1
+#define MAX_KEEP_ALIVE_PROBE_LEN 1
// target-based policy types - changes to this enum require changes to stream.h::TCP_POLICIES
enum StreamPolicy : uint8_t
#include "config.h"
#endif
-#include "tcp_module.h"
-#include "tcp_normalizer.h"
-
#include "main/snort_config.h"
#include "profiler/profiler_defs.h"
#include "stream/paf.h"
#include "stream/paf_stats.h"
#include "trace/trace.h"
+#include "trace/trace_api.h"
+#include "tcp_module.h"
+#include "tcp_normalizer.h"
#include "tcp_trace.h"
using namespace snort;
THREAD_LOCAL ProfileStats s5TcpPerfStats;
THREAD_LOCAL const Trace* stream_tcp_trace = nullptr;
+THREAD_LOCAL bool stream_tcp_trace_enabled = false;
-#ifdef DEBUG_MSGS
static const TraceOption stream_tcp_trace_options[] =
{
{ "segments", TRACE_SEGMENTS, "enable stream TCP segments trace logging" },
{ nullptr, 0, nullptr }
};
-#endif
const PegInfo tcp_pegs[] =
{
{ CountType::MAX, "max_bytes", "maximum number of bytes queued in any flow" },
{ CountType::SUM, "zero_len_tcp_opt", "number of zero length tcp options" },
{ CountType::SUM, "zero_win_probes", "number of tcp zero window probes" },
+ { CountType::SUM, "keep_alive_probes", "number of tcp keep-alive probes" },
{ CountType::SUM, "proxy_mode_flows", "number of flows set to proxy normalization policy" },
{ CountType::SUM, "full_retransmits", "number of fully retransmitted segments" },
{ CountType::SUM, "flush_on_asymmetric_flow", "number of flushes on asymmetric flows" },
}
void StreamTcpModule::set_trace(const Trace* trace) const
-{ stream_tcp_trace = trace; }
+{
+ stream_tcp_trace = trace;
+ stream_tcp_trace_enabled = trace_enabled(stream_tcp_trace, TRACE_SEGMENTS) ||
+ trace_enabled(stream_tcp_trace, TRACE_STATE);
+}
const TraceOption* StreamTcpModule::get_trace_options() const
{
-#ifndef DEBUG_MSGS
- return nullptr;
-#else
return stream_tcp_trace_options;
-#endif
}
const RuleMap* StreamTcpModule::get_rules() const
PegCount max_bytes;
PegCount zero_len_tcp_opt;
PegCount zero_win_probes;
+ PegCount keep_alive_probes;
PegCount proxy_mode_flows;
PegCount full_retransmits;
PegCount flush_on_asymmetric_flow;
};
extern THREAD_LOCAL struct TcpStats tcpStats;
+extern THREAD_LOCAL bool stream_tcp_trace_enabled;
//-------------------------------------------------------------------------
// stream_tcp module
#include "tcp_normalizer.h"
-#include "stream/stream.h"
+#include "detection/detection_engine.h"
#include "packet_io/packet_tracer.h"
+#include "stream/stream.h"
+#include "trace/trace.h"
+#include "trace/trace_api.h"
-#include "tcp_module.h"
-#include "tcp_stream_session.h"
+#include "tcp_session.h"
#include "tcp_stream_tracker.h"
using namespace snort;
// drop packet if sequence num is invalid
if ( !tns.tracker->is_segment_seq_valid(tsd) )
{
+ if ( is_keep_alive_probe(tns, tsd) )
+ return NORM_BAD_SEQ;
+
bool inline_mode = tsd.is_nap_policy_inline();
tcpStats.invalid_seq_num++;
log_drop_reason(tns, tsd, inline_mode, "stream", "Normalizer: Sequence number is invalid\n");
tcpStats.zero_win_probes++;
set_zwp_seq(tns, seq);
log_drop_reason(tns, tsd, inline_mode, "stream",
- "Normalizer: Maximum Zero Window Probe length supported at a time is 1 byte\n");
+ "Normalizer: Maximum Zero Window Probe length supported at a time is 1 byte\n");
trim_win_payload(tns, tsd, MAX_ZERO_WIN_PROBE_LEN, inline_mode);
}
}
tsd.get_pkt()->active->set_drop_reason(issuer);
if (PacketTracer::is_active())
PacketTracer::log("%s", log.c_str());
+ if (stream_tcp_trace_enabled)
+ trace_logf(TRACE_WARNING_LEVEL, stream_tcp_trace, DEFAULT_TRACE_OPTION_ID, tsd.get_pkt(), "%s", log.c_str());
}
}
+bool TcpNormalizer::is_keep_alive_probe(TcpNormalizerState& tns, const TcpSegmentDescriptor& tsd)
+{
+ if ( (tns.tracker->r_win_base - tsd.get_seq()) == MAX_KEEP_ALIVE_PROBE_LEN
+ and tsd.get_len() <= MAX_KEEP_ALIVE_PROBE_LEN and
+ !(tsd.get_tcph()->th_flags & (TH_SYN|TH_FIN|TH_RST)) )
+ {
+ tcpStats.keep_alive_probes++;
+ return true;
+ }
+
+ return false;
+}
+
uint16_t TcpNormalizer::set_urg_offset(
TcpNormalizerState&, const tcp::TCPHdr* tcph, uint16_t dsize)
{
#include "normalize/norm_stats.h"
#include "protocols/tcp_options.h"
-class TcpStreamSession;
+class TcpSession;
class TcpStreamTracker;
class TcpSegmentDescriptor;
+class TcpNormalizer;
struct TcpNormalizerState
{
- TcpStreamSession* session = nullptr;
+ TcpSession* session = nullptr;
TcpStreamTracker* tracker = nullptr;
TcpStreamTracker* peer_tracker = nullptr;
+ TcpNormalizer* prev_norm = nullptr;
StreamPolicy os_policy = StreamPolicy::OS_DEFAULT;
virtual ~TcpNormalizer() = default;
virtual void init(State&) { }
- virtual void init(TcpNormalizer*) { }
virtual NormStatus apply_normalizations(
State&, TcpSegmentDescriptor&, uint32_t seq, bool stream_is_inorder);
virtual int handle_repeated_syn(State&, TcpSegmentDescriptor&) = 0;
virtual uint16_t set_urg_offset(State&, const snort::tcp::TCPHdr* tcph, uint16_t dsize);
virtual void set_zwp_seq(State&, uint32_t seq);
- virtual void log_drop_reason(State&, const TcpSegmentDescriptor&, bool inline_mode, const char *issuer, const std::string& log);
+ virtual void log_drop_reason(State&, const TcpSegmentDescriptor&, bool inline_mode,
+ const char *issuer, const std::string& log);
+ virtual bool is_keep_alive_probe(State&, const TcpSegmentDescriptor&);
static void reset_stats();
virtual int handle_paws_no_timestamps(State&, TcpSegmentDescriptor&);
std::string my_name;
- TcpNormalizer* prev_norm = nullptr;
};
#endif
#include "tcp_module.h"
#include "tcp_segment_descriptor.h"
-#include "tcp_stream_session.h"
+#include "tcp_session.h"
#include "tcp_stream_tracker.h"
using namespace snort;
TcpNormalizerMissed3whs()
{ my_name = "Missed3whs"; }
- void init(TcpNormalizer* prev) override
- { prev_norm = prev; }
-
TcpNormalizer::NormStatus apply_normalizations(
TcpNormalizerState&, TcpSegmentDescriptor&, uint32_t seq, bool stream_is_inorder) override;
bool validate_rst(TcpNormalizerState&, TcpSegmentDescriptor&) override;
static inline int handle_repeated_syn_mswin(
TcpStreamTracker* talker, TcpStreamTracker* listener,
- const TcpSegmentDescriptor& tsd, TcpStreamSession* session)
+ const TcpSegmentDescriptor& tsd, TcpSession* session)
{
/* Windows has some strange behavior here. If the sequence of the reset is the
* next expected sequence, it Resets. Otherwise it ignores the 2nd SYN.
}
static inline int handle_repeated_syn_bsd(
- TcpStreamTracker* talker, const TcpSegmentDescriptor& tsd, TcpStreamSession* session)
+ TcpStreamTracker* talker, const TcpSegmentDescriptor& tsd, TcpSession* session)
{
/* If its not a retransmission of the actual SYN... RESET */
if ( !SEQ_EQ(tsd.get_seq(), talker->get_iss()) )
return NORM_OK;
}
-bool TcpNormalizerProxy::validate_rst(
- TcpNormalizerState&, TcpSegmentDescriptor&)
-{
- return true;
-}
+bool TcpNormalizerProxy::validate_rst(TcpNormalizerState&, TcpSegmentDescriptor&)
+{ return true; }
-int TcpNormalizerProxy::handle_paws(
- TcpNormalizerState&, TcpSegmentDescriptor&)
-{
- return ACTION_NOTHING;
-}
+int TcpNormalizerProxy::handle_paws(TcpNormalizerState&, TcpSegmentDescriptor&)
+{ return ACTION_NOTHING; }
-int TcpNormalizerProxy::handle_repeated_syn(
- TcpNormalizerState&, TcpSegmentDescriptor&)
-{
- return ACTION_NOTHING;
-}
+int TcpNormalizerProxy::handle_repeated_syn(TcpNormalizerState&, TcpSegmentDescriptor&)
+{ return ACTION_NOTHING; }
TcpNormalizer::NormStatus TcpNormalizerMissed3whs::apply_normalizations(
TcpNormalizerState&, TcpSegmentDescriptor&, uint32_t, bool)
TcpNormalizerState& tns, TcpSegmentDescriptor& tsd)
{
if ( tns.session->flow->two_way_traffic() )
- return prev_norm->validate_rst(tns, tsd);
+ return tns.prev_norm->validate_rst(tns, tsd);
- if ( !prev_norm->get_name().compare("OS_Hpux11") )
+ if ( !tns.prev_norm->get_name().compare("OS_Hpux11") )
return validate_rst_seq_geq(tns, tsd);
return true;
}
-int TcpNormalizerMissed3whs::handle_paws(
- TcpNormalizerState& tns, TcpSegmentDescriptor& tsd)
-{
- return ACTION_NOTHING;
-}
+int TcpNormalizerMissed3whs::handle_paws(TcpNormalizerState&, TcpSegmentDescriptor&)
+{ return ACTION_NOTHING; }
int TcpNormalizerMissed3whs::handle_repeated_syn(
- TcpNormalizerState& tns, TcpSegmentDescriptor& tsd)
+ TcpNormalizerState& tns, TcpSegmentDescriptor& tsd)
{
- return prev_norm->handle_repeated_syn(tns, tsd);
+ return tns.prev_norm->handle_repeated_syn(tns, tsd);
}
-void TcpNormalizerPolicy::init(StreamPolicy os, TcpStreamSession* ssn, TcpStreamTracker* trk, TcpStreamTracker* peer)
+void TcpNormalizerPolicy::init(StreamPolicy os, TcpSession* ssn, TcpStreamTracker* trk, TcpStreamTracker* peer)
{
- TcpNormalizer* prev_norm = nullptr;
- if ( os == StreamPolicy::MISSED_3WHS and os != tns.os_policy )
- prev_norm = TcpNormalizerFactory::get_instance(tns.os_policy);
+ if ( os == StreamPolicy::MISSED_3WHS and os == tns.os_policy)
+ tns.prev_norm = TcpNormalizerFactory::get_instance(StreamPolicy::OS_DEFAULT);
+ else
+ tns.prev_norm = TcpNormalizerFactory::get_instance(tns.os_policy);
tns.os_policy = os;
tns.session = ssn;
tns.opt_block = Normalize_GetMode(NORM_TCP_OPT);
norm = TcpNormalizerFactory::get_instance(os);
-
- if ( prev_norm )
- norm->init(prev_norm);
- else
- norm->init(tns);
+ norm->init(tns);
}
TcpNormalizer* TcpNormalizerFactory::normalizers[StreamPolicy::OS_END_OF_LIST];
#include "stream/tcp/tcp_normalizer.h"
-class TcpStreamSession;
-class TcpStreamSession;
+class TcpSession;
+class TcpSession;
class TcpNormalizerFactory
{
TcpNormalizerPolicy() = default;
~TcpNormalizerPolicy() = default;
- void init(StreamPolicy os, TcpStreamSession* ssn, TcpStreamTracker* trk, TcpStreamTracker* peer);
+ void init(StreamPolicy os, TcpSession* ssn, TcpStreamTracker* trk, TcpStreamTracker* peer);
void reset()
{ init(StreamPolicy::OS_DEFAULT, nullptr, nullptr, nullptr); }
void set_zwp_seq(uint32_t seq)
{ return norm->set_zwp_seq(tns, seq); }
- void log_drop_reason(const TcpSegmentDescriptor& tsd, bool inline_mode, const char *issuer, const std::string& log)
- { return norm->log_drop_reason(tns, tsd , inline_mode, issuer, log); }
+ void log_drop_reason(const TcpSegmentDescriptor& tsd, bool inline_mode,
+ const char *issuer, const std::string& log)
+ { return norm->log_drop_reason(tns, tsd, inline_mode, issuer, log); }
+
+ bool is_keep_alive_probe(const TcpSegmentDescriptor& tsd)
+ { return norm->is_keep_alive_probe(tns, tsd); }
uint16_t set_urg_offset(const snort::tcp::TCPHdr* tcph, uint16_t dsize)
{ return norm->set_urg_offset(tns, tcph, dsize); }
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_overlap_resolver.cc author davis mcpherson <davmcphe@cisco.com>
+// Created on: Oct 11, 2015
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "tcp_overlap_resolver.h"
+
+#include "detection/detection_engine.h"
+
+#include "tcp_module.h"
+#include "tcp_normalizers.h"
+#include "tcp_segment_node.h"
+#include "tcp_session.h"
+
+using namespace snort;
+
+TcpOverlapState::TcpOverlapState(TcpReassemblySegments& seglist)
+ : seglist(seglist)
+{
+ tcp_ips_data = Normalize_GetMode(NORM_TCP_IPS);
+}
+
+void TcpOverlapState::init(TcpSegmentDescriptor& tsd)
+{
+ int32_t dist_head = 0, dist_tail = 0;
+
+ if ( seglist.head && seglist.tail )
+ {
+ if ( SEQ_GT(tsd.get_seq(), seglist.head->start_seq()) )
+ dist_head = tsd.get_seq() - seglist.head->start_seq();
+ else
+ dist_head = seglist.head->start_seq() - tsd.get_seq();
+
+ if ( SEQ_GT(tsd.get_seq(), seglist.tail->start_seq()) )
+ dist_tail = tsd.get_seq() - seglist.tail->start_seq();
+ else
+ dist_tail = seglist.tail->start_seq() - tsd.get_seq();
+ }
+
+ left = right = nullptr;
+ if ( dist_head < dist_tail )
+ {
+ right = seglist.head;
+ while ( right and SEQ_LT(right->start_seq(), tsd.get_seq()) )
+ {
+ left = right;
+ right = right->next;
+ }
+ }
+ else
+ {
+ left = seglist.tail;
+ while ( left and SEQ_GEQ(left->start_seq(), tsd.get_seq()) )
+ {
+ right = left;
+ left = left->prev;
+ }
+ }
+
+ this->tsd = &tsd;
+ seq = tsd.get_seq();
+ seq_end = tsd.get_end_seq();
+ len = tsd.get_len();
+
+ overlap = 0;
+ slide = 0;
+ trunc_len = 0;
+
+ rdata = tsd.get_pkt()->data;
+ rsize = tsd.get_len();
+ rseq = tsd.get_seq();
+
+ keep_segment = true;
+}
+
+bool TcpOverlapResolver::is_segment_retransmit(TcpOverlapState& tos, bool* full_retransmit)
+{
+ // Don't want to count retransmits as overlaps or do anything
+ // else with them. Account for retransmits of multiple PDUs
+ // in one segment.
+ bool* pb = (tos.rseq == tos.tsd->get_seq()) ? full_retransmit : nullptr;
+
+ if ( tos.right->is_retransmit(tos.rdata, tos.rsize,
+ tos.rseq, tos.right->length, pb) )
+ {
+ tos.tsd->set_retransmit_flag();
+
+ if ( !(*full_retransmit) )
+ {
+ tos.rdata += tos.right->length;
+ tos.rsize -= tos.right->length;
+ tos.rseq += tos.right->length;
+ tos.slide += tos.right->length;
+ tos.left = tos.right;
+ tos.right = tos.right->next;
+ }
+ else
+ tos.rsize = 0;
+
+ if ( tos.rsize == 0 )
+ {
+ // All data was retransmitted
+ snort::DetectionEngine::disable_content(tos.tsd->get_pkt());
+ tos.keep_segment = false;
+ }
+
+ return true;
+ }
+
+ return false;
+}
+
+void TcpOverlapResolver::eval_left(TcpOverlapState& tos)
+{
+ if ( tos.left )
+ insert_left_overlap(tos);
+}
+
+void TcpOverlapResolver::eval_right(TcpOverlapState& tos)
+{
+ while ( tos.right && SEQ_LT(tos.right->start_seq(), tos.seq_end) )
+ {
+ tos.trunc_len = 0;
+
+ assert(SEQ_LEQ(tos.slide_seq(), tos.right->start_seq()));
+ tos.overlap = ( int )( tos.seq_end - tos.right->start_seq() );
+
+ // Treat sequence number overlap as a retransmission,
+ // only check right side since left side happens rarely
+ tos.seglist.session->flow->call_handlers(tos.tsd->get_pkt(), false);
+ if ( tos.overlap < tos.right->length )
+ {
+ if ( tos.right->is_retransmit(tos.rdata, tos.rsize,
+ tos.rseq, tos.right->length, nullptr) )
+ {
+ // All data was retransmitted
+ tos.tsd->set_retransmit_flag();
+ snort::DetectionEngine::disable_content(tos.tsd->get_pkt());
+ tos.keep_segment = false;
+ tcpStats.full_retransmits++;
+ }
+ else
+ {
+ tcpStats.overlaps++;
+ tos.seglist.overlap_count++;
+ insert_right_overlap(tos);
+ }
+
+ break;
+ }
+ else // Full overlap
+ {
+ bool full_retransmit = false;
+ // Don't want to count retransmits as overlaps or do anything
+ // else with them. Account for retransmits of multiple PDUs
+ // in one segment.
+ if ( is_segment_retransmit(tos, &full_retransmit) )
+ {
+ if ( full_retransmit )
+ {
+ tcpStats.full_retransmits++;
+ break;
+ }
+ continue;
+ }
+
+ tcpStats.overlaps++;
+ tos.seglist.overlap_count++;
+ insert_full_overlap(tos);
+
+ if ( !tos.keep_segment )
+ return;
+ }
+ }
+}
+
+void TcpOverlapResolver::drop_old_segment(TcpOverlapState& tos)
+{
+ TcpSegmentNode* drop_seg = tos.right;
+ tos.right = tos.right->next;
+ tos.seglist.delete_reassembly_segment(drop_seg);
+}
+
+void TcpOverlapResolver::left_overlap_keep_first(TcpOverlapState& tos)
+{
+ // NOTE that overlap will always be less than left->size since
+ // seq is always greater than left->seq
+ assert(SEQ_GT(tos.seq, tos.left->start_seq()));
+
+ tos.overlap = tos.left->next_seq() - tos.seq;
+ if ( tos.len < tos.overlap )
+ tos.overlap = tos.len;
+
+ if ( tos.overlap > 0 )
+ {
+ tcpStats.overlaps++;
+ tos.seglist.overlap_count++;
+
+ if ( SEQ_GT(tos.left->next_seq(), tos.seq_end) )
+ {
+ if (tos.tcp_ips_data == NORM_MODE_ON)
+ {
+ unsigned offset = tos.tsd->get_seq() - tos.left->start_seq();
+ tos.tsd->rewrite_payload(0, tos.left->payload() + offset);
+ }
+ norm_stats[PC_TCP_IPS_DATA][tos.tcp_ips_data]++;
+ }
+ else
+ {
+ if ( tos.tcp_ips_data == NORM_MODE_ON )
+ {
+ unsigned offset = tos.tsd->get_seq() - tos.left->start_seq();
+ unsigned length = tos.left->next_seq() - tos.tsd->get_seq();
+ tos.tsd->rewrite_payload(0, tos.left->payload() + offset, length);
+ }
+
+ norm_stats[PC_TCP_IPS_DATA][tos.tcp_ips_data]++;
+ }
+
+ tos.slide = tos.overlap;
+ }
+}
+
+void TcpOverlapResolver::left_overlap_trim_first(TcpOverlapState& tos)
+{
+ assert(SEQ_GT(tos.seq, tos.left->start_seq()));
+
+ tos.overlap = tos.left->next_seq() - tos.seq;
+ if ( tos.overlap > 0 )
+ {
+ tcpStats.overlaps++;
+ tos.seglist.overlap_count++;
+
+ if ( SEQ_GEQ(tos.left->next_seq(), tos.seq + tos.len) )
+ {
+ // existing packet overlaps new on both sides. Drop the new data.
+ tos.slide += tos.len;
+ }
+ else
+ {
+ /* Otherwise, trim the old data accordingly */
+ tos.left->length -= ( int16_t )tos.overlap;
+ tos.seglist.seg_bytes_logical -= tos.overlap;
+ }
+ }
+}
+
+void TcpOverlapResolver::left_overlap_keep_last(TcpOverlapState& tos)
+{
+ assert(SEQ_GT(tos.seq, tos.left->seq));
+
+ tos.overlap = tos.left->next_seq() - tos.seq;
+ if ( tos.overlap > 0 )
+ {
+ tcpStats.overlaps++;
+ tos.seglist.overlap_count++;
+
+ /* True "Last" policy" */
+ if (SEQ_GT(tos.left->next_seq(), tos.seq + tos.len) )
+ {
+ /* New data is overlapped on both sides by existing data. Existing data needs to be
+ * split and the new data inserted in the middle.
+ * Need to duplicate left. Adjust that seq by + (seq + len) and
+ * size by - (seq + len - left->start_seq()).
+ */
+ tos.seglist.dup_reassembly_segment(tos.left, &tos.right);
+
+ tos.left->length -= tos.overlap;
+ uint16_t delta = tos.seq_end - tos.left->start_seq();
+ tos.right->length -= delta;
+ tos.right->offset += delta;
+ tos.seglist.seg_bytes_logical -= tos.len;
+ }
+ else
+ {
+ tos.left->length -= (int16_t)tos.overlap;
+ tos.seglist.seg_bytes_logical -= tos.overlap;
+ }
+ }
+}
+
+void TcpOverlapResolver::right_overlap_truncate_existing(TcpOverlapState& tos)
+{
+ if ( SEQ_EQ(tos.right->start_seq(), tos.slide_seq()) )
+ {
+ tos.slide += tos.overlap;
+ }
+ else
+ {
+ /* partial overlap */
+ tos.right->offset += tos.overlap;
+ tos.right->length -= ( int16_t )tos.overlap;
+ tos.seglist.seg_bytes_logical -= tos.overlap;
+ }
+}
+
+void TcpOverlapResolver::right_overlap_truncate_new(TcpOverlapState& tos)
+{
+ if (tos.tcp_ips_data == NORM_MODE_ON)
+ {
+ unsigned offset = tos.right->start_seq() - tos.tsd->get_seq();
+ unsigned length = tos.tsd->get_seq() + tos.tsd->get_len() - tos.right->start_seq();
+ tos.tsd->rewrite_payload(offset, tos.right->payload(), length);
+ }
+
+ norm_stats[PC_TCP_IPS_DATA][tos.tcp_ips_data]++;
+ tos.trunc_len = tos.overlap;
+}
+
+// REASSEMBLY_POLICY_FIRST:
+// REASSEMBLY_POLICY_VISTA:
+void TcpOverlapResolver::full_right_overlap_truncate_new(TcpOverlapState& tos)
+{
+
+ if ( tos.tcp_ips_data == NORM_MODE_ON )
+ {
+ unsigned offset = tos.right->start_seq() - tos.tsd->get_seq();
+ if ( !offset && zwp_data_mismatch(tos, *tos.tsd, tos.right->length))
+ {
+ tos.seglist.tracker->normalizer.session_blocker(*tos.tsd);
+ tos.keep_segment = false;
+ return;
+ }
+
+ tos.tsd->rewrite_payload(offset, tos.right->payload(), tos.right->length);
+ }
+
+ norm_stats[PC_TCP_IPS_DATA][tos.tcp_ips_data]++;
+
+ if ( SEQ_EQ(tos.right->start_seq(), tos.slide_seq()) )
+ {
+ // Overlap is greater than or equal to right->size slide gets set before insertion
+ tos.slide += tos.right->length;
+ tos.left = tos.right;
+ tos.right = tos.right->next;
+ }
+ else
+ {
+ // seq is less than right->start_seq(), set trunc length and slide
+ // and insert chunk before current right segment...
+ tos.trunc_len = tos.overlap;
+ tos.seglist.add_reassembly_segment(*tos.tsd, tos.len, tos.slide,
+ tos.trunc_len, tos.seq, tos.left);
+
+ // adjust slide and trunc_len and move to next node to the right...
+ tos.slide += tos.right->next_seq() - tos.slide_seq();
+ tos.trunc_len = 0;
+ tos.left = tos.right;
+ tos.right = tos.right->next;
+ }
+}
+
+// REASSEMBLY_POLICY_WINDOWS:
+// REASSEMBLY_POLICY_WINDOWS2K3:
+// REASSEMBLY_POLICY_BSD:
+// REASSEMBLY_POLICY_MACOS:
+void TcpOverlapResolver::full_right_overlap_os1(TcpOverlapState& tos)
+{
+ if ( SEQ_GEQ(tos.seq_end, tos.right->next_seq()) and
+ SEQ_LT(tos.slide_seq(), tos.right->start_seq()) )
+ {
+ drop_old_segment(tos);
+ }
+ else
+ full_right_overlap_truncate_new(tos);
+}
+
+// REASSEMBLY_POLICY_LINUX:
+// REASSEMBLY_POLICY_HPUX10:
+// REASSEMBLY_POLICY_IRIX:
+void TcpOverlapResolver::full_right_overlap_os2(TcpOverlapState& tos)
+{
+ if ( SEQ_GEQ(tos.seq_end, tos.right->next_seq()) and
+ SEQ_LT(tos.slide_seq(), tos.right->start_seq()) )
+ {
+ drop_old_segment(tos);
+ }
+ else if ( SEQ_GT(tos.seq_end, tos.right->next_seq()) and
+ SEQ_EQ(tos.slide_seq(), tos.right->start_seq()) )
+ {
+ drop_old_segment(tos);
+ }
+ else
+ full_right_overlap_truncate_new(tos);
+}
+
+// REASSEMBLY_POLICY_HPUX11:
+// REASSEMBLY_POLICY_SOLARIS:
+void TcpOverlapResolver::full_right_overlap_os3(TcpOverlapState& tos)
+{
+ // If this packet is wholly overlapping and the same size as a previous one and we have not
+ // received the one immediately preceding, we take the FIRST.
+ if ( SEQ_EQ(tos.right->start_seq(), tos.seq) && (tos.right->length == tos.len)
+ && (tos.left && !SEQ_EQ(tos.left->next_seq(), tos.seq)) )
+ {
+ right_overlap_truncate_new(tos);
+
+ tos.rdata += tos.right->length;
+ tos.rsize -= tos.right->length;
+ tos.rseq += tos.right->length;
+ tos.left = tos.right;
+ tos.right = tos.right->next;
+ }
+ else
+ drop_old_segment(tos);
+}
+
+// REASSEMBLY_POLICY_OLD_LINUX:
+// REASSEMBLY_POLICY_LAST:
+void TcpOverlapResolver::full_right_overlap_os4(TcpOverlapState& tos)
+{ drop_old_segment(tos); }
+
+void TcpOverlapResolver::full_right_overlap_os5(TcpOverlapState& tos)
+{
+ full_right_overlap_truncate_new(tos);
+}
+
+bool TcpOverlapResolver::zwp_data_mismatch(TcpOverlapState& tos, TcpSegmentDescriptor& tsd, uint32_t overlap)
+{
+ if ( overlap == MAX_ZERO_WIN_PROBE_LEN
+ and tos.right->start_seq() == tos.seglist.tracker->normalizer.get_zwp_seq()
+ and (tos.right->data[0] != tsd.get_pkt()->data[0]) )
+ {
+ return tsd.is_nap_policy_inline();
+ }
+
+ return false;
+}
+
+class TcpOverlapResolverFirst : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverFirst()
+ { overlap_policy = StreamPolicy::OS_FIRST; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_new(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os5(tos); }
+};
+
+class TcpOverlapResolverLast : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverLast()
+ { overlap_policy = StreamPolicy::OS_LAST; }
+
+private:
+ void right_overlap_truncate_existing(TcpOverlapState& tos) override
+ {
+ tos.right->offset += tos.overlap;
+ tos.right->length -= ( int16_t )tos.overlap;
+ tos.seglist.seg_bytes_logical -= tos.overlap;
+ }
+
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_last(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os4(tos); }
+};
+
+class TcpOverlapResolverLinux : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverLinux()
+ { overlap_policy = StreamPolicy::OS_LINUX; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os2(tos); }
+};
+
+class TcpOverlapResolverOldLinux : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverOldLinux()
+ { overlap_policy = StreamPolicy::OS_OLD_LINUX; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os4(tos); }
+};
+
+class TcpOverlapResolverBSD : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverBSD()
+ { overlap_policy = StreamPolicy::OS_BSD; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os1(tos); }
+};
+
+class TcpOverlapResolverMacOS : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverMacOS()
+ { overlap_policy = StreamPolicy::OS_MACOS; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os1(tos); }
+};
+
+class TcpOverlapResolverSolaris : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverSolaris()
+ { overlap_policy = StreamPolicy::OS_SOLARIS; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_trim_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_new(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os3(tos); }
+};
+
+class TcpOverlapResolverIrix : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverIrix()
+ { overlap_policy = StreamPolicy::OS_IRIX; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os2(tos); }
+};
+
+class TcpOverlapResolverHpux11 : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverHpux11()
+ { overlap_policy = StreamPolicy::OS_HPUX11; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_trim_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_new(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os3(tos); }
+};
+
+class TcpOverlapResolverHpux10 : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverHpux10()
+ { overlap_policy = StreamPolicy::OS_HPUX10; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os2(tos); }
+};
+
+class TcpOverlapResolverWindows : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverWindows()
+ { overlap_policy = StreamPolicy::OS_WINDOWS; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os1(tos); }
+};
+
+class TcpOverlapResolverWindows2K3 : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverWindows2K3()
+ { overlap_policy = StreamPolicy::OS_WINDOWS2K3; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_existing(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os1(tos); }
+};
+
+class TcpOverlapResolverVista : public TcpOverlapResolver
+{
+public:
+ TcpOverlapResolverVista()
+ { overlap_policy = StreamPolicy::OS_VISTA; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_new(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os5 (tos); }
+};
+
+class TcpOverlapResolverProxy : public TcpOverlapResolverFirst
+{
+public:
+ TcpOverlapResolverProxy()
+ { overlap_policy = StreamPolicy::OS_PROXY; }
+
+private:
+ void insert_left_overlap(TcpOverlapState& tos) override
+ { left_overlap_keep_first(tos); }
+
+ void insert_right_overlap(TcpOverlapState& tos) override
+ { right_overlap_truncate_new(tos); }
+
+ void insert_full_overlap(TcpOverlapState& tos) override
+ { full_right_overlap_os5(tos); }
+};
+
+TcpOverlapResolver* TcpOverlapResolverFactory::overlap_resolvers[StreamPolicy::OS_END_OF_LIST];
+
+void TcpOverlapResolverFactory::initialize()
+{
+ overlap_resolvers[StreamPolicy::OS_FIRST] = new TcpOverlapResolverFirst;
+ overlap_resolvers[StreamPolicy::OS_LAST] = new TcpOverlapResolverLast;
+ overlap_resolvers[StreamPolicy::OS_LINUX] = new TcpOverlapResolverLinux;
+ overlap_resolvers[StreamPolicy::OS_OLD_LINUX] = new TcpOverlapResolverOldLinux;
+ overlap_resolvers[StreamPolicy::OS_BSD] = new TcpOverlapResolverBSD;
+ overlap_resolvers[StreamPolicy::OS_MACOS] = new TcpOverlapResolverMacOS;
+ overlap_resolvers[StreamPolicy::OS_SOLARIS] = new TcpOverlapResolverSolaris;
+ overlap_resolvers[StreamPolicy::OS_IRIX] = new TcpOverlapResolverIrix;
+ overlap_resolvers[StreamPolicy::OS_HPUX11] = new TcpOverlapResolverHpux11;
+ overlap_resolvers[StreamPolicy::OS_HPUX10] = new TcpOverlapResolverHpux10;
+ overlap_resolvers[StreamPolicy::OS_WINDOWS] = new TcpOverlapResolverWindows;
+ overlap_resolvers[StreamPolicy::OS_WINDOWS2K3] = new TcpOverlapResolverWindows2K3;
+ overlap_resolvers[StreamPolicy::OS_VISTA] = new TcpOverlapResolverVista;
+ overlap_resolvers[StreamPolicy::OS_PROXY] = new TcpOverlapResolverProxy;
+}
+
+void TcpOverlapResolverFactory::term()
+{
+ for ( auto sp = StreamPolicy::OS_FIRST; sp <= StreamPolicy::OS_PROXY; sp++ )
+ delete overlap_resolvers[sp];
+}
+
+TcpOverlapResolver* TcpOverlapResolverFactory::get_instance(StreamPolicy os_policy)
+{
+ NormMode tcp_ips_data = Normalize_GetMode(NORM_TCP_IPS);
+ StreamPolicy sp = (tcp_ips_data == NORM_MODE_ON) ? StreamPolicy::OS_FIRST : os_policy;
+
+ assert( sp <= StreamPolicy::OS_PROXY );
+ return overlap_resolvers[sp];
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_overlap_resolver.h author davis mcpherson <davmcphe@cisco.com>
+// Created on: Oct 11, 2015
+
+#ifndef TCP_OVERLAP_RESOLVER_H
+#define TCP_OVERLAP_RESOLVER_H
+
+#include <vector>
+
+#include "normalize/norm_stats.h"
+#include "stream/stream.h"
+
+#include "tcp_defs.h"
+
+class TcpReassemblySegments;
+class TcpSegmentDescriptor;
+class TcpSegmentNode;
+class TcpSession;
+class TcpStreamTracker;
+
+class TcpOverlapState
+{
+public:
+ TcpOverlapState(TcpReassemblySegments& seglist);
+ ~TcpOverlapState() = default;
+
+ void init(TcpSegmentDescriptor&);
+
+ uint32_t slide_seq() const
+ { return seq + slide; }
+
+ TcpReassemblySegments& seglist;
+ TcpSegmentDescriptor* tsd = nullptr;
+
+ TcpSegmentNode* left = nullptr;
+ TcpSegmentNode* right = nullptr;
+ const uint8_t* rdata = nullptr;
+
+ uint32_t seq = 0;
+ uint32_t seq_end = 0;
+ uint32_t rseq = 0;
+
+ int32_t overlap = 0;
+ int32_t slide = 0;
+ int32_t trunc_len = 0;
+
+ uint16_t len = 0;
+ uint16_t rsize = 0;
+ int8_t tcp_ips_data = 0;
+
+ bool keep_segment = true;
+};
+
+class TcpOverlapResolver
+{
+public:
+ TcpOverlapResolver() = default;
+ virtual ~TcpOverlapResolver() = default;
+
+ void eval_left(TcpOverlapState&);
+ void eval_right(TcpOverlapState&);
+
+ StreamPolicy get_overlap_policy()
+ { return overlap_policy; }
+
+protected:
+ virtual bool is_segment_retransmit(TcpOverlapState&, bool*);
+ virtual void drop_old_segment(TcpOverlapState&);
+ virtual bool zwp_data_mismatch(TcpOverlapState&, TcpSegmentDescriptor&, uint32_t);
+
+ virtual void left_overlap_keep_first(TcpOverlapState&);
+ virtual void left_overlap_trim_first(TcpOverlapState&);
+ virtual void left_overlap_keep_last(TcpOverlapState&);
+ virtual void right_overlap_truncate_existing(TcpOverlapState&);
+ virtual void right_overlap_truncate_new(TcpOverlapState&);
+ virtual void full_right_overlap_truncate_new(TcpOverlapState&);
+ virtual void full_right_overlap_os1(TcpOverlapState&);
+ virtual void full_right_overlap_os2(TcpOverlapState&);
+ virtual void full_right_overlap_os3(TcpOverlapState&);
+ virtual void full_right_overlap_os4(TcpOverlapState&);
+ virtual void full_right_overlap_os5(TcpOverlapState&);
+
+ virtual void insert_left_overlap(TcpOverlapState&) = 0;
+ virtual void insert_right_overlap(TcpOverlapState&) = 0;
+ virtual void insert_full_overlap(TcpOverlapState&) = 0;
+
+ StreamPolicy overlap_policy = StreamPolicy::OS_DEFAULT;
+};
+
+class TcpOverlapResolverFactory
+{
+public:
+ static void initialize();
+ static void term();
+ static TcpOverlapResolver* get_instance(StreamPolicy);
+
+private:
+ TcpOverlapResolverFactory() = delete;
+
+ static TcpOverlapResolver* overlap_resolvers[StreamPolicy::OS_END_OF_LIST];
+};
+
+#endif
#include "packet_io/packet_tracer.h"
#include "profiler/profiler.h"
#include "protocols/packet_manager.h"
+#include "stream/stream_splitter.h"
#include "time/packet_time.h"
#include "tcp_module.h"
#include "tcp_normalizers.h"
+#include "tcp_segment_node.h"
#include "tcp_session.h"
-#include "tcp_stream_tracker.h"
using namespace snort;
-static THREAD_LOCAL Packet* last_pdu = nullptr;
-
-static void purge_alerts_callback_ackd(IpsContext* c)
+void TcpReassembler::init(bool server, StreamSplitter* ss)
{
- TcpSession* session = (TcpSession*)c->packet->flow->session;
-
- if ( c->packet->is_from_server() )
- session->client.reassembler.purge_alerts();
+ splitter = ss;
+ paf.paf_setup(ss);
+ if ( seglist.cur_rseg )
+ seglist.cur_sseg = seglist.cur_rseg;
else
- session->server.reassembler.purge_alerts();
-}
+ seglist.cur_sseg = seglist.head;
-static void purge_alerts_callback_ips(IpsContext* c)
-{
- TcpSession* session = (TcpSession*)c->packet->flow->session;
+ server_side = server;
- if ( c->packet->is_from_server() )
- session->server.reassembler.purge_alerts();
+ if ( server_side )
+ {
+ ignore_dir = SSN_DIR_FROM_CLIENT;
+ packet_dir = PKT_FROM_CLIENT;
+ }
else
- session->client.reassembler.purge_alerts();
-}
-
-bool TcpReassembler::is_segment_pending_flush(const TcpReassemblerState& trs) const
-{
- return ( get_pending_segment_count(trs, 1) > 0 );
-}
-
-uint32_t TcpReassembler::get_pending_segment_count(const TcpReassemblerState& trs, unsigned max) const
-{
- uint32_t n = trs.sos.seg_count - trs.flush_count;
-
- if ( !n || max == 1 )
- return n;
-
- n = 0;
- const TcpSegmentNode* tsn = trs.sos.seglist.head;
- while ( tsn )
{
- if ( tsn->c_len && SEQ_LT(tsn->c_seq, trs.tracker->r_win_base) )
- n++;
-
- if ( max && n == max )
- return n;
-
- tsn = tsn->next;
+ ignore_dir = SSN_DIR_FROM_SERVER;
+ packet_dir = PKT_FROM_SERVER;
}
-
- return n;
-}
-
-bool TcpReassembler::next_no_gap(const TcpSegmentNode& tsn)
-{
- return tsn.next and (tsn.next->i_seq == tsn.i_seq + tsn.i_len);
-}
-
-bool TcpReassembler::next_no_gap_c(const TcpSegmentNode& tsn)
-{
- return tsn.next and (tsn.next->c_seq == tsn.c_seq + tsn.c_len);
-}
-
-bool TcpReassembler::next_acked_no_gap_c(const TcpSegmentNode& tsn, const TcpReassemblerState& trs)
-{
- return tsn.next and (tsn.next->c_seq == tsn.c_seq + tsn.c_len)
- and SEQ_LT(tsn.next->c_seq, trs.tracker->r_win_base);
}
-bool TcpReassembler::fin_no_gap(const TcpSegmentNode& tsn, const TcpReassemblerState& trs)
+bool TcpReassembler::fin_no_gap(const TcpSegmentNode& tsn)
{
- return trs.tracker->fin_seq_status >= TcpStreamTracker::FIN_WITH_SEQ_SEEN
- and SEQ_GEQ(tsn.i_seq + tsn.i_len, trs.tracker->get_fin_i_seq());
+ return tracker.fin_seq_status >= FIN_WITH_SEQ_SEEN
+ and SEQ_GEQ(tsn.next_seq(), tracker.get_fin_i_seq());
}
-bool TcpReassembler::fin_acked_no_gap(const TcpSegmentNode& tsn, const TcpReassemblerState& trs)
+bool TcpReassembler::fin_acked_no_gap(const TcpSegmentNode& tsn)
{
- return trs.tracker->fin_seq_status >= TcpStreamTracker::FIN_WITH_SEQ_ACKED
- and SEQ_GEQ(tsn.i_seq + tsn.i_len, trs.tracker->get_fin_i_seq());
-}
-
-void TcpReassembler::update_next(TcpReassemblerState& trs, const TcpSegmentNode& tsn)
-{
- trs.sos.seglist.cur_rseg = next_no_gap(tsn) ? tsn.next : nullptr;
- if ( trs.sos.seglist.cur_rseg )
- trs.sos.seglist.cur_rseg->c_seq = trs.sos.seglist.cur_rseg->i_seq;
+ return tracker.fin_seq_status >= FIN_WITH_SEQ_ACKED
+ and SEQ_GEQ(tsn.next_seq(), tracker.get_fin_i_seq());
}
// If we are skipping seglist hole, update tsn so that we can purge
-void TcpReassembler::update_skipped_bytes(uint32_t remaining_bytes, TcpReassemblerState& trs)
+void TcpReassembler::update_skipped_bytes(uint32_t remaining_bytes)
{
TcpSegmentNode* tsn;
- while ( remaining_bytes and (tsn = trs.sos.seglist.cur_rseg) )
+ while ( remaining_bytes and (tsn = seglist.cur_rseg) )
{
- auto bytes_skipped = ( tsn->c_len <= remaining_bytes ) ? tsn->c_len : remaining_bytes;
+ auto bytes_skipped = ( tsn->unscanned() <= remaining_bytes ) ? tsn->unscanned() : remaining_bytes;
remaining_bytes -= bytes_skipped;
- tsn->update_reassembly_cursor(bytes_skipped);
+ tsn->advance_cursor(bytes_skipped);
- if ( !tsn->c_len )
+ if ( !tsn->unscanned() )
{
- trs.flush_count++;
- update_next(trs, *tsn);
+ seglist.flush_count++;
+ seglist.update_next(tsn);
}
}
}
-int TcpReassembler::delete_reassembly_segment(TcpReassemblerState& trs, TcpSegmentNode* tsn)
-{
- int ret;
- assert(tsn);
-
- trs.sos.seglist.remove(tsn);
- trs.sos.seg_bytes_total -= tsn->i_len;
- trs.sos.seg_bytes_logical -= tsn->i_len;
- ret = tsn->i_len;
-
- if ( !tsn->c_len )
- {
- tcpStats.segs_used++;
- trs.flush_count--;
- }
-
- if ( trs.sos.seglist.cur_sseg == tsn )
- trs.sos.seglist.cur_sseg = tsn->next;
-
- if ( trs.sos.seglist.cur_rseg == tsn )
- update_next(trs, *tsn);
-
- tsn->term();
- trs.sos.seg_count--;
-
- return ret;
-}
-
-void TcpReassembler::queue_reassembly_segment(
- TcpReassemblerState& trs, TcpSegmentNode* prev, TcpSegmentNode* tsn)
-{
- trs.sos.seglist.insert(prev, tsn);
-
- if ( !trs.sos.seglist.cur_sseg )
- trs.sos.seglist.cur_sseg = tsn;
- else if ( SEQ_LT(tsn->c_seq, trs.sos.seglist.cur_sseg->c_seq) )
- {
- trs.sos.seglist.cur_sseg = tsn;
- if ( SEQ_LT(tsn->c_seq, trs.sos.seglist_base_seq) )
- trs.sos.seglist_base_seq = tsn->c_seq;
-
- if ( trs.sos.seglist.cur_rseg && SEQ_LT(tsn->c_seq, trs.sos.seglist.cur_rseg->c_seq) )
- trs.sos.seglist.cur_rseg = tsn;
- }
-
- trs.sos.seg_count++;
- trs.sos.seg_bytes_total += tsn->i_len;
- trs.sos.total_segs_queued++;
- tcpStats.segs_queued++;
-
- if ( trs.sos.seg_count > tcpStats.max_segs )
- tcpStats.max_segs = trs.sos.seg_count;
-
- if ( trs.sos.seg_bytes_total > tcpStats.max_bytes )
- tcpStats.max_bytes = trs.sos.seg_bytes_total;
-}
-
-bool TcpReassembler::is_segment_fasttrack(
- TcpReassemblerState&, TcpSegmentNode* tail, const TcpSegmentDescriptor& tsd)
+void TcpReassembler::purge_to_seq(uint32_t flush_seq)
{
- if ( SEQ_EQ(tsd.get_seq(), tail->i_seq + tail->i_len) )
- return true;
-
- return false;
-}
-
-void TcpReassembler::add_reassembly_segment(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd, uint16_t len, uint32_t slide,
- uint32_t trunc_len, uint32_t seq, TcpSegmentNode* left)
-{
- const int32_t new_size = len - slide - trunc_len;
- assert(new_size >= 0);
-
- // if trimming will delete all data, don't insert this segment in the queue
- if ( new_size <= 0 )
- {
- tcpStats.payload_fully_trimmed++;
- trs.tracker->normalizer.trim_win_payload(tsd);
- return;
- }
-
- // FIXIT-L don't allocate overlapped part
- TcpSegmentNode* const tsn = TcpSegmentNode::init(tsd);
-
- tsn->offset = slide;
- tsn->o_offset = slide;
- tsn->c_len = (uint16_t)new_size;
- tsn->i_len = (uint16_t)new_size;
- tsn->i_seq = tsn->c_seq = seq;
- tsn->ts = tsd.get_timestamp();
-
- // FIXIT-M the urgent ptr handling is broken... urg_offset could be set here but currently
- // not actually referenced anywhere else. In 2.9.7 the FlushStream function did reference
- // this field but that code has been lost... urg ptr handling needs to be reviewed and fixed
- // tsn->urg_offset = trs.tracker->normalizer.set_urg_offset(tsd.get_tcph(), tsd.get_seg_len());
-
- queue_reassembly_segment(trs, left, tsn);
-
- trs.sos.seg_bytes_logical += tsn->c_len;
- trs.sos.total_bytes_queued += tsn->c_len;
- tsd.set_packet_flags(PKT_STREAM_INSERT);
-}
-
-void TcpReassembler::dup_reassembly_segment(
- TcpReassemblerState& trs, TcpSegmentNode* left, TcpSegmentNode** retSeg)
-{
- TcpSegmentNode* tsn = TcpSegmentNode::init(*left);
- tcpStats.segs_split++;
-
- // twiddle the values for overlaps
- tsn->c_len = left->c_len;
- tsn->i_seq = tsn->c_seq = left->i_seq;
- queue_reassembly_segment(trs, left, tsn);
-
- *retSeg = tsn;
-}
-
-bool TcpReassembler::add_alert(TcpReassemblerState& trs, uint32_t gid, uint32_t sid)
-{
- assert(trs.alerts.size() <=
- (uint32_t)(get_ips_policy()->rules_loaded + get_ips_policy()->rules_shared));
-
- if (!this->check_alerted(trs, gid, sid))
- trs.alerts.emplace_back(gid, sid);
-
- return true;
-}
-
-bool TcpReassembler::check_alerted(TcpReassemblerState& trs, uint32_t gid, uint32_t sid)
-{
- return std::any_of(trs.alerts.cbegin(), trs.alerts.cend(),
- [gid, sid](const StreamAlertInfo& alert){ return alert.gid == gid && alert.sid == sid; });
-}
-
-int TcpReassembler::update_alert(TcpReassemblerState& trs, uint32_t gid, uint32_t sid,
- uint32_t event_id, uint32_t event_second)
-{
- // FIXIT-M comparison of seq_num is wrong, compare value is always 0, should be seq_num of wire packet
- uint32_t seq_num = 0;
-
- auto it = std::find_if(trs.alerts.begin(), trs.alerts.end(),
- [gid, sid, seq_num](const StreamAlertInfo& alert)
- { return alert.gid == gid && alert.sid == sid && SEQ_EQ(alert.seq, seq_num); });
- if (it != trs.alerts.end())
- {
- (*it).event_id = event_id;
- (*it).event_second = event_second;
- return 0;
- }
-
- return -1;
-}
-
-void TcpReassembler::purge_alerts(TcpReassemblerState& trs)
-{
- Flow* flow = trs.sos.session->flow;
-
- for ( auto& alert : trs.alerts )
- Stream::log_extra_data(flow, trs.xtradata_mask, alert);
-
- if ( !flow->is_suspended() )
- trs.alerts.clear();
-}
-
-void TcpReassembler::purge_to_seq(TcpReassemblerState& trs, uint32_t flush_seq)
-{
- assert( trs.sos.seglist.head );
- uint32_t last_ts = 0;
-
- TcpSegmentNode* tsn = trs.sos.seglist.head;
- while ( tsn && SEQ_LT(tsn->i_seq, flush_seq))
- {
- if ( tsn->c_len )
- break;
-
- TcpSegmentNode* dump_me = tsn;
- tsn = tsn->next;
- if (dump_me->ts > last_ts)
- last_ts = dump_me->ts;
-
- delete_reassembly_segment(trs, dump_me);
- }
-
- if ( SEQ_LT(trs.tracker->rcv_nxt, flush_seq) )
- trs.tracker->rcv_nxt = flush_seq;
+ seglist.purge_flushed_segments(flush_seq);
if ( last_pdu )
{
- if ( trs.tracker->normalizer.is_tcp_ips_enabled() )
- last_pdu->context->register_post_callback(purge_alerts_callback_ips);
- else
- last_pdu->context->register_post_callback(purge_alerts_callback_ackd);
-
+ tracker.tcp_alerts.purge_alerts(*last_pdu, tracker.normalizer.is_tcp_ips_enabled());
last_pdu = nullptr;
}
else
- purge_alerts(trs);
-
- if ( trs.sos.seglist.head == nullptr )
- trs.sos.seglist.tail = nullptr;
-
- /* Update the "last" time stamp seen from the other side
- * to be the most recent timestamp (largest) that was removed
- * from the queue. This will ensure that as we go forward,
- * last timestamp is the highest one that we had stored and
- * purged and handle the case when packets arrive out of order,
- * such as:
- * P1: seq 10, length 10, timestamp 10
- * P3: seq 30, length 10, timestamp 30
- * P2: seq 20, length 10, timestamp 20
- *
- * Without doing it this way, the timestamp would be 20. With
- * the next packet to arrive (P4, seq 40), the ts_last value
- * wouldn't be updated for the talker in ProcessTcp() since that
- * code specifically looks for the NEXT sequence number.
- */
- if ( last_ts )
- {
- if ( trs.server_side )
- {
- int32_t delta = last_ts - trs.sos.session->client.get_ts_last();
- if ( delta > 0 )
- trs.sos.session->client.set_ts_last(last_ts);
- }
- else
- {
- int32_t delta = last_ts - trs.sos.session->server.get_ts_last();
- if ( delta > 0 )
- trs.sos.session->server.set_ts_last(last_ts);
- }
- }
+ tracker.tcp_alerts.purge_alerts(seglist.session->flow);
}
// must only purge flushed and acked bytes we may flush partial segments
// part of a segment
// * FIXIT-L need flag to mark any reassembled packets that have a gap
// (if we reassemble such)
-void TcpReassembler::purge_flushed_ackd(TcpReassemblerState& trs)
+void TcpReassembler::purge_flushed_ackd()
{
- TcpSegmentNode* tsn = trs.sos.seglist.head;
- uint32_t seq;
-
- if (!trs.sos.seglist.head)
+ if ( !seglist.head )
return;
- seq = trs.sos.seglist.head->i_seq;
-
- while ( tsn && !tsn->c_len )
+ uint32_t seq = seglist.head->start_seq();
+ TcpSegmentNode* tsn = seglist.head;
+ while ( tsn && !tsn->unscanned() )
{
- uint32_t end = tsn->i_seq + tsn->i_len;
+ uint32_t end = tsn->next_seq();
- if ( SEQ_GT(end, trs.tracker->r_win_base) )
+ if ( SEQ_GT(end, tracker.r_win_base) )
break;
seq = end;
tsn = tsn->next;
}
- if ( seq != trs.sos.seglist.head->i_seq )
- purge_to_seq(trs, seq);
+ if ( !SEQ_EQ(seq, seglist.head->start_seq()) )
+ purge_to_seq(seq);
}
-void TcpReassembler::show_rebuilt_packet(const TcpReassemblerState& trs, Packet* pkt)
+void TcpReassembler::show_rebuilt_packet(Packet* pkt)
{
- if ( trs.sos.session->tcp_config->flags & STREAM_CONFIG_SHOW_PACKETS )
+ if ( seglist.session->tcp_config->flags & STREAM_CONFIG_SHOW_PACKETS )
{
// FIXIT-L setting conf here is required because this is called before context start
pkt->context->conf = SnortConfig::get_conf();
}
}
-int TcpReassembler::flush_data_segments(TcpReassemblerState& trs, uint32_t flush_len, Packet* pdu)
+int TcpReassembler::flush_data_segments(uint32_t flush_len, Packet* pdu)
{
uint32_t flags = PKT_PDU_HEAD;
- uint32_t to_seq = trs.sos.seglist.cur_rseg->c_seq + flush_len;
+
+ uint32_t to_seq = seglist.cur_rseg->scan_seq() + flush_len;
uint32_t remaining_bytes = flush_len;
uint32_t total_flushed = 0;
while ( remaining_bytes )
{
- TcpSegmentNode* tsn = trs.sos.seglist.cur_rseg;
- unsigned bytes_to_copy = ( tsn->c_len <= remaining_bytes ) ? tsn->c_len : remaining_bytes;
+ TcpSegmentNode* tsn = seglist.cur_rseg;
+ unsigned bytes_to_copy = ( tsn->unscanned() <= remaining_bytes ) ? tsn->unscanned() : remaining_bytes;
remaining_bytes -= bytes_to_copy;
if ( !remaining_bytes )
flags |= PKT_PDU_TAIL;
else
- assert( bytes_to_copy >= tsn->c_len );
+ assert( bytes_to_copy >= tsn->unscanned() );
unsigned bytes_copied = 0;
- const StreamBuffer sb = trs.tracker->get_splitter()->reassemble(
- trs.sos.session->flow, flush_len, total_flushed, tsn->payload(),
- bytes_to_copy, flags, bytes_copied);
+ const StreamBuffer sb = splitter->reassemble(seglist.session->flow, flush_len, total_flushed,
+ tsn->paf_data(), bytes_to_copy, flags, bytes_copied);
if ( sb.data )
{
}
total_flushed += bytes_copied;
- tsn->update_reassembly_cursor(bytes_copied);
+ tsn->advance_cursor(bytes_copied);
flags = 0;
- if ( !tsn->c_len )
+ if ( !tsn->unscanned() )
{
- trs.flush_count++;
- update_next(trs, *tsn);
+ seglist.flush_count++;
+ seglist.update_next(tsn);
}
/* Check for a gap/missing packet */
// FIXIT-L FIN may be in to_seq causing bogus gap counts.
- if ( tsn->is_packet_missing(to_seq) or trs.paf_state.paf == StreamSplitter::SKIP )
+ if ( tsn->is_packet_missing(to_seq) or paf.state == StreamSplitter::SKIP )
{
// FIXIT-H // assert(false); find when this scenario happens
// FIXIT-L this is suboptimal - better to exclude fin from to_seq
- if ( !trs.tracker->is_fin_seq_set() or
- SEQ_LEQ(to_seq, trs.tracker->get_fin_final_seq()) )
+ if ( !tracker.is_fin_seq_set() or
+ SEQ_LEQ(to_seq, tracker.get_fin_final_seq()) )
{
- trs.tracker->set_tf_flags(TF_MISSING_PKT);
+ tracker.set_tf_flags(TF_MISSING_PKT);
}
break;
}
- if ( sb.data || !trs.sos.seglist.cur_rseg )
+ if ( sb.data || !seglist.cur_rseg )
break;
}
- if ( trs.paf_state.paf == StreamSplitter::SKIP )
- update_skipped_bytes(remaining_bytes, trs);
+ if ( paf.state == StreamSplitter::SKIP )
+ update_skipped_bytes(remaining_bytes);
return total_flushed;
}
-static inline bool both_splitters_aborted(Flow* flow)
-{
- uint32_t both_splitters_yoinked = (SSNFLAG_ABORT_CLIENT | SSNFLAG_ABORT_SERVER);
- return (flow->get_session_flags() & both_splitters_yoinked) == both_splitters_yoinked;
-}
-
// FIXIT-L consolidate encode format, update, and this into new function?
-void TcpReassembler::prep_pdu(
- TcpReassemblerState&, Flow* flow, Packet* p, uint32_t pkt_flags, Packet* pdu)
+void TcpReassembler::prep_pdu(Flow* flow, Packet* p, uint32_t pkt_flags, Packet* pdu)
{
pdu->ptrs.set_pkt_type(PktType::PDU);
pdu->proto_bits |= PROTO_BIT__TCP;
}
}
-Packet* TcpReassembler::initialize_pdu(
- TcpReassemblerState& trs, Packet* p, uint32_t pkt_flags, struct timeval tv)
+Packet* TcpReassembler::initialize_pdu(Packet* p, uint32_t pkt_flags, struct timeval tv)
{
// partial flushes already set the pdu for http_inspect splitter processing
Packet* pdu = p->was_set() ? p : DetectionEngine::set_next_packet(p);
EncodeFlags enc_flags = 0;
DAQ_PktHdr_t pkth;
- trs.sos.session->get_packet_header_foo(&pkth, p->pkth, pkt_flags);
+ seglist.session->get_packet_header_foo(&pkth, p->pkth, pkt_flags);
PacketManager::format_tcp(enc_flags, p, pdu, PSEUDO_PKT_TCP, &pkth, pkth.opaque);
- prep_pdu(trs, trs.sos.session->flow, p, pkt_flags, pdu);
+ prep_pdu(seglist.session->flow, p, pkt_flags, pdu);
assert(pdu->pkth == pdu->context->pkth);
pdu->context->pkth->ts = tv;
pdu->dsize = 0;
}
// flush a seglist up to the given point, generate a pseudopacket, and fire it thru the system.
-int TcpReassembler::flush_to_seq(
- TcpReassemblerState& trs, uint32_t bytes, Packet* p, uint32_t pkt_flags)
+int TcpReassembler::flush_to_seq(uint32_t bytes, Packet* p, uint32_t pkt_flags)
{
- assert( p && trs.sos.seglist.cur_rseg);
+ assert( p && seglist.cur_rseg);
- trs.tracker->clear_tf_flags(TF_MISSING_PKT | TF_MISSING_PREV_PKT);
+ tracker.clear_tf_flags(TF_MISSING_PKT | TF_MISSING_PREV_PKT);
- TcpSegmentNode* tsn = trs.sos.seglist.cur_rseg;
- assert( trs.sos.seglist_base_seq == tsn->c_seq);
+ TcpSegmentNode* tsn = seglist.cur_rseg;
+ assert( seglist.seglist_base_seq == tsn->scan_seq());
- Packet* pdu = initialize_pdu(trs, p, pkt_flags, tsn->tv);
- int32_t flushed_bytes = flush_data_segments(trs, bytes, pdu);
+ Packet* pdu = initialize_pdu(p, pkt_flags, tsn->tv);
+ int32_t flushed_bytes = flush_data_segments(bytes, pdu);
assert( flushed_bytes );
- trs.sos.seglist_base_seq += flushed_bytes;
+ seglist.seglist_base_seq += flushed_bytes;
if ( pdu->data )
{
else
pdu->packet_flags |= ( PKT_REBUILT_STREAM | PKT_STREAM_EST );
- show_rebuilt_packet(trs, pdu);
+ show_rebuilt_packet(pdu);
tcpStats.rebuilt_packets++;
tcpStats.rebuilt_bytes += flushed_bytes;
else
last_pdu = nullptr;
- trs.tracker->finalize_held_packet(p);
+ tracker.finalize_held_packet(p);
}
else
{
}
// FIXIT-L abort should be by PAF callback only since recovery may be possible
- if ( trs.tracker->get_tf_flags() & TF_MISSING_PKT )
+ if ( tracker.get_tf_flags() & TF_MISSING_PKT )
{
- trs.tracker->set_tf_flags(TF_MISSING_PREV_PKT | TF_PKT_MISSED);
- trs.tracker->clear_tf_flags(TF_MISSING_PKT);
+ tracker.set_tf_flags(TF_MISSING_PREV_PKT | TF_PKT_MISSED);
+ tracker.clear_tf_flags(TF_MISSING_PKT);
tcpStats.gaps++;
}
else
- trs.tracker->clear_tf_flags(TF_MISSING_PREV_PKT);
+ tracker.clear_tf_flags(TF_MISSING_PREV_PKT);
return flushed_bytes;
}
-// flush a seglist up to the given point, generate a pseudopacket, and fire it thru the system.
-int TcpReassembler::do_zero_byte_flush(TcpReassemblerState& trs, Packet* p, uint32_t pkt_flags)
+int TcpReassembler::do_zero_byte_flush(Packet* p, uint32_t pkt_flags)
{
unsigned bytes_copied = 0;
- const StreamBuffer sb = trs.tracker->get_splitter()->reassemble(
- trs.sos.session->flow, 0, 0, nullptr, 0, (PKT_PDU_HEAD | PKT_PDU_TAIL), bytes_copied);
+ const StreamBuffer sb = splitter->reassemble(seglist.session->flow, 0, 0,
+ nullptr, 0, (PKT_PDU_HEAD | PKT_PDU_TAIL), bytes_copied);
if ( sb.data )
{
- Packet* pdu = initialize_pdu(trs, p, pkt_flags, p->pkth->ts);
+ Packet* pdu = initialize_pdu(p, pkt_flags, p->pkth->ts);
/* setup the pseudopacket payload */
pdu->data = sb.data;
pdu->dsize = sb.length;
pdu->packet_flags |= (PKT_REBUILT_STREAM | PKT_STREAM_EST | PKT_PDU_HEAD | PKT_PDU_TAIL);
- trs.flush_count++;
- show_rebuilt_packet(trs, pdu);
+ show_rebuilt_packet(pdu);
DetectionEngine de;
de.inspect(pdu);
return bytes_copied;
}
-// get the footprint for the current trs.sos.seglist, the difference
+// get the footprint for the current seglist, the difference
// between our base sequence and the last ack'd sequence we received
-uint32_t TcpReassembler::get_q_footprint(TcpReassemblerState& trs)
+uint32_t TcpReassembler::get_q_footprint()
{
int32_t footprint = 0;
int32_t sequenced = 0;
- if ( SEQ_GT(trs.tracker->r_win_base, trs.sos.seglist_base_seq) )
- footprint = trs.tracker->r_win_base - trs.sos.seglist_base_seq;
+ if ( SEQ_GT(tracker.r_win_base, seglist.seglist_base_seq) )
+ footprint = tracker.r_win_base - seglist.seglist_base_seq;
if ( footprint )
- sequenced = get_q_sequenced(trs);
+ sequenced = get_q_sequenced();
return ( footprint > sequenced ) ? sequenced : footprint;
}
// FIXIT-P get_q_sequenced() performance could possibly be
-// boosted by tracking sequenced bytes as trs.sos.seglist is updated
+// boosted by tracking sequenced bytes as seglist is updated
// to avoid the while loop, etc. below.
-uint32_t TcpReassembler::get_q_sequenced(TcpReassemblerState& trs)
+uint32_t TcpReassembler::get_q_sequenced()
{
- TcpSegmentNode* tsn = trs.sos.seglist.cur_rseg;
+ TcpSegmentNode* tsn = seglist.cur_rseg;
if ( !tsn )
{
- tsn = trs.sos.seglist.head;
+ tsn = seglist.head;
- if ( !tsn || SEQ_LT(trs.tracker->r_win_base, tsn->c_seq) )
+ if ( !tsn || SEQ_LT(tracker.r_win_base, tsn->scan_seq()) )
return 0;
- trs.sos.seglist.cur_rseg = tsn;
+ seglist.cur_rseg = tsn;
}
uint32_t len = 0;
- const uint32_t limit = trs.tracker->get_splitter()->max();
- while ( len < limit and next_no_gap(*tsn) )
+ const uint32_t limit = splitter->max();
+ while ( len < limit and tsn->next_no_gap() )
{
- if ( !tsn->c_len )
- trs.sos.seglist.cur_rseg = tsn->next;
+
+ if ( !tsn->unscanned() )
+ seglist.cur_rseg = tsn->next;
else
- len += tsn->c_len;
+ len += tsn->unscanned();
tsn = tsn->next;
}
- if ( tsn->c_len )
- len += tsn->c_len;
+ if ( tsn->unscanned() )
+ len += tsn->unscanned();
- trs.sos.seglist_base_seq = trs.sos.seglist.cur_rseg->c_seq;
+ seglist.seglist_base_seq = seglist.cur_rseg->scan_seq();
return len;
}
-bool TcpReassembler::is_q_sequenced(TcpReassemblerState& trs)
+bool TcpReassembler::is_q_sequenced()
{
- TcpSegmentNode* tsn = trs.sos.seglist.cur_rseg;
+ TcpSegmentNode* tsn = seglist.cur_rseg;
if ( !tsn )
{
- tsn = trs.sos.seglist.head;
-
- if ( !tsn || SEQ_LT(trs.tracker->r_win_base, tsn->c_seq) )
+ tsn = seglist.head;
+ if ( !tsn || SEQ_LT(tracker.r_win_base, tsn->scan_seq()) )
return false;
- trs.sos.seglist.cur_rseg = tsn;
+ seglist.cur_rseg = tsn;
}
- while ( next_no_gap(*tsn) )
+ while ( tsn->next_no_gap() )
{
- if ( tsn->c_len )
+ if ( tsn->unscanned() )
break;
- tsn = trs.sos.seglist.cur_rseg = tsn->next;
+ tsn = seglist.cur_rseg = tsn->next;
}
- trs.sos.seglist_base_seq = tsn->c_seq;
+ seglist.seglist_base_seq = tsn->scan_seq();
- return (tsn->c_len != 0);
+ return (tsn->unscanned() != 0);
}
-int TcpReassembler::flush_stream(
- TcpReassemblerState& trs, Packet* p, uint32_t dir, bool final_flush)
+void TcpReassembler::final_flush(Packet* p, uint32_t dir)
{
- // this is not always redundant; stream_reassemble rule option causes trouble
- if ( !trs.tracker->is_reassembly_enabled() )
- return 0;
+ tracker.set_tf_flags(TF_FORCE_FLUSH);
- if ( trs.sos.session->flow->two_way_traffic()
- or (trs.tracker->get_tcp_state() == TcpStreamTracker::TCP_MID_STREAM_RECV) )
+ if ( flush_stream(p, dir, true) )
{
- uint32_t bytes = 0;
-
- if ( trs.tracker->normalizer.is_tcp_ips_enabled() )
- bytes = get_q_sequenced(trs); // num bytes in pre-ack mode
- else
- bytes = get_q_footprint(trs); // num bytes in post-ack mode
-
- if ( bytes )
- return flush_to_seq(trs, bytes, p, dir);
- }
-
- if ( final_flush )
- return do_zero_byte_flush(trs, p, dir);
-
- return 0;
-}
-
-void TcpReassembler::final_flush(TcpReassemblerState& trs, Packet* p, uint32_t dir)
-{
- trs.tracker->set_tf_flags(TF_FORCE_FLUSH);
-
- if ( flush_stream(trs, p, dir, true) )
- {
- if ( trs.server_side )
+ if ( server_side )
tcpStats.server_cleanups++;
else
tcpStats.client_cleanups++;
- purge_flushed_ackd(trs);
+ purge_flushed_ackd();
}
- trs.tracker->clear_tf_flags(TF_FORCE_FLUSH);
+ tracker.clear_tf_flags(TF_FORCE_FLUSH);
}
static Packet* get_packet(Flow* flow, uint32_t flags, bool c2s)
return p;
}
-void TcpReassembler::finish_and_final_flush(
- TcpReassemblerState& trs, Flow* flow, bool clear, Packet* p)
-{
- bool pending = clear and paf_initialized(&trs.paf_state)
- and trs.tracker->splitter_finish(flow);
-
- if ( pending and !(flow->ssn_state.ignore_direction & trs.ignore_dir) )
- final_flush(trs, p, trs.packet_dir);
-}
-
-// Call this only from outside reassembly.
-void TcpReassembler::flush_queued_segments(
- TcpReassemblerState& trs, Flow* flow, bool clear, const Packet* p)
+bool TcpReassembler::splitter_finish(snort::Flow* flow)
{
- if ( p )
- {
- finish_and_final_flush(trs, flow, clear, const_cast<Packet*>(p));
- }
- else
- {
- Packet* pdu = get_packet(flow, trs.packet_dir, trs.server_side);
-
- bool pending = clear and paf_initialized(&trs.paf_state);
- if ( pending )
- {
- DetectionEngine de;
- pending = trs.tracker->splitter_finish(flow);
- }
-
- if ( pending and !(flow->ssn_state.ignore_direction & trs.ignore_dir) )
- final_flush(trs, pdu, trs.packet_dir);
- }
-}
-
-// see scan_data_post_ack() for details
-// the key difference is that we operate on forward moving data
-// because we don't wait until it is acknowledged
-int32_t TcpReassembler::scan_data_pre_ack(TcpReassemblerState& trs, uint32_t* flags, Packet* p)
-{
- assert(trs.sos.session->flow == p->flow);
-
- int32_t ret_val = FINAL_FLUSH_HOLD;
-
- if ( SEQ_GT(trs.sos.seglist.head->c_seq, trs.sos.seglist_base_seq) )
- return ret_val;
-
- if ( !trs.sos.seglist.cur_rseg )
- trs.sos.seglist.cur_rseg = trs.sos.seglist.cur_sseg;
-
- if ( !is_q_sequenced(trs) )
- return ret_val;
-
- TcpSegmentNode* tsn = trs.sos.seglist.cur_sseg;
- uint32_t total = tsn->c_seq - trs.sos.seglist_base_seq;
+ if (!splitter)
+ return true;
- ret_val = FINAL_FLUSH_OK;
- while ( tsn && *flags )
+ if (!splitter_finish_flag)
{
- total += tsn->c_len;
-
- uint32_t end = tsn->c_seq + tsn->c_len;
- uint32_t pos = paf_position(&trs.paf_state);
-
- if ( paf_initialized(&trs.paf_state) && SEQ_LEQ(end, pos) )
- {
- if ( !next_no_gap(*tsn) )
- {
- ret_val = FINAL_FLUSH_HOLD;
- break;
- }
-
- tsn = tsn->next;
- continue;
- }
-
- if ( next_no_gap_c(*tsn) )
- *flags |= PKT_MORE_TO_FLUSH;
- else
- *flags &= ~PKT_MORE_TO_FLUSH;
- int32_t flush_pt = paf_check(
- trs.tracker->get_splitter(), &trs.paf_state, p, tsn->payload(),
- tsn->c_len, total, tsn->c_seq, flags);
-
- if (flush_pt >= 0)
- {
- trs.sos.seglist.cur_sseg = tsn;
- update_rcv_nxt(trs, *tsn);
- return flush_pt;
- }
-
- if (!next_no_gap(*tsn) || (trs.paf_state.paf == StreamSplitter::STOP))
- {
- if ( !(next_no_gap(*tsn) || fin_no_gap(*tsn, trs)) )
- ret_val = FINAL_FLUSH_HOLD;
- break;
- }
-
- tsn = tsn->next;
+ splitter_finish_flag = true;
+ return splitter->finish(flow);
}
-
- trs.sos.seglist.cur_sseg = tsn;
-
- if ( tsn )
- update_rcv_nxt(trs, *tsn);
-
- return ret_val;
+ // there shouldn't be any un-flushed data beyond this point,
+ // returning false here, discards it
+ return false;
}
-static inline void fallback(TcpStreamTracker& trk, bool server_side, uint16_t max)
+void TcpReassembler::finish_and_final_flush(Flow* flow, bool clear, Packet* p)
{
-#ifndef NDEBUG
- StreamSplitter* splitter = trk.get_splitter();
- assert(splitter);
-
- // FIXIT-L: consolidate these 3
- bool to_server = splitter->to_server();
- assert(server_side == to_server && server_side == !trk.client_tracker);
-#endif
+ bool pending = clear and paf.paf_initialized() and splitter_finish(flow);
- trk.set_splitter(new AtomSplitter(server_side, max));
- tcpStats.partial_fallbacks++;
+ if ( pending and !(flow->ssn_state.ignore_direction & ignore_dir) )
+ final_flush(p, packet_dir);
}
-void TcpReassembler::fallback(TcpStreamTracker& tracker, bool server_side)
+// Call this only from outside reassembly.
+void TcpReassembler::flush_queued_segments(Flow* flow, bool clear, Packet* p)
{
- uint16_t max = tracker.session->tcp_config->paf_max;
- ::fallback(tracker, server_side, max);
-
- Flow* flow = tracker.session->flow;
- if ( server_side )
- flow->set_session_flags(SSNFLAG_ABORT_SERVER);
- else
- flow->set_session_flags(SSNFLAG_ABORT_CLIENT);
-
- if ( flow->gadget and both_splitters_aborted(flow) )
+ if ( p )
{
- flow->clear_gadget();
- tcpStats.inspector_fallbacks++;
+ finish_and_final_flush(flow, clear, p);
}
-}
-
-bool TcpReassembler::segment_within_seglist_window(TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
-{
- if ( !trs.sos.seglist.head )
- return true;
-
- // Left side
- uint32_t start;
- if ( SEQ_LT(trs.sos.seglist_base_seq, trs.sos.seglist.head->i_seq) )
- start = trs.sos.seglist_base_seq;
else
- start = trs.sos.seglist.head->i_seq;
-
- if ( SEQ_LEQ(tsd.get_end_seq(), start) )
- return false;
-
- // Right side
- uint32_t end = (trs.sos.seglist.tail->i_seq + trs.sos.seglist.tail->i_len);
- if ( SEQ_GEQ(tsd.get_seq(), end) )
- return false;
-
- return true;
-}
-
-void TcpReassembler::check_first_segment_hole(TcpReassemblerState& trs)
-{
- if ( SEQ_LT(trs.sos.seglist_base_seq, trs.sos.seglist.head->c_seq)
- and SEQ_EQ(trs.sos.seglist_base_seq, trs.tracker->rcv_nxt) )
- {
- trs.sos.seglist_base_seq = trs.sos.seglist.head->c_seq;
- trs.tracker->rcv_nxt = trs.tracker->r_win_base;
- trs.paf_state.paf = StreamSplitter::START;
- }
-}
-
-void TcpReassembler::update_rcv_nxt(TcpReassemblerState& trs, TcpSegmentNode& tsn)
-{
- uint32_t temp = (tsn.i_seq + tsn.i_len);
-
- if ( SEQ_GT(temp, trs.tracker->rcv_nxt) )
- trs.tracker->rcv_nxt = temp;
-}
-
-bool TcpReassembler::has_seglist_hole(TcpReassemblerState& trs, TcpSegmentNode& tsn, PAF_State& ps,
- uint32_t& total, uint32_t& flags)
-{
- if ( !tsn.prev or SEQ_GEQ(tsn.prev->c_seq + tsn.prev->c_len, tsn.c_seq) or
- SEQ_GEQ(tsn.c_seq, trs.tracker->r_win_base) )
- {
- check_first_segment_hole(trs);
- return false;
- }
-
- // safety - prevent seq + total < seq
- if ( total > 0x7FFFFFFF )
- total = 0x7FFFFFFF;
-
- if ( !ps.tot )
- flags |= PKT_PDU_HEAD;
-
- ps.paf = StreamSplitter::SKIP;
- return true;
-}
-
-void TcpReassembler::purge_segments_left_of_hole(TcpReassemblerState& trs, const TcpSegmentNode* end_tsn)
-{
- uint32_t packets_skipped = 0;
-
- TcpSegmentNode* cur_tsn = trs.sos.seglist.head;
- do
- {
- TcpSegmentNode* drop_tsn = cur_tsn;
- cur_tsn = cur_tsn->next;
- delete_reassembly_segment(trs, drop_tsn);
- ++packets_skipped;
- } while( cur_tsn and cur_tsn != end_tsn );
-
- if (PacketTracer::is_active())
- PacketTracer::log("Stream: Skipped %u packets before seglist hole)\n", packets_skipped);
-}
-
-void TcpReassembler::reset_asymmetric_flow_reassembly(TcpReassemblerState& trs)
-{
- TcpSegmentNode* tsn = trs.sos.seglist.head;
- // if there is a hole at the beginning, skip it...
- if ( SEQ_GT(tsn->i_seq, trs.sos.seglist_base_seq) )
{
- trs.sos.seglist_base_seq = tsn->i_seq;
- if (PacketTracer::is_active())
- PacketTracer::log("Stream: Skipped hole at beginning of the seglist\n");
- }
+ Packet* pdu = get_packet(flow, packet_dir, server_side);
- while ( tsn )
- {
- if ( tsn->next and SEQ_GT(tsn->next->i_seq, tsn->i_seq + tsn->i_len) )
+ bool pending = clear and paf.paf_initialized();
+ if ( pending )
{
- tsn = tsn->next;
- purge_segments_left_of_hole(trs, tsn);
- trs.sos.seglist_base_seq = trs.sos.seglist.head->i_seq;
+ DetectionEngine de;
+ pending = splitter_finish(flow);
}
- else
- tsn = tsn->next;
- }
- if ( trs.tracker->is_splitter_paf() )
- fallback(*trs.tracker, trs.server_side);
- else
- paf_reset(&trs.paf_state);
+ if ( pending and !(flow->ssn_state.ignore_direction & ignore_dir) )
+ final_flush(pdu, packet_dir);
+ }
}
-void TcpReassembler::skip_midstream_pickup_seglist_hole(TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
-{
- uint32_t ack = tsd.get_ack();
-
- TcpSegmentNode* tsn = trs.sos.seglist.head;
- while ( tsn )
- {
- if ( SEQ_GEQ( tsn->i_seq + tsn->i_len, ack) )
- break;
- if ( tsn->next and SEQ_GT(tsn->next->i_seq, tsn->i_seq + tsn->i_len) )
- {
- tsn = tsn->next;
- purge_segments_left_of_hole(trs, tsn);
- trs.sos.seglist_base_seq = trs.sos.seglist.head->i_seq;
- }
- else if ( !tsn->next and SEQ_LT(tsn->i_seq + tsn->i_len, ack) )
- {
- tsn = tsn->next;
- purge_segments_left_of_hole(trs, tsn);
- trs.sos.seglist_base_seq = ack;
- }
- else
- tsn = tsn->next;
- }
-
- tsn = trs.sos.seglist.head;
- if ( tsn )
+void TcpReassembler::check_first_segment_hole()
+{
+ if ( SEQ_LT(seglist.seglist_base_seq, seglist.head->start_seq()) )
{
- paf_initialize(&trs.paf_state, tsn->i_seq);
-
- while ( next_no_gap(*tsn) )
- tsn = tsn->next;
- trs.tracker->rcv_nxt = tsn->i_seq + tsn->i_len;
+ seglist.seglist_base_seq = seglist.head->start_seq();
+ seglist.advance_rcv_nxt();
+ paf.state = StreamSplitter::START;
}
- else
- trs.tracker->rcv_nxt = ack;
}
-bool TcpReassembler::flush_on_asymmetric_flow(const TcpReassemblerState &trs, uint32_t flushed, snort::Packet *p)
+uint32_t TcpReassembler::perform_partial_flush(Flow* flow, Packet*& p)
{
- bool asymmetric = flushed && trs.sos.seg_count && !p->flow->two_way_traffic() && !p->ptrs.tcph->is_syn();
- if ( asymmetric )
- {
- TcpStreamTracker::TcpState peer = trs.tracker->session->get_peer_state(trs.tracker);
- asymmetric = ( peer == TcpStreamTracker::TCP_SYN_SENT || peer == TcpStreamTracker::TCP_SYN_RECV
- || peer == TcpStreamTracker::TCP_MID_STREAM_SENT );
- }
-
- return asymmetric;
+ p = get_packet(flow, packet_dir, server_side);
+ return perform_partial_flush(p);
}
-// iterate over trs.sos.seglist and scan all new acked bytes
-// - new means not yet scanned
-// - must use trs.sos.seglist data (not packet) since this packet may plug a
-// hole and enable paf scanning of following segments
-// - if we reach a flush point
-// - return bytes to flush if data available (must be acked)
-// - return zero if not yet received or received but not acked
-// - if we reach a skip point
-// - jump ahead and resume scanning any available data
-// - must stop if we reach a gap
-// - one segment may lead to multiple checks since
-// it may contain multiple encapsulated PDUs
-// - if we partially scan a segment we must save state so we
-// know where we left off and can resume scanning the remainder
-int32_t TcpReassembler::scan_data_post_ack(TcpReassemblerState& trs, uint32_t* flags, Packet* p)
+// No error checking here, so the caller must ensure that p, p->flow are not null.
+uint32_t TcpReassembler::perform_partial_flush(Packet* p, uint32_t flushed)
{
- assert(trs.sos.session->flow == p->flow);
-
- int32_t ret_val = FINAL_FLUSH_HOLD;
-
- if ( !trs.sos.seglist.cur_sseg || SEQ_GEQ(trs.sos.seglist_base_seq, trs.tracker->r_win_base) )
- return ret_val ;
-
- if ( !trs.sos.seglist.cur_rseg )
- trs.sos.seglist.cur_rseg = trs.sos.seglist.cur_sseg;
-
- StreamSplitter* splitter = trs.tracker->get_splitter();
-
- uint32_t total = 0;
- TcpSegmentNode* tsn = trs.sos.seglist.cur_sseg;
- if ( paf_initialized(&trs.paf_state) )
- {
- uint32_t end_seq = tsn->c_seq + tsn->c_len;
- if ( SEQ_EQ(end_seq, paf_position(&trs.paf_state)) )
- {
- total = end_seq - trs.sos.seglist_base_seq;
- tsn = tsn->next;
- }
- else
- total = tsn->c_seq - trs.sos.seglist.cur_rseg->c_seq;
- }
-
- ret_val = FINAL_FLUSH_OK;
- while (tsn && *flags && SEQ_LT(tsn->c_seq, trs.tracker->r_win_base))
+ if ( splitter->init_partial_flush(p->flow) )
{
- // only flush acked data that fits in pdu reassembly buffer...
- uint32_t end = tsn->c_seq + tsn->c_len;
- uint32_t flush_len;
- int32_t flush_pt;
-
- if ( SEQ_GT(end, trs.tracker->r_win_base))
- flush_len = trs.tracker->r_win_base - tsn->c_seq;
- else
- flush_len = tsn->c_len;
-
- if ( next_acked_no_gap_c(*tsn, trs) )
- *flags |= PKT_MORE_TO_FLUSH;
- else
- *flags &= ~PKT_MORE_TO_FLUSH;
-
- if ( has_seglist_hole(trs, *tsn, trs.paf_state, total, *flags) )
- flush_pt = total;
- else
- {
- total += flush_len;
- flush_pt = paf_check(splitter, &trs.paf_state, p, tsn->payload(),
- flush_len, total, tsn->c_seq, flags);
- }
-
- // Get splitter from tracker as paf check may change it.
- splitter = trs.tracker->get_splitter();
- trs.sos.seglist.cur_sseg = tsn;
-
- if ( flush_pt >= 0 )
- {
- trs.sos.seglist_base_seq = trs.sos.seglist.cur_rseg->c_seq;
- return flush_pt;
- }
-
- if (flush_len < tsn->c_len || (splitter->is_paf() and !next_no_gap(*tsn)) ||
- (trs.paf_state.paf == StreamSplitter::STOP))
+ flushed += flush_stream(p, packet_dir, false);
+ paf.paf_jump(flushed);
+ tcpStats.partial_flushes++;
+ tcpStats.partial_flush_bytes += flushed;
+ if ( seglist.seg_count )
{
- if ( !(next_no_gap(*tsn) || fin_acked_no_gap(*tsn, trs)) )
- ret_val = FINAL_FLUSH_HOLD;
- break;
+ purge_to_seq(seglist.head->start_seq() + flushed);
+ tracker.r_win_base = seglist.seglist_base_seq;
}
-
- tsn = tsn->next;
}
-
- return ret_val;
+ return flushed;
}
// we are on a FIN, the data has been scanned, it has no gaps,
// but somehow we are waiting for more data - do final flush here
// FIXIT-M this convoluted expression needs some refactoring to simplify
-bool TcpReassembler::final_flush_on_fin(const TcpReassemblerState &trs, int32_t flush_amt, Packet *p)
+bool TcpReassembler::final_flush_on_fin(int32_t flush_amt, Packet *p, FinSeqNumStatus fin_status)
{
- return trs.tracker->fin_seq_status >= TcpStreamTracker::FIN_WITH_SEQ_SEEN
+ return tracker.fin_seq_status >= fin_status
&& -1 <= flush_amt && flush_amt <= 0
- && trs.paf_state.paf == StreamSplitter::SEARCH
+ && paf.state == StreamSplitter::SEARCH
&& !p->flow->searching_for_service();
}
-int TcpReassembler::flush_on_data_policy(TcpReassemblerState& trs, Packet* p)
-{
- uint32_t flushed = 0;
- last_pdu = nullptr;
-
- switch ( trs.tracker->get_flush_policy() )
- {
- case STREAM_FLPOLICY_IGNORE:
- return 0;
-
- case STREAM_FLPOLICY_ON_ACK:
- break;
-
- case STREAM_FLPOLICY_ON_DATA:
- if ( trs.sos.seglist.head )
- {
- uint32_t flags;
- int32_t flush_amt;
- do
- {
- flags = trs.packet_dir;
- flush_amt = scan_data_pre_ack(trs, &flags, p);
- if ( flush_amt <= 0 )
- break;
-
- flushed += flush_to_seq(trs, flush_amt, p, flags);
- } while ( trs.sos.seglist.head and !p->flow->is_inspection_disabled() );
-
- if ( (trs.paf_state.paf == StreamSplitter::ABORT) && trs.tracker->is_splitter_paf() )
- {
- fallback(*trs.tracker, trs.server_side);
- return flush_on_data_policy(trs, p);
- }
- else if ( final_flush_on_fin(trs, flush_amt, p) )
- finish_and_final_flush(trs, p->flow, true, p);
- }
- break;
- }
-
- if ( !trs.sos.seglist.head )
- return flushed;
-
- if ( trs.tracker->is_retransmit_of_held_packet(p) )
- flushed = perform_partial_flush(trs, p, flushed);
-
- if ( flush_on_asymmetric_flow(trs, flushed, p) )
- {
- purge_to_seq(trs, trs.sos.seglist.head->i_seq + flushed);
- trs.tracker->r_win_base = trs.sos.seglist_base_seq;
- tcpStats.flush_on_asymmetric_flow++;
- }
-
- return flushed;
-}
-
-void TcpReassembler::skip_seglist_hole(TcpReassemblerState& trs, Packet* p, uint32_t flags,
- int32_t flush_amt)
-{
- if ( trs.tracker->is_splitter_paf() )
- {
- if ( flush_amt > 0 )
- update_skipped_bytes(flush_amt, trs);
- fallback(*trs.tracker, trs.server_side);
- }
- else
- {
- if ( flush_amt > 0 )
- flush_to_seq(trs, flush_amt, p, flags);
- trs.paf_state.paf = StreamSplitter::START;
- }
-
- if ( trs.sos.seglist.head )
- {
- if ( flush_amt > 0 )
- purge_to_seq(trs, trs.sos.seglist_base_seq + flush_amt);
- trs.sos.seglist_base_seq = trs.sos.seglist.head->c_seq;
- }
- else
- trs.sos.seglist_base_seq = trs.tracker->r_win_base;
-
- trs.tracker->rcv_nxt = trs.tracker->r_win_base;
- trs.sos.seglist.cur_rseg = trs.sos.seglist.head;
-}
-
-int TcpReassembler::flush_on_ack_policy(TcpReassemblerState& trs, Packet* p)
-{
- uint32_t flushed = 0;
- last_pdu = nullptr;
-
- switch ( trs.tracker->get_flush_policy() )
- {
- case STREAM_FLPOLICY_IGNORE:
- return 0;
-
- case STREAM_FLPOLICY_ON_ACK:
- {
- int32_t flush_amt;
- uint32_t flags;
-
- do
- {
- flags = trs.packet_dir;
- flush_amt = scan_data_post_ack(trs, &flags, p);
- if ( flush_amt <= 0 or trs.paf_state.paf == StreamSplitter::SKIP )
- break;
-
- // for consistency with other cases, should return total
- // but that breaks flushing pipelined pdus
- flushed += flush_to_seq(trs, flush_amt, p, flags);
- assert( flushed );
-
- // ideally we would purge just once after this loop but that throws off base
- if ( trs.sos.seglist.head )
- purge_to_seq(trs, trs.sos.seglist_base_seq);
- }
- while ( trs.sos.seglist.head and !p->flow->is_inspection_disabled() );
-
- if ( (trs.paf_state.paf == StreamSplitter::ABORT) && trs.tracker->is_splitter_paf() )
- {
- fallback(*trs.tracker, trs.server_side);
- return flush_on_ack_policy(trs, p);
- }
- else if ( trs.paf_state.paf == StreamSplitter::SKIP )
- {
- skip_seglist_hole(trs, p, flags, flush_amt);
- return flush_on_ack_policy(trs, p);
- }
- else if ( -1 <= flush_amt and flush_amt <= 0 and
- trs.paf_state.paf == StreamSplitter::SEARCH and
- trs.tracker->fin_seq_status == TcpStreamTracker::FIN_WITH_SEQ_ACKED and
- !p->flow->searching_for_service() )
- {
- // we are acknowledging a FIN, the data has been scanned, it has no gaps,
- // but somehow we are waiting for more data - do final flush here
- finish_and_final_flush(trs, p->flow, true, p);
- }
- }
- break;
-
- case STREAM_FLPOLICY_ON_DATA:
- purge_flushed_ackd(trs);
- break;
- }
-
- return flushed;
-}
-
-void TcpReassembler::purge_segment_list(TcpReassemblerState& trs)
-{
- trs.sos.seglist.reset();
- trs.sos.seg_count = 0;
- trs.sos.seg_bytes_total = 0;
- trs.sos.seg_bytes_logical = 0;
- trs.flush_count = 0;
-}
-
-void TcpReassembler::insert_segment_in_empty_seglist(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
-{
- uint32_t overlap = 0;
-
- if ( SEQ_GT(trs.sos.seglist_base_seq, tsd.get_seq()) )
- {
- overlap = trs.sos.seglist_base_seq - tsd.get_seq();
- if ( overlap >= tsd.get_len() )
- return;
- }
-
- add_reassembly_segment(
- trs, tsd, tsd.get_len(), overlap, 0, tsd.get_seq() + overlap, nullptr);
-}
-
-void TcpReassembler::init_overlap_editor(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
-{
- TcpSegmentNode* left = nullptr, *right = nullptr, *tsn = nullptr;
- int32_t dist_head = 0, dist_tail = 0;
-
- if ( trs.sos.seglist.head && trs.sos.seglist.tail )
- {
- if ( SEQ_GT(tsd.get_seq(), trs.sos.seglist.head->i_seq) )
- dist_head = tsd.get_seq() - trs.sos.seglist.head->i_seq;
- else
- dist_head = trs.sos.seglist.head->i_seq - tsd.get_seq();
-
- if ( SEQ_GT(tsd.get_seq(), trs.sos.seglist.tail->i_seq) )
- dist_tail = tsd.get_seq() - trs.sos.seglist.tail->i_seq;
- else
- dist_tail = trs.sos.seglist.tail->i_seq - tsd.get_seq();
- }
-
- if ( SEQ_LEQ(dist_head, dist_tail) )
- {
- for ( tsn = trs.sos.seglist.head; tsn; tsn = tsn->next )
- {
- right = tsn;
- if ( SEQ_GEQ(right->i_seq, tsd.get_seq() ) )
- break;
-
- left = right;
- }
-
- if ( tsn == nullptr )
- right = nullptr;
- }
- else
- {
- for ( tsn = trs.sos.seglist.tail; tsn; tsn = tsn->prev )
- {
- left = tsn;
- if ( SEQ_LT(left->i_seq, tsd.get_seq() ) )
- break;
-
- right = left;
- }
-
- if (tsn == nullptr)
- left = nullptr;
- }
-
- trs.sos.init_soe(tsd, left, right);
-}
-
-void TcpReassembler::insert_segment_in_seglist(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
+bool TcpReassembler::flush_on_asymmetric_flow(uint32_t flushed, snort::Packet *p)
{
- // NORM fast tracks are in sequence - no norms
- if ( trs.sos.seglist.tail && is_segment_fasttrack(trs, trs.sos.seglist.tail, tsd) )
- {
- /* segment fit cleanly at the end of the segment list */
- add_reassembly_segment(
- trs, tsd, tsd.get_len(), 0, 0, tsd.get_seq(), trs.sos.seglist.tail);
- return;
- }
-
- init_overlap_editor(trs, tsd);
- eval_left(trs);
- eval_right(trs);
-
- if ( trs.sos.keep_segment )
- {
- if ( !trs.sos.left and trs.sos.right and
- paf_initialized(&trs.paf_state) and SEQ_GT(trs.paf_state.pos, tsd.get_seq()) )
- {
- return;
- }
-
- // slide is current seq number - initial seq number unless all data
- // truncated from right, then slide is 0
- if ( trs.sos.len - trs.sos.trunc_len > 0 )
- trs.sos.slide = trs.sos.seq - tsd.get_seq();
- else
- trs.sos.slide = 0;
-
- add_reassembly_segment(
- trs, tsd, trs.sos.len, trs.sos.slide, trs.sos.trunc_len, trs.sos.seq, trs.sos.left);
- }
-}
-
-void TcpReassembler::queue_packet_for_reassembly(
- TcpReassemblerState& trs, TcpSegmentDescriptor& tsd)
-{
- if ( trs.sos.seg_count == 0 )
- {
- insert_segment_in_empty_seglist(trs, tsd);
- return;
- }
-
- if ( SEQ_GT(trs.tracker->r_win_base, tsd.get_seq() ) )
+ bool asymmetric = flushed && seglist.seg_count && !p->flow->two_way_traffic() && !p->ptrs.tcph->is_syn();
+ if ( asymmetric )
{
- const int32_t offset = trs.tracker->r_win_base - tsd.get_seq();
-
- if ( offset < tsd.get_len() )
- {
- tsd.slide_segment_in_rcv_window(offset);
- insert_segment_in_seglist(trs, tsd);
- tsd.slide_segment_in_rcv_window(-offset);
- }
+ TcpStreamTracker::TcpState peer = tracker.session->get_peer_state(&tracker);
+ asymmetric = ( peer == TcpStreamTracker::TCP_SYN_SENT || peer == TcpStreamTracker::TCP_SYN_RECV
+ || peer == TcpStreamTracker::TCP_MID_STREAM_SENT );
}
- else
- insert_segment_in_seglist(trs, tsd);
-}
-uint32_t TcpReassembler::perform_partial_flush(TcpReassemblerState& trs, Flow* flow, Packet*& p)
-{
- p = get_packet(flow, trs.packet_dir, trs.server_side);
- return perform_partial_flush(trs, p);
-}
-
-// No error checking here, so the caller must ensure that p, p->flow and context
-// are not null.
-uint32_t TcpReassembler::perform_partial_flush(TcpReassemblerState& trs, Packet* p, uint32_t flushed)
-{
- if ( trs.tracker->get_splitter()->init_partial_flush(p->flow) )
- {
- flushed += flush_stream(trs, p, trs.packet_dir, false);
- paf_jump(&trs.paf_state, flushed);
- tcpStats.partial_flushes++;
- tcpStats.partial_flush_bytes += flushed;
- if ( trs.sos.seg_count )
- {
- purge_to_seq(trs, trs.sos.seglist.head->i_seq + flushed);
- trs.tracker->r_win_base = trs.sos.seglist_base_seq;
- }
- }
- return flushed;
+ return asymmetric;
}
// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
//--------------------------------------------------------------------------
-// tcp_reassembly.h author davis mcpherson <davmcphe@cisco.com>
+// tcp_reassembler.h author davis mcpherson <davmcphe@cisco.com>
// Created on: Jul 31, 2015
#ifndef TCP_REASSEMBLER_H
#define TCP_REASSEMBLER_H
+
+#include <cstdint>
+
+#include "flow/flow.h"
+#include "protocols/packet.h"
+#include "stream/pafng.h"
#include "stream/stream.h"
-#include "segment_overlap_editor.h"
+#include "tcp_reassembly_segments.h"
-class TcpReassembler : public SegmentOverlapEditor
+namespace snort
+{
+ class StreamSplitter;
+}
+
+class TcpSegmentDescriptor;
+class TcpSegmentNode;
+class TcpStreamTracker;
+enum FinSeqNumStatus : uint8_t;
+
+class TcpReassembler
{
public:
FINAL_FLUSH_OK = -1
};
- virtual void queue_packet_for_reassembly(TcpReassemblerState&, TcpSegmentDescriptor&);
- virtual void purge_segment_list(TcpReassemblerState&);
- virtual void purge_flushed_ackd(TcpReassemblerState&);
- virtual int flush_stream(
- TcpReassemblerState&, snort::Packet* p, uint32_t dir, bool final_flush = false);
- virtual void flush_queued_segments(
- TcpReassemblerState&, snort::Flow* flow, bool clear, const snort::Packet* = nullptr);
- void finish_and_final_flush(
- TcpReassemblerState&, snort::Flow* flow, bool clear, snort::Packet*);
- virtual bool is_segment_pending_flush(const TcpReassemblerState&) const;
- virtual int flush_on_data_policy(TcpReassemblerState&, snort::Packet*);
- virtual int flush_on_ack_policy(TcpReassemblerState&, snort::Packet*);
- virtual bool add_alert(TcpReassemblerState&, uint32_t gid, uint32_t sid);
- virtual bool check_alerted(TcpReassemblerState&, uint32_t gid, uint32_t sid);
- virtual int update_alert(TcpReassemblerState&, uint32_t gid, uint32_t sid,
- uint32_t event_id, uint32_t event_second);
- virtual void purge_alerts(TcpReassemblerState&);
- virtual bool segment_within_seglist_window(TcpReassemblerState&, TcpSegmentDescriptor&);
- void reset_asymmetric_flow_reassembly(TcpReassemblerState&);
- void skip_midstream_pickup_seglist_hole(TcpReassemblerState&, TcpSegmentDescriptor&);
- void initialize_paf(TcpReassemblerState& trs)
+ TcpReassembler(TcpStreamTracker& trk, TcpReassemblySegments& seglist)
+ : tracker(trk), seglist(seglist)
+ { }
+
+ virtual ~TcpReassembler()
+ { }
+
+ virtual void init(bool server, snort::StreamSplitter* ss);
+
+ virtual int eval_flush_policy_on_ack(snort::Packet*) = 0;
+ virtual int eval_flush_policy_on_data(snort::Packet*) = 0;
+ virtual int flush_stream(snort::Packet*, uint32_t dir, bool final_flush = false) = 0;
+ void flush_queued_segments(snort::Flow* flow, bool clear, snort::Packet* = nullptr);
+ void finish_and_final_flush(snort::Flow* flow, bool clear, snort::Packet*);
+ uint32_t perform_partial_flush(snort::Flow*, snort::Packet*&);
+ void purge_flushed_ackd();
+
+ void release_splitter()
+ { splitter = nullptr; }
+
+ snort::StreamSplitter* get_splitter()
+ { return splitter; }
+
+ bool is_splitter_paf() const
+ { return splitter && splitter->is_paf(); }
+
+ bool segment_already_scanned(uint32_t seq)
+ {
+ if ( paf.paf_initialized() and SEQ_GT(paf.pos, seq) )
+ return true;
+ else
+ return false;
+ }
+
+ void initialize_paf()
{
// only initialize if we have a data segment queued
- if ( !trs.sos.seglist.head )
+ if ( !seglist.head )
return;
- if ( !paf_initialized(&trs.paf_state) or SEQ_GT(trs.paf_state.seq, trs.sos.seglist.head->i_seq) )
- paf_initialize(&trs.paf_state, trs.sos.seglist.head->i_seq);
+ if ( !paf.paf_initialized() or !SEQ_EQ(paf.seq_num, seglist.head->start_seq()) )
+ paf.paf_initialize(seglist.head->start_seq());
}
- uint32_t perform_partial_flush(TcpReassemblerState&, snort::Flow*, snort::Packet*&);
+ void reset_paf()
+ { paf.paf_reset(); }
+
+ void clear_paf()
+ { paf.paf_clear(); }
+
+ virtual FlushPolicy get_flush_policy() const = 0;
protected:
- TcpReassembler() = default;
-
- void add_reassembly_segment(
- TcpReassemblerState&, TcpSegmentDescriptor&, uint16_t len, uint32_t slide,
- uint32_t trunc, uint32_t seq, TcpSegmentNode* left) override;
-
- void dup_reassembly_segment(
- TcpReassemblerState&, TcpSegmentNode* left, TcpSegmentNode** retSeg) override;
- int delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*) override;
- virtual void insert_segment_in_empty_seglist(TcpReassemblerState&, TcpSegmentDescriptor&);
- virtual void insert_segment_in_seglist(TcpReassemblerState&, TcpSegmentDescriptor&);
- virtual uint32_t get_pending_segment_count(const TcpReassemblerState&, unsigned max) const;
- int trim_delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*, uint32_t flush_seq);
- void queue_reassembly_segment(TcpReassemblerState&, TcpSegmentNode* prev, TcpSegmentNode*);
- void init_overlap_editor(TcpReassemblerState&, TcpSegmentDescriptor&);
- bool is_segment_fasttrack
- (TcpReassemblerState&, TcpSegmentNode* tail, const TcpSegmentDescriptor&);
- void show_rebuilt_packet(const TcpReassemblerState&, snort::Packet*);
- int flush_data_segments(TcpReassemblerState&, uint32_t flush_len, snort::Packet* pdu);
- void prep_pdu(
- TcpReassemblerState&, snort::Flow*, snort::Packet*, uint32_t pkt_flags, snort::Packet*);
- snort::Packet* initialize_pdu(
- TcpReassemblerState&, snort::Packet*, uint32_t pkt_flags, struct timeval);
- int flush_to_seq(TcpReassemblerState&, uint32_t bytes, snort::Packet*, uint32_t pkt_flags);
- int do_zero_byte_flush(TcpReassemblerState&, snort::Packet*, uint32_t pkt_flags);
- uint32_t get_q_footprint(TcpReassemblerState&);
- uint32_t get_q_sequenced(TcpReassemblerState&);
- bool is_q_sequenced(TcpReassemblerState&);
- void final_flush(TcpReassemblerState&, snort::Packet*, uint32_t dir);
- uint32_t get_reverse_packet_dir(TcpReassemblerState&, const snort::Packet*);
- uint32_t get_forward_packet_dir(TcpReassemblerState&, const snort::Packet*);
- int32_t scan_data_pre_ack(TcpReassemblerState&, uint32_t*, snort::Packet*);
- void fallback(TcpStreamTracker&, bool server_side);
- int32_t scan_data_post_ack(TcpReassemblerState&, uint32_t* flags, snort::Packet*);
- void purge_to_seq(TcpReassemblerState&, uint32_t flush_seq);
- void purge_segments_left_of_hole(TcpReassemblerState&, const TcpSegmentNode*);
-
- bool next_no_gap(const TcpSegmentNode&);
- bool next_no_gap_c(const TcpSegmentNode&);
- bool next_acked_no_gap_c(const TcpSegmentNode&, const TcpReassemblerState&);
- bool fin_no_gap(const TcpSegmentNode&, const TcpReassemblerState&);
- bool fin_acked_no_gap(const TcpSegmentNode&, const TcpReassemblerState&);
- void update_next(TcpReassemblerState&, const TcpSegmentNode&);
- void update_skipped_bytes(uint32_t, TcpReassemblerState&);
- void check_first_segment_hole(TcpReassemblerState&);
- void update_rcv_nxt(TcpReassemblerState&, TcpSegmentNode&);
- bool has_seglist_hole(TcpReassemblerState&, TcpSegmentNode&, PAF_State&, uint32_t& total,
- uint32_t& flags);
- void skip_seglist_hole(TcpReassemblerState&, snort::Packet*, uint32_t flags,
- int32_t flush_amt);
-
- uint32_t perform_partial_flush(TcpReassemblerState&, snort::Packet*, uint32_t flushed = 0);
-
-private:
- bool final_flush_on_fin(const TcpReassemblerState&, int32_t flush_amt, snort::Packet*);
- bool flush_on_asymmetric_flow(const TcpReassemblerState &trs, uint32_t flushed, snort::Packet *p);
+ void show_rebuilt_packet(snort::Packet*);
+ int flush_data_segments(uint32_t flush_len, snort::Packet* pdu);
+ void prep_pdu(snort::Flow*, snort::Packet*, uint32_t pkt_flags, snort::Packet*);
+ snort::Packet* initialize_pdu(snort::Packet*, uint32_t pkt_flags, struct timeval);
+ int flush_to_seq(uint32_t bytes, snort::Packet*, uint32_t pkt_flags);
+ int do_zero_byte_flush(snort::Packet*, uint32_t pkt_flags);
+ uint32_t get_q_footprint();
+ uint32_t get_q_sequenced();
+ bool is_q_sequenced();
+ void final_flush(snort::Packet*, uint32_t dir);
+ bool splitter_finish(snort::Flow* flow);
+ void purge_to_seq(uint32_t flush_seq);
+
+ bool fin_no_gap(const TcpSegmentNode&);
+ bool fin_acked_no_gap(const TcpSegmentNode&);
+ void update_skipped_bytes(uint32_t);
+ void check_first_segment_hole();
+ uint32_t perform_partial_flush(snort::Packet*, uint32_t flushed = 0);
+ bool final_flush_on_fin(int32_t flush_amt, snort::Packet*, FinSeqNumStatus);
+ bool flush_on_asymmetric_flow(uint32_t flushed, snort::Packet *p);
+
+ ProtocolAwareFlusher paf;
+ TcpStreamTracker& tracker;
+ TcpReassemblySegments& seglist;
+ snort::StreamSplitter* splitter = nullptr;
+
+ snort::Packet* last_pdu = nullptr;
+ uint8_t ignore_dir = 0;
+ uint8_t packet_dir = 0;
+ bool server_side = true;
+ bool splitter_finish_flag = false;
+};
+
+class TcpReassemblerIgnore : public TcpReassembler
+{
+public:
+ TcpReassemblerIgnore(TcpStreamTracker& trk, TcpReassemblySegments& sl)
+ : TcpReassembler(trk, sl)
+ { }
+
+ void init(bool, snort::StreamSplitter*) override
+ { }
+
+ int eval_flush_policy_on_ack(snort::Packet*) override
+ { return 0; }
+
+ int eval_flush_policy_on_data(snort::Packet*) override
+ { return 0; }
+
+ int flush_stream(snort::Packet*, uint32_t, bool) override
+ { return 0; }
+
+ FlushPolicy get_flush_policy() const override
+ { return STREAM_FLPOLICY_IGNORE; }
};
#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembler_ids.cc author davis mcpherson <davmcphe@cisco.com>
+// Created on: Jul 31, 2015
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "tcp_reassembler_ids.h"
+
+#include <cassert>
+
+#include "detection/detection_engine.h"
+#include "log/log.h"
+#include "main/analyzer.h"
+#include "packet_io/active.h"
+#include "profiler/profiler.h"
+#include "protocols/packet_manager.h"
+#include "time/packet_time.h"
+
+#include "tcp_module.h"
+#include "tcp_normalizers.h"
+#include "tcp_segment_node.h"
+#include "tcp_session.h"
+#include "tcp_stream_tracker.h"
+
+using namespace snort;
+
+bool TcpReassemblerIds::has_seglist_hole(TcpSegmentNode& tsn, uint32_t& total, uint32_t& flags)
+{
+ if ( !tsn.prev or SEQ_GEQ(tsn.prev->scan_seq() + tsn.prev->unscanned(), tsn.scan_seq())
+ or SEQ_GEQ(tsn.scan_seq(), tracker.r_win_base) )
+ {
+ check_first_segment_hole();
+ return false;
+ }
+
+ // safety - prevent seq + total < seq
+ if ( total > 0x7FFFFFFF )
+ total = 0x7FFFFFFF;
+
+ if ( !paf.tot )
+ flags |= PKT_PDU_HEAD;
+
+ paf.state = StreamSplitter::SKIP;
+ return true;
+}
+
+void TcpReassemblerIds::skip_seglist_hole(Packet* p, uint32_t flags, int32_t flush_amt)
+{
+ if ( is_splitter_paf() )
+ {
+ if ( flush_amt > 0 )
+ update_skipped_bytes(flush_amt);
+ tracker.fallback();
+ }
+ else
+ {
+ if ( flush_amt > 0 )
+ flush_to_seq(flush_amt, p, flags);
+ paf.state = StreamSplitter::START;
+ }
+
+ if ( seglist.head )
+ {
+ if ( flush_amt > 0 )
+ purge_to_seq(seglist.seglist_base_seq + flush_amt);
+ seglist.seglist_base_seq = seglist.head->scan_seq();
+ }
+ else
+ seglist.seglist_base_seq = tracker.r_win_base; // FIXIT-H - do we need to set rcv_nxt here?
+
+ seglist.cur_rseg = seglist.head;
+ tracker.set_order(TcpStreamTracker::OUT_OF_SEQUENCE);
+}
+
+// iterate over seglist and scan all new acked bytes
+// - new means not yet scanned
+// - must use seglist data (not packet) since this packet may plug a
+// hole and enable paf scanning of following segments
+// - if we reach a flush point
+// - return bytes to flush if data available (must be acked)
+// - return zero if not yet received or received but not acked
+// - if we reach a skip point
+// - jump ahead and resume scanning any available data
+// - must stop if we reach a gap
+// - one segment may lead to multiple checks since
+// it may contain multiple encapsulated PDUs
+// - if we partially scan a segment we must save state so we
+// know where we left off and can resume scanning the remainder
+int32_t TcpReassemblerIds::scan_data_post_ack(uint32_t* flags, Packet* p)
+{
+ assert(seglist.session->flow == p->flow);
+
+ int32_t ret_val = FINAL_FLUSH_HOLD;
+
+ if ( !seglist.cur_sseg || SEQ_GEQ(seglist.seglist_base_seq, tracker.r_win_base) )
+ return ret_val ;
+
+ if ( !seglist.cur_rseg )
+ seglist.cur_rseg = seglist.cur_sseg;
+
+ uint32_t total = 0;
+ TcpSegmentNode* tsn = seglist.cur_sseg;
+ if ( paf.paf_initialized() )
+ {
+ uint32_t end_seq = tsn->scan_seq() + tsn->unscanned();
+ if ( SEQ_EQ(end_seq, paf.paf_position()) )
+ {
+ total = end_seq - seglist.seglist_base_seq;
+ tsn = tsn->next;
+ }
+ else
+ total = tsn->scan_seq() - seglist.cur_rseg->scan_seq();
+ }
+
+ ret_val = FINAL_FLUSH_OK;
+ while (tsn && *flags && SEQ_LT(tsn->scan_seq(), tracker.r_win_base))
+ {
+ // only flush acked data that fits in pdu reassembly buffer...
+ uint32_t end = tsn->scan_seq() + tsn->unscanned();
+ uint32_t flush_len;
+ int32_t flush_pt;
+
+ if ( SEQ_GT(end, tracker.r_win_base))
+ flush_len = tracker.r_win_base - tsn->scan_seq();
+ else
+ flush_len = tsn->unscanned();
+
+ if ( tsn->next_acked_no_gap(tracker.r_win_base) )
+ *flags |= PKT_MORE_TO_FLUSH;
+ else
+ *flags &= ~PKT_MORE_TO_FLUSH;
+
+ if ( has_seglist_hole(*tsn, total, *flags) )
+ flush_pt = total;
+ else
+ {
+ total += flush_len;
+ flush_pt = paf.paf_check(p, tsn->paf_data(), flush_len, total, tsn->scan_seq(), flags);
+ }
+
+ // Get splitter from tracker as paf check may change it.
+ seglist.cur_sseg = tsn;
+
+ if ( flush_pt >= 0 )
+ {
+ seglist.seglist_base_seq = seglist.cur_rseg->scan_seq();
+ return flush_pt;
+ }
+
+ if (flush_len < tsn->unscanned() || (splitter->is_paf() and !tsn->next_no_gap()) ||
+ (paf.state == StreamSplitter::STOP))
+ {
+ if ( !(tsn->next_no_gap() || fin_acked_no_gap(*tsn)) )
+ ret_val = FINAL_FLUSH_HOLD;
+ break;
+ }
+
+ tsn = tsn->next;
+ }
+
+ return ret_val;
+}
+
+int TcpReassemblerIds::eval_flush_policy_on_ack(Packet* p)
+{
+ uint32_t flushed = 0;
+ int32_t flush_amt;
+ uint32_t flags;
+
+ last_pdu = nullptr;
+
+ do
+ {
+ flags = packet_dir;
+ flush_amt = scan_data_post_ack(&flags, p);
+ if ( flush_amt <= 0 or paf.state == StreamSplitter::SKIP )
+ break;
+
+ // for consistency with other cases, should return total
+ // but that breaks flushing pipelined pdus
+ flushed += flush_to_seq(flush_amt, p, flags);
+ assert( flushed );
+
+ // ideally we would purge just once after this loop but that throws off base
+ if ( seglist.head )
+ purge_to_seq(seglist.seglist_base_seq);
+ } while ( seglist.head and !p->flow->is_inspection_disabled() );
+
+ if ( (paf.state == StreamSplitter::ABORT) && is_splitter_paf() )
+ {
+ tracker.fallback();
+ return eval_flush_policy_on_ack(p);
+ }
+ else if ( paf.state == StreamSplitter::SKIP )
+ {
+ skip_seglist_hole(p, flags, flush_amt);
+ return eval_flush_policy_on_ack(p);
+ }
+ else if ( final_flush_on_fin(flush_amt, p, FIN_WITH_SEQ_ACKED) )
+ finish_and_final_flush(p->flow, true, p);
+
+ return flushed;
+}
+
+int TcpReassemblerIds::eval_flush_policy_on_data(Packet* p)
+{
+ uint32_t flushed = 0;
+
+ if ( !seglist.head )
+ return flushed;
+
+ if ( tracker.is_retransmit_of_held_packet(p) )
+ flushed = perform_partial_flush(p, flushed);
+
+ if ( flush_on_asymmetric_flow(flushed, p) )
+ {
+ purge_to_seq(seglist.head->start_seq() + flushed);
+ tracker.r_win_base = seglist.seglist_base_seq;
+ tcpStats.flush_on_asymmetric_flow++;
+ }
+
+ return flushed;
+}
+
+int TcpReassemblerIds::flush_stream(Packet* p, uint32_t dir, bool final_flush)
+{
+ if ( seglist.session->flow->two_way_traffic()
+ or (tracker.get_tcp_state() == TcpStreamTracker::TCP_MID_STREAM_RECV) )
+ {
+ uint32_t bytes = get_q_footprint(); // num bytes in post-ack mode
+ if ( bytes )
+ return flush_to_seq(bytes, p, dir);
+ }
+
+ if ( final_flush )
+ return do_zero_byte_flush(p, dir);
+
+ return 0;
+}
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembler_ids.h author davis mcpherson <davmcphe@cisco.com>
+// Created on: Jul 31, 2015
+
+#ifndef TCP_REASSEMBLER_IDS_H
+#define TCP_REASSEMBLER_IDS_H
+
+#include <cstdint>
+
+#include "protocols/packet.h"
+#include "stream/stream.h"
+
+#include "tcp_reassembler.h"
+#include "tcp_reassembly_segments.h"
+
+class TcpSegmentDescriptor;
+class TcpSegmentNode;
+
+class TcpReassemblerIds : public TcpReassembler
+{
+public:
+
+
+ TcpReassemblerIds(TcpStreamTracker& trk, TcpReassemblySegments& sl)
+ : TcpReassembler(trk, sl)
+ { }
+
+ ~TcpReassemblerIds() override
+ { }
+
+ int eval_flush_policy_on_ack(snort::Packet*) override;
+ int eval_flush_policy_on_data(snort::Packet*) override;
+ int flush_stream(snort::Packet*, uint32_t dir, bool final_flush = false) override;
+
+ FlushPolicy get_flush_policy() const override
+ { return STREAM_FLPOLICY_ON_ACK; }
+
+private:
+ int32_t scan_data_post_ack(uint32_t* flags, snort::Packet*);
+ bool has_seglist_hole(TcpSegmentNode&, uint32_t& total, uint32_t& flags);
+ void skip_seglist_hole(snort::Packet*, uint32_t flags, int32_t flush_amt);
+};
+
+#endif
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembler_ips.cc author davis mcpherson <davmcphe@cisco.com>
+// Created on: Jul 31, 2015
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "tcp_reassembler_ips.h"
+
+#include <cassert>
+
+#include "detection/detection_engine.h"
+#include "log/log.h"
+#include "main/analyzer.h"
+#include "packet_io/active.h"
+#include "profiler/profiler.h"
+#include "protocols/packet_manager.h"
+#include "time/packet_time.h"
+
+#include "tcp_module.h"
+#include "tcp_normalizers.h"
+#include "tcp_segment_node.h"
+#include "tcp_session.h"
+#include "tcp_stream_tracker.h"
+
+using namespace snort;
+
+// see scan_data_post_ack() for details
+// the key difference is that we operate on forward moving data
+// because we don't wait until it is acknowledged
+int32_t TcpReassemblerIps::scan_data_pre_ack(uint32_t* flags, Packet* p)
+{
+ assert(seglist.session->flow == p->flow);
+
+ int32_t ret_val = FINAL_FLUSH_HOLD;
+
+ if ( SEQ_GT(seglist.head->scan_seq(), seglist.seglist_base_seq) )
+ return ret_val;
+
+ if ( !seglist.cur_rseg )
+ seglist.cur_rseg = seglist.cur_sseg;
+
+ if ( !is_q_sequenced() )
+ return ret_val;
+
+ TcpSegmentNode* tsn = seglist.cur_sseg;
+ uint32_t total = tsn->scan_seq() - seglist.seglist_base_seq;
+
+ ret_val = FINAL_FLUSH_OK;
+ while ( tsn && *flags )
+ {
+ total += tsn->unscanned();
+
+ uint32_t end = tsn->scan_seq() + tsn->unscanned();
+ uint32_t pos = paf.paf_position();
+
+ if ( paf.paf_initialized() && SEQ_LEQ(end, pos) )
+ {
+ if ( !tsn->next_no_gap() )
+ {
+ ret_val = FINAL_FLUSH_HOLD;
+ break;
+ }
+
+ tsn = tsn->next;
+ continue;
+ }
+
+ if ( tsn->next_no_gap() )
+ *flags |= PKT_MORE_TO_FLUSH;
+ else
+ *flags &= ~PKT_MORE_TO_FLUSH;
+ int32_t flush_pt = paf.paf_check(p, tsn->paf_data(), tsn->unscanned(),
+ total, tsn->scan_seq(), flags);
+
+ if (flush_pt >= 0)
+ {
+ seglist.cur_sseg = tsn;
+ return flush_pt;
+ }
+
+ if (!tsn->next_no_gap() || (paf.state == StreamSplitter::STOP))
+ {
+ if ( !(tsn->next_no_gap() || fin_no_gap(*tsn)) )
+ ret_val = FINAL_FLUSH_HOLD;
+ break;
+ }
+
+ tsn = tsn->next;
+ }
+
+ seglist.cur_sseg = tsn;
+
+ return ret_val;
+}
+
+int TcpReassemblerIps::eval_flush_policy_on_ack(Packet*)
+{
+ purge_flushed_ackd();
+
+ return 0;
+}
+
+int TcpReassemblerIps::eval_flush_policy_on_data(Packet* p)
+{
+ uint32_t flushed = 0;
+ last_pdu = nullptr;
+
+ if ( seglist.head )
+ {
+ uint32_t flags;
+ int32_t flush_amt;
+ do
+ {
+ flags = packet_dir;
+ flush_amt = scan_data_pre_ack(&flags, p);
+ if ( flush_amt <= 0 )
+ break;
+
+ flushed += flush_to_seq(flush_amt, p, flags);
+ } while ( seglist.head and !p->flow->is_inspection_disabled() );
+
+ if ( (paf.state == StreamSplitter::ABORT) && is_splitter_paf() )
+ {
+ tracker.fallback();
+ return eval_flush_policy_on_data(p);
+ }
+ else if ( final_flush_on_fin(flush_amt, p, FIN_WITH_SEQ_SEEN) )
+ finish_and_final_flush(p->flow, true, p);
+ }
+
+ if ( !seglist.head )
+ return flushed;
+
+ if ( tracker.is_retransmit_of_held_packet(p) )
+ flushed = perform_partial_flush(p, flushed);
+
+ if ( flush_on_asymmetric_flow(flushed, p) )
+ {
+ purge_to_seq(seglist.head->start_seq() + flushed);
+ tracker.r_win_base = seglist.seglist_base_seq;
+ tcpStats.flush_on_asymmetric_flow++;
+ }
+
+ return flushed;
+}
+
+int TcpReassemblerIps::flush_stream(Packet* p, uint32_t dir, bool final_flush)
+{
+ if ( seglist.session->flow->two_way_traffic()
+ or (tracker.get_tcp_state() == TcpStreamTracker::TCP_MID_STREAM_RECV) )
+ {
+ uint32_t bytes = get_q_sequenced(); // num bytes in pre-ack mode
+ if ( bytes )
+ return flush_to_seq(bytes, p, dir);
+ }
+
+ if ( final_flush )
+ return do_zero_byte_flush(p, dir);
+
+ return 0;
+}
+
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2024-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembler_ips.h author davis mcpherson <davmcphe@cisco.com>
+// Created on: Jul 31, 2015
+
+#ifndef TCP_REASSEMBLER_IPS_H
+#define TCP_REASSEMBLER_IPS_H
+
+#include <cstdint>
+
+#include "flow/flow.h"
+#include "protocols/packet.h"
+#include "stream/stream.h"
+
+#include "tcp_reassembler.h"
+#include "tcp_reassembly_segments.h"
+
+class TcpSegmentDescriptor;
+class TcpSegmentNode;
+
+class TcpReassemblerIps : public TcpReassembler
+{
+public:
+ TcpReassemblerIps(TcpStreamTracker& trk, TcpReassemblySegments& sl)
+ : TcpReassembler(trk, sl)
+ { }
+
+ ~TcpReassemblerIps() override
+ { }
+
+ int eval_flush_policy_on_ack(snort::Packet*) override;
+ int eval_flush_policy_on_data(snort::Packet*) override;
+ int flush_stream(snort::Packet*, uint32_t dir, bool final_flush = false) override;
+
+ FlushPolicy get_flush_policy() const override
+ { return STREAM_FLPOLICY_ON_DATA; }
+
+private:
+ int32_t scan_data_pre_ack(uint32_t*, snort::Packet*);
+};
+
+#endif
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// tcp_reassemblers.cc author davis mcpherson <davmcphe@cisco.com>
-// Created on: Oct 9, 2015
-
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
-#include "tcp_reassemblers.h"
-
-#include "tcp_defs.h"
-#include "tcp_stream_tracker.h"
-
-class TcpReassemblerFirst : public TcpReassembler
-{
-public:
- TcpReassemblerFirst() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os5(trs); }
-};
-
-class TcpReassemblerLast : public TcpReassembler
-{
-public:
- TcpReassemblerLast() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_last(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os4(trs); }
-};
-
-class TcpReassemblerLinux : public TcpReassembler
-{
-public:
- TcpReassemblerLinux() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os2(trs); }
-};
-
-class TcpReassemblerOldLinux : public TcpReassembler
-{
-public:
- TcpReassemblerOldLinux() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os4(trs); }
-};
-
-class TcpReassemblerBSD : public TcpReassembler
-{
-public:
- TcpReassemblerBSD() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os1(trs); }
-};
-
-class TcpReassemblerMacOS : public TcpReassembler
-{
-public:
- TcpReassemblerMacOS() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os1(trs); }
-};
-
-class TcpReassemblerSolaris : public TcpReassembler
-{
-public:
- TcpReassemblerSolaris() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_trim_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os3(trs); }
-};
-
-class TcpReassemblerIrix : public TcpReassembler
-{
-public:
- TcpReassemblerIrix() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os2(trs); }
-};
-
-class TcpReassemblerHpux11 : public TcpReassembler
-{
-public:
- TcpReassemblerHpux11() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_trim_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os3(trs); }
-};
-
-class TcpReassemblerHpux10 : public TcpReassembler
-{
-public:
- TcpReassemblerHpux10() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os2(trs); }
-};
-
-class TcpReassemblerWindows : public TcpReassembler
-{
-public:
- TcpReassemblerWindows() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os1(trs); }
-};
-
-class TcpReassemblerWindows2K3 : public TcpReassembler
-{
-public:
- TcpReassemblerWindows2K3() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_existing(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os1(trs); }
-};
-
-class TcpReassemblerVista : public TcpReassembler
-{
-public:
- TcpReassemblerVista() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os5 (trs); }
-};
-
-class TcpReassemblerProxy : public TcpReassemblerFirst
-{
-public:
- TcpReassemblerProxy() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os5(trs); }
-};
-
-class TcpReassemblerMissed3whs : public TcpReassemblerFirst
-{
-public:
- TcpReassemblerMissed3whs() = default;
-
-private:
- void insert_left_overlap(TcpReassemblerState& trs) override
- { left_overlap_keep_first(trs); }
-
- void insert_right_overlap(TcpReassemblerState& trs) override
- { right_overlap_truncate_new(trs); }
-
- void insert_full_overlap(TcpReassemblerState& trs) override
- { full_right_overlap_os5(trs); }
-};
-
-void TcpReassemblerPolicy::init(TcpSession* ssn, TcpStreamTracker* trk, StreamPolicy pol, bool server)
-{
- trs.sos.init_sos(ssn, pol);
- setup_paf();
- trs.server_side = server;
- trs.tracker = trk;
-
- if ( trs.server_side )
- {
- trs.ignore_dir = SSN_DIR_FROM_CLIENT;
- trs.packet_dir = PKT_FROM_CLIENT;
- }
- else
- {
- trs.ignore_dir = SSN_DIR_FROM_SERVER;
- trs.packet_dir = PKT_FROM_SERVER;
- }
-
- trs.flush_count = 0;
- trs.xtradata_mask = 0;
- trs.alerts.clear();
-
- reassembler = TcpReassemblerFactory::get_instance(pol);
-}
-
-void TcpReassemblerPolicy::reset()
-{ init(nullptr, nullptr, StreamPolicy::OS_DEFAULT, false); }
-
-TcpReassembler* TcpReassemblerFactory::reassemblers[StreamPolicy::OS_END_OF_LIST];
-
-void TcpReassemblerFactory::initialize()
-{
- reassemblers[StreamPolicy::OS_FIRST] = new TcpReassemblerFirst;
- reassemblers[StreamPolicy::OS_LAST] = new TcpReassemblerLast;
- reassemblers[StreamPolicy::OS_LINUX] = new TcpReassemblerLinux;
- reassemblers[StreamPolicy::OS_OLD_LINUX] = new TcpReassemblerOldLinux;
- reassemblers[StreamPolicy::OS_BSD] = new TcpReassemblerBSD;
- reassemblers[StreamPolicy::OS_MACOS] = new TcpReassemblerMacOS;
- reassemblers[StreamPolicy::OS_SOLARIS] = new TcpReassemblerSolaris;
- reassemblers[StreamPolicy::OS_IRIX] = new TcpReassemblerIrix;
- reassemblers[StreamPolicy::OS_HPUX11] = new TcpReassemblerHpux11;
- reassemblers[StreamPolicy::OS_HPUX10] = new TcpReassemblerHpux10;
- reassemblers[StreamPolicy::OS_WINDOWS] = new TcpReassemblerWindows;
- reassemblers[StreamPolicy::OS_WINDOWS2K3] = new TcpReassemblerWindows2K3;
- reassemblers[StreamPolicy::OS_VISTA] = new TcpReassemblerVista;
- reassemblers[StreamPolicy::OS_PROXY] = new TcpReassemblerProxy;
- reassemblers[StreamPolicy::MISSED_3WHS] = new TcpReassemblerMissed3whs;
-}
-
-void TcpReassemblerFactory::term()
-{
- for ( auto sp = StreamPolicy::OS_FIRST; sp < StreamPolicy::OS_END_OF_LIST; sp++ )
- delete reassemblers[sp];
-}
-
-TcpReassembler* TcpReassemblerFactory::get_instance(StreamPolicy os_policy)
-{
- assert( os_policy < StreamPolicy::OS_END_OF_LIST );
- return reassemblers[os_policy];
-}
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// tcp_reassemblers.h author davis mcpherson <davmcphe@cisco.com>
-// Created on: Oct 9, 2015
-
-#ifndef TCP_REASSEMBLERS_H
-#define TCP_REASSEMBLERS_H
-
-#include "tcp_reassembler.h"
-
-class TcpReassemblerFactory
-{
-public:
- static void initialize();
- static void term();
- static TcpReassembler* get_instance(StreamPolicy);
-
-private:
- TcpReassemblerFactory() = delete;
-
- static TcpReassembler* reassemblers[StreamPolicy::OS_END_OF_LIST];
-};
-
-class TcpReassemblerPolicy
-{
-public:
- TcpReassemblerPolicy() = default;
- ~TcpReassemblerPolicy() = default;
-
- void init(TcpSession* ssn, TcpStreamTracker* trk, StreamPolicy pol, bool server);
- void reset();
-
- void queue_packet_for_reassembly(TcpSegmentDescriptor& tsd)
- { reassembler->queue_packet_for_reassembly(trs, tsd); }
-
- bool add_alert(uint32_t gid, uint32_t sid)
- { return reassembler->add_alert(trs, gid, sid); }
-
- bool check_alerted(uint32_t gid, uint32_t sid)
- { return reassembler->check_alerted(trs, gid, sid); }
-
- int update_alert(uint32_t gid, uint32_t sid, uint32_t event_id, uint32_t event_second)
- { return reassembler->update_alert(trs, gid, sid, event_id, event_second); }
-
- void purge_alerts()
- { reassembler->purge_alerts(trs); }
-
- void purge_segment_list()
- { reassembler->purge_segment_list(trs); }
-
- void purge_flushed_ackd()
- { return reassembler->purge_flushed_ackd(trs); }
-
- int flush_stream(snort::Packet* p, uint32_t dir, bool final_flush = false)
- { return reassembler->flush_stream(trs, p, dir, final_flush); }
-
- void finish_and_final_flush(snort::Flow* flow, bool clear, snort::Packet* p)
- { reassembler->finish_and_final_flush(trs, flow, clear, p); }
-
- void flush_queued_segments(snort::Flow* flow, bool clear, const snort::Packet* p = nullptr)
- { reassembler->flush_queued_segments(trs, flow, clear, p); }
-
- bool is_segment_pending_flush() const
- { return reassembler->is_segment_pending_flush(trs); }
-
- void skip_midstream_pickup_seglist_hole(TcpSegmentDescriptor& tsd)
- { reassembler->skip_midstream_pickup_seglist_hole(trs, tsd); }
-
- void reset_asymmetric_flow_reassembly()
- { reassembler->reset_asymmetric_flow_reassembly(trs); }
-
- void initialize_paf()
- { reassembler->initialize_paf(trs); }
-
- int flush_on_data_policy(snort::Packet* p)
- { return reassembler->flush_on_data_policy(trs, p); }
-
- int flush_on_ack_policy(snort::Packet* p)
- { return reassembler->flush_on_ack_policy(trs, p); }
-
- void set_seglist_base_seq(uint32_t seglist_base_seq)
- { trs.sos.seglist_base_seq = seglist_base_seq; }
-
- uint32_t get_seglist_base_seq() const
- { return trs.sos.seglist_base_seq; }
-
- void set_xtradata_mask(uint32_t xtradata_mask)
- { trs.xtradata_mask = xtradata_mask; }
-
- uint32_t get_xtradata_mask() const
- { return trs.xtradata_mask; }
-
- bool data_was_queued() const
- { return trs.sos.total_bytes_queued > 0; }
-
- uint32_t get_seg_count() const
- { return trs.sos.seg_count; }
-
- uint32_t get_seg_bytes_total() const
- { return trs.sos.seg_bytes_total; }
-
- uint32_t get_overlap_count() const
- { return trs.sos.overlap_count; }
-
- void set_overlap_count(uint32_t overlap_count)
- { trs.sos.overlap_count = overlap_count; }
-
- uint32_t get_flush_count() const
- { return trs.flush_count; }
-
- uint32_t get_seg_bytes_logical() const
- { return trs.sos.seg_bytes_logical; }
-
- StreamPolicy get_reassembly_policy() const
- { return trs.sos.reassembly_policy; }
-
- void set_norm_mode_test()
- { trs.sos.tcp_ips_data = NORM_MODE_TEST; }
-
- bool segment_within_seglist_window(TcpSegmentDescriptor& tsd)
- { return reassembler->segment_within_seglist_window(trs, tsd); }
-
- uint32_t perform_partial_flush(snort::Flow* flow, snort::Packet*& p)
- { return reassembler->perform_partial_flush(trs, flow, p); }
-
- void reset_paf()
- { paf_reset(&trs.paf_state); }
-
- void clear_paf()
- { paf_clear(&trs.paf_state); }
-
- void setup_paf()
- {
- paf_setup(&trs.paf_state);
- if ( trs.sos.seglist.cur_rseg )
- trs.sos.seglist.cur_sseg = trs.sos.seglist.cur_rseg;
- else
- trs.sos.seglist.cur_sseg = trs.sos.seglist.head;
- }
-
-private:
- TcpReassembler* reassembler = nullptr;
- TcpReassemblerState trs = {};
- friend inline void TraceSegments(const TcpReassemblerPolicy&, const snort::Packet* p);
-};
-#endif
-
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembly_segments.cc author davis mcpherson <davmcphe@cisco.com>
+// Created on: Oct 9, 2015
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "tcp_reassembly_segments.h"
+
+#include "log/messages.h"
+#include "packet_io/packet_tracer.h"
+#include "protocols/tcp.h"
+
+#include "tcp_module.h"
+#include "tcp_overlap_resolver.h"
+#include "tcp_segment_descriptor.h"
+#include "tcp_segment_node.h"
+#include "tcp_session.h"
+#include "tcp_stream_tracker.h"
+#include "tcp_overlap_resolver.h"
+
+using namespace snort;
+
+TcpReassemblySegments::~TcpReassemblySegments()
+{
+ delete tos;
+}
+
+void TcpReassemblySegments::init(TcpSession* ssn, TcpStreamTracker* trk, StreamPolicy pol)
+{
+ session = ssn;
+ tracker = trk;
+ overlap_resolver = TcpOverlapResolverFactory::get_instance(pol);
+ if ( tos )
+ delete tos;
+ tos = new TcpOverlapState(*this);
+}
+
+void TcpReassemblySegments::reset()
+{
+ purge();
+ seglist_base_seq = 0;
+}
+
+void TcpReassemblySegments::update_next(TcpSegmentNode* tsn)
+{
+ cur_rseg = tsn->next_no_gap() ? tsn->next : nullptr;
+}
+
+bool TcpReassemblySegments::is_segment_pending_flush() const
+{
+ return ( get_pending_segment_count(1) > 0 );
+}
+
+uint32_t TcpReassemblySegments::get_pending_segment_count(const unsigned max) const
+{
+ uint32_t n = seg_count - flush_count;
+
+ if ( !n || max == 1 )
+ return n;
+
+ n = 0;
+ const TcpSegmentNode* tsn = head;
+ while ( tsn )
+ {
+ if ( tsn->unscanned() && SEQ_LT(tsn->scan_seq(), tracker->r_win_base) )
+ n++;
+
+ if ( max && n == max )
+ return n;
+
+ tsn = tsn->next;
+ }
+
+ return n;
+}
+
+bool TcpReassemblySegments::segment_within_seglist_window(TcpSegmentDescriptor& tsd)
+{
+ if ( !head )
+ return true;
+
+ // Left side
+ uint32_t start;
+ if ( SEQ_LT(seglist_base_seq, head->start_seq()) )
+ start = seglist_base_seq;
+ else
+ start = head->seq;
+
+ if ( SEQ_LEQ(tsd.get_end_seq(), start) )
+ return false;
+
+ // Right side
+ if ( SEQ_GEQ(tsd.get_seq(), tail->next_seq()) )
+ return false;
+
+ return true;
+}
+
+void TcpReassemblySegments::queue_reassembly_segment(TcpSegmentDescriptor& tsd)
+{
+ if ( seg_count == 0 )
+ {
+ insert_segment_in_empty_seglist(tsd);
+ }
+ else if ( SEQ_GT(tracker->r_win_base, tsd.get_seq() ) )
+ {
+ const int32_t offset = tracker->r_win_base - tsd.get_seq();
+
+ if ( offset < tsd.get_len() )
+ {
+ tsd.slide_segment_in_rcv_window(offset);
+ insert_segment_in_seglist(tsd);
+ tsd.slide_segment_in_rcv_window(-offset);
+ }
+ }
+ else
+ insert_segment_in_seglist(tsd);
+}
+
+void TcpReassemblySegments::insert_segment_in_empty_seglist(TcpSegmentDescriptor& tsd)
+{
+ uint32_t overlap = 0;
+
+ if ( SEQ_GT(seglist_base_seq, tsd.get_seq()) )
+ {
+ overlap = seglist_base_seq - tsd.get_seq();
+ if ( overlap >= tsd.get_len() )
+ return;
+ }
+
+ add_reassembly_segment(tsd, tsd.get_len(), overlap, 0, tsd.get_seq(), nullptr);
+}
+
+bool TcpReassemblySegments::is_segment_fasttrack(TcpSegmentNode* tail, const TcpSegmentDescriptor& tsd)
+{
+ if ( SEQ_EQ(tsd.get_seq(), tail->next_seq()) )
+ return true;
+
+ return false;
+}
+
+void TcpReassemblySegments::insert_segment_in_seglist(TcpSegmentDescriptor& tsd)
+{
+ // NORM fast tracks are in sequence - no norms
+ if ( tail && is_segment_fasttrack(tail, tsd) )
+ {
+ /* segment fit cleanly at the end of the segment list */
+ add_reassembly_segment(tsd, tsd.get_len(), 0, 0, tsd.get_seq(), tail);
+ return;
+ }
+
+ tos->init(tsd);
+ overlap_resolver->eval_left(*tos);
+ overlap_resolver->eval_right(*tos);
+
+ if ( tos->keep_segment )
+ {
+ // FIXIT-L - is this skipping the add if the segment is first and already scan
+ if ( !tos->left and tos->right and tracker->reassembler->segment_already_scanned(tsd.get_seq()) )
+ {
+ return;
+ }
+
+ add_reassembly_segment(tsd, tos->len, tos->slide, tos->trunc_len, tos->seq, tos->left);
+ }
+}
+
+void TcpReassemblySegments::insert_segment_data(TcpSegmentNode* prev, TcpSegmentNode* tsn)
+{
+ insert(prev, tsn);
+
+ if ( !cur_sseg )
+ cur_sseg = tsn;
+ else if ( SEQ_LT(tsn->scan_seq(), cur_sseg->scan_seq()) )
+ {
+ cur_sseg = tsn;
+ if ( SEQ_LT(tsn->scan_seq(), seglist_base_seq) )
+ seglist_base_seq = tsn->scan_seq();
+
+ if ( cur_rseg && SEQ_LT(tsn->scan_seq(), cur_rseg->scan_seq()) )
+ cur_rseg = tsn;
+ }
+
+ // FIXIT-M - increment seg_count here?
+ seg_bytes_total += tsn->size;
+ total_segs_queued++;
+ tcpStats.segs_queued++;
+
+ if ( seg_count > tcpStats.max_segs )
+ tcpStats.max_segs = seg_count;
+
+ if ( seg_bytes_total > tcpStats.max_bytes )
+ tcpStats.max_bytes = seg_bytes_total;
+}
+
+void TcpReassemblySegments::add_reassembly_segment(TcpSegmentDescriptor& tsd, uint16_t len,
+ uint32_t slide, uint32_t trunc_len, uint32_t seq, TcpSegmentNode* left)
+{
+ const int32_t new_size = len - slide - trunc_len;
+ assert(new_size >= 0);
+
+ // if trimming will delete all data, don't insert this segment in the queue
+ if ( new_size <= 0 )
+ {
+ tcpStats.payload_fully_trimmed++;
+ tracker->normalizer.trim_win_payload(tsd);
+ return;
+ }
+
+ // FIXIT-L don't allocate overlapped part
+ TcpSegmentNode* tsn = TcpSegmentNode::init(tsd);
+
+ tsn->seq = seq;
+ tsn->offset = slide;
+ tsn->length = (uint16_t)new_size;
+ tsn->cursor = 0;
+ tsn->ts = tsd.get_timestamp();
+
+ // FIXIT-M the urgent ptr handling is broken... urg_offset could be set here but currently
+ // not actually referenced anywhere else. In 2.9.7 the FlushStream function did reference
+ // this field but that code has been lost... urg ptr handling needs to be reviewed and fixed
+ // tsn->urg_offset = tracker->normalizer.set_urg_offset(tsd.get_tcph(), tsd.get_seg_len());
+
+ insert_segment_data(left, tsn);
+
+ seg_bytes_logical += tsn->length;
+ total_bytes_queued += tsn->size;
+ tsd.set_packet_flags(PKT_STREAM_INSERT);
+
+ if( tsd.is_packet_inorder()
+ or (SEQ_LEQ(tsn->start_seq(), tracker->get_rcv_nxt())
+ and SEQ_GEQ(tsn->next_seq(), tracker->get_rcv_nxt())) )
+ advance_rcv_nxt(tsn);
+}
+
+void TcpReassemblySegments::dup_reassembly_segment(TcpSegmentNode* left, TcpSegmentNode** retSeg)
+{
+ TcpSegmentNode* tsn = TcpSegmentNode::init(*left);
+ tcpStats.segs_split++;
+
+ // twiddle the values for overlaps
+ tsn->cursor = left->cursor;
+ tsn->seq = left->seq;
+ insert_segment_data(left, tsn);
+
+ *retSeg = tsn;
+}
+
+int TcpReassemblySegments::delete_reassembly_segment(TcpSegmentNode* tsn)
+{
+ int ret;
+ assert(tsn);
+
+ remove(tsn);
+ seg_bytes_total -= tsn->size;
+ seg_bytes_logical -= tsn->length;
+ ret = tsn->length;
+
+ if ( !tsn->unscanned() )
+ {
+ tcpStats.segs_used++;
+ flush_count--;
+ }
+
+ if ( cur_sseg == tsn )
+ cur_sseg = tsn->next;
+
+ if ( cur_rseg == tsn )
+ update_next(tsn);
+
+ tsn->term();
+
+ return ret;
+}
+
+void TcpReassemblySegments::purge_flushed_segments(uint32_t flush_seq)
+{
+ assert( head );
+ uint32_t last_ts = 0;
+
+ TcpSegmentNode* tsn = head;
+ while ( tsn && SEQ_LT(tsn->start_seq(), flush_seq))
+ {
+ if ( tsn->unscanned() )
+ break;
+
+ TcpSegmentNode* dump_me = tsn;
+ tsn = tsn->next;
+ if (dump_me->ts > last_ts)
+ last_ts = dump_me->ts;
+
+ delete_reassembly_segment(dump_me);
+ }
+
+ if ( tsn and SEQ_LT(tracker->rcv_nxt, tsn->next_seq()) )
+ advance_rcv_nxt(tsn);
+
+ /* Update the "last" time stamp seen from the other side
+ * to be the most recent timestamp (largest) that was removed
+ * from the queue. This will ensure that as we go forward,
+ * last timestamp is the highest one that we had stored and
+ * purged and handle the case when packets arrive out of order,
+ * such as:
+ * P1: seq 10, length 10, timestamp 10
+ * P3: seq 30, length 10, timestamp 30
+ * P2: seq 20, length 10, timestamp 20
+ *
+ * Without doing it this way, the timestamp would be 20. With
+ * the next packet to arrive (P4, seq 40), the ts_last value
+ * wouldn't be updated for the talker in ProcessTcp() since that
+ * code specifically looks for the NEXT sequence number.
+ */
+ if ( last_ts )
+ {
+ if ( tracker->client_tracker )
+ {
+ int32_t delta = last_ts - session->server.get_ts_last();
+ if ( delta > 0 )
+ session->server.set_ts_last(last_ts);
+ }
+ else
+ {
+ int32_t delta = last_ts - session->client.get_ts_last();
+ if ( delta > 0 )
+ session->client.set_ts_last(last_ts);
+ }
+ }
+}
+
+void TcpReassemblySegments::purge_segments_left_of_hole(const TcpSegmentNode* end_tsn)
+{
+ uint32_t packets_skipped = 0;
+
+ TcpSegmentNode* cur_tsn = head;
+ do
+ {
+ TcpSegmentNode* drop_tsn = cur_tsn;
+ cur_tsn = cur_tsn->next;
+ delete_reassembly_segment(drop_tsn);
+ ++packets_skipped;
+ } while( cur_tsn and cur_tsn != end_tsn );
+
+ tracker->set_order(TcpStreamTracker::OUT_OF_SEQUENCE);
+
+ if (PacketTracer::is_active())
+ PacketTracer::log("Stream: Skipped %u packets before seglist hole)\n", packets_skipped);
+}
+
+void TcpReassemblySegments::advance_rcv_nxt(TcpSegmentNode *tsn)
+{
+ if ( !tsn )
+ {
+ if ( !head )
+ return;
+ tsn = head;
+ }
+
+ while (tsn->next_no_gap())
+ tsn = tsn->next;
+ tracker->set_rcv_nxt(tsn->next_seq());
+}
+
+void TcpReassemblySegments::skip_holes()
+{
+ TcpSegmentNode* tsn = head;
+ // if there is a hole at the beginning, skip it...
+ if ( SEQ_GT(tsn->seq, seglist_base_seq) )
+ {
+ seglist_base_seq = tsn->seq;
+ tracker->set_order(TcpStreamTracker::OUT_OF_SEQUENCE);
+
+ if (PacketTracer::is_active())
+ PacketTracer::log("Stream: Skipped hole at beginning of the seglist\n");
+ }
+
+ while ( tsn )
+ {
+ if ( tsn->next and SEQ_GT(tsn->next->start_seq(), tsn->next_seq()) )
+ {
+ tsn = tsn->next;
+ purge_segments_left_of_hole(tsn);
+ seglist_base_seq = head->start_seq();
+ }
+ else
+ tsn = tsn->next;
+ }
+
+ advance_rcv_nxt();
+ tracker->set_order(TcpStreamTracker::OUT_OF_SEQUENCE);
+}
+
+void TcpReassemblySegments::skip_midstream_pickup_seglist_hole(TcpSegmentDescriptor& tsd)
+{
+ uint32_t ack = tsd.get_ack();
+
+ TcpSegmentNode* tsn = head;
+ while ( tsn )
+ {
+ if ( SEQ_GEQ( tsn->next_seq(), ack) )
+ break;
+
+ if ( tsn->next and SEQ_GT(tsn->next->start_seq(), tsn->next_seq()) )
+ {
+ tsn = tsn->next;
+ purge_segments_left_of_hole(tsn);
+ seglist_base_seq = head->start_seq();
+ }
+ else if ( !tsn->next and SEQ_LT(tsn->next_seq(), ack) )
+ {
+ tsn = tsn->next;
+ purge_segments_left_of_hole(tsn);
+ seglist_base_seq = ack;
+ }
+ else
+ tsn = tsn->next;
+ }
+
+ tsn = head;
+ if ( tsn )
+ {
+ tracker->reassembler->initialize_paf();
+ advance_rcv_nxt(tsn);
+ }
+ else
+ tracker->set_rcv_nxt(ack);
+}
+
+void TcpReassemblySegments::purge_segment_list()
+{
+ purge();
+}
--- /dev/null
+//--------------------------------------------------------------------------
+// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
+//
+// This program is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License Version 2 as published
+// by the Free Software Foundation. You may not use, modify or distribute
+// this program under any other version of the GNU General Public License.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, write to the Free Software Foundation, Inc.,
+// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+//--------------------------------------------------------------------------
+
+// tcp_reassembly_segments.h author davis mcpherson <davmcphe@cisco.com>
+// Created on: Oct 9, 2015
+
+#ifndef TCP_REASSEMBLERS_H
+#define TCP_REASSEMBLERS_H
+
+#include <cstdint>
+
+#include "tcp_defs.h"
+
+#include "tcp_segment_node.h"
+
+class TcpOverlapResolver;
+class TcpOverlapState;
+class TcpSegmentDescriptor;
+class TcpSession;
+class TcpStreamTracker;
+
+class TcpReassemblySegments
+{
+public:
+ TcpReassemblySegments() = default;
+ ~TcpReassemblySegments();
+
+ void init(TcpSession* ssn, TcpStreamTracker* trk, StreamPolicy pol);
+ void reset();
+
+ void update_next(TcpSegmentNode*);
+ bool segment_within_seglist_window(TcpSegmentDescriptor&);
+ void queue_reassembly_segment(TcpSegmentDescriptor&);
+ void add_reassembly_segment(TcpSegmentDescriptor&, uint16_t, uint32_t,
+ uint32_t, uint32_t, TcpSegmentNode*);
+ void dup_reassembly_segment(TcpSegmentNode*, TcpSegmentNode**);
+ int delete_reassembly_segment(TcpSegmentNode*);
+ void advance_rcv_nxt(TcpSegmentNode *tsn = nullptr);
+ void purge_flushed_segments(uint32_t flush_seq);
+ void purge_segments_left_of_hole(const TcpSegmentNode*);
+ void skip_holes();
+ void skip_midstream_pickup_seglist_hole(TcpSegmentDescriptor&);
+ void purge_segment_list();
+
+ bool is_segment_pending_flush() const;
+
+ void set_seglist_base_seq(uint32_t base_seq)
+ { seglist_base_seq = base_seq; }
+
+ uint32_t get_seglist_base_seq() const
+ { return seglist_base_seq; }
+
+ bool data_was_queued() const
+ { return total_bytes_queued > 0; }
+
+ uint32_t get_seg_count() const
+ { return seg_count; }
+
+ uint32_t get_seg_bytes_total() const
+ { return seg_bytes_total; }
+
+ uint32_t get_overlap_count() const
+ { return overlap_count; }
+
+ void set_overlap_count(uint32_t count)
+ { overlap_count = count; }
+
+ uint32_t get_flush_count() const
+ { return flush_count; }
+
+ uint32_t get_seg_bytes_logical() const
+ { return seg_bytes_logical; }
+
+private:
+ void insert_segment_data(TcpSegmentNode* prev, TcpSegmentNode*);
+
+
+ void insert(TcpSegmentNode* prev, TcpSegmentNode* ss)
+ {
+ if ( prev )
+ {
+ ss->next = prev->next;
+ ss->prev = prev;
+ prev->next = ss;
+
+ if ( ss->next )
+ ss->next->prev = ss;
+ else
+ tail = ss;
+ }
+ else
+ {
+ ss->next = head;
+
+ if ( ss->next )
+ ss->next->prev = ss;
+ else
+ tail = ss;
+ head = ss;
+ }
+
+ seg_count++;
+ }
+
+ void remove(TcpSegmentNode* ss)
+ {
+ if ( ss->prev )
+ ss->prev->next = ss->next;
+ else
+ head = ss->next;
+
+ if ( ss->next )
+ ss->next->prev = ss->prev;
+ else
+ tail = ss->prev;
+
+ seg_count--;
+ }
+
+ uint32_t purge()
+ {
+ int i = 0;
+
+ while ( head )
+ {
+ i++;
+ TcpSegmentNode* dump_me = head;
+ head = head->next;
+ dump_me->term();
+ }
+
+ head = tail = cur_rseg = cur_sseg = nullptr;
+ seg_count = 0;
+ flush_count = 0;
+ seg_bytes_total = 0;
+ seg_bytes_logical = 0;
+ total_bytes_queued = 0;
+ total_segs_queued = 0;
+ overlap_count = 0;
+ return i;
+ }
+
+public:
+ TcpSegmentNode* head = nullptr;
+ TcpSegmentNode* tail = nullptr;
+
+ TcpSegmentNode* cur_rseg = nullptr;
+ TcpSegmentNode* cur_sseg = nullptr;
+
+ uint32_t seg_count = 0; /* number of current queued segments */
+ uint32_t flush_count = 0; /* queued segments already flushed */
+
+ uint32_t seglist_base_seq = 0; /* seq of first queued segment */
+ uint32_t seg_bytes_total = 0; /* total bytes currently queued */
+ uint32_t seg_bytes_logical = 0; /* logical bytes queued (total - overlaps) */
+ uint32_t total_bytes_queued = 0; /* total bytes queued (life of session) */
+ uint32_t total_segs_queued = 0; /* number of segments queued (life) */
+ uint32_t overlap_count = 0; /* overlaps encountered */
+
+ TcpSession* session = nullptr;
+ TcpStreamTracker* tracker = nullptr;
+ TcpOverlapResolver* overlap_resolver = nullptr;
+ TcpOverlapState* tos = nullptr;
+
+private:
+ void insert_segment_in_empty_seglist(TcpSegmentDescriptor&);
+ void insert_segment_in_seglist(TcpSegmentDescriptor&);
+
+ bool is_segment_fasttrack(TcpSegmentNode*, const TcpSegmentDescriptor&);
+ uint32_t get_pending_segment_count(const unsigned max) const;
+
+};
+
+#endif
+
#include "detection/rules.h"
#include "packet_io/packet_tracer.h"
#include "protocols/tcp_options.h"
-#include "stream/tcp/tcp_defs.h"
-#include "stream/tcp/tcp_stream_tracker.h"
+
+#include "tcp_defs.h"
+#include "tcp_event_logger.h"
+#include "tcp_stream_tracker.h"
using namespace snort;
pkt->pkth = p->pkth;
pkt->ptrs = p->ptrs;
pkt->ptrs.ip_api.set(*p->ptrs.ip_api.get_dst(), *p->ptrs.ip_api.get_src());
+ pkt->ptrs.dp = p->ptrs.sp;
+ pkt->ptrs.sp = p->ptrs.dp;
pkt->active = p->active_inst;
pkt->action = &p->action_inst;
if( p->is_from_client() )
pkt->flow = p->flow;
pkt->context = p->context;
pkt->dsize = 0;
+ pkt->daq_msg = p->daq_msg;
seq = tcph->seq();
ack = tcph->ack();
wnd = tcph->win();
end_seq = seq;
timestamp_option = 0;
- src_port = tcph->src_port();
- dst_port = tcph->dst_port();
+ src_port = tcph->dst_port();
+ dst_port = tcph->src_port();
packet_timestamp = p->pkth->ts.tv_sec;
packet_from_client = !p->is_from_client();
#include <cassert>
-#include <daq_common.h>
-
#include "flow/flow.h"
#include "detection/ips_context.h"
#include "main/snort_config.h"
#include "packet_io/active.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
-#include "stream/tcp/tcp_event_logger.h"
+class TcpEventLogger;
class TcpStreamTracker;
class TcpSegmentDescriptor
void set_talker(TcpStreamTracker& tracker)
{ talker = &tracker; }
+ bool is_packet_inorder() const
+ {
+ return packet_inorder;
+ }
+
+ void set_packet_inorder(bool inorder)
+ {
+ packet_inorder = inorder;
+ }
+
private:
snort::Flow* const flow;
snort::Packet* const pkt;
uint16_t dst_port;
uint32_t packet_timestamp;
bool packet_from_client;
+ bool packet_inorder = false;
bool meta_ack_packet = false;
};
#include "utils/util.h"
-#include "segment_overlap_editor.h"
#include "tcp_module.h"
+#include "tcp_segment_descriptor.h"
+
+using namespace snort;
#define USE_RESERVE
#ifdef USE_RESERVE
tcpStats.mem_in_use += len;
}
tsn->tv = tv;
- tsn->i_len = tsn->c_len = len;
+ tsn->length = len;
memcpy(tsn->data, payload, len);
tsn->prev = tsn->next = nullptr;
- tsn->i_seq = tsn->c_seq = 0;
- tsn->offset = tsn->o_offset = 0;
+
+ tsn->seq = 0;
+ tsn->offset = 0;
+ tsn->cursor = 0;
tsn->ts = 0;
return tsn;
return create(tsd.get_pkt()->pkth->ts, tsd.get_pkt()->data, tsd.get_len());
}
-TcpSegmentNode* TcpSegmentNode::init(TcpSegmentNode& tns)
+TcpSegmentNode* TcpSegmentNode::init(TcpSegmentNode& tsn)
{
- return create(tns.tv, tns.payload(), tns.c_len);
+ return create(tsn.tv, tsn.payload(), tsn.length);
}
void TcpSegmentNode::term()
uint32_t rseq, uint16_t orig_dsize, bool *full_retransmit)
{
// retransmit must have same payload at same place
- if ( !SEQ_EQ(i_seq, rseq) )
+ if ( !SEQ_EQ(seq, rseq) )
return false;
- if ( orig_dsize == c_len )
+ if ( orig_dsize == unscanned() )
{
- uint16_t cmp_len = ( c_len <= rsize ) ? c_len : rsize;
+ uint16_t cmp_len = ( length <= rsize ) ? length : rsize;
if ( !memcmp(data, rdata, cmp_len) )
return true;
}
// tcp_segment_node.h author davis mcpherson <davmcphe@cisco.com>
// Created on: Sep 21, 2015
-#ifndef TCP_SEGMENT_H
-#define TCP_SEGMENT_H
+#ifndef TCP_SEGMENT_NODE_H
+#define TCP_SEGMENT_NODE_H
+
+#include <cassert>
+
+#include "protocols/packet.h"
+#include "protocols/tcp.h"
-#include "tcp_segment_descriptor.h"
#include "tcp_defs.h"
class TcpSegmentDescriptor;
uint8_t* payload()
{ return data + offset; }
+ uint8_t* paf_data()
+ { return data + offset + cursor; }
+
+ uint32_t start_seq() const
+ { return seq + offset; }
+
+ uint32_t next_seq() const
+ { return start_seq() + length; }
+
+ uint32_t scan_seq() const
+ { return start_seq() + cursor; }
+
bool is_packet_missing(uint32_t to_seq)
{
if ( next )
- return !(SEQ_EQ((i_seq + i_len), next->i_seq));
+ return !(SEQ_EQ(next_seq(), next->start_seq()));
else
- return SEQ_LT((c_seq + c_len), to_seq);
+ return SEQ_LT(next_seq(), to_seq);
}
- void update_reassembly_cursor(uint16_t bytes)
+ void advance_cursor(uint16_t bytes)
+ { cursor += bytes; }
+
+ unsigned unscanned() const
{
- c_seq += bytes;
- c_len -= bytes;
- offset += bytes;
+ assert(cursor <= length);
+ return length - cursor;
}
-public:
- TcpSegmentNode* prev;
- TcpSegmentNode* next;
-
- struct timeval tv;
- uint32_t ts;
- uint32_t i_seq; // initial seq # of the data segment
- uint32_t c_seq; // current seq # of data for reassembly
- uint16_t i_len; // initial length of the data segment
- uint16_t c_len; // length of data remaining for reassembly
- uint16_t offset; // current offset into the data buffer for reassembly
- uint16_t o_offset; // offset into the data buffer due to overlaps
- uint16_t size; // actual allocated size (overlaps cause i_len to differ)
- uint8_t data[1];
-};
-
-class TcpSegmentList
-{
-public:
- uint32_t reset()
+ bool next_no_gap()
{
- int i = 0;
-
- while ( head )
- {
- i++;
- TcpSegmentNode* dump_me = head;
- head = head->next;
- dump_me->term();
- }
-
- head = tail = cur_rseg = cur_sseg = nullptr;
- count = 0;
- return i;
+ return next and SEQ_EQ(next_seq(), next->start_seq());
}
- void insert(TcpSegmentNode* prev, TcpSegmentNode* ss)
+ bool next_acked_no_gap(uint32_t seq_acked)
{
- if ( prev )
- {
- ss->next = prev->next;
- ss->prev = prev;
- prev->next = ss;
-
- if ( ss->next )
- ss->next->prev = ss;
- else
- tail = ss;
- }
- else
- {
- ss->next = head;
-
- if ( ss->next )
- ss->next->prev = ss;
- else
- tail = ss;
- head = ss;
- }
+ if ( !next_no_gap() )
+ return false;
- count++;
+ return SEQ_LT(next->start_seq() + next->cursor, seq_acked);
}
- void remove(TcpSegmentNode* ss)
- {
- if ( ss->prev )
- ss->prev->next = ss->next;
- else
- head = ss->next;
-
- if ( ss->next )
- ss->next->prev = ss->prev;
- else
- tail = ss->prev;
+public:
+ TcpSegmentNode* prev;
+ TcpSegmentNode* next;
- count--;
- }
+ struct timeval tv;
+ uint32_t ts;
- TcpSegmentNode* head = nullptr;
- TcpSegmentNode* tail = nullptr;
- TcpSegmentNode* cur_rseg = nullptr;
- TcpSegmentNode* cur_sseg = nullptr;
- uint32_t count = 0;
+ uint32_t seq; // initial seq # of the data segment (fixed)
+ uint16_t length; // working length of the segment data (relative to offset)
+ uint16_t offset; // working start of segment data
+ uint16_t cursor; // scan position (relative to offset)
+ uint16_t size; // allocated payload size
+ uint8_t data[1];
};
#endif
#include "detection/detection_engine.h"
#include "detection/rules.h"
+#include "framework/data_bus.h"
#include "log/log.h"
#include "packet_io/packet_tracer.h"
#include "profiler/profiler.h"
#include "protocols/eth.h"
#include "pub_sub/intrinsic_event_ids.h"
+#include "pub_sub/stream_event_ids.h"
+#include "stream/stream.h"
#include "stream_tcp.h"
#include "tcp_ha.h"
#include "tcp_module.h"
#include "tcp_normalizers.h"
-#include "tcp_reassemblers.h"
#include "tcp_segment_node.h"
#include "tcp_state_machine.h"
#include "tcp_trace.h"
+#include "trace/trace_api.h"
using namespace snort;
TcpSegmentNode::clear();
}
-TcpSession::TcpSession(Flow* f) : TcpStreamSession(f)
+TcpSession::TcpSession(Flow* f)
+ : Session(f), client(true), server(false)
{
tsm = TcpStateMachine::get_instance();
- splitter_init = false;
+ client.init_tcp_state(this);
+ server.init_tcp_state(this);
- client.session = this;
- server.session = this;
tcpStats.instantiated++;
}
bool TcpSession::setup(Packet*)
{
- client.init_tcp_state();
- server.init_tcp_state();
- lws_init = tcp_init = false;
+ client.init_tcp_state(this);
+ server.init_tcp_state(this);
+ tcp_init = false;
generate_3whs_alert = true;
cleaning = false;
splitter_init = false;
return true;
}
-// FIXIT-M once TcpReassembler interface is abstract class move this to base class
void TcpSession::restart(Packet* p)
{
// sanity check since this is called externally
if ( talker->midstream_initial_ack_flush )
{
talker->midstream_initial_ack_flush = false;
- talker->reassembler.flush_on_data_policy(p);
+ talker->eval_flush_policy_on_data(p);
}
if (p->dsize > 0)
- listener->reassembler.flush_on_data_policy(p);
+ listener->eval_flush_policy_on_data(p);
if (p->ptrs.tcph->is_ack())
- talker->reassembler.flush_on_ack_policy(p);
+ talker->eval_flush_policy_on_ack(p);
tcpStats.restarts++;
}
{
assert(!p or p->flow == flow);
if ( !tcp_init )
- {
- if ( lws_init )
- tcpStats.no_pickups++;
return;
- }
- lws_init = false;
tcp_init = false;
tcpStats.released++;
if ( !flow->two_way_traffic() and free_flow_data )
tcpStats.asymmetric_flows++;
- if ( flush_segments )
- {
- client.reassembler.flush_queued_segments(flow, true, p);
- server.reassembler.flush_queued_segments(flow, true, p);
- }
-
- if ( p )
- {
- client.finalize_held_packet(p);
- server.finalize_held_packet(p);
- }
- else
- {
- client.finalize_held_packet(flow);
- server.finalize_held_packet(flow);
- }
-
- client.reassembler.purge_segment_list();
- server.reassembler.purge_segment_list();
-
+ client.clear_tracker(flow, p, flush_segments, restart);
+ server.clear_tracker(flow, p, flush_segments, restart);
update_perf_base_state(TcpStreamTracker::TCP_CLOSED);
-
- set_splitter(true, nullptr);
- set_splitter(false, nullptr);
+ tel.log_internal_event(SESSION_EVENT_CLEAR);
if ( restart )
- {
flow->restart(free_flow_data);
- client.reassembler.reset_paf();
- server.reassembler.reset_paf();
- }
else
- {
flow->clear(free_flow_data);
- client.reassembler.clear_paf();
- server.reassembler.clear_paf();
- }
-
- tel.log_internal_event(SESSION_EVENT_CLEAR);
}
void TcpSession::update_perf_base_state(char newState)
server.normalizer.init(StreamPolicy::MISSED_3WHS, this, &server, &client);
}
-void TcpSession::update_stream_order(const TcpSegmentDescriptor& tsd, bool aligned)
-{
- TcpStreamTracker* listener = tsd.get_listener();
- uint32_t seq = tsd.get_seq();
-
- switch ( listener->order )
- {
- case TcpStreamTracker::IN_SEQUENCE:
- if ( aligned )
- tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
- else if ( SEQ_GT(seq, listener->rcv_nxt) )
- {
- listener->order = TcpStreamTracker::NONE;
- listener->hole_left_edge = listener->rcv_nxt;
- listener->hole_right_edge = seq - 1;
- }
- break;
-
- case TcpStreamTracker::NONE:
- if ( aligned )
- {
- tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
- if ( SEQ_GT(tsd.get_end_seq(), listener->hole_right_edge) )
- listener->order = TcpStreamTracker::OUT_OF_SEQUENCE;
- else
- listener->hole_left_edge = tsd.get_end_seq();
- }
- else
- {
- if ( SEQ_LEQ(seq, listener->hole_right_edge) )
- {
- if ( SEQ_GT(seq, listener->hole_left_edge) )
- listener->hole_right_edge = seq - 1;
- else if ( SEQ_GT(tsd.get_end_seq(), listener->hole_left_edge) )
- {
- listener->hole_left_edge = tsd.get_end_seq();
- tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
- }
- }
- // accounting for overlaps when not aligned
- if ( SEQ_GT(listener->hole_left_edge, listener->hole_right_edge) )
- listener->order = TcpStreamTracker::OUT_OF_SEQUENCE;
- }
- break;
-
- case TcpStreamTracker::OUT_OF_SEQUENCE:
- tsd.set_packet_flags(PKT_STREAM_ORDER_BAD);
- }
-}
-
void TcpSession::set_os_policy()
{
StreamPolicy client_os_policy = flow->ssn_policy ?
server_os_policy = StreamPolicy::OS_FIRST;
}
- client.reassembler.init(this, &client, client_os_policy, false);
- server.reassembler.init(this, &server, server_os_policy, true);
+ client.seglist.init(this, &client, client_os_policy);
+ server.seglist.init(this, &server, server_os_policy);
}
// FIXIT-M this is no longer called (but should be)
}
}
-void TcpSession::update_session_on_rst(TcpSegmentDescriptor& tsd, bool flush)
+void TcpSession::update_session_on_rst(const TcpSegmentDescriptor& tsd, bool flush)
{
Packet* p = tsd.get_pkt();
set_pkt_action_flag(ACTION_BAD_PKT);
}
-int32_t TcpSession::kickstart_asymmetric_flow(const TcpSegmentDescriptor& tsd, TcpStreamTracker* listener)
-{
- listener->reassembler.reset_asymmetric_flow_reassembly();
- listener->reassembler.flush_on_data_policy(tsd.get_pkt());
-
- int32_t space_left =
- tcp_config->max_queued_bytes - listener->reassembler.get_seg_bytes_total();
-
- if ( listener->get_tcp_state() == TcpStreamTracker::TCP_MID_STREAM_RECV )
- {
- listener->set_tcp_state(TcpStreamTracker::TCP_ESTABLISHED);
- if (PacketTracer::is_active())
- PacketTracer::log("Stream: Kickstart of midstream asymmetric flow! Seglist queue space: %u\n",
- space_left );
- }
- else
- {
- if (PacketTracer::is_active())
- PacketTracer::log("Stream: Kickstart of asymmetric flow! Seglist queue space: %u\n",
- space_left );
- }
-
- return space_left;
-}
-
bool TcpSession::check_reassembly_queue_thresholds(TcpSegmentDescriptor& tsd, TcpStreamTracker* listener)
{
// if this packet fits within the current queue limit window then it's good
- if( listener->reassembler.segment_within_seglist_window(tsd) )
+ if( listener->seglist.segment_within_seglist_window(tsd) )
return false;
bool inline_mode = tsd.is_nap_policy_inline();
if ( tcp_config->max_queued_bytes )
{
int32_t space_left =
- tcp_config->max_queued_bytes - listener->reassembler.get_seg_bytes_total();
+ tcp_config->max_queued_bytes - listener->seglist.get_seg_bytes_total();
if ( space_left < (int32_t)tsd.get_len() )
{
// and flush to free up seglist space
if ( tsd.is_ips_policy_inline() && !tsd.get_pkt()->flow->two_way_traffic() )
{
- space_left = kickstart_asymmetric_flow(tsd, listener);
+ space_left = listener->kickstart_asymmetric_flow(tsd, tcp_config->max_queued_bytes);
if ( space_left >= (int32_t)tsd.get_len() )
return false;
}
if ( tcp_config->max_queued_segs )
{
- if ( listener->reassembler.get_seg_count() + 1 > tcp_config->max_queued_segs )
+ if ( listener->seglist.get_seg_count() + 1 > tcp_config->max_queued_segs )
{
tcpStats.exceeded_max_segs++;
// and flush to free up seglist space
if ( tsd.is_ips_policy_inline() && !tsd.get_pkt()->flow->two_way_traffic() )
{
- kickstart_asymmetric_flow(tsd, listener);
- if ( listener->reassembler.get_seg_count() + 1 <= tcp_config->max_queued_segs )
+ listener->kickstart_asymmetric_flow(tsd, tcp_config->max_queued_bytes);
+ if ( listener->seglist.get_seg_count() + 1 <= tcp_config->max_queued_segs )
return false;
}
else
server.set_tcp_options_len(tcp_options_len);
- bool stream_is_inorder = ( tsd.get_seq() == listener->rcv_nxt );
+ tsd.set_packet_inorder(tsd.get_seq() == listener->rcv_nxt );
- int rc = listener->normalizer.apply_normalizations(tsd, tsd.get_seq(), stream_is_inorder);
+ int rc = listener->normalizer.apply_normalizations(tsd, tsd.get_seq(), tsd.is_packet_inorder());
switch ( rc )
{
case TcpNormalizer::NORM_OK:
- if ( stream_is_inorder )
- listener->rcv_nxt = tsd.get_end_seq();
-
- update_stream_order(tsd, stream_is_inorder);
check_small_segment_threshold(tsd, listener);
// don't queue data if we are ignoring or queue thresholds are exceeded
if ( filter_packet_for_reassembly(tsd, listener) )
{
set_packet_header_foo(tsd);
- listener->reassembler.queue_packet_for_reassembly(tsd);
+ listener->seglist.queue_reassembly_segment(tsd);
// Alert if overlap limit exceeded
if ( (tcp_config->overlap_limit)
- && (listener->reassembler.get_overlap_count() > tcp_config->overlap_limit) )
+ && (listener->seglist.get_overlap_count() > tcp_config->overlap_limit) )
{
tel.set_tcp_event(EVENT_EXCESSIVE_OVERLAP);
- listener->reassembler.set_overlap_count(0);
+ listener->seglist.set_overlap_count(0);
}
}
+ else if ( tsd.is_packet_inorder() )
+ listener->set_rcv_nxt(tsd.get_end_seq());
+
+ listener->update_stream_order(tsd, tsd.is_packet_inorder());
+
break;
case TcpNormalizer::NORM_TRIMMED:
default:
assert(false);
break;
-
}
}
if ( flush )
- listener->reassembler.flush_on_data_policy(tsd.get_pkt());
+ listener->eval_flush_policy_on_data(tsd.get_pkt());
else
- listener->reassembler.initialize_paf();
+ listener->reassembler->initialize_paf();
}
-TcpStreamTracker::TcpState TcpSession::get_talker_state(TcpSegmentDescriptor& tsd)
+TcpStreamTracker::TcpState TcpSession::get_talker_state(const TcpSegmentDescriptor& tsd)
{
return tsd.get_talker()->get_tcp_state();
}
-TcpStreamTracker::TcpState TcpSession::get_listener_state(TcpSegmentDescriptor& tsd)
+TcpStreamTracker::TcpState TcpSession::get_listener_state(const TcpSegmentDescriptor& tsd)
{
return tsd.get_listener()->get_tcp_state();
}
return; // We'll check & clear the TF_FORCE_FLUSH next time through
// Need to convert the addresses to network order
- if ( server.reassembler.flush_stream(p, PKT_FROM_SERVER) )
- server.reassembler.purge_flushed_ackd();
+ if ( server.reassembler->flush_stream(p, PKT_FROM_SERVER) )
+ server.reassembler->purge_flushed_ackd();
server.clear_tf_flags(TF_FORCE_FLUSH);
}
if ( p->packet_flags & PKT_REBUILT_STREAM )
return; // TF_FORCE_FLUSH checked & cleared next time through
- if ( client.reassembler.flush_stream(p, PKT_FROM_CLIENT) )
- client.reassembler.purge_flushed_ackd();
+ if ( client.reassembler->flush_stream(p, PKT_FROM_CLIENT) )
+ client.reassembler->purge_flushed_ackd();
client.clear_tf_flags(TF_FORCE_FLUSH);
}
return;
tracker.set_tf_flags(TF_FORCE_FLUSH);
- if ( tracker.reassembler.flush_stream(p, dir, final_flush) )
- tracker.reassembler.purge_flushed_ackd();
+ if ( tracker.reassembler->flush_stream(p, dir, final_flush) )
+ tracker.reassembler->purge_flushed_ackd();
tracker.clear_tf_flags(TF_FORCE_FLUSH);
}
if ( !tcp_init )
return;
- //FIXIT-L Cleanup tcp_init and lws_init as they have some side effect in TcpSession::clear_session
- lws_init = false;
+ //FIXIT-L Cleanup tcp_init has some side effect in TcpSession::clear_session
tcp_init = false;
- client.reassembler.flush_queued_segments(flow, true);
- server.reassembler.flush_queued_segments(flow, true);
+ client.reassembler->flush_queued_segments(flow, true);
+ server.reassembler->flush_queued_segments(flow, true);
- lws_init = true;
tcp_init = true;
}
void TcpSession::set_extra_data(Packet* p, uint32_t xid)
{
TcpStreamTracker& st = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ? server : client;
- st.reassembler.set_xtradata_mask(st.reassembler.get_xtradata_mask() | BIT(xid));
+ st.tcp_alerts.set_xtradata_mask(st.tcp_alerts.get_xtradata_mask() | BIT(xid));
}
static inline void set_window_scale(TcpSegmentDescriptor& tsd)
if ( Stream::blocked_flow(p) )
return true;
- if ( flow->ssn_state.ignore_direction != SSN_DIR_NONE )
+ if ( flow->ssn_state.ignore_direction == SSN_DIR_BOTH )
{
- server.set_flush_policy(STREAM_FLPOLICY_IGNORE);
- client.set_flush_policy(STREAM_FLPOLICY_IGNORE);
+ server.set_splitter((StreamSplitter*)nullptr);
+ client.set_splitter((StreamSplitter*)nullptr);
return true;
}
client.set_splitter(tsd.get_flow());
server.set_splitter(tsd.get_flow());
- client.init_flush_policy();
- server.init_flush_policy();
-
if ( tsd.is_packet_from_client() ) // Important if the 3-way handshake's ACK contains data
flow->set_session_flags(SSNFLAG_SEEN_CLIENT);
else
flow->set_session_flags(SSNFLAG_SEEN_SERVER);
check_flow_missed_3whs();
-
set_no_ack(tcp_config->no_ack);
}
tsm->eval(tsd);
check_events_and_actions(tsd);
- S5TraceTCP(tsd, p);
+ if ( stream_tcp_trace_enabled )
+ S5TraceTCP(tsd, p);
return ACTION_NOTHING;
}
if ( tcp_mack )
{
TcpSegmentDescriptor ma_tsd(flow, p, tcp_mack->tcp_ack_seq_num, tcp_mack->tcp_window_size);
+ assert( ma_tsd.get_pkt()->daq_msg && ma_tsd.get_pkt()->daq_msg == p->daq_msg );
+
init_tcp_packet_analysis(ma_tsd);
process_tcp_packet(ma_tsd, p);
tcpStats.meta_acks++;
return process_tcp_packet(tsd, p);
}
+
+void TcpSession::init_new_tcp_session(TcpSegmentDescriptor& tsd)
+{
+ Packet* p = tsd.get_pkt();
+
+ flow->pkt_type = p->type();
+ flow->ip_proto = (uint8_t)p->get_ip_proto_next();
+
+ /* New session, previous was marked as reset. Clear the reset flag. */
+ flow->clear_session_flags(SSNFLAG_RESET);
+
+ flow->set_expire(p, flow->default_session_timeout);
+
+ update_perf_base_state(TcpStreamTracker::TCP_SYN_SENT);
+
+ tcp_init = true;
+}
+
+void TcpSession::update_session_on_server_packet(TcpSegmentDescriptor& tsd)
+{
+ flow->set_session_flags(SSNFLAG_SEEN_SERVER);
+ tsd.set_talker(server);
+ tsd.set_listener(client);
+
+ if ( !flow->inner_server_ttl && !tsd.is_meta_ack_packet() )
+ flow->set_ttl(tsd.get_pkt(), false);
+}
+
+void TcpSession::update_session_on_client_packet(TcpSegmentDescriptor& tsd)
+{
+ /* if we got here we have seen the SYN already... */
+ flow->set_session_flags(SSNFLAG_SEEN_CLIENT);
+ tsd.set_talker(client);
+ tsd.set_listener(server);
+
+ if ( !flow->inner_client_ttl && !tsd.is_meta_ack_packet() )
+ flow->set_ttl(tsd.get_pkt(), true);
+}
+
+bool TcpSession::can_set_no_ack()
+{
+ return ( server.get_flush_policy() == STREAM_FLPOLICY_ON_DATA and
+ client.get_flush_policy() == STREAM_FLPOLICY_ON_DATA );
+}
+
+bool TcpSession::set_no_ack(bool b)
+{
+ if ( can_set_no_ack() )
+ {
+ no_ack = b;
+ return true;
+ }
+ else
+ return false;
+}
+
+void TcpSession::disable_reassembly(Flow* f)
+{
+ client.disable_reassembly(f);
+ server.disable_reassembly(f);
+}
+
+bool TcpSession::is_sequenced(uint8_t dir) const
+{
+ if ( dir & SSN_DIR_FROM_CLIENT )
+ {
+ if ( server.get_tf_flags() & ( TF_MISSING_PREV_PKT | TF_PKT_MISSED ) )
+ return false;
+ }
+
+ if ( dir & SSN_DIR_FROM_SERVER )
+ {
+ if ( client.get_tf_flags() & ( TF_MISSING_PREV_PKT | TF_PKT_MISSED ) )
+ return false;
+ }
+
+ return true;
+}
+
+/* This will falsely return SSN_MISSING_BEFORE on the first reassembled
+ * packet if reassembly for this direction was set mid-session */
+uint8_t TcpSession::missing_in_reassembled(uint8_t dir) const
+{
+ if ( dir & SSN_DIR_FROM_CLIENT )
+ {
+ if ( (server.get_tf_flags() & TF_MISSING_PKT)
+ && (server.get_tf_flags() & TF_MISSING_PREV_PKT) )
+ return SSN_MISSING_BOTH;
+ else if ( server.get_tf_flags() & TF_MISSING_PREV_PKT )
+ return SSN_MISSING_BEFORE;
+ else if ( server.get_tf_flags() & TF_MISSING_PKT )
+ return SSN_MISSING_AFTER;
+ }
+ else if ( dir & SSN_DIR_FROM_SERVER )
+ {
+ if ( (client.get_tf_flags() & TF_MISSING_PKT)
+ && (client.get_tf_flags() & TF_MISSING_PREV_PKT) )
+ return SSN_MISSING_BOTH;
+ else if ( client.get_tf_flags() & TF_MISSING_PREV_PKT )
+ return SSN_MISSING_BEFORE;
+ else if ( client.get_tf_flags() & TF_MISSING_PKT )
+ return SSN_MISSING_AFTER;
+ }
+
+ return SSN_MISSING_NONE;
+}
+
+bool TcpSession::are_packets_missing(uint8_t dir) const
+{
+ if ( dir & SSN_DIR_FROM_CLIENT )
+ {
+ if ( server.get_tf_flags() & TF_PKT_MISSED )
+ return true;
+ }
+
+ if ( dir & SSN_DIR_FROM_SERVER )
+ {
+ if ( client.get_tf_flags() & TF_PKT_MISSED )
+ return true;
+ }
+
+ return false;
+}
+
+bool TcpSession::are_client_segments_queued() const
+{
+ return client.seglist.is_segment_pending_flush();
+}
+
+bool TcpSession::add_alert(Packet* p, uint32_t gid, uint32_t sid)
+{
+ TcpStreamTracker& trk = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
+ server : client;
+
+ return trk.tcp_alerts.add_alert(gid, sid);
+}
+
+bool TcpSession::check_alerted(Packet* p, uint32_t gid, uint32_t sid)
+{
+ // only check for alert on wire packet, skip this when processing a rebuilt packet
+ if ( !(p->packet_flags & PKT_REBUILT_STREAM) )
+ return false;
+
+ TcpStreamTracker& trk = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
+ server : client;
+
+ return trk.tcp_alerts.check_alerted(gid, sid);
+}
+
+int TcpSession::update_alert(Packet* p, uint32_t gid, uint32_t sid,
+ uint32_t event_id, uint32_t event_second)
+{
+ TcpStreamTracker& trk = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
+ server : client;
+
+ return trk.tcp_alerts.update_alert(gid, sid, event_id, event_second);
+}
+
+bool TcpSession::set_packet_action_to_hold(Packet* p)
+{
+ if ( p->is_from_client() )
+ {
+ held_packet_dir = SSN_DIR_FROM_CLIENT;
+ return server.set_held_packet(p);
+ }
+ else
+ {
+ held_packet_dir = SSN_DIR_FROM_SERVER;
+ return client.set_held_packet(p);
+ }
+}
+
+void TcpSession::set_packet_header_foo(const TcpSegmentDescriptor& tsd)
+{
+ const Packet* p = tsd.get_pkt();
+
+ if ( tsd.is_packet_from_client() || (p->pkth->egress_index == DAQ_PKTHDR_UNKNOWN
+ && p->pkth->egress_group == DAQ_PKTHDR_UNKNOWN) )
+ {
+ ingress_index = p->pkth->ingress_index;
+ ingress_group = p->pkth->ingress_group;
+ // ssn egress may be unknown, but will be correct
+ egress_index = p->pkth->egress_index;
+ egress_group = p->pkth->egress_group;
+ }
+ else
+ {
+ egress_index = p->pkth->ingress_index;
+ egress_group = p->pkth->ingress_group;
+ ingress_index = p->pkth->egress_index;
+ ingress_group = p->pkth->egress_group;
+ }
+
+ daq_flags = p->pkth->flags;
+ address_space_id = p->pkth->address_space_id;
+}
+
+void TcpSession::get_packet_header_foo(DAQ_PktHdr_t* pkth, const DAQ_PktHdr_t* orig, uint32_t dir)
+{
+ if ( (dir & PKT_FROM_CLIENT) || (egress_index == DAQ_PKTHDR_UNKNOWN &&
+ egress_group == DAQ_PKTHDR_UNKNOWN) )
+ {
+ pkth->ingress_index = ingress_index;
+ pkth->ingress_group = ingress_group;
+ pkth->egress_index = egress_index;
+ pkth->egress_group = egress_group;
+ }
+ else
+ {
+ pkth->ingress_index = egress_index;
+ pkth->ingress_group = egress_group;
+ pkth->egress_index = ingress_index;
+ pkth->egress_group = ingress_group;
+ }
+ pkth->opaque = 0;
+ pkth->flags = daq_flags;
+ pkth->address_space_id = address_space_id;
+ pkth->tenant_id = orig->tenant_id;
+}
+
+void TcpSession::reset()
+{
+ if ( tcp_init )
+ clear_session(true, false, false );
+}
+
+void TcpSession::cleanup(Packet* p)
+{
+ if ( cleaning )
+ return;
+
+ cleaning = true;
+ clear_session(true, true, false, p);
+ client.reset();
+ server.reset();
+ cleaning = false;
+}
+
+void TcpSession::clear()
+{
+ if ( tcp_init )
+ clear_session( true, false, false );
+
+ TcpHAManager::process_deletion(*flow);
+}
+
+void TcpSession::set_splitter(bool to_server, StreamSplitter* ss)
+{
+ TcpStreamTracker& trk = ( to_server ) ? server : client;
+
+ trk.set_splitter(ss);
+}
+
+uint16_t TcpSession::get_mss(bool to_server) const
+{
+ const TcpStreamTracker& trk = (to_server) ? client : server;
+
+ return trk.get_mss();
+}
+
+uint8_t TcpSession::get_tcp_options_len(bool to_server) const
+{
+ const TcpStreamTracker& trk = (to_server) ? client : server;
+
+ return trk.get_tcp_options_len();
+}
+
+StreamSplitter* TcpSession::get_splitter(bool to_server)
+{
+ if ( to_server )
+ return server.get_splitter();
+ else
+ return client.get_splitter();
+}
+
+void TcpSession::start_proxy()
+{
+ if ( PacketTracer::is_active() )
+ PacketTracer::log("Stream TCP normalization policy set to Proxy mode. Normalizations will be skipped\n");
+
+ tcp_config->policy = StreamPolicy::OS_PROXY;
+ client.normalizer.init(tcp_config->policy, this, &client, &server);
+ server.normalizer.init(tcp_config->policy, this, &server, &client);
+ ++tcpStats.proxy_mode_flows;
+}
+
+void TcpSession::set_established(const TcpSegmentDescriptor& tsd)
+{
+ update_perf_base_state(TcpStreamTracker::TCP_ESTABLISHED);
+ flow->session_state |= STREAM_STATE_ESTABLISHED;
+ flow->set_idle_timeout(this->tcp_config->idle_timeout);
+ if (SSNFLAG_ESTABLISHED != (SSNFLAG_ESTABLISHED & flow->get_session_flags()))
+ {
+ flow->set_session_flags(SSNFLAG_ESTABLISHED);
+ // Only send 1 event
+ if (SSNFLAG_TCP_PSEUDO_EST != (SSNFLAG_TCP_PSEUDO_EST & flow->get_session_flags()))
+ DataBus::publish(Stream::get_pub_id(), StreamEventIds::TCP_ESTABLISHED, tsd.get_pkt());
+ }
+}
+
+void TcpSession::set_pseudo_established(Packet* p)
+{
+ p->flow->ssn_state.session_flags |= SSNFLAG_TCP_PSEUDO_EST;
+ DataBus::publish(Stream::get_pub_id(), StreamEventIds::TCP_ESTABLISHED, p);
+}
+
+bool TcpSession::check_for_one_sided_session(Packet* p)
+{
+ Flow& flow = *p->flow;
+ if ( 0 == ( (SSNFLAG_ESTABLISHED | SSNFLAG_TCP_PSEUDO_EST) & flow.ssn_state.session_flags )
+ && p->is_from_client_originally() )
+ {
+ uint64_t initiator_packets;
+ uint64_t responder_packets;
+ if (flow.flags.client_initiated)
+ {
+ initiator_packets = flow.flowstats.client_pkts;
+ responder_packets = flow.flowstats.server_pkts;
+ }
+ else
+ {
+ initiator_packets = flow.flowstats.server_pkts;
+ responder_packets = flow.flowstats.client_pkts;
+ }
+
+ if ( !responder_packets )
+ {
+ // handle case where traffic is only in one direction, but the sequence numbers
+ // are changing indicating an asynchronous session
+ uint32_t watermark = p->ptrs.tcph->seq() + p->ptrs.tcph->ack();
+ if ( 1 == initiator_packets )
+ initiator_watermark = watermark;
+ else if ( initiator_watermark != watermark )
+ {
+ set_pseudo_established(p);
+ return true;
+ }
+ }
+ }
+ return false;
+}
+
+void TcpSession::check_for_pseudo_established(Packet* p)
+{
+ Flow& flow = *p->flow;
+ if ( 0 == ( (SSNFLAG_ESTABLISHED | SSNFLAG_TCP_PSEUDO_EST) & flow.ssn_state.session_flags ) )
+ {
+ if ( check_for_one_sided_session(p) )
+ return;
+ if ( 0 < flow.flowstats.client_pkts && 0 < flow.flowstats.server_pkts )
+ set_pseudo_established(p);
+ }
+}
+
+
#ifndef TCP_SESSION_H
#define TCP_SESSION_H
+#include "flow/flow.h"
+#include "flow/session.h"
+#include "protocols/packet.h"
+
+#include "tcp_event_logger.h"
#include "tcp_state_machine.h"
-#include "tcp_stream_session.h"
+#include "tcp_stream_config.h"
#include "tcp_stream_tracker.h"
namespace snort
class Flow;
struct Packet;
}
-class TcpEventLogger;
+class TcpSegmentDescriptor;
-class TcpSession : public TcpStreamSession
+class TcpSession : public Session
{
public:
TcpSession(snort::Flow*);
void precheck(snort::Packet* p) override;
int process(snort::Packet*) override;
+ void clear() override;
+ void cleanup(snort::Packet* = nullptr) override;
+
+ void set_splitter(bool, snort::StreamSplitter*) override;
+ snort::StreamSplitter* get_splitter(bool) override;
+
+ void disable_reassembly(snort::Flow*) override;
+ uint8_t missing_in_reassembled(uint8_t dir) const override;
+ bool are_client_segments_queued() const override;
+ bool is_sequenced(uint8_t dir) const override;
+ bool are_packets_missing(uint8_t dir) const override;
+ bool set_packet_action_to_hold(snort::Packet*) override;
+
+ bool add_alert(snort::Packet*, uint32_t gid, uint32_t sid) override;
+ bool check_alerted(snort::Packet*, uint32_t gid, uint32_t sid) override;
+ int update_alert(snort::Packet*, uint32_t gid, uint32_t sid,
+ uint32_t event_id, uint32_t event_second) override;
+ void set_extra_data(snort::Packet*, uint32_t /*flag*/) override;
+
void flush() override;
void flush_client(snort::Packet*) override;
void flush_server(snort::Packet*) override;
void flush_talker(snort::Packet*, bool final_flush = false) override;
void flush_listener(snort::Packet*, bool final_flush = false) override;
// cppcheck-suppress virtualCallInConstructor
- void clear_session(bool free_flow_data, bool flush_segments, bool restart, snort::Packet* p = nullptr) override;
- void set_extra_data(snort::Packet*, uint32_t /*flag*/) override;
- void update_perf_base_state(char new_state) override;
- TcpStreamTracker::TcpState get_talker_state(TcpSegmentDescriptor& tsd) override;
- TcpStreamTracker::TcpState get_listener_state(TcpSegmentDescriptor& tsd) override;
- void update_timestamp_tracking(TcpSegmentDescriptor&) override;
- void update_session_on_rst(TcpSegmentDescriptor&, bool) override;
- bool handle_syn_on_reset_session(TcpSegmentDescriptor&) override;
- void handle_data_on_syn(TcpSegmentDescriptor&) override;
- void update_ignored_session(TcpSegmentDescriptor&) override;
- void update_paws_timestamps(TcpSegmentDescriptor&) override;
- void check_for_repeated_syn(TcpSegmentDescriptor&) override;
- void check_for_session_hijack(TcpSegmentDescriptor&) override;
- bool check_for_window_slam(TcpSegmentDescriptor& tsd) override;
- void mark_packet_for_drop(TcpSegmentDescriptor&) override;
- void handle_data_segment(TcpSegmentDescriptor&, bool flush = true);
- bool validate_packet_established_session(TcpSegmentDescriptor&) override;
+
+ void reset();
+ void start_proxy();
+ void clear_session(bool free_flow_data, bool flush_segments, bool restart, snort::Packet* p = nullptr);
+ TcpStreamTracker::TcpState get_talker_state(const TcpSegmentDescriptor& tsd);
+ TcpStreamTracker::TcpState get_listener_state(const TcpSegmentDescriptor& tsd);
+ TcpStreamTracker::TcpState get_peer_state(const TcpStreamTracker* me)
+ { return me == &client ? server.get_tcp_state() : client.get_tcp_state(); }
+
+ void init_new_tcp_session(TcpSegmentDescriptor&);
+ void update_perf_base_state(char new_state);
+ void update_timestamp_tracking(TcpSegmentDescriptor&);
+ void update_paws_timestamps(TcpSegmentDescriptor&);
+ void update_session_on_rst(const TcpSegmentDescriptor&, bool);
+ bool handle_syn_on_reset_session(TcpSegmentDescriptor&);
+ void handle_data_on_syn(TcpSegmentDescriptor&);
+ void update_ignored_session(TcpSegmentDescriptor&);
bool is_midstream_allowed(const TcpSegmentDescriptor& tsd)
{ return tcp_config->midstream_allowed(tsd.get_pkt()); }
+ uint16_t get_mss(bool to_server) const;
+ uint8_t get_tcp_options_len(bool to_server) const;
+
+ void get_packet_header_foo(DAQ_PktHdr_t*, const DAQ_PktHdr_t* orig, uint32_t dir);
+ bool can_set_no_ack();
+ bool set_no_ack(bool);
+ bool no_ack_mode_enabled() { return no_ack; }
+
+ void generate_no_3whs_event()
+ {
+ if ( generate_3whs_alert && flow->two_way_traffic())
+ {
+ tel.set_tcp_event(EVENT_NO_3WHS);
+ generate_3whs_alert = false;
+ }
+ }
+
+ void set_pkt_action_flag(uint32_t flag)
+ { pkt_action_mask |= flag; }
+
+ void set_established(const TcpSegmentDescriptor&);
+ void set_pseudo_established(snort::Packet*);
+ void check_for_pseudo_established(snort::Packet*);
+ bool check_for_one_sided_session(snort::Packet*);
+
+ void check_for_repeated_syn(TcpSegmentDescriptor&);
+ void check_for_session_hijack(TcpSegmentDescriptor&);
+ bool check_for_window_slam(TcpSegmentDescriptor& tsd);
+ void mark_packet_for_drop(TcpSegmentDescriptor&);
+ void handle_data_segment(TcpSegmentDescriptor&, bool flush = true);
+ bool validate_packet_established_session(TcpSegmentDescriptor&);
+
+ TcpStreamTracker client;
+ TcpStreamTracker server;
+ TcpStreamConfig* tcp_config = nullptr;
+ TcpEventLogger tel;
+ bool tcp_init = false;
+ uint32_t pkt_action_mask = ACTION_NOTHING;
+ uint32_t initiator_watermark = 0;
+ int32_t ingress_index = DAQ_PKTHDR_UNKNOWN;
+ int32_t egress_index = DAQ_PKTHDR_UNKNOWN;
+ int16_t ingress_group = DAQ_PKTHDR_UNKNOWN;
+ int16_t egress_group = DAQ_PKTHDR_UNKNOWN;
+ uint32_t daq_flags = 0;
+ uint32_t address_space_id = 0;
+ bool generate_3whs_alert = true;
+ bool cleaning = false;
+ uint8_t held_packet_dir = SSN_DIR_NONE;
+ uint8_t ecn = 0;
+
private:
int process_tcp_packet(TcpSegmentDescriptor&, const snort::Packet*);
- void set_os_policy() override;
- void update_stream_order(const TcpSegmentDescriptor&, bool aligned);
+ void set_os_policy();
void swap_trackers();
void init_session_on_syn(TcpSegmentDescriptor&);
void init_session_on_synack(TcpSegmentDescriptor&);
bool check_reassembly_queue_thresholds(TcpSegmentDescriptor&, TcpStreamTracker*);
bool filter_packet_for_reassembly(TcpSegmentDescriptor&, TcpStreamTracker*);
void check_small_segment_threshold(const TcpSegmentDescriptor&, TcpStreamTracker*);
- int32_t kickstart_asymmetric_flow(const TcpSegmentDescriptor&, TcpStreamTracker*);
void check_flow_missed_3whs();
-private:
+ void set_packet_header_foo(const TcpSegmentDescriptor&);
+ void update_session_on_server_packet(TcpSegmentDescriptor&);
+ void update_session_on_client_packet(TcpSegmentDescriptor&);
+
TcpStateMachine* tsm;
- bool splitter_init;
+ bool splitter_init = false;
+ bool no_ack = false;
};
#endif
#include "tcp_session.h"
-#ifdef UNIT_TEST
-#include "catch/snort_catch.h"
-#endif
-
using namespace snort;
TcpStateClosed::TcpStateClosed(TcpStateMachine& tsm) :
#include "tcp_state_machine.h"
-#ifdef UNIT_TEST
-#include "catch/snort_catch.h"
-#endif
-
using namespace std;
TcpStateHandler::TcpStateHandler(TcpStreamTracker::TcpState state, TcpStateMachine& tsm)
{
if ( trk.normalizer.is_tcp_ips_enabled() )
{
- trk.reassembler.skip_midstream_pickup_seglist_hole(tsd);
- trk.reassembler.flush_on_data_policy(tsd.get_pkt());
+ trk.seglist.skip_midstream_pickup_seglist_hole(tsd);
+ trk.eval_flush_policy_on_data(tsd.get_pkt());
trk.midstream_initial_ack_flush = true;
}
{
if ( trk.normalizer.is_tcp_ips_enabled() )
{
- trk.reassembler.skip_midstream_pickup_seglist_hole(tsd);
- trk.reassembler.flush_on_data_policy(tsd.get_pkt());
+ trk.seglist.skip_midstream_pickup_seglist_hole(tsd);
+ trk.eval_flush_policy_on_data(tsd.get_pkt());
trk.midstream_initial_ack_flush = true;
}
{
if ( trk.normalizer.is_tcp_ips_enabled() )
{
- trk.reassembler.skip_midstream_pickup_seglist_hole(tsd);
- trk.reassembler.flush_on_data_policy(tsd.get_pkt());
+ trk.seglist.skip_midstream_pickup_seglist_hole(tsd);
+ trk.eval_flush_policy_on_data(tsd.get_pkt());
trk.midstream_initial_ack_flush = true;
}
{
if ( trk.normalizer.is_tcp_ips_enabled() )
{
- trk.reassembler.skip_midstream_pickup_seglist_hole(tsd);
- trk.reassembler.flush_on_data_policy(tsd.get_pkt());
+ trk.seglist.skip_midstream_pickup_seglist_hole(tsd);
+ trk.eval_flush_policy_on_data(tsd.get_pkt());
trk.midstream_initial_ack_flush = true;
}
bool TcpStateMidStreamSent::data_seg_recv(TcpSegmentDescriptor& tsd, TcpStreamTracker& trk)
{
trk.update_tracker_ack_recv(tsd);
- trk.reassembler.set_seglist_base_seq(tsd.get_seq());
+ trk.seglist.set_seglist_base_seq(tsd.get_seq());
trk.session->handle_data_segment(tsd);
trk.session->set_established(tsd);
trk.set_tcp_state(TcpStreamTracker::TCP_ESTABLISHED);
bool TcpStateSynSent::ack_sent(TcpSegmentDescriptor& tsd, TcpStreamTracker& trk)
{
trk.update_tracker_ack_sent(tsd);
+ if ( SEQ_GT(tsd.get_ack(), trk.get_rcv_nxt()) )
+ trk.set_rcv_nxt(tsd.get_ack());
trk.session->update_timestamp_tracking(tsd);
if ( trk.session->flow->two_way_traffic() )
{
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// tcp_stream_session.cc author davis mcpherson <davmcphe@cisco.com>
-// Created on: Feb 18, 2016
-
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
-#include "tcp_stream_session.h"
-
-#include "framework/data_bus.h"
-#include "packet_io/packet_tracer.h"
-#include "pub_sub/stream_event_ids.h"
-#include "stream/stream.h"
-#include "stream/tcp/tcp_ha.h"
-
-using namespace snort;
-
-TcpStreamSession::TcpStreamSession(Flow* f)
- : Session(f), client(true), server(false)
-{ }
-
-TcpStreamSession::~TcpStreamSession() = default;
-
-void TcpStreamSession::init_new_tcp_session(TcpSegmentDescriptor& tsd)
-{
- Packet* p = tsd.get_pkt();
-
- flow->pkt_type = p->type();
- flow->ip_proto = (uint8_t)p->get_ip_proto_next();
-
- /* New session, previous was marked as reset. Clear the reset flag. */
- flow->clear_session_flags(SSNFLAG_RESET);
-
- flow->set_expire(p, flow->default_session_timeout);
-
- update_perf_base_state(TcpStreamTracker::TCP_SYN_SENT);
-
- tcp_init = true;
- lws_init = true;
-}
-
-void TcpStreamSession::update_session_on_server_packet(TcpSegmentDescriptor& tsd)
-{
- flow->set_session_flags(SSNFLAG_SEEN_SERVER);
- tsd.set_talker(server);
- tsd.set_listener(client);
-
- if ( !flow->inner_server_ttl && !tsd.is_meta_ack_packet() )
- flow->set_ttl(tsd.get_pkt(), false);
-}
-
-void TcpStreamSession::update_session_on_client_packet(TcpSegmentDescriptor& tsd)
-{
- /* if we got here we have seen the SYN already... */
- flow->set_session_flags(SSNFLAG_SEEN_CLIENT);
- tsd.set_talker(client);
- tsd.set_listener(server);
-
- if ( !flow->inner_client_ttl && !tsd.is_meta_ack_packet() )
- flow->set_ttl(tsd.get_pkt(), true);
-}
-
-bool TcpStreamSession::can_set_no_ack()
-{
- return ( server.get_flush_policy() == STREAM_FLPOLICY_ON_DATA and
- client.get_flush_policy() == STREAM_FLPOLICY_ON_DATA );
-}
-
-bool TcpStreamSession::set_no_ack(bool b)
-{
- if ( can_set_no_ack() )
- {
- no_ack = b;
- return true;
- }
- else
- return false;
-}
-
-void TcpStreamSession::disable_reassembly(Flow* f)
-{
- client.set_splitter((StreamSplitter*)nullptr);
- server.set_splitter((StreamSplitter*)nullptr);
-
- client.reassembler.purge_segment_list();
- server.reassembler.purge_segment_list();
-
- client.set_flush_policy(STREAM_FLPOLICY_IGNORE);
- server.set_flush_policy(STREAM_FLPOLICY_IGNORE);
-
- client.finalize_held_packet(f);
- server.finalize_held_packet(f);
-}
-
-uint8_t TcpStreamSession::get_reassembly_direction() const
-{
- uint8_t dir = SSN_DIR_NONE;
-
- if ( server.get_flush_policy() != STREAM_FLPOLICY_IGNORE )
- dir |= SSN_DIR_FROM_CLIENT;
-
- if ( client.get_flush_policy() != STREAM_FLPOLICY_IGNORE )
- dir |= SSN_DIR_FROM_SERVER;
-
- return dir;
-}
-
-bool TcpStreamSession::is_sequenced(uint8_t dir) const
-{
- if ( dir & SSN_DIR_FROM_CLIENT )
- {
- if ( server.get_tf_flags() & ( TF_MISSING_PREV_PKT | TF_PKT_MISSED ) )
- return false;
- }
-
- if ( dir & SSN_DIR_FROM_SERVER )
- {
- if ( client.get_tf_flags() & ( TF_MISSING_PREV_PKT | TF_PKT_MISSED ) )
- return false;
- }
-
- return true;
-}
-
-/* This will falsely return SSN_MISSING_BEFORE on the first reassembled
- * packet if reassembly for this direction was set mid-session */
-uint8_t TcpStreamSession::missing_in_reassembled(uint8_t dir) const
-{
- if ( dir & SSN_DIR_FROM_CLIENT )
- {
- if ( (server.get_tf_flags() & TF_MISSING_PKT)
- && (server.get_tf_flags() & TF_MISSING_PREV_PKT) )
- return SSN_MISSING_BOTH;
- else if ( server.get_tf_flags() & TF_MISSING_PREV_PKT )
- return SSN_MISSING_BEFORE;
- else if ( server.get_tf_flags() & TF_MISSING_PKT )
- return SSN_MISSING_AFTER;
- }
- else if ( dir & SSN_DIR_FROM_SERVER )
- {
- if ( (client.get_tf_flags() & TF_MISSING_PKT)
- && (client.get_tf_flags() & TF_MISSING_PREV_PKT) )
- return SSN_MISSING_BOTH;
- else if ( client.get_tf_flags() & TF_MISSING_PREV_PKT )
- return SSN_MISSING_BEFORE;
- else if ( client.get_tf_flags() & TF_MISSING_PKT )
- return SSN_MISSING_AFTER;
- }
-
- return SSN_MISSING_NONE;
-}
-
-bool TcpStreamSession::are_packets_missing(uint8_t dir) const
-{
- if ( dir & SSN_DIR_FROM_CLIENT )
- {
- if ( server.get_tf_flags() & TF_PKT_MISSED )
- return true;
- }
-
- if ( dir & SSN_DIR_FROM_SERVER )
- {
- if ( client.get_tf_flags() & TF_PKT_MISSED )
- return true;
- }
-
- return false;
-}
-
-bool TcpStreamSession::are_client_segments_queued() const
-{
- return client.reassembler.is_segment_pending_flush();
-}
-
-bool TcpStreamSession::add_alert(Packet* p, uint32_t gid, uint32_t sid)
-{
- TcpReassemblerPolicy& trp = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
- server.reassembler : client.reassembler;
-
- return trp.add_alert(gid, sid);
-}
-
-bool TcpStreamSession::check_alerted(Packet* p, uint32_t gid, uint32_t sid)
-{
- // only check for alert on wire packet if this when processing a rebuilt packet
- if ( !(p->packet_flags & PKT_REBUILT_STREAM) )
- return false;
-
- TcpReassemblerPolicy& trp = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
- server.reassembler : client.reassembler;
-
- return trp.check_alerted(gid, sid);
-}
-
-int TcpStreamSession::update_alert(Packet* p, uint32_t gid, uint32_t sid,
- uint32_t event_id, uint32_t event_second)
-{
- TcpReassemblerPolicy& trp = p->ptrs.ip_api.get_src()->equals(flow->client_ip) ?
- server.reassembler : client.reassembler;
-
- return trp.update_alert(gid, sid, event_id, event_second);
-}
-
-bool TcpStreamSession::set_packet_action_to_hold(Packet* p)
-{
- if ( p->is_from_client() )
- {
- held_packet_dir = SSN_DIR_FROM_CLIENT;
- return server.set_held_packet(p);
- }
- else
- {
- held_packet_dir = SSN_DIR_FROM_SERVER;
- return client.set_held_packet(p);
- }
-}
-
-void TcpStreamSession::set_packet_header_foo(const TcpSegmentDescriptor& tsd)
-{
- const Packet* p = tsd.get_pkt();
-
- if ( tsd.is_packet_from_client() || (p->pkth->egress_index == DAQ_PKTHDR_UNKNOWN
- && p->pkth->egress_group == DAQ_PKTHDR_UNKNOWN) )
- {
- ingress_index = p->pkth->ingress_index;
- ingress_group = p->pkth->ingress_group;
- // ssn egress may be unknown, but will be correct
- egress_index = p->pkth->egress_index;
- egress_group = p->pkth->egress_group;
- }
- else
- {
- egress_index = p->pkth->ingress_index;
- egress_group = p->pkth->ingress_group;
- ingress_index = p->pkth->egress_index;
- ingress_group = p->pkth->egress_group;
- }
-
- daq_flags = p->pkth->flags;
- address_space_id = p->pkth->address_space_id;
-}
-
-void TcpStreamSession::get_packet_header_foo(DAQ_PktHdr_t* pkth, const DAQ_PktHdr_t* orig, uint32_t dir)
-{
- if ( (dir & PKT_FROM_CLIENT) || (egress_index == DAQ_PKTHDR_UNKNOWN &&
- egress_group == DAQ_PKTHDR_UNKNOWN) )
- {
- pkth->ingress_index = ingress_index;
- pkth->ingress_group = ingress_group;
- pkth->egress_index = egress_index;
- pkth->egress_group = egress_group;
- }
- else
- {
- pkth->ingress_index = egress_index;
- pkth->ingress_group = egress_group;
- pkth->egress_index = ingress_index;
- pkth->egress_group = ingress_group;
- }
- pkth->opaque = 0;
- pkth->flags = daq_flags;
- pkth->address_space_id = address_space_id;
- pkth->tenant_id = orig->tenant_id;
-}
-
-void TcpStreamSession::reset()
-{
- if ( tcp_init )
- clear_session(true, false, false );
-}
-
-void TcpStreamSession::cleanup(Packet* p)
-{
- if ( cleaning )
- return;
-
- cleaning = true;
- clear_session(true, true, false, p);
- client.normalizer.reset();
- server.normalizer.reset();
- client.reassembler.reset();
- server.reassembler.reset();
- cleaning = false;
-}
-
-void TcpStreamSession::clear()
-{
- if ( tcp_init )
- clear_session( true, false, false );
-
- TcpHAManager::process_deletion(*flow);
-}
-
-void TcpStreamSession::set_splitter(bool to_server, StreamSplitter* ss)
-{
- TcpStreamTracker& trk = ( to_server ) ? server : client;
-
- trk.set_splitter(ss);
-}
-
-uint16_t TcpStreamSession::get_mss(bool to_server) const
-{
- const TcpStreamTracker& trk = (to_server) ? client : server;
-
- return trk.get_mss();
-}
-
-uint8_t TcpStreamSession::get_tcp_options_len(bool to_server) const
-{
- const TcpStreamTracker& trk = (to_server) ? client : server;
-
- return trk.get_tcp_options_len();
-}
-
-StreamSplitter* TcpStreamSession::get_splitter(bool to_server)
-{
- if ( to_server )
- return server.get_splitter();
- else
- return client.get_splitter();
-}
-
-void TcpStreamSession::start_proxy()
-{
- if ( PacketTracer::is_active() )
- PacketTracer::log("Stream TCP normalization policy set to Proxy mode. Normalizations will be skipped\n");
-
- tcp_config->policy = StreamPolicy::OS_PROXY;
- client.normalizer.init(tcp_config->policy, this, &client, &server);
- server.normalizer.init(tcp_config->policy, this, &server, &client);
- ++tcpStats.proxy_mode_flows;
-}
-
-void TcpStreamSession::set_established(const TcpSegmentDescriptor& tsd)
-{
- update_perf_base_state(TcpStreamTracker::TCP_ESTABLISHED);
- flow->session_state |= STREAM_STATE_ESTABLISHED;
- flow->set_idle_timeout(this->tcp_config->idle_timeout);
- if (SSNFLAG_ESTABLISHED != (SSNFLAG_ESTABLISHED & flow->get_session_flags()))
- {
- flow->set_session_flags(SSNFLAG_ESTABLISHED);
- // Only send 1 event
- if (SSNFLAG_TCP_PSEUDO_EST != (SSNFLAG_TCP_PSEUDO_EST & flow->get_session_flags()))
- DataBus::publish(Stream::get_pub_id(), StreamEventIds::TCP_ESTABLISHED, tsd.get_pkt());
- }
-}
-
-void TcpStreamSession::set_pseudo_established(Packet* p)
-{
- p->flow->ssn_state.session_flags |= SSNFLAG_TCP_PSEUDO_EST;
- DataBus::publish(Stream::get_pub_id(), StreamEventIds::TCP_ESTABLISHED, p);
-}
-
-bool TcpStreamSession::check_for_one_sided_session(Packet* p)
-{
- Flow& flow = *p->flow;
- if ( 0 == ( (SSNFLAG_ESTABLISHED | SSNFLAG_TCP_PSEUDO_EST) & flow.ssn_state.session_flags )
- && p->is_from_client_originally() )
- {
- uint64_t initiator_packets;
- uint64_t responder_packets;
- if (flow.flags.client_initiated)
- {
- initiator_packets = flow.flowstats.client_pkts;
- responder_packets = flow.flowstats.server_pkts;
- }
- else
- {
- initiator_packets = flow.flowstats.server_pkts;
- responder_packets = flow.flowstats.client_pkts;
- }
-
- if ( !responder_packets )
- {
- // handle case where traffic is only in one direction, but the sequence numbers
- // are changing indicating an asynchronous session
- uint32_t watermark = p->ptrs.tcph->seq() + p->ptrs.tcph->ack();
- if ( 1 == initiator_packets )
- initiator_watermark = watermark;
- else if ( initiator_watermark != watermark )
- {
- set_pseudo_established(p);
- return true;
- }
- }
- }
- return false;
-}
-
-void TcpStreamSession::check_for_pseudo_established(Packet* p)
-{
- Flow& flow = *p->flow;
- if ( 0 == ( (SSNFLAG_ESTABLISHED | SSNFLAG_TCP_PSEUDO_EST) & flow.ssn_state.session_flags ) )
- {
- if ( check_for_one_sided_session(p) )
- return;
- if ( 0 < flow.flowstats.client_pkts && 0 < flow.flowstats.server_pkts )
- set_pseudo_established(p);
- }
-}
-
+++ /dev/null
-//--------------------------------------------------------------------------
-// Copyright (C) 2015-2024 Cisco and/or its affiliates. All rights reserved.
-//
-// This program is free software; you can redistribute it and/or modify it
-// under the terms of the GNU General Public License Version 2 as published
-// by the Free Software Foundation. You may not use, modify or distribute
-// this program under any other version of the GNU General Public License.
-//
-// This program is distributed in the hope that it will be useful, but
-// WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-// General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License along
-// with this program; if not, write to the Free Software Foundation, Inc.,
-// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-//--------------------------------------------------------------------------
-
-// tcp_stream_session.h author davis mcpherson <davmcphe@cisco.com>
-// Created on: Feb 18, 2016
-
-#ifndef TCP_STREAM_SESSION_H
-#define TCP_STREAM_SESSION_H
-
-#include "detection/detection_engine.h"
-#include "flow/session.h"
-#include "protocols/ipv6.h"
-
-#include "tcp_stream_config.h"
-#include "tcp_stream_tracker.h"
-
-// FIXIT-L session tracking could be split from reassembly
-// into a separate module a la ip_session.cc and ip_defrag.cc
-// (of course defrag should also be cleaned up)
-class TcpStreamSession : public Session
-{
-public:
- ~TcpStreamSession() override;
-
- void clear() override;
- void cleanup(snort::Packet* = nullptr) override;
-
- void set_splitter(bool, snort::StreamSplitter*) override;
- snort::StreamSplitter* get_splitter(bool) override;
-
- bool is_sequenced(uint8_t dir) const override;
- bool are_packets_missing(uint8_t dir) const override;
-
- void disable_reassembly(snort::Flow*) override;
- uint8_t get_reassembly_direction() const override;
- uint8_t missing_in_reassembled(uint8_t dir) const override;
- bool are_client_segments_queued() const override;
-
- bool add_alert(snort::Packet*, uint32_t gid, uint32_t sid) override;
- bool check_alerted(snort::Packet*, uint32_t gid, uint32_t sid) override;
- int update_alert(snort::Packet*, uint32_t gid, uint32_t sid,
- uint32_t event_id, uint32_t event_second) override;
-
- bool set_packet_action_to_hold(snort::Packet*) override;
-
- uint16_t get_mss(bool to_server) const;
- uint8_t get_tcp_options_len(bool to_server) const;
-
- void reset();
- void start_proxy();
-
- void set_packet_header_foo(const TcpSegmentDescriptor&);
- void get_packet_header_foo(DAQ_PktHdr_t*, const DAQ_PktHdr_t* orig, uint32_t dir);
- bool can_set_no_ack();
- bool set_no_ack(bool);
- bool no_ack_mode_enabled() { return no_ack; }
- virtual void update_perf_base_state(char) = 0;
- virtual void clear_session(
- bool free_flow_data, bool flush_segments, bool restart, snort::Packet* p = nullptr) = 0;
- virtual TcpStreamTracker::TcpState get_talker_state(TcpSegmentDescriptor&) = 0;
- virtual TcpStreamTracker::TcpState get_listener_state(TcpSegmentDescriptor&) = 0;
- TcpStreamTracker::TcpState get_peer_state(const TcpStreamTracker* me)
- { return me == &client ? server.get_tcp_state() : client.get_tcp_state(); }
-
- virtual void init_new_tcp_session(TcpSegmentDescriptor&);
- virtual void update_timestamp_tracking(TcpSegmentDescriptor&) = 0;
- virtual void update_session_on_server_packet(TcpSegmentDescriptor&);
- virtual void update_session_on_client_packet(TcpSegmentDescriptor&);
- virtual void update_session_on_rst(TcpSegmentDescriptor&, bool) = 0;
- virtual bool handle_syn_on_reset_session(TcpSegmentDescriptor&) = 0;
- virtual void handle_data_on_syn(TcpSegmentDescriptor&) = 0;
- virtual void update_ignored_session(TcpSegmentDescriptor&) = 0;
- void generate_no_3whs_event()
- {
- if ( generate_3whs_alert && flow->two_way_traffic())
- {
- tel.set_tcp_event(EVENT_NO_3WHS);
- generate_3whs_alert = false;
- }
- }
-
- void set_pkt_action_flag(uint32_t flag)
- { pkt_action_mask |= flag; }
-
- void set_established(const TcpSegmentDescriptor&);
- void set_pseudo_established(snort::Packet*);
- bool check_for_one_sided_session(snort::Packet*);
- void check_for_pseudo_established(snort::Packet*);
-
- virtual void update_paws_timestamps(TcpSegmentDescriptor&) = 0;
- virtual void check_for_repeated_syn(TcpSegmentDescriptor&) = 0;
- virtual void check_for_session_hijack(TcpSegmentDescriptor&) = 0;
- virtual bool check_for_window_slam(TcpSegmentDescriptor&) = 0;
- virtual void mark_packet_for_drop(TcpSegmentDescriptor&) = 0;
- virtual bool validate_packet_established_session(TcpSegmentDescriptor&) = 0;
-
- TcpStreamTracker client;
- TcpStreamTracker server;
- bool lws_init = false;
- bool tcp_init = false;
- uint32_t pkt_action_mask = ACTION_NOTHING;
- uint32_t initiator_watermark = 0;
- int32_t ingress_index = DAQ_PKTHDR_UNKNOWN;
- int32_t egress_index = DAQ_PKTHDR_UNKNOWN;
- int16_t ingress_group = DAQ_PKTHDR_UNKNOWN;
- int16_t egress_group = DAQ_PKTHDR_UNKNOWN;
- uint32_t daq_flags = 0;
- uint32_t address_space_id = 0;
- bool generate_3whs_alert = true;
- TcpStreamConfig* tcp_config = nullptr;
- TcpEventLogger tel;
- bool cleaning = false;
- uint8_t held_packet_dir = SSN_DIR_NONE;
- uint8_t ecn = 0;
-
-private:
- bool no_ack = false;
-
-protected:
- TcpStreamSession(snort::Flow*);
- virtual void set_os_policy() = 0;
-};
-
-#endif
-
#include "main/analyzer.h"
#include "main/snort.h"
#include "packet_io/active.h"
+#include "packet_io/packet_tracer.h"
#include "profiler/profiler_defs.h"
#include "protocols/eth.h"
#include "pub_sub/stream_event_ids.h"
#include "stream/stream.h"
#include "held_packet_queue.h"
-#include "segment_overlap_editor.h"
+#include "tcp_overlap_resolver.h"
#include "tcp_normalizers.h"
-#include "tcp_reassemblers.h"
+#include "tcp_reassembler.h"
+#include "tcp_reassembler_ids.h"
+#include "tcp_reassembler_ips.h"
#include "tcp_session.h"
using namespace snort;
TcpStreamTracker::TcpStreamTracker(bool client) :
tcp_state(client ? TCP_STATE_NONE : TCP_LISTEN), client_tracker(client),
held_packet(null_iterator)
-{ }
+{
+ reassembler = new TcpReassemblerIgnore(*this, seglist);
+ reassembler->init(!client_tracker, nullptr);
+}
TcpStreamTracker::~TcpStreamTracker()
-{ if (splitter != nullptr) splitter->go_away(); }
+{
+ delete reassembler;
+
+ if( oaitw_reassembler )
+ {
+ delete oaitw_reassembler;
+ oaitw_reassembler = nullptr;
+ }
+
+ if ( splitter )
+ splitter->go_away();
+}
+
+void TcpStreamTracker::reset()
+{
+ tcp_alerts.clear();
+ normalizer.reset();
+ seglist.reset();
+ reassembler->reset_paf();
+}
+
+void TcpStreamTracker::clear_tracker(snort::Flow* flow, snort::Packet* p, bool flush_segments, bool restart)
+{
+ if ( flush_segments )
+ reassembler->flush_queued_segments(flow, true, p);
+
+ if ( p )
+ finalize_held_packet(p);
+ else
+ finalize_held_packet(flow);
+
+ seglist.purge_segment_list();
+
+ if ( restart )
+ reassembler->reset_paf();
+ else
+ reassembler->clear_paf();
+
+ set_splitter((StreamSplitter*)nullptr);
+}
+
+int TcpStreamTracker::eval_flush_policy_on_ack(snort::Packet* p)
+{
+ if( oaitw_reassembler )
+ {
+ delete oaitw_reassembler;
+ oaitw_reassembler = nullptr;
+ }
+
+ reassembler->eval_flush_policy_on_ack(p);
+
+ return 0;
+}
+
+int TcpStreamTracker::eval_flush_policy_on_data(snort::Packet* p)
+{
+ if( oaitw_reassembler )
+ {
+ delete oaitw_reassembler;
+ oaitw_reassembler = nullptr;
+ }
+
+ reassembler->eval_flush_policy_on_data(p);
+
+ return 0;
+}
TcpStreamTracker::TcpEvent TcpStreamTracker::set_tcp_event(const TcpSegmentDescriptor& tsd)
{
}
}
-
void TcpStreamTracker::set_fin_seq_status_seen(const TcpSegmentDescriptor& tsd)
{
if ( !fin_seq_set and SEQ_GEQ(tsd.get_end_seq(), r_win_base) )
fin_i_seq = tsd.get_seq();
fin_final_seq = tsd.get_end_seq();
fin_seq_set = true;
- fin_seq_status = TcpStreamTracker::FIN_WITH_SEQ_SEEN;
+ fin_seq_status = FIN_WITH_SEQ_SEEN;
}
}
-void TcpStreamTracker::init_tcp_state()
+void TcpStreamTracker::init_tcp_state(TcpSession* s)
{
+ session = s;
tcp_state = ( client_tracker ) ?
TcpStreamTracker::TCP_STATE_NONE : TcpStreamTracker::TCP_LISTEN;
mac_addr_valid = false;
fin_i_seq = 0;
fin_final_seq = 0;
- fin_seq_status = TcpStreamTracker::FIN_NOT_SEEN;
+ fin_seq_status = FIN_NOT_SEEN;
fin_seq_set = false;
rst_pkt_sent = false;
order = TcpStreamTracker::IN_SEQUENCE;
held_packet = null_iterator;
+
flush_policy = STREAM_FLPOLICY_IGNORE;
- reassembler.reset();
- splitter_finish_flag = false;
+ if( oaitw_reassembler )
+ {
+ delete oaitw_reassembler;
+ oaitw_reassembler = nullptr;
+ }
+ if ( reassembler )
+ delete reassembler;
+ reassembler = new TcpReassemblerIgnore(*this, seglist);
+ reassembler->init(!client_tracker, nullptr);
+
+ normalizer.reset();
+ seglist.reset();
+ tcp_alerts.clear();
}
-//-------------------------------------------------------------------------
-// flush policy stuff
-//-------------------------------------------------------------------------
+void TcpStreamTracker::update_stream_order(const TcpSegmentDescriptor& tsd, bool aligned)
+{
+ uint32_t seq = tsd.get_seq();
+
+ switch ( order )
+ {
+ case TcpStreamTracker::IN_SEQUENCE:
+ if ( aligned )
+ tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
+ else if ( SEQ_GT(seq, rcv_nxt) )
+ {
+ order = TcpStreamTracker::NONE;
+ hole_left_edge = rcv_nxt;
+ hole_right_edge = seq - 1;
+ }
+ break;
-void TcpStreamTracker::init_flush_policy()
+ case TcpStreamTracker::NONE:
+ if ( aligned )
+ {
+ tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
+ if ( SEQ_GT(tsd.get_end_seq(), hole_right_edge) )
+ order = TcpStreamTracker::OUT_OF_SEQUENCE;
+ else
+ hole_left_edge = tsd.get_end_seq();
+ }
+ else
+ {
+ if ( SEQ_LEQ(seq, hole_right_edge) )
+ {
+ if ( SEQ_GT(seq, hole_left_edge) )
+ hole_right_edge = seq - 1;
+ else if ( SEQ_GT(tsd.get_end_seq(), hole_left_edge) )
+ {
+ hole_left_edge = tsd.get_end_seq();
+ tsd.set_packet_flags(PKT_STREAM_ORDER_OK);
+ }
+ }
+ // accounting for overlaps when not aligned
+ if ( SEQ_GT(hole_left_edge, hole_right_edge) )
+ order = TcpStreamTracker::OUT_OF_SEQUENCE;
+ }
+ break;
+
+ case TcpStreamTracker::OUT_OF_SEQUENCE:
+ tsd.set_packet_flags(PKT_STREAM_ORDER_BAD);
+ }
+}
+
+void TcpStreamTracker::update_flush_policy(StreamSplitter* splitter)
{
- if ( !splitter )
- flush_policy = STREAM_FLPOLICY_IGNORE;
- else if ( normalizer.is_tcp_ips_enabled() )
- flush_policy = STREAM_FLPOLICY_ON_DATA;
+ if( oaitw_reassembler )
+ {
+ delete oaitw_reassembler;
+ oaitw_reassembler = nullptr;
+ }
+
+ if ( reassembler and flush_policy == reassembler->get_flush_policy() )
+ {
+ reassembler->init(!client_tracker, splitter);
+ return;
+ }
+
+ if ( flush_policy == STREAM_FLPOLICY_IGNORE )
+ {
+ // switching to Ignore flush policy...save pointer to current reassembler to delete later
+ if ( reassembler )
+ oaitw_reassembler = reassembler;
+
+ reassembler = new TcpReassemblerIgnore(*this, seglist);
+ reassembler->init(!client_tracker, splitter);
+ }
+ else if ( flush_policy == STREAM_FLPOLICY_ON_DATA )
+ {
+ if ( reassembler )
+ {
+ // update from IDS -> IPS is not supported
+ assert( reassembler->get_flush_policy() != STREAM_FLPOLICY_ON_ACK );
+ delete reassembler;
+ }
+
+ reassembler = new TcpReassemblerIps(*this, seglist);
+ reassembler->init(!client_tracker, splitter);
+ }
else
- flush_policy = STREAM_FLPOLICY_ON_ACK;
+ {
+ if ( reassembler )
+ {
+ // update from IPS -> IDS is not supported
+ assert( reassembler->get_flush_policy() != STREAM_FLPOLICY_ON_DATA );
+ delete reassembler;
+ }
+
+ reassembler = new TcpReassemblerIds(*this, seglist);
+ reassembler->init(!client_tracker, splitter);
+ }
}
void TcpStreamTracker::set_splitter(StreamSplitter* ss)
{
if ( splitter )
+ {
+ reassembler->release_splitter();
splitter->go_away();
+ }
splitter = ss;
-
- if ( !splitter )
- flush_policy = STREAM_FLPOLICY_IGNORE;
+ if ( ss )
+ {
+ if ( normalizer.is_tcp_ips_enabled() )
+ flush_policy = STREAM_FLPOLICY_ON_DATA;
+ else
+ flush_policy = STREAM_FLPOLICY_ON_ACK;
+ }
else
- reassembler.setup_paf();
+ flush_policy = STREAM_FLPOLICY_IGNORE;
+
+ update_flush_policy(ss);
}
void TcpStreamTracker::set_splitter(const Flow* flow)
ins = flow->clouseau;
if ( ins )
- set_splitter(ins->get_splitter(!client_tracker) );
+ set_splitter(ins->get_splitter(!client_tracker));
else
- set_splitter(new AtomSplitter(!client_tracker) );
+ set_splitter(new AtomSplitter(!client_tracker));
}
-bool TcpStreamTracker::splitter_finish(snort::Flow* flow)
+static inline bool both_splitters_aborted(Flow* flow)
{
- if (!splitter)
- return true;
+ uint32_t both_splitters_yoinked = (SSNFLAG_ABORT_CLIENT | SSNFLAG_ABORT_SERVER);
+ return (flow->get_session_flags() & both_splitters_yoinked) == both_splitters_yoinked;
+}
- if (!splitter_finish_flag)
+void TcpStreamTracker::fallback()
+{
+#ifndef NDEBUG
+ assert(splitter);
+
+ // FIXIT-L: consolidate these 3
+ //bool to_server = splitter->to_server();
+ //assert(server_side == to_server && server_side == !tracker.client_tracker);
+#endif
+
+ set_splitter(new AtomSplitter(!client_tracker));
+ tcpStats.partial_fallbacks++;
+
+ Flow* flow = session->flow;
+ if ( !client_tracker )
+ flow->set_session_flags(SSNFLAG_ABORT_SERVER);
+ else
+ flow->set_session_flags(SSNFLAG_ABORT_CLIENT);
+
+ if ( flow->gadget and both_splitters_aborted(flow) )
{
- splitter_finish_flag = true;
- return splitter->finish(flow);
+ flow->clear_gadget();
+
+ if (flow->clouseau)
+ flow->clear_clouseau();
+
+ tcpStats.inspector_fallbacks++;
}
- // there shouldn't be any un-flushed data beyond this point,
- // returning false here, discards it
- return false;
+}
+
+void TcpStreamTracker::disable_reassembly(Flow* f)
+{
+ set_splitter((StreamSplitter*)nullptr);
+ seglist.reset();
+ reassembler->reset_paf();
+ finalize_held_packet(f);
}
void TcpStreamTracker::init_on_syn_sent(TcpSegmentDescriptor& tsd)
tcpStats.sessions_on_syn++;
}
-void TcpStreamTracker::init_on_syn_recv(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::init_on_syn_recv(const TcpSegmentDescriptor& tsd)
{
irs = tsd.get_seq();
rcv_nxt = tsd.get_seq() + 1;
r_win_base = tsd.get_seq() + 1;
- reassembler.set_seglist_base_seq(tsd.get_seq() + 1);
+ seglist.set_seglist_base_seq(tsd.get_seq() + 1);
cache_mac_address(tsd, FROM_CLIENT);
tcp_state = TcpStreamTracker::TCP_SYN_RECV;
r_win_base = tsd.get_ack();
rcv_nxt = tsd.get_ack();
- reassembler.set_seglist_base_seq(tsd.get_ack() );
+ seglist.set_seglist_base_seq(tsd.get_ack() );
ts_last_packet = tsd.get_packet_timestamp();
tf_flags |= normalizer.get_tcp_timestamp(tsd, false);
tcpStats.sessions_on_syn_ack++;
}
-void TcpStreamTracker::init_on_synack_recv(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::init_on_synack_recv(const TcpSegmentDescriptor& tsd)
{
iss = tsd.get_ack() - 1;
irs = tsd.get_seq();
rcv_nxt = tsd.get_seq() + 1;
r_win_base = tsd.get_seq() + 1;
- reassembler.set_seglist_base_seq(tsd.get_seq() + 1);
+ seglist.set_seglist_base_seq(tsd.get_seq() + 1);
cache_mac_address(tsd, FROM_SERVER);
if ( TcpStreamTracker::TCP_SYN_SENT == tcp_state )
r_win_base = tsd.get_ack();
rcv_nxt = tsd.get_ack();
- reassembler.set_seglist_base_seq(tsd.get_ack());
+ seglist.set_seglist_base_seq(tsd.get_ack());
ts_last_packet = tsd.get_packet_timestamp();
tf_flags |= normalizer.get_tcp_timestamp(tsd, false);
tcp_state = TcpStreamTracker::TCP_MID_STREAM_SENT;
}
-void TcpStreamTracker::init_on_data_seg_recv(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::init_on_data_seg_recv(const TcpSegmentDescriptor& tsd)
{
iss = tsd.get_ack() - 1;
irs = tsd.get_seq() - 1;
rcv_nxt = tsd.get_seq();
r_win_base = tsd.get_seq();
- reassembler.set_seglist_base_seq(tsd.get_seq());
+ seglist.set_seglist_base_seq(tsd.get_seq());
cache_mac_address(tsd, tsd.get_direction() );
tcpStats.sessions_on_data++;
tf_flags |= ( tsd.init_mss(&mss) | tsd.init_wscale(&wscale) );
}
-void TcpStreamTracker::finish_client_init(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::finish_client_init(const TcpSegmentDescriptor& tsd)
{
Flow* flow = tsd.get_flow();
rcv_nxt = tsd.get_end_seq();
- if ( reassembler.data_was_queued() )
+ if ( seglist.data_was_queued() )
return; // we already have state, don't mess it up
if ( !( flow->session_state & STREAM_STATE_MIDSTREAM ) )
{
if ( tsd.get_tcph()->is_syn() )
- reassembler.set_seglist_base_seq(tsd.get_seq() + 1);
+ seglist.set_seglist_base_seq(tsd.get_seq() + 1);
else
- reassembler.set_seglist_base_seq(tsd.get_seq());
+ seglist.set_seglist_base_seq(tsd.get_seq());
r_win_base = tsd.get_end_seq();
}
else
{
- reassembler.set_seglist_base_seq(tsd.get_seq());
+ seglist.set_seglist_base_seq(tsd.get_seq());
r_win_base = tsd.get_seq();
}
}
if ( SEQ_LT(snd_nxt, snd_una) )
snd_nxt = snd_una;
}
- if ( !tsd.get_len() and snd_wnd == 0
- and SEQ_LT(tsd.get_seq(), r_win_base) )
- tcpStats.zero_win_probes++;
+ if ( !tsd.get_len() and SEQ_LT(tsd.get_seq(), r_win_base) )
+ {
+ if ( snd_wnd == 0 )
+ tcpStats.zero_win_probes++;
+ else if ( (r_win_base - tsd.get_seq()) == MAX_KEEP_ALIVE_PROBE_LEN
+ and !(tsd.get_tcph()->th_flags & (TH_SYN|TH_FIN|TH_RST)) )
+ tcpStats.keep_alive_probes++;
+ }
}
// In no-ack policy, data is implicitly acked immediately.
-void TcpStreamTracker::update_tracker_no_ack_recv(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::update_tracker_no_ack_recv(const TcpSegmentDescriptor& tsd)
{
snd_una = snd_nxt = tsd.get_end_seq();
}
-void TcpStreamTracker::update_tracker_no_ack_sent(TcpSegmentDescriptor& tsd)
+void TcpStreamTracker::update_tracker_no_ack_sent(const TcpSegmentDescriptor& tsd)
{
r_win_base = tsd.get_end_seq();
- reassembler.flush_on_ack_policy(tsd.get_pkt());
+ eval_flush_policy_on_ack(tsd.get_pkt());
}
void TcpStreamTracker::update_tracker_ack_sent(TcpSegmentDescriptor& tsd)
snd_wnd = tsd.get_wnd();
}
- if ( ( fin_seq_status == TcpStreamTracker::FIN_WITH_SEQ_SEEN )
+ if ( flush_policy == STREAM_FLPOLICY_IGNORE
+ and SEQ_GT(tsd.get_ack(), rcv_nxt) )
+ rcv_nxt = tsd.get_ack();
+
+ if ( ( fin_seq_status == FIN_WITH_SEQ_SEEN )
&& SEQ_GEQ(tsd.get_ack(), fin_final_seq + 1) && !(tsd.is_meta_ack_packet()) )
{
- fin_seq_status = TcpStreamTracker::FIN_WITH_SEQ_ACKED;
+ fin_seq_status = FIN_WITH_SEQ_ACKED;
}
- reassembler.flush_on_ack_policy(tsd.get_pkt());
+ eval_flush_policy_on_ack(tsd.get_pkt());
}
bool TcpStreamTracker::update_on_3whs_ack(TcpSegmentDescriptor& tsd)
return true;
}
+int32_t TcpStreamTracker::kickstart_asymmetric_flow(const TcpSegmentDescriptor& tsd, uint32_t max_queued_bytes)
+{
+ seglist.skip_holes();
+
+ if ( reassembler->is_splitter_paf() )
+ fallback();
+ else
+ reassembler->reset_paf();
+
+ eval_flush_policy_on_data(tsd.get_pkt());
+
+ int32_t space_left = max_queued_bytes - seglist.get_seg_bytes_total();
+
+ if ( get_tcp_state() == TcpStreamTracker::TCP_MID_STREAM_RECV )
+ {
+ set_tcp_state(TcpStreamTracker::TCP_ESTABLISHED);
+ if (PacketTracer::is_active())
+ PacketTracer::log("Stream: Kickstart of midstream asymmetric flow! Seglist queue space: %u\n",
+ space_left );
+ }
+ else
+ {
+ if (PacketTracer::is_active())
+ PacketTracer::log("Stream: Kickstart of asymmetric flow! Seglist queue space: %u\n",
+ space_left );
+ }
+
+ return space_left;
+}
+
void TcpStreamTracker::perform_fin_recv_flush(TcpSegmentDescriptor& tsd)
{
if ( tsd.is_data_segment() )
if ( flush_policy == STREAM_FLPOLICY_ON_DATA and SEQ_EQ(tsd.get_end_seq(), rcv_nxt)
and !tsd.get_flow()->searching_for_service() )
- reassembler.finish_and_final_flush(tsd.get_flow(), true, tsd.get_pkt());
+ reassembler->finish_and_final_flush(tsd.get_flow(), true, tsd.get_pkt());
}
uint32_t TcpStreamTracker::perform_partial_flush()
if ( held_packet != null_iterator )
{
Packet* p;
- flushed = reassembler.perform_partial_flush(session->flow, p);
+ flushed = reassembler->perform_partial_flush(session->flow, p);
// If the held_packet hasn't been released by perform_partial_flush(),
// call finalize directly.
tcpStats.held_packets_passed++;
}
- TcpStreamSession* tcp_session = (TcpStreamSession*)cp->flow->session;
+ TcpSession* tcp_session = (TcpSession*)cp->flow->session;
tcp_session->held_packet_dir = SSN_DIR_NONE;
}
}
else
{
- TcpStreamSession* tcp_session = (TcpStreamSession*)flow->session;
+ TcpSession* tcp_session = (TcpSession*)flow->session;
tcp_session->held_packet_dir = SSN_DIR_NONE;
Analyzer::get_local_analyzer()->finalize_daq_message(msg, DAQ_VERDICT_PASS);
tcpStats.held_packets_passed++;
#include <cstdint>
#include <list>
-#include "stream/paf.h"
-
-#include "segment_overlap_editor.h"
+#include "tcp_alerts.h"
#include "tcp_defs.h"
#include "tcp_module.h"
#include "tcp_normalizers.h"
-#include "tcp_reassemblers.h"
+#include "tcp_reassembler.h"
+#include "tcp_reassembly_segments.h"
+#include "tcp_segment_node.h"
#include "tcp_segment_descriptor.h"
extern const char* tcp_state_names[];
}
class HeldPacket;
-class TcpReassembler;
class TcpSession;
+enum FinSeqNumStatus : uint8_t { FIN_NOT_SEEN, FIN_WITH_SEQ_SEEN, FIN_WITH_SEQ_ACKED };
+
class TcpStreamTracker
{
public:
enum PacketOrder : uint8_t { OUT_OF_SEQUENCE, IN_SEQUENCE, NONE };
- enum FinSeqNumStatus : uint8_t { FIN_NOT_SEEN, FIN_WITH_SEQ_SEEN, FIN_WITH_SEQ_ACKED };
-
TcpStreamTracker(bool client);
- virtual ~TcpStreamTracker();
+ ~TcpStreamTracker();
+
+ void reset();
+ void clear_tracker(snort::Flow*, snort::Packet*, bool flush_segments, bool restart);
+
+ int eval_flush_policy_on_ack(snort::Packet*);
+ int eval_flush_policy_on_data(snort::Packet*);
+ void update_stream_order(const TcpSegmentDescriptor&, bool aligned);
+
+ void fallback();
bool is_client_tracker() const
{ return client_tracker; }
bool is_rst_pkt_sent() const
{ return rst_pkt_sent; }
- void set_flush_policy(FlushPolicy policy)
- { flush_policy = policy; }
-
+ void set_flush_policy(FlushPolicy policy);
FlushPolicy get_flush_policy() const
{ return flush_policy; }
- virtual void init_tcp_state();
- virtual void init_flush_policy();
- virtual void set_splitter(snort::StreamSplitter* ss);
- virtual void set_splitter(const snort::Flow* flow);
+ void set_order(uint8_t order)
+ { this->order = order; }
+
+ void init_tcp_state(TcpSession*);
+ void set_splitter(snort::StreamSplitter* ss);
+ void set_splitter(const snort::Flow* flow);
snort::StreamSplitter* get_splitter()
{ return splitter; }
- bool is_splitter_paf() const
- { return splitter && splitter->is_paf(); }
-
- bool splitter_finish(snort::Flow* flow);
-
- bool is_reassembly_enabled() const
- { return ( splitter and (flush_policy != STREAM_FLPOLICY_IGNORE) ); }
-
- virtual void init_on_syn_sent(TcpSegmentDescriptor&);
- virtual void init_on_syn_recv(TcpSegmentDescriptor&);
- virtual void init_on_synack_sent(TcpSegmentDescriptor&);
- virtual void init_on_synack_recv(TcpSegmentDescriptor&);
- virtual void init_on_data_seg_sent(TcpSegmentDescriptor&);
- virtual void init_on_data_seg_recv(TcpSegmentDescriptor&);
- virtual void finish_server_init(TcpSegmentDescriptor&);
- virtual void finish_client_init(TcpSegmentDescriptor&);
-
- virtual void update_tracker_ack_recv(TcpSegmentDescriptor&);
- virtual void update_tracker_ack_sent(TcpSegmentDescriptor&);
- virtual void update_tracker_no_ack_recv(TcpSegmentDescriptor&);
- virtual void update_tracker_no_ack_sent(TcpSegmentDescriptor&);
- virtual bool update_on_3whs_ack(TcpSegmentDescriptor&);
- virtual bool update_on_rst_recv(TcpSegmentDescriptor&);
- virtual void update_on_rst_sent();
- virtual bool update_on_fin_recv(TcpSegmentDescriptor&);
- virtual bool update_on_fin_sent(TcpSegmentDescriptor&);
- virtual bool is_segment_seq_valid(TcpSegmentDescriptor&);
+ void disable_reassembly(snort::Flow* f);
+
+ void init_on_syn_sent(TcpSegmentDescriptor&);
+ void init_on_syn_recv(const TcpSegmentDescriptor&);
+ void init_on_synack_sent(TcpSegmentDescriptor&);
+ void init_on_synack_recv(const TcpSegmentDescriptor&);
+ void init_on_data_seg_sent(TcpSegmentDescriptor&);
+ void init_on_data_seg_recv(const TcpSegmentDescriptor&);
+ void finish_server_init(TcpSegmentDescriptor&);
+ void finish_client_init(const TcpSegmentDescriptor&);
+
+ void update_tracker_ack_recv(TcpSegmentDescriptor&);
+ void update_tracker_ack_sent(TcpSegmentDescriptor&);
+ void update_tracker_no_ack_recv(const TcpSegmentDescriptor&);
+ void update_tracker_no_ack_sent(const TcpSegmentDescriptor&);
+ bool update_on_3whs_ack(TcpSegmentDescriptor&);
+ bool update_on_rst_recv(TcpSegmentDescriptor&);
+ void update_on_rst_sent();
+ bool update_on_fin_recv(TcpSegmentDescriptor&);
+ bool update_on_fin_sent(TcpSegmentDescriptor&);
+ bool is_segment_seq_valid(TcpSegmentDescriptor&);
bool set_held_packet(snort::Packet*);
bool is_retransmit_of_held_packet(snort::Packet*);
void finalize_held_packet(snort::Packet*);
void finalize_held_packet(snort::Flow*);
void perform_fin_recv_flush(TcpSegmentDescriptor&);
+ int32_t kickstart_asymmetric_flow(const TcpSegmentDescriptor& tsd, uint32_t max_queued_bytes);
uint32_t perform_partial_flush();
- bool is_holding_packet() const { return held_packet != null_iterator; }
+ bool is_holding_packet() const
+ { return held_packet != null_iterator; }
// max_remove < 0 means time out all eligible packets.
// Return whether there are more packets that need to be released.
static void thread_term();
public:
- uint32_t snd_una = 0; // SND.UNA - send unacknowledged
- uint32_t snd_nxt = 0; // SND.NXT - send next
- uint32_t snd_wnd = 0; // SND.WND - send window
- uint32_t snd_wl1 = 0; // SND.WL1 - segment sequence number used for last window update
- uint32_t snd_wl2 = 0; // SND.WL2 - segment acknowledgment number used for last window update
- uint32_t iss = 0; // ISS - initial send sequence number
+ uint32_t snd_una = 0; // SND.UNA - send unacknowledged
+ uint32_t snd_nxt = 0; // SND.NXT - send next
+ uint32_t snd_wnd = 0; // SND.WND - send window
+ uint32_t snd_wl1 = 0; // SND.WL1 - segment sequence number used for last window update
+ uint32_t snd_wl2 = 0; // SND.WL2 - segment acknowledgment number used for last window update
+ uint32_t iss = 0; // ISS - initial send sequence number
- uint32_t rcv_nxt = 0; // RCV.NXT - receive next
- uint32_t rcv_wnd = 0; // RCV.WND - receive window
- uint32_t irs = 0; // IRS - initial receive sequence number
+ uint32_t rcv_nxt = 0; // RCV.NXT - receive next
+ uint32_t rcv_wnd = 0; // RCV.WND - receive window
+ uint32_t irs = 0; // IRS - initial receive sequence number
- uint16_t snd_up = 0; // SND.UP - send urgent pointer
- uint16_t rcv_up = 0; // RCV.UP - receive urgent pointer
+ uint16_t snd_up = 0; // SND.UP - send urgent pointer
+ uint16_t rcv_up = 0; // RCV.UP - receive urgent pointer
uint32_t held_pkt_seq = 0;
- uint32_t hole_left_edge = 0; // First left hole
- uint32_t hole_right_edge = 0;
TcpState tcp_state;
TcpEvent tcp_event = TCP_MAX_EVENTS;
// FIXIT-L make these non-public
public:
TcpNormalizerPolicy normalizer;
- TcpReassemblerPolicy reassembler;
+ TcpReassemblySegments seglist;
+ TcpReassembler* reassembler = nullptr;
+ TcpReassembler* oaitw_reassembler = nullptr;
TcpSession* session = nullptr;
+ TcpAlerts tcp_alerts;
uint32_t r_win_base = 0; // remote side window base sequence number (the last ack we got)
uint32_t small_seg_count = 0;
- uint8_t order = IN_SEQUENCE;
- FinSeqNumStatus fin_seq_status = TcpStreamTracker::FIN_NOT_SEEN;
+ FinSeqNumStatus fin_seq_status = FIN_NOT_SEEN;
+
+private:
+ void update_flush_policy(snort::StreamSplitter*);
-protected:
+ snort::StreamSplitter* splitter = nullptr;
static const std::list<HeldPacket>::iterator null_iterator;
std::list<HeldPacket>::iterator held_packet;
- snort::StreamSplitter* splitter = nullptr;
uint32_t ts_last_packet = 0;
uint32_t ts_last = 0; // last timestamp (for PAWS)
uint32_t fin_final_seq = 0;
FlushPolicy flush_policy = STREAM_FLPOLICY_IGNORE;
bool mac_addr_valid = false;
bool fin_seq_set = false; // FIXIT-M should be obviated by tcp state
- bool splitter_finish_flag = false;
+
+ uint8_t order = IN_SEQUENCE;
+ uint32_t hole_left_edge = 0; // First left hole
+ uint32_t hole_right_edge = 0;
};
// <--- note -- the 'state' parameter must be a reference
return statext[t.get_tcp_state()];
}
-#ifndef DEBUG_MSGS
-void S5TraceTCP(const TcpSegmentDescriptor&, const snort::Packet*) { }
-#else
#define LCL(p, x) ((p).x() - (p).get_iss())
#define RMT(p, x, q) ((p).x - (q).get_iss())
uint32_t rseq = txd ? tsd.get_seq() - txd : tsd.get_seq();
uint32_t rack = rxd ? tsd.get_ack() - rxd : tsd.get_ack();
- debug_logf(stream_tcp_trace, TRACE_STATE, p,
+ trace_logf(DEFAULT_TRACE_LOG_LEVEL, stream_tcp_trace, TRACE_STATE, p,
FMTu64("-3") " %s %s=0x%02x Seq=%-4u Ack=%-4u Win=%-4u Len=%-4hu%s\n",
tsd.get_packet_number(), meta_ack_marker, flags, h->th_flags, rseq, rack, tsd.get_wnd(),
tsd.get_len(), order);
inline void TraceSession(const snort::Flow* flow, const snort::Packet* p)
{
- debug_logf(stream_tcp_trace, TRACE_STATE, p,
+ trace_logf(DEFAULT_TRACE_LOG_LEVEL, stream_tcp_trace, TRACE_STATE, p,
" LWS: ST=0x%x SF=0x%x CP=%hu SP=%hu\n", (unsigned)flow->session_state,
flow->ssn_state.session_flags, flow->client_port, flow->server_port);
}
-inline void TraceSegments(const TcpReassemblerPolicy& trp, const snort::Packet* p)
+inline void TraceSegments(const TcpReassemblySegments& seglist, const snort::Packet* p)
{
- const TcpSegmentNode* tsn = trp.trs.sos.seglist.head;
- uint32_t sx = trp.trs.tracker->r_win_base;
- unsigned segs = 0;
- unsigned bytes = 0;
- std::stringstream ss;
-
if ( !trace_enabled(stream_tcp_trace, TRACE_SEGMENTS) )
return;
+ const TcpSegmentNode* tsn = seglist.head;
+ uint32_t sx = seglist.tracker->r_win_base;
+ unsigned segs = 0;
+ std::stringstream ss;
+
while ( tsn )
{
- if ( SEQ_LT(sx, tsn->i_seq) )
- ss << " +" << tsn->i_seq - sx;
- else if ( SEQ_GT(sx, tsn->i_seq) )
- ss << " -" << sx - tsn->i_seq;
+ uint32_t seq = tsn->start_seq();
+
+ if ( SEQ_LT(sx, seq) )
+ ss << " +" << seq - sx;
- ss << " " << tsn->i_len;
+ else if ( SEQ_GT(sx, seq) )
+ ss << " -" << sx - seq;
- if ( tsn->c_len and tsn->c_len != tsn->i_len )
- {
- ss << "(" << tsn->offset << "|" << tsn->c_len;
- ss << "|" << tsn->i_len-tsn->offset-tsn->c_len << ")";
- }
+ ss << " " << tsn->length;
+
+ if ( tsn->cursor and tsn->unscanned() > 0 )
+ ss << "(" << tsn->cursor << "|" << tsn->unscanned() << ")";
segs++;
- bytes += tsn->i_len;
- sx = tsn->i_seq + tsn->i_len;
+ sx = seq + tsn->length;
tsn = tsn->next;
}
if ( !ss.str().empty() )
- debug_logf(stream_tcp_trace, TRACE_SEGMENTS, p, " %s\n", ss.str().c_str());
-
- assert(trp.trs.sos.seg_count == segs);
- assert(trp.trs.sos.seg_bytes_logical == bytes);
+ trace_logf(DEFAULT_TRACE_LOG_LEVEL, stream_tcp_trace, TRACE_SEGMENTS, p, " %s\n", ss.str().c_str());
}
inline void TraceState(const TcpStreamTracker& a, const TcpStreamTracker& b, const char* s,
uint32_t ua = a.get_snd_una() ? LCL(a, get_snd_una) : 0;
uint32_t ns = a.get_snd_nxt() ? LCL(a, get_snd_nxt) : 0;
- debug_logf(stream_tcp_trace, TRACE_STATE, p,
+ trace_logf(DEFAULT_TRACE_LOG_LEVEL, stream_tcp_trace, TRACE_STATE, p,
" %s ST=%s UA=%-4u NS=%-4u LW=%-5u RN=%-4u RW=%-4u ISS=%-4u IRS=%-4u\n",
s, stream_tcp_state_to_str(a), ua, ns, a.get_snd_wnd( ),
RMT(a, rcv_nxt, b), RMT(a, r_win_base, b), a.get_iss(), a.get_irs());
- unsigned paf = a.is_splitter_paf() ? 2 : 0;
+ unsigned paf = a.reassembler->is_splitter_paf() ? 2 : 0;
unsigned fpt = a.get_flush_policy() ? 192 : 0;
- debug_logf(stream_tcp_trace, TRACE_STATE, p,
+ trace_logf(DEFAULT_TRACE_LOG_LEVEL, stream_tcp_trace, TRACE_STATE, p,
" FP=%s:%-4u SC=%-4u FL=%-4u SL=%-5u BS=%-4u\n",
flushxt[a.get_flush_policy() + paf], fpt,
- a.reassembler.get_seg_count(), a.reassembler.get_flush_count(),
- a.reassembler.get_seg_bytes_logical(),
- a.reassembler.get_seglist_base_seq() - b.get_iss());
+ a.seglist.get_seg_count(), a.seglist.get_flush_count(),
+ a.seglist.get_seg_bytes_logical(),
+ a.seglist.get_seglist_base_seq() - b.get_iss());
- TraceSegments(a.reassembler, p);
+ TraceSegments(a.seglist, p);
}
void S5TraceTCP(const TcpSegmentDescriptor& tsd, const snort::Packet* p)
TraceEvent(tsd, txd, rxd, p);
- if ( ssn->lws_init )
+ if ( ssn->tcp_init )
TraceSession(tsd.get_flow(), p);
TraceState(cli, srv, cdir, p);
TraceState(srv, cli, sdir, p);
}
-#endif // DEBUG_MSGS
void TcpSession::set_extra_data(Packet*, unsigned int){ }
bool TcpSession::is_sequenced(unsigned char) const { return true; }
bool TcpSession::are_packets_missing(unsigned char) const { return false; }
-uint8_t TcpSession::get_reassembly_direction() const { return 0; }
uint8_t TcpSession::missing_in_reassembled(unsigned char) const { return 0; }
class TcpSessionMock : public TcpSession
This directory contains the trace logger framework.
-* TraceLogger
+==== TraceLogger
- The TraceLogger encapsulates the logging method for traces. A particular thread-local logger
- is created per each packet thread and one for the main thread. The logging configuration
- happens in the module. The logger factory is used to init/cleanup loggers.
+The TraceLogger encapsulates the logging method for traces. A particular thread-local logger
+is created per each packet thread and one for the main thread. The logging configuration
+happens in the module. The logger factory is used to init/cleanup loggers.
- Include "trace_logger.h" to get TraceLogger base class.
- Built-in loggers are defined in "trace_loggers.h/trace_loggers.cc".
+Include "trace_logger.h" to get TraceLogger base class.
+Built-in loggers are defined in "trace_loggers.h"/"trace_loggers.cc".
-* TraceLoggerFactory
+==== TraceLoggerFactory
- The base TraceLoggerFactory is used to create a particular TraceLogger instance per each
- thread. One factory instance exists which used to init/cleanup loggers and placed
- into TraceConfig. The factory object instantiates in the module due to configuration.
+The base TraceLoggerFactory is used to create a particular TraceLogger instance per each
+thread. One factory instance exists which used to init/cleanup loggers and placed
+into TraceConfig. The factory object instantiates in the module due to configuration.
- Include "trace_logger.h" to get TraceLoggerFactory base class and template function
- to create particular objects. Built-in factories are defined in "trace_loggers.h/trace_loggers.cc".
+Include "trace_logger.h" to get TraceLoggerFactory base class and template function
+to create particular objects. Built-in factories are defined in "trace_loggers.h/trace_loggers.cc".
-* TraceConfig
+==== TraceConfig
- This is a class that contains pointer on TraceLoggerFactory object and Trace list.
- By default all Traces are off. TraceModule fills Traces from configuration.
+This is a class that contains pointer on TraceLoggerFactory object and Trace list.
+By default all Traces are off. TraceModule fills Traces from configuration.
- TraceConfig placed into "trace_config.h".
+TraceConfig placed into "trace_config.h".
-* TraceModule
+==== TraceModule
- This module provides configuration for trace logs:
- output - create a concrete logger factory based on the output value (stdout/syslog).
- constraints - set packet constraints to use for trace filtering.
- modules - set modules trace level verbosity.
- ntuple - on/off packet n-tuple info logging.
- timestamp - on/off message time stamps.
+This module provides configuration for trace logs:
- This is a built-in module (from coreinit.lua)
+* `output` - create a concrete logger factory based on the output value (stdout/syslog).
+* `constraints` - set packet constraints to use for trace filtering.
+* `modules` - set modules trace level verbosity.
+* `ntuple` - on/off packet n-tuple info logging.
+* `timestamp` - on/off message time stamps.
- The module placed into "trace_module.h/trace_module.cc".
+This is a built-in module (from coreinit.lua)
+The module placed into "trace_module.h"/"trace_module.cc".
- TraceModule ctor should be called after all existed modules due to TraceModule
- dynamic params restriction.
+TraceModule ctor should be called after all existed modules due to TraceModule
+dynamic params restriction.
-* TraceSwap
+==== TraceSwap
- This class extends snort::AnalyzerCommand and represents control channel CLI commands
- for setting/clearing trace configuration from the shell. Commands parameters are encapsulated in
- the TraceSwapParams class.
- Available commands:
- 1) trace.set({ modules = {}, constraints = {}, ntuple = true/false, timestamp = true/false })
- -- set new modules traces, constraints, ntuple and timestamp options;
- 2) trace.clear() -- clear modules traces and constraints.
+This class extends snort::AnalyzerCommand and represents control channel CLI commands
+for setting/clearing trace configuration from the shell. Commands parameters are encapsulated in
+the TraceSwapParams class.
+Available commands:
-* TraceSwapParams
+* `trace.set({ modules = {}, constraints = {}, ntuple = true/false, timestamp = true/false })`
+ -- set new modules traces, constraints, ntuple and timestamp options;
+* `trace.clear()` -- clear modules traces and constraints.
- This is a helper class for TraceSwap which encapsulates dynamic initialization and storing of
- snort::Parameter and snort::Command lists based on TraceModule's parameters.
+==== TraceSwapParams
-* TraceParser
+This is a helper class for TraceSwap which encapsulates dynamic initialization and storing of
+snort::Parameter and snort::Command lists based on TraceModule's parameters.
- This class encapsulates module trace options and packet constraints parsing and setting logic.
+==== TraceParser
-* TraceApi
+This class encapsulates module trace options and packet constraints parsing and setting logic.
- TraceApi is a facade API class used to init/reinit/term thread-local trace logger and module's
- trace pointers and to match packets and flows against trace filtering constraints.
+==== TraceApi
- TraceApi placed into "trace_api.h/trace_api.cc"
+TraceApi is a facade API class used to init/reinit/term thread-local trace logger and module's
+trace pointers and to match packets and flows against trace filtering constraints.
-* TraceOption
+TraceApi placed into "trace_api.h"/"trace_api.cc"
- Represents a targeted module's trace option.
- The targeted module provides a list of its trace options,
- which control the verbosity level of the module's trace messages.
- Since the messages can be present in release-build and/or debug-build,
- one should pay attention which option should be in the options list.
- TraceOptionID indexes (for the module's trace options) must start from zero.
+==== TraceOption
-* Trace
+Represents a targeted module's trace option.
+The targeted module provides a list of its trace options,
+which control the verbosity level of the module's trace messages.
+Since the messages can be present in release-build and/or debug-build,
+one should pay attention which option should be in the options list.
+TraceOptionID indexes (for the module's trace options) must start from zero.
- This class encapsulates trace verbosity functionality.
+==== Trace
-* Configuration override
+This class encapsulates trace verbosity functionality.
- By default, the factory will be initialized based on Snort run mode (stdout or syslog).
- But this can be explicitly overwritten in TraceModule's configuration to specify
- which kind of logger to use.
+==== Configuration override
- Changes will be applied in case of reloading too.
+By default, the factory will be initialized based on Snort run mode (stdout or syslog).
+But this can be explicitly overwritten in TraceModule's configuration to specify
+which kind of logger to use.
-* External dependencies
+Changes will be applied in case of reloading too.
- Include TraceApi header somewhere in the code where it needs to make trace logs.
- Make sure that TraceApi::thread_init()/thread_term() are provided for thread where
- TraceApi::log() is going used.
+==== External dependencies
- TraceConfig should be configured in SnortConfig before TraceApi init.
+Include TraceApi header somewhere in the code where it needs to make trace logs.
+Make sure that TraceApi::thread_init()/thread_term() are provided for thread where
+TraceApi::log() is going used.
- To create specific TraceLogger/TraceLoggerFactory pair just inherit base classes placed
- into "trace_logger.h" and init TraceConfig with a new factory during configuration.
+TraceConfig should be configured in SnortConfig before TraceApi init.
-* Extending the trace logger framework with TraceLogger plugins
+To create specific TraceLogger/TraceLoggerFactory pair just inherit base classes placed
+into "trace_logger.h" and init TraceConfig with a new factory during configuration.
- It's possible to create a trace logger as an inspector plugin to handle a custom logic of trace
- messages printing. The workflow here is to implement the custom logger and logger factory by
- inheriting from the Snort TraceLogger and TraceLoggerFactory classes, put them into a separate
- plugin, and call TraceApi::override_logger_factory() during the plugin configuration to
- initialize the framework with the custom logger factory.
+==== Extending the trace logger framework with TraceLogger plugins
-* Disabling packet constraints matching
+It's possible to create a trace logger as an inspector plugin to handle a custom logic of trace
+messages printing. The workflow here is to implement the custom logger and logger factory by
+inheriting from the Snort TraceLogger and TraceLoggerFactory classes, put them into a separate
+plugin, and call TraceApi::override_logger_factory() during the plugin configuration to
+initialize the framework with the custom logger factory.
- Constraints matching can be disabled by setting "trace.constraints.match = false" during
- configuration. This is effectively saying all packets don't pass the trace filter. It may be
- useful in case of using external packet filtering, such as filtering by the DAQ, or to block
- printing for all trace messages in the context of a packet.
+==== Disabling packet constraints matching
+
+Constraints matching can be disabled by setting "trace.constraints.match = false" during
+configuration. This is effectively saying all packets don't pass the trace filter. It may be
+useful in case of using external packet filtering, such as filtering by the DAQ, or to block
+printing for all trace messages in the context of a packet.
#include "util.h"
+#include <sys/resource.h>
#include <sys/stat.h>
#include <chrono>
return true;
}
-#if defined(NOCOREFILE)
-void SetNoCores()
-{
- struct rlimit rlim;
-
- getrlimit(RLIMIT_CORE, &rlim);
- rlim.rlim_max = 0;
- setrlimit(RLIMIT_CORE, &rlim);
-}
-#endif
-
namespace snort
{
const char* get_error(int errnum)
// data (variables, includes, etcs), one api for creating tables. Hoever,
// the reason they are together is because this class is not static, and I
// did not want to be pass three pointers to the three API's when creating
-// new conversion states. There are comments in in all caps which show the
+// new conversion states. There are comments in all caps which show the
// separate the sections.
// The first section of this file is really DataApi creation and
projects / thirdparty / snort3.git / commitdiff
