HTTP reserved GZIP flags are set
+119:289
+
+Too many partial flushes. Partial depth is enabled and 20 or more partial flushes are
+made before a regular flush.
+
121:1
Invalid flag set on HTTP/2 frame header
This feature is off by default. script_detection = true will activate it.
+===== partial_depth
+
+Partial depth detection is a feature that enables Snort to more quickly detect
+and block malicious requests. It is configured by the partial_depth parameter
+which can take values in the range -1-16384 bytes. The feature is enabled by
+setting partial_depth to some non zero value. When the feature is enabled and
+either, the number of bytes received in the request body is below the value
+specified by partial_depth, or partial_depth is set to -1, unlimited; it
+immediately forwards the available part of the message body for early detection.
+This enables earlier threat detection but consumes somewhat more of the sensor's
+resources.
+
+This feature is turned off by default by setting partial_depth = 0. To activate
+it, set partial_depth to the desired value.
+
===== gzip
http_inspect by default decompresses deflate and gzip message bodies
The nature of splitting allows packets to be forwarded before they are aggregated into a message
section and inspected. This may lead to problems when the target consumes a partial message
body even though the end of the message body was never received because Snort blocked it.
-
-Script detection is a feature developed to solve this problem for message bodies containing
-Javascripts. The stream splitter scan() method searches its input for the end-of-script tag
-"</script>". When necessary this requires scan() to unzip the data. This is an extra unzip as
-storage limitations preclude saving the unzipped version of the data for subsequent reassembly.
-
-Update: the previous sentence has been discovered to be incorrect. The memory requirements of
-zlib are very large. It would save a lot of memory and some processing time for script detection
-to unzip one time in scan() and store the result for eventual use by reassemble(). The memory
-lost by storing partial message sections in HI while waiting for reassemble() would be more than
-compensated for by not having two instances of zlib.
+Partial inspection was developed to solve this problem.
HttpFlowData is a data class representing all HI information relating to a flow. It serves as
persistent memory between invocations of HI by the framework. It also glues together the inspector,
-When the end of a script is found and the normal flush point has not been found, the current TCP
-segment and all previous segments for the current message section are flushed using a special
-procedure known as partial inspection. From the perspective of Stream (or H2I) a partial inspection
-is a regular flush in every respect.
-
-scan() calls prep_partial_flush() to prepare for the partial inspection. Then it returns a normal
-flush point to Stream at the end of the current TCP segment. Partial inspections perform all of the
-functions of a regular inspection including forwarding data to file processing and detection.
-
The difference between a partial inspection and a regular inspection is reassemble() saves the
input data for future reuse. Eventually there will be a regular full inspection of the entire
message section. reassemble() will accomplish this by combining the input data for the partial
-inspection with later data that complete the message section.
+inspection with later data that completes the message section.
+
+scan() calls prep_partial_flush() to prepare for the partial inspection. Then it returns a normal
+flush point to Stream at the end of the current TCP segment. Partial inspections perform all of the
+functions of a regular inspection including forwarding data to file processing and detection. From
+the perspective of Stream (or H2I) a partial inspection is a regular flush in every respect.
Correct and efficient execution of a full inspection following a partial inspection requires
special handling of certain functions. Unzipping is only done once in reassemble(). The stored
will not miss anything. The benefits of partial inspection are in addition to the benefits of a
full inspection.
-The http_inspect partial inspection mechanism is also used by http2_inspect to manage frame
-boundaries. When inspecting HTTP/2, a partial inspection by http_inspect may occur because script
-detection triggered it, because H2I wanted it, or both.
+Partial inspection is used in multiple scenarios, described below, and in combinations of them.
+
+Script detection uses partial inspection for message bodies containing Javascripts. The stream
+splitter scan() method searches its input for the end-of-script tag "</script>". When the end
+of a script is found and the normal flush point has not been found, the current TCP segment and
+all previous segments for the current message section are flushed using partial inspection.
+
+Searching for the end-of-script tag may require scan() to unzip the data. This is an extra unzip
+as storage limitations preclude saving the unzipped version of the data for subsequent reassembly.
+
+Update: the previous sentence has been discovered to be incorrect. The memory requirements of
+zlib are very large. It would save a lot of memory and some processing time for script detection
+to unzip one time in scan() and store the result for eventual use by reassemble(). The memory
+lost by storing partial message sections in HI while waiting for reassemble() would be more than
+compensated for by not having two instances of zlib.
+
+For request bodies, when partial_depth parameter is set to a non zero value, a partial body will
+be subjected to partial inspection if its length is below partial_depth value. When the partial_depth
+parameter is set to -1, the entire body will be subjected to inspection regardless of its length.
+
+The http_inspect partial inspection mechanism is invoked by http2_inspect on frame boundaries.
-Some applications may be affected by blocks too late scenarios related to seeing part of the
-zero-length chunk. For example a TCP packet that ends with:
+With chunking some applications may be affected by blocks too late scenarios related to seeing part
+of the zero-length chunk. For example a TCP packet that ends with:
8<CR><LF>abcdefgh<CR><LF>0
SEC_REQUEST = 2, SEC_STATUS, SEC_HEADER, SEC_BODY_CL, SEC_BODY_CHUNK, SEC_TRAILER,
SEC_BODY_OLD, SEC_BODY_HX };
+inline bool is_body(SectionType st)
+{
+ return (st == SEC_BODY_CL || st == SEC_BODY_CHUNK || st == SEC_BODY_OLD || st == SEC_BODY_HX);
+}
+
// Caters to all extended versions of HTTP, i.e. HTTP/2, HTTP/3
enum HXBodyState { HX_BODY_NOT_COMPLETE, HX_BODY_LAST_SEG, HX_BODY_COMPLETE,
HX_BODY_COMPLETE_EXPECT_TRAILERS, HX_BODY_NO_BODY };
EVENT_UNEXPECTED_H2_PREFACE = 286,
EVENT_DISALLOWED_METHOD = 287,
EVENT_GZIP_RESERVED_FLAGS = 288,
+ EVENT_MAX_PARTIAL_FLUSH = 289,
EVENT__MAX_VALUE
};
uint32_t num_good_chunks[2] = { 0, 0 };
uint32_t octets_expected[2] = { 0, 0 };
bool is_broken_chunk[2] = { false, false };
+ uint64_t partial_flush_counter = 0;
// *** StreamSplitter => Inspector (facts about the most recent message section)
HttpCommon::SectionType section_type[2] = { HttpCommon::SEC__NOT_COMPUTE,
ConfigLogger::log_limit("request_depth", params->request_depth, -1);
ConfigLogger::log_limit("response_depth", params->response_depth, -1);
+ ConfigLogger::log_limit("partial_depth", params->partial_depth, -1, 0);
ConfigLogger::log_flag("unzip", params->unzip);
ConfigLogger::log_flag("normalize_utf", params->normalize_utf);
ConfigLogger::log_flag("decompress_pdf", params->decompress_pdf);
{ "response_depth", Parameter::PT_INT, "-1:max53", "-1",
"maximum response message body bytes to examine (-1 no limit)" },
+ { "partial_depth", Parameter::PT_INT, "-1:16384", "0",
+ "maximum request body to send to early detection (0 disabled, -1 no limit)" },
+
{ "unzip", Parameter::PT_BOOL, nullptr, "true",
"decompress gzip and deflate message bodies" },
{
params->response_depth = val.get_int64();
}
+ else if (val.is("partial_depth"))
+ {
+ params->partial_depth = val.get_int64();
+ }
else if (val.is("unzip"))
{
params->unzip = val.get_bool();
~HttpParaList();
int64_t request_depth = -1;
int64_t response_depth = -1;
+ int64_t partial_depth = 0;
bool unzip = true;
bool normalize_utf = true;
HttpModule::increment_peg_counts(PEG_REASSEMBLE);
- const bool is_body =
- (session_data->section_type[source_id] == SEC_BODY_CHUNK) ||
- (session_data->section_type[source_id] == SEC_BODY_CL) ||
- (session_data->section_type[source_id] == SEC_BODY_OLD) ||
- (session_data->section_type[source_id] == SEC_BODY_HX);
-
uint8_t*& buffer = session_data->section_buffer[source_id];
if (buffer == nullptr)
{
// Body sections need extra space to accommodate unzipping
- if (is_body)
+ if (is_body(session_data->section_type[source_id]))
buffer = new uint8_t[MAX_OCTETS];
else
{
}
const uint32_t max_length = MAX_OCTETS - cutter->get_octets_seen();
- const ScanResult cut_result = cutter->cut(data, (length <= max_length) ? length :
+ ScanResult cut_result = cutter->cut(data, (length <= max_length) ? length :
max_length, session_data->get_infractions(source_id), session_data->events[source_id],
session_data->section_size_target[source_id],
session_data->stretch_section_to_packet[source_id],
return status_value(StreamSplitter::ABORT);
}
+ if (is_body(type) && source_id == SRC_CLIENT &&
+ (my_inspector->params->partial_depth == -1 ||
+ (cutter->get_octets_seen() < my_inspector->params->partial_depth && cutter->get_num_flush() == 0)))
+ {
+ static const uint64_t MAX_PARTIAL_FLUSH_COUNTER = 20;
+ if (++session_data->partial_flush_counter == MAX_PARTIAL_FLUSH_COUNTER)
+ session_data->events[source_id]->create_event(HttpEnums::EVENT_MAX_PARTIAL_FLUSH);
+ cut_result = SCAN_NOT_FOUND_ACCELERATE;
+ }
+
if (cut_result == SCAN_NOT_FOUND_ACCELERATE)
{
prep_partial_flush(flow, length);
case SCAN_FOUND:
case SCAN_FOUND_PIECE:
{
+ session_data->partial_flush_counter = 0;
const uint32_t flush_octets = cutter->get_num_flush();
prepare_flush(session_data, flush_offset, type, flush_octets, cutter->get_num_excess(),
cutter->get_num_head_lines(), cutter->get_is_broken_chunk(),
{ EVENT_DISALLOWED_METHOD, "HTTP request method is not on allowed methods list or is on "
"disallowed methods list" },
{ EVENT_GZIP_RESERVED_FLAGS, "HTTP gzip body with reserved flag set" },
+ { EVENT_MAX_PARTIAL_FLUSH, "Too many partial flushes" },
{ 0, nullptr }
};
projects / thirdparty / snort3.git / commitdiff
