From: Tom Peters (thopeter) Date: Thu, 29 Apr 2021 21:36:05 +0000 (+0000) Subject: Merge pull request #2868 in SNORT/snort3 from ~KATHARVE/snort3:h2i_window_update... X-Git-Tag: 3.1.5.0~20 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=fa58cde9576a57adfe28b87901846fb5c91bf4cb;p=thirdparty%2Fsnort3.git Merge pull request #2868 in SNORT/snort3 from ~KATHARVE/snort3:h2i_window_update to master Squashed commit of the following: commit f80eef948c70811e81155a64745aeb9e92be74e3 Author: Katura Harvey Date: Fri Apr 23 10:20:09 2021 -0400 http2_inspect: implement window_update frame --- diff --git a/src/service_inspectors/http2_inspect/CMakeLists.txt b/src/service_inspectors/http2_inspect/CMakeLists.txt index f8f7b3d60..b888f3c46 100644 --- a/src/service_inspectors/http2_inspect/CMakeLists.txt +++ b/src/service_inspectors/http2_inspect/CMakeLists.txt @@ -56,6 +56,8 @@ set (FILE_LIST http2_tables.cc http2_utils.cc http2_utils.h + http2_window_update_frame.cc + http2_window_update_frame.h ips_http2.cc ips_http2.h ) diff --git a/src/service_inspectors/http2_inspect/dev_notes.txt b/src/service_inspectors/http2_inspect/dev_notes.txt index d36a149c1..37e042eb7 100644 --- a/src/service_inspectors/http2_inspect/dev_notes.txt +++ b/src/service_inspectors/http2_inspect/dev_notes.txt @@ -1,13 +1,15 @@ +*** HTTP/2 frame processing overview *** The HTTP/2 inspector (H2I) converts HTTP/2 frames into HTTP/1.1 message sections and feeds them to the new HTTP inspector (NHI) for further processing. -The Http2StreamSplitter splits HTTP/2 traffic on frame boundaries with two exceptions. First, any +The Http2StreamSplitter splits HTTP/2 traffic on frame boundaries with two exceptions. First, any Continuation frames following a Headers frame are aggregated and sent through detection as a single -header block. Second, long Data frames are split into 16kb chunks for inspection as message bodies -are in NHI. The Http2StreamSplitter strips the frame headers from the frame data and stores them in +header block. Second, long Data frames are split into 16kb chunks for inspection as message bodies +are in NHI. The Http2StreamSplitter strips the frame headers from the frame data and stores them in separate buffers. In the case of a Headers frame followed by Continuation frames, the Headers frame header is stored and the Continuation frame headers are discarded. +*** HTTP/2 Frame Classes *** HTTP/2 frames are stored in frame objects for processing. 1. Http2Frame - top level class 2. Http2HeadersFrame : Http2Frame - common elements of frames containing headers @@ -17,11 +19,41 @@ HTTP/2 frames are stored in frame objects for processing. 6. Http2HeadersFrameTrailer : Http2HeadersFrame - headers frame containing trailers 7. Http2DataFrame : Http2Frame 8. Http2SettingsFrame : Http2Frame +9. Http2RstStreamFrame : Http2Frame +10. Http2WindowUpdateFrame: Http2Frame +Http2Frame virtual methods: +1. bool valid_sequence(Http2Enums::StreamState state) - Returns if the current frame has been sent + on a valid stream in a valid state for the frame. +2. void analyze_http1() - Contains the interface with http_inspect if necessary for the frame type +3. void clear() - Cleanup after processing is complete +4. const Field& get_buf(unsigned id) - Returns any buffers associated with this frame type +5. void update_stream_state() - Updates the state of the current stream based on the contents of + the current packet. +6. bool is_detection_required() - Returns if the data in this frame be sent through the detection + engine. + +*** HTTP/2 Stream States *** +Http2 stream states are directional and take the form {, }. States can have +the following values: +1. STREAM_EXPECT_HEADERS - No frames have yet been seen on this stream in this direction. A stream + with state {STREAM_EXPECT_HEADERS, STREAM_EXPECT_HEADERS} is what the HTTP/2 RFC refers to as + "idle." +2. STREAM_EXPECT_BODY - Headers are complete and we are expecting to see a Data frame +3. STREAM_BODY - There has been at least one Data frame in this direction already, but we have not + yet seen the END_STREAM flag so are still expecting more Data or Headers frames. +4. STREAM_COMPLETE - The END_STREAM flag has been seen. No more Data or Headers frames are + expected. +5. STREAM_ERROR - An error has occurred processing this stream. This is a terminal stream state. + Further frames on this stream in this direction may be checked for proper formatting, but will + not impact the stream state or interact with http_inspect. For information on error processing + see the section below. + +*** HTTP/2 Header Decoding *** HTTP/2 headers frames can come at the start of a stream and contain the header block, or at the end of a stream and contain trailers. H2I contains headers frame subclasses Http2HeadersFrameHeader and Http2HeadersFrameTrailer to support these two types of headers frames. Http2PushPromise frames begin -server pushed streams and also contain the header block. Headers frames containing the header block +server pushed streams and also contain the header block. Headers frames containing the header block will contain pseudo-headers that must be converted into an HTTP/1.1 start line in addition to regular HTTP/1.1 headers. The two Http2StartLine subclasses, Http2RequestLine and Http2ResponseLine perform this translation and generate the start-line, which is stored in a new buffer inside the @@ -44,6 +76,7 @@ literal. The resulting header line is then added to the dynamic table. The third is literal not to be indexed, which is the same as literal to be indexed, except the header line is not added to the dynamic table. +*** Error Processing *** H2I has two levels of failure for flow processing. Fatal errors include failures in frame splitting and errors in header decoding that compromise the HPACK dictionary. A fatal error will trigger an immediate EVENT_MISFORMATTED_HTTP2 and will cause scan() to return ABORT the next time it is called @@ -56,7 +89,8 @@ processing of that stream in that direction will end. H2I supports the NHI test tool. See ../http_inspect/dev_notes.txt for usage instructions. -Memory requirements: Http2FlowData represents all H2I information in a flow. It does not account +*** Memory requirements *** +Http2FlowData represents all H2I information in a flow. It does not account for the entries in the hpack dynamic table. The formula below estimates the size of an entry in the dynamic table: name.length() + value.length() + RFC_ENTRY_OVERHEAD (32 as defined by the RFC) diff --git a/src/service_inspectors/http2_inspect/http2_enum.h b/src/service_inspectors/http2_inspect/http2_enum.h index 2596e8565..86d6b9396 100644 --- a/src/service_inspectors/http2_inspect/http2_enum.h +++ b/src/service_inspectors/http2_inspect/http2_enum.h @@ -87,6 +87,8 @@ enum EventSid EVENT_INVALID_RST_STREAM_FRAME = 28, EVENT_BAD_RST_STREAM_SEQUENCE = 29, EVENT_HEADER_UPPERCASE = 30, + EVENT_INVALID_WINDOW_UPDATE_FRAME = 31, + EVENT_WINDOW_UPDATE_FRAME_ZERO_INCREMENT = 32, EVENT__MAX_VALUE }; @@ -140,6 +142,8 @@ enum Infraction INF_INVALID_RST_STREAM_FRAME = 43, INF_BAD_RST_STREAM_SEQUENCE = 44, INF_HEADER_UPPERCASE = 45, + INF_INVALID_WINDOW_UPDATE_FRAME = 46, + INF_WINDOW_UPDATE_FRAME_ZERO_INCREMENT = 47, INF__MAX_VALUE }; diff --git a/src/service_inspectors/http2_inspect/http2_flow_data.h b/src/service_inspectors/http2_inspect/http2_flow_data.h index cb53000cb..740b4e09a 100644 --- a/src/service_inspectors/http2_inspect/http2_flow_data.h +++ b/src/service_inspectors/http2_inspect/http2_flow_data.h @@ -81,6 +81,7 @@ public: friend class Http2StatusLine; friend class Http2Stream; friend class Http2StreamSplitter; + friend class Http2WindowUpdateFrame; friend void finish_msg_body(Http2FlowData* session_data, HttpCommon::SourceId source_id); size_t size_of() override; diff --git a/src/service_inspectors/http2_inspect/http2_frame.cc b/src/service_inspectors/http2_inspect/http2_frame.cc index 9508dfb5c..23ef47c82 100644 --- a/src/service_inspectors/http2_inspect/http2_frame.cc +++ b/src/service_inspectors/http2_inspect/http2_frame.cc @@ -33,6 +33,7 @@ #include "http2_rst_stream_frame.h" #include "http2_settings_frame.h" #include "http2_stream.h" +#include "http2_window_update_frame.h" #include "service_inspectors/http_inspect/http_field.h" using namespace HttpCommon; @@ -72,23 +73,27 @@ Http2Frame* Http2Frame::new_frame(const uint8_t* header, const uint32_t header_l case FT_SETTINGS: frame = new Http2SettingsFrame(header, header_len, data, data_len, session_data, source_id, stream); - break; + break; case FT_DATA: frame = new Http2DataFrame(header, header_len, data, data_len, session_data, source_id, stream); - break; + break; case FT_PUSH_PROMISE: frame = new Http2PushPromiseFrame(header, header_len, data, data_len, session_data, source_id, stream); - break; + break; case FT_PING: frame = new Http2PingFrame(header, header_len, data, data_len, session_data, source_id, stream); - break; + break; case FT_RST_STREAM: frame = new Http2RstStreamFrame(header, header_len, data, data_len, session_data, source_id, stream); break; + case FT_WINDOW_UPDATE: + frame = new Http2WindowUpdateFrame(header, header_len, data, data_len, session_data, + source_id, stream); + break; default: frame = new Http2Frame(header, header_len, data, data_len, session_data, source_id, stream); diff --git a/src/service_inspectors/http2_inspect/http2_frame.h b/src/service_inspectors/http2_inspect/http2_frame.h index 4f71c2a62..9c9b25484 100644 --- a/src/service_inspectors/http2_inspect/http2_frame.h +++ b/src/service_inspectors/http2_inspect/http2_frame.h @@ -43,8 +43,7 @@ public: static Http2Frame* new_frame(const uint8_t* header_buffer, const uint32_t header_len, const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* session_data, HttpCommon::SourceId source_id, Http2Stream* stream); - virtual bool valid_sequence(Http2Enums::StreamState state) - { return state != Http2Enums::STREAM_ERROR; } + virtual bool valid_sequence(Http2Enums::StreamState) { return true; } virtual void analyze_http1() { } virtual void clear() { } virtual const Field& get_buf(unsigned id); diff --git a/src/service_inspectors/http2_inspect/http2_tables.cc b/src/service_inspectors/http2_inspect/http2_tables.cc index 6d562267e..e46a31e37 100644 --- a/src/service_inspectors/http2_inspect/http2_tables.cc +++ b/src/service_inspectors/http2_inspect/http2_tables.cc @@ -61,6 +61,8 @@ const RuleMap Http2Module::http2_events[] = { EVENT_INVALID_RST_STREAM_FRAME, "invalid HTTP/2 rst stream frame" }, { EVENT_BAD_RST_STREAM_SEQUENCE, "HTTP/2 rst stream frame sent at invalid time" }, { EVENT_HEADER_UPPERCASE, "uppercase HTTP/2 header field name" }, + { EVENT_INVALID_WINDOW_UPDATE_FRAME, "invalid HTTP/2 window update frame" }, + { EVENT_WINDOW_UPDATE_FRAME_ZERO_INCREMENT, "HTTP/2 window update frame with zero increment" }, { 0, nullptr } }; diff --git a/src/service_inspectors/http2_inspect/http2_window_update_frame.cc b/src/service_inspectors/http2_inspect/http2_window_update_frame.cc new file mode 100644 index 000000000..580a08da5 --- /dev/null +++ b/src/service_inspectors/http2_inspect/http2_window_update_frame.cc @@ -0,0 +1,69 @@ +//-------------------------------------------------------------------------- +// Copyright (C) 2021-2021 Cisco and/or its affiliates. All rights reserved. +// +// This program is free software; you can redistribute it and/or modify it +// under the terms of the GNU General Public License Version 2 as published +// by the Free Software Foundation. You may not use, modify or distribute +// this program under any other version of the GNU General Public License. +// +// This program is distributed in the hope that it will be useful, but +// WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +// General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +//-------------------------------------------------------------------------- +// http2_window_update_frame.cc author Katura Harvey + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include "http2_window_update_frame.h" + +#include "service_inspectors/http_inspect/http_test_manager.h" + +#include "http2_enum.h" +#include "http2_flow_data.h" + +using namespace HttpCommon; +using namespace Http2Enums; + +Http2WindowUpdateFrame::Http2WindowUpdateFrame(const uint8_t* header_buffer, const uint32_t header_len, + const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data, + HttpCommon::SourceId src_id, Http2Stream* stream_) : Http2Frame(header_buffer, header_len, + data_buffer, data_len, ssn_data, src_id, stream_) +{ + if (data.length() != 4) + { + session_data->events[source_id]->create_event(EVENT_INVALID_WINDOW_UPDATE_FRAME); + *session_data->infractions[source_id] += INF_INVALID_WINDOW_UPDATE_FRAME; + } + else + { + static const uint32_t CLEAR_FIRST_BIT_MASK = 0x7fffffff; + const uint32_t increment = (data.start()[0] << 24 | data.start()[1] << 16 | + data.start()[2] << 8 | data.start()[3]) & CLEAR_FIRST_BIT_MASK; + if (increment == 0) + { + session_data->events[source_id]->create_event(EVENT_WINDOW_UPDATE_FRAME_ZERO_INCREMENT); + *session_data->infractions[source_id] += INF_WINDOW_UPDATE_FRAME_ZERO_INCREMENT; + } + } +} + +bool Http2WindowUpdateFrame::valid_sequence(Http2Enums::StreamState) +{ + //FIXIT-E Not valid on streams in idle state; add check once we track completed streams + return true; +} + +#ifdef REG_TEST +void Http2WindowUpdateFrame::print_frame(FILE* output) +{ + fprintf(output, "window_update frame\n"); + Http2Frame::print_frame(output); +} +#endif diff --git a/src/service_inspectors/http2_inspect/http2_window_update_frame.h b/src/service_inspectors/http2_inspect/http2_window_update_frame.h new file mode 100644 index 000000000..8fe6c0cc7 --- /dev/null +++ b/src/service_inspectors/http2_inspect/http2_window_update_frame.h @@ -0,0 +1,45 @@ +//-------------------------------------------------------------------------- +// Copyright (C) 2021-2021 Cisco and/or its affiliates. All rights reserved. +// +// This program is free software; you can redistribute it and/or modify it +// under the terms of the GNU General Public License Version 2 as published +// by the Free Software Foundation. You may not use, modify or distribute +// this program under any other version of the GNU General Public License. +// +// This program is distributed in the hope that it will be useful, but +// WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +// General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +//-------------------------------------------------------------------------- +// http2_window_update_frame.h author Katura Harvey + +#ifndef HTTP2_WINDOW_UPDATE_FRAME_H +#define HTTP2_WINDOW_UPDATE_FRAME_H + +#include "http2_frame.h" + +class Http2Frame; + +class Http2WindowUpdateFrame : public Http2Frame +{ +public: + friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*, + const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream); + bool is_detection_required() const override { return false; } + bool valid_sequence(Http2Enums::StreamState state) override; + +#ifdef REG_TEST + void print_frame(FILE* output) override; +#endif + +private: + Http2WindowUpdateFrame(const uint8_t* header_buffer, const uint32_t header_len, + const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data, + HttpCommon::SourceId src_id, Http2Stream* stream); +}; + +#endif