From: Steve Chew (stechew) Date: Wed, 19 Oct 2022 14:07:24 +0000 (+0000) Subject: Pull request #3536: US #762655 detection: target service http rules to specific messa... X-Git-Tag: 3.1.45.0~4 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=c5093e2c781e0c15def30edee8987386c67708c9;p=thirdparty%2Fsnort3.git Pull request #3536: US #762655 detection: target service http rules to specific message sections - Part 5 Merge in SNORT/snort3 from ~MDAGON/snort3:proto_5 to master Squashed commit of the following: commit 83ef46f4c04816c433d40af59cda244aaacde1b2 Author: Tom Peters Date: Mon Mar 21 16:39:24 2022 -0400 http_inspect: remove rule option timing features --- diff --git a/doc/user/http_inspect.txt b/doc/user/http_inspect.txt index 5b9aa5dd0..0498cb9f4 100755 --- a/doc/user/http_inspect.txt +++ b/doc/user/http_inspect.txt @@ -540,8 +540,11 @@ http_inspect parses HTTP messages into their components and makes them available to the detection engine through rule options. Let's start with an example: - alert tcp any any -> any any ( msg:"URI example"; flow:established, - to_server; http_uri; content:"chocolate"; sid:1; rev:1; ) + alert tcp any any -> any any ( + msg:"URI example"; + flow:established, to_server; + http_uri; content:"chocolate"; + sid:1; rev:1; ) This rule looks for chocolate in the URI portion of the request message. Specifically, the http_uri rule option is the normalized URI with all the @@ -555,15 +558,20 @@ and It is also possible to search the unnormalized URI - alert tcp any any -> any any ( msg:"Raw URI example"; flow:established, - to_server; http_raw_uri; content:"chocolate"; sid:2; rev:1; ) + alert tcp any any -> any any ( + msg:"Raw URI example"; + flow:established, to_server; + http_raw_uri; content:"chocolate"; + sid:2; rev:1; ) will match the first message but not the second. If you want to detect someone who is trying to hide his request for chocolate then - alert tcp any any -> any any ( msg:"Raw URI example"; flow:established, - to_server; http_raw_uri; content:"%63%68$6F%63%6F%6C%61%74%65"; - sid:3; rev:1; ) + alert tcp any any -> any any ( + msg:"Raw URI example"; + flow:established, to_server; + http_raw_uri; content:"%63%68$6F%63%6F%6C%61%74%65"; + sid:3; rev:1; ) will do the trick. @@ -571,9 +579,11 @@ Let's look at possible ways of writing a rule to match HTTP response messages with the Content-Language header set to "da" (Danish). You could write: - alert tcp any any -> any any ( msg:"whole header search"; - flow:established, to_client; http_header; content: - "Content-Language: da", nocase; sid:4; rev:1; ) + alert tcp any any -> any any ( + msg:"whole header search"; + flow:established, to_client; + http_header; content:"Content-Language: da", nocase; + sid:4; rev:1; ) This rule leaves much to be desired. Modern headers are often thousands of bytes and seem to get longer every year. Searching all of the headers @@ -590,9 +600,11 @@ the match. A better way to write this rule is: - alert tcp any any -> any any ( msg:"individual header search"; - flow:established, to_client; http_header: field content-language; - content:"da", nocase; sid:4; rev:2; ) + alert tcp any any -> any any ( + msg:"individual header search"; + flow:established, to_client; + http_header: field content-language; content:"da", nocase; + sid:4; rev:2; ) The field option improves performance by narrowing the search to the Content-Language field of the header. Because it uses the header parsing @@ -646,8 +658,11 @@ appeared in the message and the normalized form is determined by the URI normalization options you selected. In addition to searching the entire URI there are six components that can be searched individually: - alert tcp any any -> any any ( msg:"URI path"; flow:established, - to_server; http_uri: path; content:"chocolate"; sid:1; rev:2; ) + alert tcp any any -> any any ( + msg:"URI path"; + flow:established, to_server; + http_uri: path; content:"chocolate"; + sid:1; rev:2; ) By specifying "path" the search is limited to the path portion of the URI. Informally this is the part consisting of the directory path and file name. @@ -702,9 +717,11 @@ These cover all the header lines except the first one. You may specify an individual header by name using the field option as shown in this earlier example: - alert tcp any any -> any any ( msg:"individual header search"; - flow:established, to_client; http_header: field content-language; - content:"da", nocase; sid:4; rev:2; ) + alert tcp any any -> any any ( + msg:"individual header search"; + flow:established, to_client; + http_header: field content-language; content:"da", nocase; + sid:4; rev:2; ) This rule searches the value of the Content-Language header. Header names are not case sensitive and may be written in the rule in any mixture of @@ -871,41 +888,15 @@ the dividing lines between one message and the next, which request message triggered which response message, pipelines, and how many messages have been sent over the current connection. -Some rules use a single rule option: - - alert tcp any any -> any any ( msg:"URI example"; flow:established, - to_server; http_uri; content:"chocolate"; sid:1; rev:1; ) - -Whenever a new URI is available this rule will be evaluated. Nothing -complicated about that, but suppose we use more than one rule option: - - alert tcp any any -> any any ( msg:"combined example"; flow:established, - to_server; http_uri: with_body; content:"chocolate"; file_data; - content:"sinister POST data"; sid:5; rev:1; ) - -The with_body option to http_uri causes the URI to be made available with -the message body. Use with_body for header-related rule options in rules -that also examine the message body. - -The with_trailer option is analogous and causes an earlier message element -to be made available at the end of the message when the trailers following -a chunked body arrive. - - alert tcp any any -> any any ( msg:"double content-language"; - flow:established, to_client; http_header: with_trailer, field - content-language; content:"da", nocase; http_trailer: field - content-language; content:"en", nocase; sid:6; rev:1; ) - -This rule will alert if the Content-Language changes from Danish in the -headers to English in the trailers. The with_trailer option is essential to -make this rule work. - -It is also possible to write rules that examine both the client request and +It is possible to write rules that examine both the client request and the server response to it. - alert tcp any any -> any any ( msg:"request and response example"; - flow:established, to_client; http_uri: with_body; content:"chocolate"; - file_data; content:"white chocolate"; sid:7; rev:1; ) + alert tcp any any -> any any ( + msg:"request and response example"; + flow:established, to_client; + http_uri; content:"chocolate"; + file_data; content:"white chocolate"; + sid:7; rev:1; ) This rule looks for white chocolate in a response message body where the URI of the request contained chocolate. Note that this is a "to_client" @@ -915,21 +906,23 @@ rule were rewritten "to_server" it would be nonsense and not work. Snort cannot block a client request based on what the server response will be because that has not happened yet. -Another point is "with_body" for http_uri. This ensures the rule works on -the entire response body. If we were looking for white chocolate in the -response headers this would not be necessary. - Response messages do not have a URI so there was only one thing http_uri could have meant in the previous rule. It had to be referring to the request message. Sometimes that is not so clear. - alert tcp any any -> any any ( msg:"header ambiguity example 1"; - flow:established, to_client; http_header: with_body; content: - "chocolate"; file_data; content:"white chocolate"; sid:8; rev:1; ) + alert tcp any any -> any any ( + msg:"header ambiguity example 1"; + flow:established, to_client; + http_header; content:"chocolate"; + file_data; content:"white chocolate"; + sid:8; rev:1; ) - alert tcp any any -> any any ( msg:"header ambiguity example 2"; - flow:established, to_client; http_header: with_body, request; content: - "chocolate"; file_data; content:"white chocolate"; sid:8; rev:2; ) + alert tcp any any -> any any ( + msg:"header ambiguity example 2"; + flow:established, to_client; + http_header: request; content:"chocolate"; + file_data; content:"white chocolate"; + sid:8; rev:2; ) Our search for chocolate has moved from the URI to the message headers. Both the request and response messages have headers--which one are we @@ -941,32 +934,12 @@ the body. The second rule uses the "request" option to explicitly say that the http_header to be searched is the request header. -Let's put all of this together. There are six opportunities to do -detection: - -1. When the the request headers arrive. The request line and all of the -headers go through detection at the same time. - -2. When sections of the request message body arrive. If you want to combine -this with something from the request line or headers you must use the -with_body option. - -3. When the request trailers arrive. If you want to combine this with -something from the request line or headers you must use the with_trailer -option. - -4. When the response headers arrive. The status line and all of the headers -go through detection at the same time. These may be combined with elements -from the request line, request headers, or request trailers. Where -ambiguity arises use the request option. - -5. When sections of the response message body arrive. These may be combined -with the status line, response headers, request line, request headers, or -request trailers as described above. - -6. When the response trailers arrive. Again these may be combined as -described above. +Fast patterns are always searched in the current message. Rule options using +"request" option can't be used as fast patterns. Message body sections can only go through detection at the time they are received. Headers may be combined with later items but the body cannot. +The sub-options "with_header", "with_body" and "with_trailer" are deprecated, +and no longer required when mixing the different sections. + diff --git a/src/detection/fp_create.cc b/src/detection/fp_create.cc index 57c99ff48..16825b8b0 100644 --- a/src/detection/fp_create.cc +++ b/src/detection/fp_create.cc @@ -74,7 +74,7 @@ static int fpGetFinalPattern( FastPatternConfig*, PatternMatchData*, const char*& ret_pattern, unsigned& ret_bytes); static void print_nfp_info(const char*, OptTreeNode*); -static void print_fp_info(const char*, const OptTreeNode*, const PatternMatchData*); +static void print_fp_info(const char*, const OptTreeNode*, const PatternMatchData*, int sect); static OptTreeNode* fixup_tree( detection_option_tree_node_t* dot, bool branched, unsigned contents) @@ -124,7 +124,7 @@ static int finalize_detection_option_tree(SnortConfig* sc, detection_option_tree fixup_tree(root->children[i], true, 0); debug_logf(detection_trace, TRACE_OPTION_TREE, nullptr, "%3d %3d %p %4s\n", - 0, root->num_children, (void*)root, "root" ); + 0, root->num_children, (void*)root, "root"); print_option_tree(root->children[i], 0); } @@ -454,17 +454,20 @@ static int fpFinishRuleGroup(SnortConfig* sc, RuleGroup* pg) assert(pg); bool has_rules = false; - for ( auto& it : pg->pm_list ) + for ( int sect = PS_NONE; sect <= PS_MAX; sect++) { - if ( it->group.normal_mpse ) + for ( auto& it : pg->pm_list[sect] ) { - queue_mpse(it->group.normal_mpse); - has_rules = true; - } - if ( it->group.offload_mpse ) - { - queue_mpse(it->group.offload_mpse); - has_rules = true; + if ( it->group.normal_mpse && !it->group.normal_is_dup) + { + queue_mpse(it->group.normal_mpse); + has_rules = true; + } + if ( it->group.offload_mpse && !it->group.offload_is_dup) + { + queue_mpse(it->group.offload_mpse); + has_rules = true; + } } } @@ -493,8 +496,14 @@ static int fpFinishRuleGroup(SnortConfig* sc, RuleGroup* pg) return 0; } +static bool srvc_supports_section(const char* srvc) +{ + return (srvc && ( !strcmp("http", srvc) || !strcmp("http2",srvc) || !strcmp("http3",srvc))); +} + static int fpAddRuleGroupRule( - SnortConfig* sc, RuleGroup* pg, OptTreeNode* otn, FastPatternConfig* fp, bool srvc) + SnortConfig* sc, RuleGroup* pg, OptTreeNode* otn, FastPatternConfig* fp, const char* srvc = nullptr, + bool to_server = false) { const MpseApi* search_api = nullptr; const MpseApi* offload_search_api = nullptr; @@ -514,7 +523,7 @@ static int fpAddRuleGroupRule( IpsOption* opt = nullptr; bool only_literal = !MpseManager::is_regex_capable(search_api); - PatternMatchVector pmv = get_fp_content(otn, ofp, opt, srvc, only_literal, exclude); + PatternMatchVector pmv = get_fp_content(otn, ofp, opt, srvc != nullptr, only_literal, exclude); if ( !pmv.empty() ) { @@ -549,122 +558,171 @@ static int fpAddRuleGroupRule( PatternMatchData* main_pmd = pmv.back(); pmv.pop_back(); - static MpseAgent agent = - { - pmx_create_tree_normal, add_patrn_to_neg_list, - fpDeletePMX, free_detection_option_root, neg_list_free - }; - - const char* s = opt ? opt->get_name() : "pkt_data"; - auto pmt = get_pm_type(s); - PatternMatcher* pm = pg->get_pattern_matcher(pmt, s); - MpseGroup* mpg = &pm->group; - - if ( !mpg->normal_mpse ) - { - if ( !mpg->create_normal_mpse(sc, &agent) ) - { - ParseError("Failed to create normal pattern matcher for %s", pm->name); - return -1; - } - - mpse_count++; - } - if ( add_to_offload ) { ol_pmd = pmv_ol.back(); pmv_ol.pop_back(); + } - static MpseAgent agent_offload = - { - pmx_create_tree_offload, add_patrn_to_neg_list, - fpDeletePMX, free_detection_option_root, neg_list_free - }; - - // Keep the created mpse alongside the same pm type as the main pmd - if ( !mpg->offload_mpse ) - { - if ( !mpg->create_offload_mpse(sc, &agent_offload) ) - { - ParseError("Failed to create offload pattern matcher for %s", pm->name); - return -1; - } + section_flags sects = section_to_flag(PS_NONE); + const bool supports_sect = srvc_supports_section(srvc); + if (supports_sect) + sects = to_server ? otn->sections[OptTreeNode::SECT_TO_SRV] : + otn->sections[OptTreeNode::SECT_TO_CLIENT]; - offload_mpse_count++; - } + // pkt_data can work on both PS_NONE and PS_BODY + const char* s = opt ? opt->get_name() : "pkt_data"; + if (!strcmp("pkt_data", s)) + { + if (!srvc) + sects = section_to_flag(PS_NONE) | section_to_flag(PS_BODY); + else if (!supports_sect) + sects = section_to_flag(PS_NONE); + else if (has_service_rule_opt(otn)) + sects = section_to_flag(PS_BODY); + else + sects = section_to_flag(PS_NONE) | section_to_flag(PS_BODY); } bool add_rule = false; bool add_nfp_rule = false; - - if ( mpg->normal_mpse ) + bool is_first_sect = true; + const auto pmt = get_pm_type(s); + for (int sect = PS_NONE; sect <= PS_MAX; sect++) { - add_rule = true; - if ( main_pmd->is_negated() ) - add_nfp_rule = true; - - // Now add patterns - if ( fpFinishRuleGroupRule(mpg->normal_mpse, otn, main_pmd, fp, true) == 0 ) + if (sects & section_to_flag((PduSection)sect)) { - if ( make_fast_pattern_only(ofp, main_pmd) ) + PatternMatcher* pm = pg->get_pattern_matcher(pmt, s, (PduSection)sect); + MpseGroup* mpg = &pm->group; + const bool update_mpse = srvc || is_first_sect; + + if ( !mpg->normal_mpse ) { - otn->normal_fp_only = ofp; - fp_only++; + if (!update_mpse) + { + assert(!strcmp("pkt_data", s)); + PatternMatcher* pm_none = pg->get_pattern_matcher(pmt, s, (PduSection)PS_NONE); + MpseGroup* mpg_none = &pm_none->group; + mpg->normal_mpse = mpg_none->normal_mpse; + mpg->normal_is_dup = true; + } + else + { + static MpseAgent agent = + { + pmx_create_tree_normal, add_patrn_to_neg_list, + fpDeletePMX, free_detection_option_root, neg_list_free + }; + + if ( !mpg->create_normal_mpse(sc, &agent) ) + { + ParseError("Failed to create normal pattern matcher for %s", pm->name); + return -1; + } + + mpse_count++; + } } - if ( !pm->fp_opt ) - pm->fp_opt = opt; + if ( add_to_offload && !mpg->offload_mpse ) + { + // Keep the created mpse alongside the same pm type as the main pmd + if (!update_mpse) + { + PatternMatcher* pm_none = pg->get_pattern_matcher(pmt, s, (PduSection)PS_NONE); + MpseGroup* mpg_none = &pm_none->group; + mpg->offload_mpse = mpg_none->offload_mpse; + mpg->offload_is_dup = true; + } + else + { + static MpseAgent agent_offload = + { + pmx_create_tree_offload, add_patrn_to_neg_list, + fpDeletePMX, free_detection_option_root, neg_list_free + }; + + if ( !mpg->create_offload_mpse(sc, &agent_offload) ) + { + ParseError("Failed to create offload pattern matcher for %s", + pm->name); + return -1; + } + + offload_mpse_count++; + } + } - main_pmd->sticky_buf = pm->name; + if ( mpg->normal_mpse && update_mpse ) + { + add_rule = true; + if ( main_pmd->is_negated() ) + add_nfp_rule = true; - if ( fp->get_debug_print_fast_patterns() and !otn->soid ) - print_fp_info(s_group, otn, main_pmd); + // Now add patterns + if ( fpFinishRuleGroupRule(mpg->normal_mpse, otn, main_pmd, fp, true) == 0 ) + { + if ( make_fast_pattern_only(ofp, main_pmd) ) + { + otn->normal_fp_only = ofp; + fp_only++; + } - // Add Alternative patterns - for ( auto alt_pmd : pmv ) - { - fpFinishRuleGroupRule(mpg->normal_mpse, otn, alt_pmd, fp, false); - alt_pmd->sticky_buf = pm->name; + if ( !pm->fp_opt ) + pm->fp_opt = opt; - if ( fp->get_debug_print_fast_patterns() and !otn->soid ) - print_fp_info(s_group, otn, alt_pmd); - } - } - } + main_pmd->sticky_buf = pm->name; - if ( ol_pmd and mpg->offload_mpse ) - { - add_rule = true; - if ( ol_pmd->is_negated() ) - add_nfp_rule = true; + if ( fp->get_debug_print_fast_patterns() and !otn->soid ) + print_fp_info(s_group, otn, main_pmd, sect); - // Now add patterns - if ( fpFinishRuleGroupRule(mpg->offload_mpse, otn, ol_pmd, fp, true) == 0 ) - { - if ( make_fast_pattern_only(ofp_ol, ol_pmd) ) - { - otn->offload_fp_only = ofp_ol; - fp_only++; + // Add Alternative patterns + for ( auto alt_pmd : pmv ) + { + fpFinishRuleGroupRule(mpg->normal_mpse, otn, alt_pmd, fp, false); + alt_pmd->sticky_buf = pm->name; + + if ( fp->get_debug_print_fast_patterns() and !otn->soid ) + print_fp_info(s_group, otn, alt_pmd, sect); + } + } } - if ( !pm->fp_opt ) - pm->fp_opt = opt_ol; + if ( ol_pmd and mpg->offload_mpse && update_mpse ) + { + add_rule = true; + if ( ol_pmd->is_negated() ) + add_nfp_rule = true; - main_pmd->sticky_buf = pm->name; + // Now add patterns + if ( fpFinishRuleGroupRule(mpg->offload_mpse, otn, ol_pmd, fp, true) == 0 ) + { + if ( make_fast_pattern_only(ofp_ol, ol_pmd) ) + { + otn->offload_fp_only = ofp_ol; + fp_only++; + } - if ( fp->get_debug_print_fast_patterns() and !otn->soid ) - print_fp_info(s_group, otn, main_pmd); + if ( !pm->fp_opt ) + pm->fp_opt = opt_ol; - // Add Alternative patterns - for (auto alt_pmd : pmv_ol) - { - fpFinishRuleGroupRule(mpg->offload_mpse, otn, alt_pmd, fp, false); - alt_pmd->sticky_buf = pm->name; + main_pmd->sticky_buf = pm->name; - if ( fp->get_debug_print_fast_patterns() and !otn->soid ) - print_fp_info(s_group, otn, alt_pmd); + if ( fp->get_debug_print_fast_patterns() and !otn->soid ) + print_fp_info(s_group, otn, main_pmd, sect); + + // Add Alternative patterns + for (auto alt_pmd : pmv_ol) + { + fpFinishRuleGroupRule(mpg->offload_mpse, otn, alt_pmd, fp, false); + alt_pmd->sticky_buf = pm->name; + + if ( fp->get_debug_print_fast_patterns() and !otn->soid ) + print_fp_info(s_group, otn, alt_pmd, sect); + } + } } + is_first_sect = false; } } @@ -932,16 +990,19 @@ static void fpRuleGroupPrintRuleCount(RuleGroup* pg, const char* what) { LogMessage("RuleGroup rule summary (%s):\n", what); - for ( auto& it : pg->pm_list ) + for ( int sect = PS_NONE; sect <= PS_MAX; sect++) { - int count = it->group.normal_mpse ? it->group.normal_mpse->get_pattern_count() : 0; - int count_ol = it->group.offload_mpse ? it->group.offload_mpse->get_pattern_count() : 0; + for ( auto& it : pg->pm_list[sect] ) + { + int count = it->group.normal_mpse ? it->group.normal_mpse->get_pattern_count() : 0; + int count_ol = it->group.offload_mpse ? it->group.offload_mpse->get_pattern_count() : 0; - if ( count ) - LogMessage("\tNormal Pattern Matcher %s: %d\n", it->name, count); + if ( count ) + LogMessage("\tNormal Pattern Matcher %s: %d\n", it->name, count); - if ( count_ol ) - LogMessage("\tOffload Pattern Matcher %s: %d\n", it->name, count_ol); + if ( count_ol ) + LogMessage("\tOffload Pattern Matcher %s: %d\n", it->name, count_ol); + } } if ( pg->nfp_rule_count ) @@ -963,7 +1024,7 @@ static void fpDeletePMX(void* pv) */ static void fpCreatePortObject2RuleGroup(SnortConfig* sc, PortObject2* po, PortObject2* poaa) { - assert( po ); + assert(po); po->group = nullptr; FastPatternConfig* fp = sc->fast_pattern_config; @@ -1015,7 +1076,7 @@ static void fpCreatePortObject2RuleGroup(SnortConfig* sc, PortObject2* po, PortO assert(otn); if ( is_network_protocol(otn->snort_protocol_id) ) - fpAddRuleGroupRule(sc, pg, otn, fp, false); + fpAddRuleGroupRule(sc, pg, otn, fp); } if (fp->get_debug_print_rule_group_build_details()) @@ -1190,7 +1251,7 @@ static int fpCreateRuleGroups(SnortConfig* sc, RulePortTables* p) * list- list of otns for this service */ static void fpBuildServiceRuleGroupByServiceOtnList( - SnortConfig* sc, GHash* p, const char* srvc, SF_LIST* list, FastPatternConfig* fp) + SnortConfig* sc, GHash* p, const char* srvc, SF_LIST* list, FastPatternConfig* fp, bool to_server) { RuleGroup* pg = new RuleGroup; s_group = srvc; @@ -1205,7 +1266,7 @@ static void fpBuildServiceRuleGroupByServiceOtnList( otn; otn = (OptTreeNode*)sflist_next(&cursor) ) { - fpAddRuleGroupRule(sc, pg, otn, fp, true); + fpAddRuleGroupRule(sc, pg, otn, fp, srvc, to_server); } if (fpFinishRuleGroup(sc, pg) != 0) @@ -1229,7 +1290,7 @@ static void fpBuildServiceRuleGroupByServiceOtnList( * */ static void fpBuildServiceRuleGroups( - SnortConfig* sc, GHash* spg, RuleGroupVector& sopg, GHash* srm, FastPatternConfig* fp) + SnortConfig* sc, GHash* spg, RuleGroupVector& sopg, GHash* srm, FastPatternConfig* fp, bool to_server) { for (GHashNode* n = srm->find_first(); n; n = srm->find_next()) { @@ -1238,7 +1299,7 @@ static void fpBuildServiceRuleGroups( assert(list and srvc); - fpBuildServiceRuleGroupByServiceOtnList(sc, spg, srvc, list, fp); + fpBuildServiceRuleGroupByServiceOtnList(sc, spg, srvc, list, fp, to_server); /* Add this RuleGroup to the protocol-ordinal -> port_group table */ RuleGroup* pg = (RuleGroup*)spg->find(srvc); @@ -1266,10 +1327,10 @@ static void fpCreateServiceMapRuleGroups(SnortConfig* sc) sc->sopgTable = new sopg_table_t(sc->proto_ref->get_count()); fpBuildServiceRuleGroups(sc, sc->spgmmTable->to_srv, - sc->sopgTable->to_srv, sc->srmmTable->to_srv, fp); + sc->sopgTable->to_srv, sc->srmmTable->to_srv, fp, true); fpBuildServiceRuleGroups(sc, sc->spgmmTable->to_cli, - sc->sopgTable->to_cli, sc->srmmTable->to_cli, fp); + sc->sopgTable->to_cli, sc->srmmTable->to_cli, fp, false); } /* @@ -1362,10 +1423,13 @@ static void fp_sum_port_groups(RuleGroup* pg, unsigned& c) if ( !pg ) return; - for ( const auto& it : pg->pm_list ) + for ( int sect = PS_NONE; sect <= PS_MAX; sect++) { - if ( it->group.normal_mpse and it->group.normal_mpse->get_pattern_count() ) - c++; + for ( const auto& it : pg->pm_list[sect] ) + { + if ( it->group.normal_mpse and it->group.normal_mpse->get_pattern_count() ) + c++; + } } } @@ -1639,13 +1703,13 @@ void get_pattern_info(const PatternMatchData* pmd, string& hex, string& txt, str opts += " )"; } -static void print_fp_info(const char* group, const OptTreeNode* otn, const PatternMatchData* pmd) +static void print_fp_info(const char* group, const OptTreeNode* otn, const PatternMatchData* pmd, int sect) { std::string hex, txt, opts; get_pattern_info(pmd, hex, txt, opts); - LogMessage("FP %s %u:%u:%u %s[%d] = '%s' |%s| %s\n", + LogMessage("FP %s %u:%u:%u %s[%d] = '%s' |%s| %s, section %s\n", group, otn->sigInfo.gid, otn->sigInfo.sid, otn->sigInfo.rev, - pmd->sticky_buf, pmd->pattern_size, txt.c_str(), hex.c_str(), opts.c_str()); + pmd->sticky_buf, pmd->pattern_size, txt.c_str(), hex.c_str(), opts.c_str(), section_to_str[sect]); } diff --git a/src/detection/fp_detect.cc b/src/detection/fp_detect.cc index 03eb99618..b3c0e0493 100644 --- a/src/detection/fp_detect.cc +++ b/src/detection/fp_detect.cc @@ -880,9 +880,9 @@ static int fp_search(RuleGroup* port_group, Packet* p, bool srvc) p->packet_flags |= PKT_FAST_PAT_EVAL; Inspector* gadget = p->flow ? p->flow->gadget : nullptr; - debug_log(detection_trace, TRACE_RULE_EVAL, p, "Fast pattern search\n"); - - for ( const auto it : port_group->pm_list ) + debug_logf(detection_trace, TRACE_RULE_EVAL, p, "Fast pattern search, packet section %s\n", + section_to_str[p->sect]); + for ( const auto it : port_group->pm_list[p->sect] ) { switch ( it->type ) { @@ -912,7 +912,6 @@ static int fp_search(RuleGroup* port_group, Packet* p, bool srvc) { debug_logf(detection_trace, TRACE_FP_SEARCH, p, "%" PRIu64 " fp pkt_data[%u]\n", p->context->packet_number, length); - batch_search(&it->group, p, p->data, length, pc.pkt_searches); p->is_cooked() ? pc.cooked_searches++ : pc.raw_searches++; } diff --git a/src/detection/fp_utils.cc b/src/detection/fp_utils.cc index 2a559ace0..a94523918 100644 --- a/src/detection/fp_utils.cc +++ b/src/detection/fp_utils.cc @@ -56,6 +56,9 @@ using namespace snort; +// PduSection to string, used by debug traces +const char* section_to_str[] = {"NONE", "HEADER", "HEADER_BODY", "BODY", "TRAILER"}; + //-------------------------------------------------------------------------- // private utilities //-------------------------------------------------------------------------- @@ -66,6 +69,7 @@ static bool pmd_can_be_fp( switch ( cat ) { case CAT_NONE: + case CAT_READ: case CAT_ADJUST: case CAT_SET_OTHER: return false; @@ -296,7 +300,7 @@ static bool fetch(const std::string& s, uint8_t*& data, size_t& len) } static std::string make_db_name( - const std::string& path, const char* proto, const char* dir, const char* buf, const std::string& id) + const std::string& path, const char* proto, const char* dir, const char* buf, const std::string& id, int sect) { std::stringstream ss; @@ -304,7 +308,7 @@ static std::string make_db_name( ss << proto << "_"; ss << dir << "_"; ss << buf << "_"; - + ss << std::to_string(sect) << "_"; ss << std::hex << std::setfill('0') << std::setw(2); for ( auto c : id ) @@ -317,26 +321,32 @@ static std::string make_db_name( static bool db_dump(const std::string& path, const char* proto, const char* dir, RuleGroup* g) { - for ( auto it : g->pm_list ) + for ( int sect = PS_NONE; sect <= PS_MAX; sect++) { - std::string id; - it->group.normal_mpse->get_hash(id); - - std::string file = make_db_name(path, proto, dir, it->name, id); - - uint8_t* db = nullptr; - size_t len = 0; - - if ( it->group.normal_mpse->serialize(db, len) and db and len > 0 ) + for ( auto it : g->pm_list[sect] ) { - store(file, db, len); - free(db); - ++mpse_dumped; - } - else - { - ParseWarning(WARN_RULES, "Failed to serialize %s", file.c_str()); - return false; + if (it->group.normal_is_dup) + continue; + + std::string id; + it->group.normal_mpse->get_hash(id); + + std::string file = make_db_name(path, proto, dir, it->name, id, sect); + + uint8_t* db = nullptr; + size_t len = 0; + + if ( it->group.normal_mpse->serialize(db, len) and db and len > 0 ) + { + store(file, db, len); + free(db); + ++mpse_dumped; + } + else + { + ParseWarning(WARN_RULES, "Failed to serialize %s", file.c_str()); + return false; + } } } return true; @@ -344,30 +354,36 @@ static bool db_dump(const std::string& path, const char* proto, const char* dir, static bool db_load(const std::string& path, const char* proto, const char* dir, RuleGroup* g) { - for ( auto it : g->pm_list ) + for ( int sect = PS_NONE; sect <= PS_MAX; sect++) { - std::string id; - it->group.normal_mpse->get_hash(id); - - std::string file = make_db_name(path, proto, dir, it->name, id); - - uint8_t* db = nullptr; - size_t len = 0; - - if ( !fetch(file, db, len) ) + for ( auto it : g->pm_list[sect] ) { - ParseWarning(WARN_RULES, "Failed to read %s", file.c_str()); + if (it->group.normal_is_dup) + continue; + + std::string id; + it->group.normal_mpse->get_hash(id); + + std::string file = make_db_name(path, proto, dir, it->name, id, sect); + + uint8_t* db = nullptr; + size_t len = 0; + + if ( !fetch(file, db, len) ) + { + ParseWarning(WARN_RULES, "Failed to read %s", file.c_str()); + delete[] db; + return false; + } + else if ( !it->group.normal_mpse->deserialize(db, len) ) + { + ParseWarning(WARN_RULES, "Failed to deserialize %s", file.c_str()); + delete[] db; + return false; + } delete[] db; - return false; + ++mpse_loaded; } - else if ( !it->group.normal_mpse->deserialize(db, len) ) - { - ParseWarning(WARN_RULES, "Failed to deserialize %s", file.c_str()); - delete[] db; - return false; - } - delete[] db; - ++mpse_loaded; } return true; } @@ -449,6 +465,30 @@ unsigned fp_deserialize(const SnortConfig* sc, const std::string& dir) return mpse_loaded; } +bool has_service_rule_opt(OptTreeNode* otn) +{ + for (OptFpList* ofl = otn->opt_func; ofl; ofl = ofl->next) + { + if ( !ofl->ips_opt ) + continue; + + CursorActionType cat = ofl->ips_opt->get_cursor_type(); + const char* s = ofl->ips_opt->get_name(); + + if ( cat <= CAT_ADJUST ) + { + if (guess_service(s) != nullptr) + return true; + + continue; + } + + if (get_num_services(s) != 0) + return true; + } + return false; +} + void validate_services(SnortConfig* sc, OptTreeNode* otn) { std::string svc, multi_svc_buf; diff --git a/src/detection/fp_utils.h b/src/detection/fp_utils.h index 293ca6d65..f2d870046 100644 --- a/src/detection/fp_utils.h +++ b/src/detection/fp_utils.h @@ -48,6 +48,7 @@ std::vector get_fp_content( void queue_mpse(snort::Mpse*); unsigned compile_mpses(struct snort::SnortConfig*, bool parallel = false); +bool has_service_rule_opt(OptTreeNode*); void validate_services(struct snort::SnortConfig*, OptTreeNode*); unsigned fp_serialize(const struct snort::SnortConfig*, const std::string& dir); @@ -56,5 +57,6 @@ unsigned fp_deserialize(const struct snort::SnortConfig*, const std::string& dir void update_buffer_map(const char** bufs, const char* svc); void add_default_services(struct snort::SnortConfig*, const std::string&, OptTreeNode*); -#endif +extern const char* section_to_str[]; +#endif diff --git a/src/detection/service_map.cc b/src/detection/service_map.cc index 89ef5de62..2a6710c81 100644 --- a/src/detection/service_map.cc +++ b/src/detection/service_map.cc @@ -163,10 +163,10 @@ static void ServiceMapAddOtn( { assert(servicename and otn); - if ( !otn->to_server() ) + if ( !otn->to_server() && !otn->to_client_err()) ServiceMapAddOtnRaw(srmm->to_cli, servicename, otn); - if ( !otn->to_client() ) + if ( !otn->to_client() && !otn->to_server_err()) ServiceMapAddOtnRaw(srmm->to_srv, servicename, otn); } diff --git a/src/detection/treenodes.h b/src/detection/treenodes.h index c8fc3339e..6aaa2d7cd 100644 --- a/src/detection/treenodes.h +++ b/src/detection/treenodes.h @@ -27,6 +27,7 @@ #include "actions/actions.h" #include "detection/signature.h" #include "detection/rule_option_types.h" +#include "framework/pdu_section.h" #include "main/policy.h" #include "main/snort_types.h" #include "ports/port_group.h" @@ -205,6 +206,10 @@ struct OptTreeNode IpsPolicy::Enable enable; Flag flags = 0; + enum SectionDir { SECT_TO_SRV = 0, SECT_TO_CLIENT, SECT_DIR__MAX }; + snort::section_flags sections[SECT_DIR__MAX] = { section_to_flag(snort::PS_NONE), + section_to_flag(snort::PS_NONE) }; + void set_warned_fp() { flags |= WARNED_FP; } @@ -263,6 +268,12 @@ struct OptTreeNode { return (flags & SVC_ONLY) != 0; } void update_fp(snort::IpsOption*); + + bool to_client_err() const + { return sections[SECT_TO_CLIENT] == section_to_flag(snort::PS_ERROR); } + + bool to_server_err() const + { return sections[SECT_TO_SRV] == section_to_flag(snort::PS_ERROR); } }; typedef int (* RuleOptEvalFunc)(void*, Cursor&, snort::Packet*); diff --git a/src/framework/CMakeLists.txt b/src/framework/CMakeLists.txt index a4afae34f..db2db7e33 100644 --- a/src/framework/CMakeLists.txt +++ b/src/framework/CMakeLists.txt @@ -19,6 +19,7 @@ set (FRAMEWORK_INCLUDES mpse_batch.h packet_constraints.h parameter.h + pdu_section.h policy_selector.h range.h so_rule.h diff --git a/src/framework/base_api.h b/src/framework/base_api.h index ccf1b4099..1345d9de7 100644 --- a/src/framework/base_api.h +++ b/src/framework/base_api.h @@ -29,7 +29,7 @@ // this is the current version of the base api // must be prefixed to subtype version -#define BASE_API_VERSION 14 +#define BASE_API_VERSION 15 // set options to API_OPTIONS to ensure compatibility #ifndef API_OPTIONS diff --git a/src/framework/inspector.h b/src/framework/inspector.h index db3b25da4..378993de6 100644 --- a/src/framework/inspector.h +++ b/src/framework/inspector.h @@ -50,11 +50,8 @@ struct InspectionBuffer // this is the only generic rule option IBT_VBA, - // FIXIT-M all of these should be eliminated after NHI is updated + // FIXIT-M all of these should be eliminated IBT_KEY, IBT_HEADER, IBT_BODY, - IBT_RAW_KEY, IBT_RAW_HEADER, IBT_METHOD, - IBT_STAT_CODE, IBT_STAT_MSG, IBT_COOKIE, - IBT_JS_DATA, IBT_MAX }; diff --git a/src/framework/ips_option.cc b/src/framework/ips_option.cc index f3feecd9c..3462943c2 100644 --- a/src/framework/ips_option.cc +++ b/src/framework/ips_option.cc @@ -50,6 +50,11 @@ bool IpsOption::operator==(const IpsOption& ips) const return !strcmp(get_name(), ips.get_name()); } +section_flags IpsOption::get_pdu_section(bool) const +{ + return section_to_flag(PS_NONE); +} + //------------------------------------------------------------------------- // UNIT TESTS //------------------------------------------------------------------------- diff --git a/src/framework/ips_option.h b/src/framework/ips_option.h index 9264832e2..2acfaa572 100644 --- a/src/framework/ips_option.h +++ b/src/framework/ips_option.h @@ -28,6 +28,8 @@ #include "main/snort_types.h" #include "target_based/snort_protocols.h" +#include "pdu_section.h" + //------------------------------------------------------------------------- // api for class // eval and action are packet thread specific @@ -49,6 +51,7 @@ class Module; enum CursorActionType { CAT_NONE, + CAT_READ, CAT_ADJUST, CAT_SET_OTHER, CAT_SET_RAW, @@ -105,6 +108,8 @@ public: const char* get_buffer() { return buffer; } + virtual section_flags get_pdu_section(bool to_server) const; + protected: IpsOption(const char* s, option_type_t t = RULE_OPTION_TYPE_OTHER); diff --git a/src/framework/mpse_batch.cc b/src/framework/mpse_batch.cc index c1ce91fe1..516453c03 100644 --- a/src/framework/mpse_batch.cc +++ b/src/framework/mpse_batch.cc @@ -72,12 +72,14 @@ MpseGroup::~MpseGroup() { if (normal_mpse) { - MpseManager::delete_search_engine(normal_mpse); + if (!normal_is_dup) + MpseManager::delete_search_engine(normal_mpse); normal_mpse = nullptr; } if (offload_mpse) { - MpseManager::delete_search_engine(offload_mpse); + if (!offload_is_dup) + MpseManager::delete_search_engine(offload_mpse); offload_mpse = nullptr; } } diff --git a/src/framework/mpse_batch.h b/src/framework/mpse_batch.h index fd500a98e..51b11743b 100644 --- a/src/framework/mpse_batch.h +++ b/src/framework/mpse_batch.h @@ -62,6 +62,8 @@ public: public: // FIXIT-L privatize Mpse* normal_mpse; Mpse* offload_mpse; + bool normal_is_dup = false; + bool offload_is_dup = false; }; template diff --git a/src/framework/pdu_section.h b/src/framework/pdu_section.h new file mode 100644 index 000000000..ec94d3850 --- /dev/null +++ b/src/framework/pdu_section.h @@ -0,0 +1,49 @@ +//-------------------------------------------------------------------------- +// Copyright (C) 2022-2022 Cisco and/or its affiliates. All rights reserved. +// +// This program is free software; you can redistribute it and/or modify it +// under the terms of the GNU General Public License Version 2 as published +// by the Free Software Foundation. You may not use, modify or distribute +// this program under any other version of the GNU General Public License. +// +// This program is distributed in the hope that it will be useful, but +// WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +// General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +//-------------------------------------------------------------------------- +// pdu_section.h author Maya Dagon + +#ifndef PDU_SECTION_H +#define PDU_SECTION_H + +// PDU section in which an IPS option provides the buffer. +// The sections are ordered from earliest to latest. +// The latest section is used to determine the rule group. +// Currently only used by ips options that apply to HTTP. The rest default to PS_NONE. + +#include "main/snort_types.h" + +namespace snort +{ +// PS_HEADER_BODY is used for rule options that can work on both header and body. +// It was added to make the rule timing selection easier: +// - if combined with header the rule should still be evaluated in both header and body. +// - if combined with body or trailer should be evaluated at body/trailer. +// PS_ERROR is used for invalid combination of sections: +// trailer and body sections can be combined only if it's a request trailer in a to_client direction +// When updating this enum, also update section_to_str +enum PduSection { PS_NONE = 0, PS_HEADER, PS_HEADER_BODY, PS_BODY, PS_TRAILER, PS_MAX = PS_TRAILER, + PS_ERROR }; + +// Bitmask with all of supported sections +using section_flags = uint16_t; + +inline section_flags section_to_flag(PduSection sect) +{ return 1<set_service_only(); + buf_is_set = true; } if ( type != OPT_TYPE_META ) otn->num_detection_opts++; + + for (int i=0; iget_pdu_section(i==OptTreeNode::SECT_TO_SRV) : section_to_flag(PS_NONE); + // Rule option is using the cursor. The default buffer is pkt_data, belongs to BODY section + if (!buf_is_set and ((cat == CAT_ADJUST) or (cat == CAT_READ))) + sections = section_to_flag(PS_BODY); + + select_section(otn->sections[i], sections); + } } OptTreeNode* parse_rule_open(SnortConfig* sc, RuleTreeNode& rtn, bool stub) @@ -1033,6 +1060,7 @@ OptTreeNode* parse_rule_open(SnortConfig* sc, RuleTreeNode& rtn, bool stub) s_capture = sc->dump_rule_meta(); s_body = "("; + buf_is_set = false; return otn; } @@ -1217,6 +1245,19 @@ void parse_rule_close(SnortConfig* sc, RuleTreeNode& rtn, OptTreeNode* otn) } ClearIpsOptionsVars(); + + for (int i=0; isections[i] == section_to_flag(PS_HEADER_BODY)) + otn->sections[i] = section_to_flag(PS_HEADER) | section_to_flag(PS_BODY); + } + + if ((otn->to_server_err() && otn->to_server()) || + (otn->to_client_err() && otn->to_client()) || + (otn->to_server_err() && otn->to_client_err())) + ParseError("Rule cannot examine both HTTP message body and HTTP trailers, unless it is request" + " trailer with response body"); + } void parse_rule_process_rtn(RuleTreeNode* rtn) diff --git a/src/ports/port_group.cc b/src/ports/port_group.cc index f39b7b924..80c09e95f 100644 --- a/src/ports/port_group.cc +++ b/src/ports/port_group.cc @@ -35,8 +35,9 @@ void RuleGroup::add_rule() RuleGroup::~RuleGroup() { - for ( auto* it : pm_list ) - delete it; + for ( int sect = snort::PS_NONE; sect <= snort::PS_MAX; sect++) + for ( auto* it : pm_list[sect] ) + delete it; delete_nfp_rules(); free_detection_option_root(&nfp_tree); @@ -79,7 +80,7 @@ void RuleGroup::delete_nfp_rules() nfp_head = nullptr; } -PatternMatcher* RuleGroup::get_pattern_matcher(PatternMatcher::Type t, const char* s) +PatternMatcher* RuleGroup::get_pattern_matcher(PatternMatcher::Type t, const char* s, snort::PduSection sect) { bool raw = false; @@ -88,7 +89,7 @@ PatternMatcher* RuleGroup::get_pattern_matcher(PatternMatcher::Type t, const cha s = "pkt_data"; raw = true; } - for ( auto& it : pm_list ) + for ( auto& it : pm_list[sect] ) { if ( it->type == t and !strcmp(it->name, s) ) { @@ -96,7 +97,7 @@ PatternMatcher* RuleGroup::get_pattern_matcher(PatternMatcher::Type t, const cha return it; } } - pm_list.push_back(new PatternMatcher(t, s, raw)); - return pm_list.back(); + pm_list[sect].push_back(new PatternMatcher(t, s, raw)); + return pm_list[sect].back(); } diff --git a/src/ports/port_group.h b/src/ports/port_group.h index c66a52b84..56a6d7ee1 100644 --- a/src/ports/port_group.h +++ b/src/ports/port_group.h @@ -28,6 +28,7 @@ #include #include +#include "framework/pdu_section.h" #include "framework/mpse_batch.h" // RuleGroup contains a set of fast patterns in the form of an MPSE and a @@ -73,7 +74,7 @@ struct RuleGroup RULE_NODE* nfp_tail = nullptr; // pattern matchers - using PmList = std::vector; + using PmList = std::vector[snort::PS_MAX + 1]; PmList pm_list; // detection option tree @@ -86,7 +87,7 @@ struct RuleGroup bool add_nfp_rule(void*); void delete_nfp_rules(); - PatternMatcher* get_pattern_matcher(PatternMatcher::Type, const char*); + PatternMatcher* get_pattern_matcher(PatternMatcher::Type, const char*, snort::PduSection sect); }; #endif diff --git a/src/protocols/packet.cc b/src/protocols/packet.cc index f3022aeb8..d3dc5cedb 100644 --- a/src/protocols/packet.cc +++ b/src/protocols/packet.cc @@ -93,6 +93,7 @@ void Packet::reset() user_network_policy_id = 0; vlan_idx = 0; filtering_state.clear(); + sect = PS_NONE; } void Packet::release_helpers() diff --git a/src/protocols/packet.h b/src/protocols/packet.h index c605741da..3b8ca361c 100644 --- a/src/protocols/packet.h +++ b/src/protocols/packet.h @@ -24,6 +24,7 @@ #include "flow/flow.h" #include "framework/decode_data.h" +#include "framework/pdu_section.h" #include "main/snort_types.h" #include "target_based/snort_protocols.h" @@ -135,6 +136,7 @@ struct SO_PUBLIC Packet IpProtocol ip_proto_next; /* the protocol ID after IP and all IP6 extension */ bool disable_inspect; mutable FilteringState filtering_state; + PduSection sect; // nothing after this point is zeroed by reset() ... IpsContext* context; @@ -366,6 +368,9 @@ struct SO_PUBLIC Packet return DAQ_PKTHDR_UNKNOWN; } + void set_pdu_section(PduSection pdu_sect) + { sect = pdu_sect; } + private: bool allocated; }; diff --git a/src/service_inspectors/http_inspect/dev_notes.txt b/src/service_inspectors/http_inspect/dev_notes.txt index 9714577f8..c17958255 100755 --- a/src/service_inspectors/http_inspect/dev_notes.txt +++ b/src/service_inspectors/http_inspect/dev_notes.txt @@ -441,9 +441,9 @@ attachment installed as file_data. Rule options: -HttpIpsOption is the base class for http rule options. It supports the commonly used parameters: -field, request, with_body, with_header and with_trailer. HttpBufferIpsOption is a rule option that -sets a buffer. It implements most of the rule options. +HttpIpsOption is the base class for http rule options. It supports the parameters field and request +that are used by some rule options. HttpBufferIpsOption is a rule option that sets a buffer. It +implements most of the rule options. Test tool usage instructions: diff --git a/src/service_inspectors/http_inspect/http_buffer_info.cc b/src/service_inspectors/http_inspect/http_buffer_info.cc index c8443981c..15f5ac377 100644 --- a/src/service_inspectors/http_inspect/http_buffer_info.cc +++ b/src/service_inspectors/http_inspect/http_buffer_info.cc @@ -21,10 +21,13 @@ #include "config.h" #endif -#include "hash/hash_key_operations.h" #include "http_buffer_info.h" +#include "hash/hash_key_operations.h" +#include "http_enum.h" + using namespace snort; +using namespace HttpEnums; uint32_t HttpBufferInfo::hash() const { @@ -46,3 +49,7 @@ bool HttpBufferInfo::operator==(const HttpBufferInfo& rhs) const form == rhs.form); } +bool HttpBufferInfo::is_request() const +{ + return ((form & FORM_REQUEST) != 0); +} diff --git a/src/service_inspectors/http_inspect/http_buffer_info.h b/src/service_inspectors/http_inspect/http_buffer_info.h index cda4a2823..757d5f3b2 100644 --- a/src/service_inspectors/http_inspect/http_buffer_info.h +++ b/src/service_inspectors/http_inspect/http_buffer_info.h @@ -20,6 +20,8 @@ #ifndef HTTP_BUFFER_INFO_H #define HTTP_BUFFER_INFO_H +#include "main/snort_types.h" + class HttpBufferInfo { public: @@ -29,6 +31,7 @@ public: uint32_t hash() const; bool operator==(const HttpBufferInfo& rhs) const; + bool is_request() const; public: const unsigned type; diff --git a/src/service_inspectors/http_inspect/http_enum.h b/src/service_inspectors/http_inspect/http_enum.h index 4671fa1ad..28f999130 100755 --- a/src/service_inspectors/http_inspect/http_enum.h +++ b/src/service_inspectors/http_inspect/http_enum.h @@ -56,7 +56,7 @@ enum HTTP_RULE_OPT { HTTP_BUFFER_CLIENT_BODY = 1, HTTP_BUFFER_COOKIE, HTTP_BUFFE HTTP_BUFFER_RAW_HEADER, HTTP_BUFFER_RAW_REQUEST, HTTP_BUFFER_RAW_STATUS, HTTP_BUFFER_RAW_TRAILER, HTTP_BUFFER_RAW_URI, HTTP_BUFFER_STAT_CODE, HTTP_BUFFER_STAT_MSG, HTTP_BUFFER_TRAILER, HTTP_BUFFER_TRUE_IP, HTTP_BUFFER_URI, HTTP_BUFFER_VERSION, - BUFFER_JS_DATA, BUFFER_VBA_DATA , HTTP__BUFFER_MAX = BUFFER_VBA_DATA, + BUFFER_JS_DATA, BUFFER_VBA_DATA, HTTP__BUFFER_MAX = BUFFER_VBA_DATA, HTTP_RANGE_NUM_HDRS, HTTP_RANGE_NUM_TRAILERS, HTTP_VERSION_MATCH, HTTP_HEADER_TEST, HTTP_TRAILER_TEST, HTTP_RANGE_NUM_COOKIES, HTTP_RANGE_MAX_HEADER_LINE, HTTP_RANGE_MAX_TRAILER_LINE, HTTP__MAX_RULE_OPTION }; @@ -113,9 +113,6 @@ enum CompressId { CMP_NONE=2, CMP_GZIP, CMP_DEFLATE }; // GZIP magic verification state enum GzipVerificationState { GZIP_TBD, GZIP_MAGIC_BAD, GZIP_MAGIC_GOOD, GZIP_FLAGS_PROCESSED }; -// Message section in which an IPS option provides the buffer -enum InspectSection { IS_NONE, IS_HEADER, IS_FLEX_HEADER, IS_FIRST_BODY, IS_BODY, IS_TRAILER }; - // Part of the URI to be provided enum UriComponent { UC_SCHEME = 1, UC_HOST, UC_PORT, UC_PATH, UC_QUERY, UC_FRAGMENT }; diff --git a/src/service_inspectors/http_inspect/http_inspect.cc b/src/service_inspectors/http_inspect/http_inspect.cc index 91d1e3589..8d248e2da 100755 --- a/src/service_inspectors/http_inspect/http_inspect.cc +++ b/src/service_inspectors/http_inspect/http_inspect.cc @@ -200,17 +200,17 @@ void HttpInspect::show(const SnortConfig*) const ConfigLogger::log_flag("request_body_app_detection", params->publish_request_body); } -InspectSection HttpInspect::get_latest_is(const Packet* p) +PduSection HttpInspect::get_latest_is(const Packet* p) { HttpMsgSection* current_section = HttpContextData::get_snapshot(p); if (current_section == nullptr) - return IS_NONE; + return PS_NONE; // FIXIT-L revisit why we need this check. We should not be getting a current section back // for a raw packet but one of the test cases did exactly that. if (!(p->packet_flags & PKT_PSEUDO)) - return IS_NONE; + return PS_NONE; return current_section->get_inspection_section(); } @@ -233,7 +233,7 @@ bool HttpInspect::get_buf(InspectionBuffer::Type ibt, Packet* p, InspectionBuffe return get_buf(HTTP_BUFFER_URI, p, b); case InspectionBuffer::IBT_HEADER: - if (get_latest_is(p) == IS_TRAILER) + if (get_latest_is(p) == PS_TRAILER) return get_buf(HTTP_BUFFER_TRAILER, p, b); else return get_buf(HTTP_BUFFER_HEADER, p , b); @@ -241,34 +241,11 @@ bool HttpInspect::get_buf(InspectionBuffer::Type ibt, Packet* p, InspectionBuffe case InspectionBuffer::IBT_BODY: return get_buf(HTTP_BUFFER_CLIENT_BODY, p, b); - case InspectionBuffer::IBT_RAW_KEY: - return get_buf(HTTP_BUFFER_RAW_URI, p , b); - - case InspectionBuffer::IBT_RAW_HEADER: - if (get_latest_is(p) == IS_TRAILER) - return get_buf(HTTP_BUFFER_RAW_TRAILER, p, b); - else - return get_buf(HTTP_BUFFER_RAW_HEADER, p , b); - - case InspectionBuffer::IBT_METHOD: - return get_buf(HTTP_BUFFER_METHOD, p , b); - - case InspectionBuffer::IBT_STAT_CODE: - return get_buf(HTTP_BUFFER_STAT_CODE, p , b); - - case InspectionBuffer::IBT_STAT_MSG: - return get_buf(HTTP_BUFFER_STAT_MSG, p , b); - - case InspectionBuffer::IBT_COOKIE: - return get_buf(HTTP_BUFFER_COOKIE, p , b); - case InspectionBuffer::IBT_VBA: return get_buf(BUFFER_VBA_DATA, p, b); - case InspectionBuffer::IBT_JS_DATA: - return get_buf(BUFFER_JS_DATA, p, b); - default: + assert(false); return false; } } @@ -373,48 +350,11 @@ void HttpInspect::set_hx_body_state(snort::Flow* flow, HttpCommon::SourceId sour bool HttpInspect::get_fp_buf(InspectionBuffer::Type ibt, Packet* p, InspectionBuffer& b) { - if (get_latest_is(p) == IS_NONE) + assert(ibt == InspectionBuffer::IBT_VBA); + + if (get_latest_is(p) == PS_NONE) return false; - // Fast pattern buffers only supplied at specific times - switch (ibt) - { - case InspectionBuffer::IBT_KEY: - case InspectionBuffer::IBT_RAW_KEY: - // Many rules targeting POST feature http_uri fast pattern with http_client_body. We - // accept the performance hit of rerunning http_uri fast pattern with request body message - // sections - if (get_latest_src(p) != SRC_CLIENT) - return false; - break; - case InspectionBuffer::IBT_HEADER: - case InspectionBuffer::IBT_RAW_HEADER: - // http_header fast patterns for response bodies limited to first section - if ((get_latest_src(p) == SRC_SERVER) && (get_latest_is(p) == IS_BODY)) - return false; - break; - case InspectionBuffer::IBT_BODY: - case InspectionBuffer::IBT_VBA: - case InspectionBuffer::IBT_JS_DATA: - if ((get_latest_is(p) != IS_FIRST_BODY) && (get_latest_is(p) != IS_BODY)) - return false; - break; - case InspectionBuffer::IBT_METHOD: - if ((get_latest_src(p) != SRC_CLIENT) || (get_latest_is(p) == IS_BODY)) - return false; - break; - case InspectionBuffer::IBT_STAT_CODE: - case InspectionBuffer::IBT_STAT_MSG: - if ((get_latest_src(p) != SRC_SERVER) || (get_latest_is(p) != IS_HEADER)) - return false; - break; - case InspectionBuffer::IBT_COOKIE: - if (get_latest_is(p) != IS_HEADER) - return false; - break; - default: - break; - } return get_buf(ibt, p, b); } @@ -681,6 +621,12 @@ void HttpInspect::process(const uint8_t* data, const uint16_t dsize, Flow* const #endif current_section->publish(); + if (p != nullptr) + { + const PduSection pdu_section = current_section->get_inspection_section(); + p->set_pdu_section(pdu_section); + } + if (current_section->run_detection(p)) { #ifdef REG_TEST diff --git a/src/service_inspectors/http_inspect/http_inspect.h b/src/service_inspectors/http_inspect/http_inspect.h index 4835c00e7..1542d0d2b 100644 --- a/src/service_inspectors/http_inspect/http_inspect.h +++ b/src/service_inspectors/http_inspect/http_inspect.h @@ -25,6 +25,7 @@ //------------------------------------------------------------------------- #include "framework/cursor.h" +#include "framework/pdu_section.h" #include "helpers/literal_search.h" #include "log/messages.h" @@ -77,7 +78,7 @@ public: bool can_start_tls() const override { return true; } - static HttpEnums::InspectSection get_latest_is(const snort::Packet* p); + static snort::PduSection get_latest_is(const snort::Packet* p); static HttpCommon::SourceId get_latest_src(const snort::Packet* p); void disable_detection(snort::Packet* p); diff --git a/src/service_inspectors/http_inspect/http_msg_body.h b/src/service_inspectors/http_inspect/http_msg_body.h index 0fe62ddac..ab10bed5b 100644 --- a/src/service_inspectors/http_inspect/http_msg_body.h +++ b/src/service_inspectors/http_inspect/http_msg_body.h @@ -38,8 +38,8 @@ class HttpMsgBody : public HttpMsgSection public: ~HttpMsgBody() override { delete mime_bufs; } void analyze() override; - HttpEnums::InspectSection get_inspection_section() const override - { return first_body ? HttpEnums::IS_FIRST_BODY : HttpEnums::IS_BODY; } + snort::PduSection get_inspection_section() const override + { return snort::PS_BODY; } bool detection_required() const override { return (detect_data.length() > 0); } bool run_detection(snort::Packet* p) override; void clear() override; diff --git a/src/service_inspectors/http_inspect/http_msg_header.h b/src/service_inspectors/http_inspect/http_msg_header.h index 739f8129d..67642db94 100644 --- a/src/service_inspectors/http_inspect/http_msg_header.h +++ b/src/service_inspectors/http_inspect/http_msg_header.h @@ -38,8 +38,8 @@ public: HttpMsgHeader(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_, HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_, const HttpParaList* params_); - HttpEnums::InspectSection get_inspection_section() const override - { return HttpEnums::IS_HEADER; } + snort::PduSection get_inspection_section() const override + { return snort::PS_HEADER; } bool detection_required() const override { return true; } void update_flow() override; void gen_events() override; diff --git a/src/service_inspectors/http_inspect/http_msg_request.h b/src/service_inspectors/http_inspect/http_msg_request.h index 8f58df912..c3a3c54d6 100644 --- a/src/service_inspectors/http_inspect/http_msg_request.h +++ b/src/service_inspectors/http_inspect/http_msg_request.h @@ -41,8 +41,8 @@ public: ~HttpMsgRequest() override; bool detection_required() const override { return version_id == HttpEnums::VERS_0_9; } - HttpEnums::InspectSection get_inspection_section() const override - { return HttpEnums::IS_HEADER; } + snort::PduSection get_inspection_section() const override + { return snort::PS_HEADER; } void gen_events() override; void update_flow() override; void publish() override; diff --git a/src/service_inspectors/http_inspect/http_msg_section.h b/src/service_inspectors/http_inspect/http_msg_section.h index 9c20afc11..905d4586d 100644 --- a/src/service_inspectors/http_inspect/http_msg_section.h +++ b/src/service_inspectors/http_inspect/http_msg_section.h @@ -22,6 +22,7 @@ #include "detection/detection_util.h" #include "framework/cursor.h" +#include "framework/pdu_section.h" #include "protocols/packet.h" #include "http_buffer_info.h" @@ -42,8 +43,8 @@ class HttpMsgSection { public: virtual ~HttpMsgSection() = default; - virtual HttpEnums::InspectSection get_inspection_section() const - { return HttpEnums::IS_NONE; } + virtual snort::PduSection get_inspection_section() const + { return snort::PS_NONE; } virtual bool detection_required() const = 0; HttpCommon::SourceId get_source_id() const { return source_id; } HttpTransaction* get_transaction() const { return transaction; } diff --git a/src/service_inspectors/http_inspect/http_msg_trailer.h b/src/service_inspectors/http_inspect/http_msg_trailer.h index cdb5d109d..224067bcb 100644 --- a/src/service_inspectors/http_inspect/http_msg_trailer.h +++ b/src/service_inspectors/http_inspect/http_msg_trailer.h @@ -34,8 +34,8 @@ public: HttpMsgTrailer(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_, HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_, const HttpParaList* params_); - HttpEnums::InspectSection get_inspection_section() const override - { return HttpEnums::IS_TRAILER; } + snort::PduSection get_inspection_section() const override + { return snort::PS_TRAILER; } bool detection_required() const override { return (msg_text.length() > 0); } void gen_events() override; void update_flow() override; diff --git a/src/service_inspectors/http_inspect/ips_http.cc b/src/service_inspectors/http_inspect/ips_http.cc index 609c14fd3..660c928d0 100644 --- a/src/service_inspectors/http_inspect/ips_http.cc +++ b/src/service_inspectors/http_inspect/ips_http.cc @@ -42,10 +42,8 @@ using namespace HttpEnums; bool HttpRuleOptModule::begin(const char*, int, SnortConfig*) { - para_list.reset(); sub_id = 0; form = 0; - is_trailer_opt = false; return true; } @@ -55,14 +53,14 @@ bool HttpRuleOptModule::set(const char*, Value& v, SnortConfig*) { if (sub_id != 0) ParseError("Only specify one header field to match"); - para_list.field = v.get_string(); - const int32_t name_size = (para_list.field.size() <= MAX_FIELD_NAME_LENGTH) ? - para_list.field.size() : MAX_FIELD_NAME_LENGTH; + std::string field = v.get_string(); + const int32_t name_size = (field.size() <= MAX_FIELD_NAME_LENGTH) ? + field.size() : MAX_FIELD_NAME_LENGTH; uint8_t lower_name[MAX_FIELD_NAME_LENGTH]; for (int32_t k=0; k < name_size; k++) { - lower_name[k] = ((para_list.field[k] < 'A') || (para_list.field[k] > 'Z')) ? - para_list.field[k] : para_list.field[k] - ('A' - 'a'); + lower_name[k] = ((field[k] < 'A') || (field[k] > 'Z')) ? + field[k] : field[k] - ('A' - 'a'); } sub_id = str_to_code(lower_name, name_size, HttpMsgHeadShared::header_list); if (sub_id == STAT_OTHER) @@ -70,51 +68,15 @@ bool HttpRuleOptModule::set(const char*, Value& v, SnortConfig*) } else if (v.is("request")) { - para_list.request = true; form |= FORM_REQUEST; } - else if (v.is("with_header")) - { - para_list.with_header = true; - inspect_section = IS_HEADER; - } - else if (v.is("with_body")) - { - para_list.with_body = true; - inspect_section = IS_BODY; - } - else if (v.is("with_trailer")) - { - para_list.with_trailer = true; - inspect_section = IS_TRAILER; - } return true; } -bool HttpRuleOptModule::end(const char*, int, SnortConfig*) -{ - // Check for option conflicts - if (para_list.with_header + para_list.with_body + para_list.with_trailer > 1) - ParseError("Only specify one with_ option. Use the one that happens last."); - if ( is_trailer_opt && (para_list.with_header || para_list.with_body) && - !para_list.request) - ParseError("Trailers with with_ option must also specify request"); - return true; -} - -void HttpRuleOptModule::HttpRuleParaList::reset() -{ - field.clear(); - request = false; - with_header = false; - with_body = false; - with_trailer = false; -} - uint32_t HttpIpsOption::hash() const { uint32_t a = IpsOption::hash(); - uint32_t b = (uint32_t)inspect_section; + uint32_t b = (uint32_t)pdu_section; uint32_t c = buffer_info.hash(); mix(a,b,c); finalize(a,b,c); @@ -125,23 +87,14 @@ bool HttpIpsOption::operator==(const IpsOption& ips) const { const HttpIpsOption& hio = static_cast(ips); return IpsOption::operator==(ips) && - inspect_section == hio.inspect_section && + pdu_section == hio.pdu_section && buffer_info == hio.buffer_info; } -// Verify inspect_section matches. If it does get inspector pointer. +// If pdu_section isn't NONE get inspector pointer. HttpInspect const* HttpIpsOption::eval_helper(Packet* p) { - if (!p->flow || !p->flow->gadget || (HttpInspect::get_latest_is(p) == IS_NONE)) - return nullptr; - - const bool section_match = - (p->packet_flags & PKT_FAST_PAT_EVAL) || - (HttpInspect::get_latest_is(p) == inspect_section) || - ((HttpInspect::get_latest_is(p) == IS_HEADER) && (inspect_section == IS_FLEX_HEADER)) || - ((HttpInspect::get_latest_is(p) == IS_FIRST_BODY) && (inspect_section == IS_BODY)) || - ((HttpInspect::get_latest_src(p) == SRC_CLIENT) && (inspect_section == IS_FLEX_HEADER)); - if (!section_match) + if (!p->flow || !p->flow->gadget || (HttpInspect::get_latest_is(p) == PS_NONE)) return nullptr; assert(p->flow->stream_intf); @@ -154,3 +107,11 @@ HttpInspect const* HttpIpsOption::eval_helper(Packet* p) return hi; } +section_flags HttpIpsOption::get_pdu_section(bool to_server) const +{ + // Trailer with request sub-option in a rule working on the response + // should be evaluated during the response headers + if (pdu_section == PS_TRAILER && buffer_info.is_request() && !to_server) + return section_to_flag(PS_HEADER); + return section_to_flag(pdu_section); +} diff --git a/src/service_inspectors/http_inspect/ips_http.h b/src/service_inspectors/http_inspect/ips_http.h index 898113236..cab275707 100644 --- a/src/service_inspectors/http_inspect/ips_http.h +++ b/src/service_inspectors/http_inspect/ips_http.h @@ -23,6 +23,7 @@ #include #include "profiler/profiler.h" +#include "framework/pdu_section.h" #include "framework/ips_option.h" #include "framework/module.h" #include "framework/range.h" @@ -47,35 +48,20 @@ public: static void mod_dtor(snort::Module* m) { delete m; } bool begin(const char*, int, snort::SnortConfig*) override; bool set(const char*, snort::Value&, snort::SnortConfig*) override; - bool end(const char*, int, snort::SnortConfig*) override; Usage get_usage() const override { return DETECT; } protected: - HttpEnums::InspectSection inspect_section; + snort::PduSection pdu_section; const HttpEnums::HTTP_RULE_OPT rule_opt_index; const char* const key; uint64_t sub_id; - bool is_trailer_opt; // used by ::end for with_* validation private: friend class HttpIpsOption; - struct HttpRuleParaList - { - public: - std::string field; // provide buffer containing specific header field - bool request; // provide buffer from request not response - bool with_header; // provide buffer with a later section than it appears in - bool with_body; - bool with_trailer; - - void reset(); - }; - const snort::CursorActionType cat; - HttpRuleParaList para_list; uint64_t form; }; @@ -85,20 +71,22 @@ public: HttpIpsOption(const HttpRuleOptModule* cm) : snort::IpsOption(cm->key), buffer_info(cm->rule_opt_index, cm->sub_id, cm->form), - cat(cm->cat), inspect_section(cm->inspect_section) {} + cat(cm->cat), pdu_section(cm->pdu_section) {} snort::CursorActionType get_cursor_type() const override { return cat; } EvalStatus eval(Cursor&, snort::Packet*) override = 0; uint32_t hash() const override; bool operator==(const snort::IpsOption& ips) const override; + + snort::section_flags get_pdu_section(bool) const override; protected: const HttpBufferInfo buffer_info; - + const snort::CursorActionType cat; + HttpInspect const* eval_helper(snort::Packet* p); private: - const snort::CursorActionType cat; - const HttpEnums::InspectSection inspect_section; + const snort::PduSection pdu_section; }; #endif diff --git a/src/service_inspectors/http_inspect/ips_http_buffer.cc b/src/service_inspectors/http_inspect/ips_http_buffer.cc index 15f0e0718..a2be675e5 100644 --- a/src/service_inspectors/http_inspect/ips_http_buffer.cc +++ b/src/service_inspectors/http_inspect/ips_http_buffer.cc @@ -55,8 +55,6 @@ bool HttpBufferRuleOptModule::begin(const char*, int, SnortConfig*) case HTTP_BUFFER_RAW_STATUS: case HTTP_BUFFER_STAT_CODE: case HTTP_BUFFER_STAT_MSG: - inspect_section = IS_HEADER; - break; case HTTP_BUFFER_COOKIE: case HTTP_BUFFER_HEADER: case HTTP_BUFFER_METHOD: @@ -67,17 +65,16 @@ bool HttpBufferRuleOptModule::begin(const char*, int, SnortConfig*) case HTTP_BUFFER_TRUE_IP: case HTTP_BUFFER_URI: case HTTP_BUFFER_VERSION: - inspect_section = IS_FLEX_HEADER; + pdu_section = PS_HEADER; break; case HTTP_BUFFER_CLIENT_BODY: case HTTP_BUFFER_RAW_BODY: case BUFFER_JS_DATA: - inspect_section = IS_BODY; + pdu_section = PS_BODY; break; case HTTP_BUFFER_RAW_TRAILER: case HTTP_BUFFER_TRAILER: - inspect_section = IS_TRAILER; - is_trailer_opt = true; + pdu_section = PS_TRAILER; break; default: assert(false); @@ -130,40 +127,9 @@ bool HttpBufferRuleOptModule::end(const char*, int, SnortConfig*) // Check for option conflicts if (scheme + host + port + path + query + fragment > 1) ParseError("Only specify one part of the URI"); - return HttpRuleOptModule::end(nullptr, 0, nullptr); + return true; } -static InspectionBuffer::Type buf_map[] = -{ -#if 0 - BUFFER_PSI_CLIENT_BODY, BUFFER_PSI_COOKIE, BUFFER_PSI_HEADER, BUFFER_PSI_METHOD, - BUFFER_PSI_RAW_BODY, BUFFER_PSI_RAW_COOKIE, BUFFER_PSI_RAW_HEADER, BUFFER_PSI_RAW_REQUEST, - BUFFER_PSI_RAW_STATUS, BUFFER_PSI_RAW_TRAILER, BUFFER_PSI_RAW_URI, BUFFER_PSI_STAT_CODE, - BUFFER_PSI_STAT_MSG, BUFFER_PSI_TRAILER, BUFFER_PSI_TRUE_IP, BUFFER_PSI_URI, BUFFER_PSI_VERSION, - BUFFER_PSI_JS_DATA, BUFFER_PSI_VBA_DATA, BUFFER_PSI_MAX -#endif - InspectionBuffer::IBT_BODY, - InspectionBuffer::IBT_COOKIE, - InspectionBuffer::IBT_HEADER, - InspectionBuffer::IBT_METHOD, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_RAW_HEADER, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_RAW_HEADER, - InspectionBuffer::IBT_RAW_KEY, - InspectionBuffer::IBT_STAT_CODE, - InspectionBuffer::IBT_STAT_MSG, - InspectionBuffer::IBT_HEADER, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_KEY, - InspectionBuffer::IBT_MAX, - InspectionBuffer::IBT_JS_DATA, - InspectionBuffer::IBT_VBA, - InspectionBuffer::IBT_MAX -}; - IpsOption::EvalStatus HttpBufferIpsOption::eval(Cursor& c, Packet* p) { RuleProfile profile(HttpBufferRuleOptModule::http_buffer_ps[idx]); @@ -174,20 +140,26 @@ IpsOption::EvalStatus HttpBufferIpsOption::eval(Cursor& c, Packet* p) if (p->packet_flags & PKT_FAST_PAT_EVAL) { - InspectionBuffer buf; - InspectionBuffer::Type ibt = buf_map[idx]; - - if (ibt == InspectionBuffer::IBT_MAX) - return NO_MATCH; - - if (!hi->get_fp_buf(ibt, p, buf)) - return NO_MATCH; - - c.set(key, buf.data, buf.len); - return MATCH; + switch (idx) + { + case BUFFER_PSI_RAW_URI: + case BUFFER_PSI_URI: + case BUFFER_PSI_METHOD: + if (hi->get_latest_src(p) != SRC_CLIENT) + return NO_MATCH; + break; + case BUFFER_PSI_STAT_CODE: + case BUFFER_PSI_STAT_MSG: + if (hi->get_latest_src(p) != SRC_SERVER) + return NO_MATCH; + break; + default: + break; + } } + const HttpBufferInfo& cur_buf_info = (p->packet_flags & PKT_FAST_PAT_EVAL) ? fp_buffer_info : buffer_info; + const Field& http_buffer = hi->http_get_buf(p, cur_buf_info); - const Field& http_buffer = hi->http_get_buf(p, buffer_info); if (http_buffer.length() <= 0) return NO_MATCH; @@ -246,11 +218,11 @@ static const Parameter http_cookie_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the cookie from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -307,11 +279,11 @@ static const Parameter http_header_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the headers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -358,11 +330,11 @@ static const IpsApi header_api = static const Parameter http_method_params[] = { { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -451,11 +423,11 @@ static const Parameter http_raw_cookie_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the cookie from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -506,11 +478,11 @@ static const Parameter http_raw_header_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the headers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -557,11 +529,11 @@ static const IpsApi raw_header_api = static const Parameter http_raw_request_params[] = { { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -608,9 +580,9 @@ static const IpsApi raw_request_api = static const Parameter http_raw_status_params[] = { { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -661,10 +633,9 @@ static const Parameter http_raw_trailer_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the trailers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP response message headers (must be combined with request)" - }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP response message body (must be combined with request)" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -711,11 +682,11 @@ static const IpsApi raw_trailer_api = static const Parameter http_raw_uri_params[] = { { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { "scheme", Parameter::PT_IMPLIED, nullptr, nullptr, "match against scheme section of URI only" }, { "host", Parameter::PT_IMPLIED, nullptr, nullptr, @@ -774,9 +745,9 @@ static const IpsApi raw_uri_api = static const Parameter http_stat_code_params[] = { { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -823,9 +794,9 @@ static const IpsApi stat_code_api = static const Parameter http_stat_msg_params[] = { { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -875,10 +846,9 @@ static const Parameter http_trailer_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the trailers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP response message headers (must be combined with request)" - }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body (must be combined with request)" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -925,11 +895,11 @@ static const IpsApi trailer_api = static const Parameter http_true_ip_params[] = { { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; @@ -976,11 +946,11 @@ static const IpsApi true_ip_api = static const Parameter http_uri_params[] = { { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { "scheme", Parameter::PT_IMPLIED, nullptr, nullptr, "match against scheme section of URI only" }, { "host", Parameter::PT_IMPLIED, nullptr, nullptr, @@ -1041,11 +1011,11 @@ static const Parameter http_version_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the version from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; diff --git a/src/service_inspectors/http_inspect/ips_http_buffer.h b/src/service_inspectors/http_inspect/ips_http_buffer.h index 8af7d0c7f..4ec46fd80 100644 --- a/src/service_inspectors/http_inspect/ips_http_buffer.h +++ b/src/service_inspectors/http_inspect/ips_http_buffer.h @@ -33,7 +33,7 @@ enum BufferPsIdx { BUFFER_PSI_CLIENT_BODY, BUFFER_PSI_COOKIE, BUFFER_PSI_HEADER, BUFFER_PSI_RAW_BODY, BUFFER_PSI_RAW_COOKIE, BUFFER_PSI_RAW_HEADER, BUFFER_PSI_RAW_REQUEST, BUFFER_PSI_RAW_STATUS, BUFFER_PSI_RAW_TRAILER, BUFFER_PSI_RAW_URI, BUFFER_PSI_STAT_CODE, BUFFER_PSI_STAT_MSG, BUFFER_PSI_TRAILER, BUFFER_PSI_TRUE_IP, BUFFER_PSI_URI, BUFFER_PSI_VERSION, - BUFFER_PSI_JS_DATA, BUFFER_PSI_VBA_DATA, BUFFER_PSI_MAX }; + BUFFER_PSI_JS_DATA, BUFFER_PSI_MAX }; class HttpBufferRuleOptModule : public HttpRuleOptModule { @@ -70,7 +70,7 @@ class HttpBufferIpsOption : public HttpIpsOption public: HttpBufferIpsOption(const HttpBufferRuleOptModule* cm) : HttpIpsOption(cm), idx(cm->idx), - key(cm->key) {} + key(cm->key), fp_buffer_info(cm->rule_opt_index) {} EvalStatus eval(Cursor&, snort::Packet*) override; static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*) @@ -78,9 +78,13 @@ public: static void opt_dtor(snort::IpsOption* p) { delete p; } + snort::CursorActionType get_cursor_type() const override + { return buffer_info.is_request()? snort::CAT_SET_OTHER : cat; } + private: const BufferPsIdx idx; const char* const key; + const HttpBufferInfo fp_buffer_info; }; #endif diff --git a/src/service_inspectors/http_inspect/ips_http_num_hdrs.cc b/src/service_inspectors/http_inspect/ips_http_num_hdrs.cc index 90d0e07f5..dfc46c6ea 100644 --- a/src/service_inspectors/http_inspect/ips_http_num_hdrs.cc +++ b/src/service_inspectors/http_inspect/ips_http_num_hdrs.cc @@ -119,7 +119,7 @@ static const Parameter http_max_header_line_params[] = static Module* max_header_line_mod_ctor() { - return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, + return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, http_max_header_line_params); } @@ -167,7 +167,7 @@ static const Parameter http_max_trailer_line_params[] = static Module* max_trailer_line_mod_ctor() { - return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, + return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, http_max_trailer_line_params); } @@ -215,7 +215,7 @@ static const Parameter http_num_cookies_params[] = static Module* num_cookies_mod_ctor() { - return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, + return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, http_num_cookies_params); } @@ -259,17 +259,17 @@ static const Parameter http_num_hdrs_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the version from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } }; static Module* num_hdrs_mod_ctor() { - return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, + return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, http_num_hdrs_params); } @@ -308,7 +308,7 @@ static const IpsApi num_headers_api = static Module* num_trailers_mod_ctor() { - return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, + return new HttpNumRuleOptModule(IPS_OPT, IPS_HELP, http_num_hdrs_params); } diff --git a/src/service_inspectors/http_inspect/ips_http_num_hdrs.h b/src/service_inspectors/http_inspect/ips_http_num_hdrs.h index bc0e62712..f4817cc5b 100644 --- a/src/service_inspectors/http_inspect/ips_http_num_hdrs.h +++ b/src/service_inspectors/http_inspect/ips_http_num_hdrs.h @@ -73,7 +73,7 @@ private: }; // Template class for range-based rule options module -template +template class HttpNumRuleOptModule : public HttpRangeRuleOptModule { public: @@ -83,11 +83,7 @@ public: bool begin(const char*, int, snort::SnortConfig*) override { HttpRangeRuleOptModule::begin(nullptr, 0, nullptr); - inspect_section = SECTION; - if (inspect_section == HttpEnums::IS_TRAILER) - { - is_trailer_opt = true; - } + pdu_section = SECTION; return true; } @@ -95,7 +91,7 @@ private: static THREAD_LOCAL snort::ProfileStats ps; }; -template +template THREAD_LOCAL snort::ProfileStats HttpNumRuleOptModule::ps; // Template class for range-based rule options diff --git a/src/service_inspectors/http_inspect/ips_http_param.cc b/src/service_inspectors/http_inspect/ips_http_param.cc index 3db8f2ff0..28a1a8b4a 100644 --- a/src/service_inspectors/http_inspect/ips_http_param.cc +++ b/src/service_inspectors/http_inspect/ips_http_param.cc @@ -47,7 +47,7 @@ bool HttpParamRuleOptModule::begin(const char*, int, SnortConfig*) HttpRuleOptModule::begin(nullptr, 0, nullptr); param.clear(); nocase = false; - inspect_section = IS_FLEX_HEADER; + pdu_section = PS_HEADER; return true; } @@ -119,6 +119,13 @@ IpsOption::EvalStatus HttpParamIpsOption::eval(Cursor& c, Packet* p) return MATCH; } +section_flags HttpParamIpsOption::get_pdu_section(bool) const +{ + // Works on URI or client body + return section_to_flag(snort::PS_HEADER_BODY); +} + + //------------------------------------------------------------------------- // http_param //------------------------------------------------------------------------- diff --git a/src/service_inspectors/http_inspect/ips_http_param.h b/src/service_inspectors/http_inspect/ips_http_param.h index 9557363c5..c7ca347ac 100644 --- a/src/service_inspectors/http_inspect/ips_http_param.h +++ b/src/service_inspectors/http_inspect/ips_http_param.h @@ -64,6 +64,8 @@ public: static void opt_dtor(snort::IpsOption* p) { delete p; } bool retry(Cursor& , const Cursor&) override; + + snort::section_flags get_pdu_section(bool) const override; private: const char* const key; diff --git a/src/service_inspectors/http_inspect/ips_http_test.cc b/src/service_inspectors/http_inspect/ips_http_test.cc index d78d2eae0..42bef6bba 100644 --- a/src/service_inspectors/http_inspect/ips_http_test.cc +++ b/src/service_inspectors/http_inspect/ips_http_test.cc @@ -48,12 +48,9 @@ bool HttpTestRuleOptModule::begin(const char*, int, SnortConfig*) numeric = NV_UNDEFINED; absent = false; if (rule_opt_index == HTTP_HEADER_TEST) - inspect_section = IS_FLEX_HEADER; + pdu_section = PS_HEADER; else - { - inspect_section = IS_TRAILER; - is_trailer_opt = true; - } + pdu_section = PS_TRAILER; return true; } @@ -92,7 +89,7 @@ bool HttpTestRuleOptModule::end(const char*, int, SnortConfig*) (check.is_set() && numeric == NV_FALSE)) ParseWarning(WARN_RULES, "conflicting suboptions"); - return HttpRuleOptModule::end(nullptr, 0, nullptr); + return true; } uint32_t HttpTestIpsOption::hash() const @@ -143,7 +140,7 @@ IpsOption::EvalStatus HttpTestIpsOption::eval_header_test(const Field& http_buff int64_t num = 0; const int32_t length = http_buffer.length(); - if (length <= 0) + if (length < 0) is_absent = true; // Limit to 18 decimal digits, to fit comfortably into int64_t. else if (length <= 18) @@ -159,7 +156,6 @@ IpsOption::EvalStatus HttpTestIpsOption::eval_header_test(const Field& http_buff return (absent_passed && numeric_passed && range_passed) ? MATCH : NO_MATCH; } - IpsOption::EvalStatus HttpTestIpsOption::eval(Cursor&, Packet* p) { RuleProfile profile(HttpTestRuleOptModule::http_test_ps[idx]); @@ -170,6 +166,15 @@ IpsOption::EvalStatus HttpTestIpsOption::eval(Cursor&, Packet* p) const Field& http_buffer = hi->http_get_buf(p, buffer_info); + // Check if field is absent or section is not present + if (http_buffer.length() < 0) + { + const HttpBufferInfo section_info = HttpBufferInfo(buffer_info.type, 0, buffer_info.form); + const Field& section_buffer = hi->http_get_buf(p, section_info); + if (section_buffer.length() < 0) + return NO_MATCH; + } + return eval_header_test(http_buffer); } @@ -190,11 +195,11 @@ static const Parameter hdr_test_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the headers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { "check", Parameter::PT_INTERVAL, hdr_test_range.c_str(), nullptr, "range check to perform on header value" }, { "numeric", Parameter::PT_BOOL, nullptr, nullptr, @@ -245,9 +250,9 @@ static const Parameter trailer_test_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the trailers from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "check", Parameter::PT_INTERVAL, hdr_test_range.c_str(), nullptr, "range check to perform on trailer value" }, { "numeric", Parameter::PT_BOOL, nullptr, nullptr, diff --git a/src/service_inspectors/http_inspect/ips_http_version.cc b/src/service_inspectors/http_inspect/ips_http_version.cc index cd7166b67..ba5e0dff6 100644 --- a/src/service_inspectors/http_inspect/ips_http_version.cc +++ b/src/service_inspectors/http_inspect/ips_http_version.cc @@ -43,7 +43,7 @@ THREAD_LOCAL ProfileStats HttpVersionRuleOptModule::http_version_ps; bool HttpVersionRuleOptModule::begin(const char*, int, SnortConfig*) { HttpRuleOptModule::begin(nullptr, 0, nullptr); - inspect_section = IS_FLEX_HEADER; + pdu_section = PS_HEADER; version_flags = 0; return true; } @@ -144,11 +144,11 @@ static const Parameter version_match_params[] = { "request", Parameter::PT_IMPLIED, nullptr, nullptr, "match against the version from the request message even when examining the response" }, { "with_header", Parameter::PT_IMPLIED, nullptr, nullptr, - "this rule is limited to examining HTTP message headers" }, + "option is no longer used and will be removed in a future release" }, { "with_body", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message body" }, + "option is no longer used and will be removed in a future release" }, { "with_trailer", Parameter::PT_IMPLIED, nullptr, nullptr, - "parts of this rule examine HTTP message trailers" }, + "option is no longer used and will be removed in a future release" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } };