From: Christopher Faulet Date: Wed, 22 Jan 2020 08:26:35 +0000 (+0100) Subject: MEDIUM: http: Add a ruleset evaluated on all responses just before forwarding X-Git-Tag: v2.2-dev2~16 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=6d0c3dfac643d21478793759927dc7cfb8b1b3cd;p=thirdparty%2Fhaproxy.git MEDIUM: http: Add a ruleset evaluated on all responses just before forwarding This patch introduces the 'http-after-response' rules. These rules are evaluated at the end of the response analysis, just before the data forwarding, on ALL HTTP responses, the server ones but also all responses generated by HAProxy. Thanks to this ruleset, it is now possible for instance to add some headers to the responses generated by the stats applet. Following actions are supported : * allow * add-header * del-header * replace-header * replace-value * set-header * set-status * set-var * strict-mode * unset-var --- diff --git a/doc/configuration.txt b/doc/configuration.txt index 776fd70640..cfbcc5cd6b 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -2605,6 +2605,7 @@ filter - X X X fullconn X - X X grace X X X X hash-type X - X X +http-after-response - X X X http-check disable-on-404 X - X X http-check expect - - X X http-check send-state X - X X @@ -4189,6 +4190,154 @@ hash-type See also : "balance", "hash-balance-factor", "server" +http-after-response [ { if | unless } ] + Access control for all Layer 7 responses (server, applet/service and internal + ones). + + May be used in sections: defaults | frontend | listen | backend + no | yes | yes | yes + + The http-after-response statement defines a set of rules which apply to layer + 7 processing. The rules are evaluated in their declaration order when they + are met in a frontend, listen or backend section. Any rule may optionally be + followed by an ACL-based condition, in which case it will only be evaluated + if the condition is true. Since these rules apply on responses, the backend + rules are applied first, followed by the frontend's rules. + + Unlike http-response rules, these ones are applied on all responses, the + server ones but also to all responses generated by HAProxy. These rules are + evaluated at the end of the responses analysis, before the data forwarding. + + The first keyword is the rule's action. The supported actions are described + below. + + There is no limit to the number of http-after-response statements per + instance. + + Example: + http-after-response set-header Strict-Transport-Security "max-age=31536000" + http-after-response set-header Cache-Control "no-store,no-cache,private" + http-after-response set-header Pragma "no-cache" + +http-after-response add-header [ { if | unless } ] + + This appends an HTTP header field whose name is specified in and whose + value is defined by which follows the log-format rules (see Custom Log + Format in section 8.2.4). This may be used to send a cookie to a client for + example, or to pass some internal information. + This rule is not final, so it is possible to add other similar rules. + Note that header addition is performed immediately, so one rule might reuse + the resulting header from a previous rule. + +http-after-response allow [ { if | unless } ] + + This stops the evaluation of the rules and lets the response pass the check. + No further "http-after-response" rules are evaluated. + +http-after-response del-header [ { if | unless } ] + + This removes all HTTP header fields whose name is specified in . + +http-after-response replace-header + [ { if | unless } ] + + This works like "http-response replace-header". + + Example: + http-after-response replace-header Set-Cookie (C=[^;]*);(.*) \1;ip=%bi;\2 + + # applied to: + Set-Cookie: C=1; expires=Tue, 14-Jun-2016 01:40:45 GMT + + # outputs: + Set-Cookie: C=1;ip=192.168.1.20; expires=Tue, 14-Jun-2016 01:40:45 GMT + + # assuming the backend IP is 192.168.1.20. + +http-after-response replace-value + [ { if | unless } ] + + This works like "http-response replace-value". + + Example: + http-after-response replace-value Cache-control ^public$ private + + # applied to: + Cache-Control: max-age=3600, public + + # outputs: + Cache-Control: max-age=3600, private + +http-after-response set-header [ { if | unless } ] + + This does the same as "add-header" except that the header name is first + removed if it existed. This is useful when passing security information to + the server, where the header must not be manipulated by external users. + +http-after-response set-status [reason ] + [ { if | unless } ] + + This replaces the response status code with which must be an integer + between 100 and 999. Optionally, a custom reason text can be provided defined + by , or the default reason for the specified code will be used as a + fallback. + + Example: + # return "431 Request Header Fields Too Large" + http-response set-status 431 + # return "503 Slow Down", custom reason + http-response set-status 503 reason "Slow Down" + +http-after-response set-var() [ { if | unless } ] + + This is used to set the contents of a variable. The variable is declared + inline. + + Arguments: + The name of the variable starts with an indication about its + scope. The scopes allowed are: + "proc" : the variable is shared with the whole process + "sess" : the variable is shared with the whole session + "txn" : the variable is shared with the transaction + (request and response) + "req" : the variable is shared only during request + processing + "res" : the variable is shared only during response + processing + This prefix is followed by a name. The separator is a '.'. + The name may only contain characters 'a-z', 'A-Z', '0-9', '.' + and '_'. + + Is a standard HAProxy expression formed by a sample-fetch + followed by some converters. + + Example: + http-after-response set-var(sess.last_redir) res.hdr(location) + +http-after-response strict-mode { on | off } + + This enables or disables the strict rewriting mode for following rules. It + does not affect rules declared before it and it is only applicable on rules + performing a rewrite on the responses. When the strict mode is enabled, any + rewrite failure triggers an internal error. Otherwise, such errors are + silently ignored. The purpose of the strict rewriting mode is to make some + rewrites optionnal while others must be performed to continue the response + processing. + + By default, the strict rewriting mode is enabled. Its value is also reset + when a ruleset evaluation ends. So, for instance, if you change the mode on + the bacnkend, the default mode is restored when HAProxy starts the frontend + rules evaluation. + +http-after-response unset-var() [ { if | unless } ] + + This is used to unset a variable. See "http-after-response set-var" for + details about . + + Example: + http-after-response unset-var(sess.last_redir) + + http-check disable-on-404 Enable a maintenance mode upon HTTP/404 response to health-checks May be used in sections : defaults | frontend | listen | backend diff --git a/include/proto/http_ana.h b/include/proto/http_ana.h index 2b35388b14..62fd74d7e0 100644 --- a/include/proto/http_ana.h +++ b/include/proto/http_ana.h @@ -40,6 +40,7 @@ int http_process_res_common(struct stream *s, struct channel *rep, int an_bit, s int http_request_forward_body(struct stream *s, struct channel *req, int an_bit); int http_response_forward_body(struct stream *s, struct channel *res, int an_bit); int http_apply_redirect_rule(struct redirect_rule *rule, struct stream *s, struct http_txn *txn); +int http_eval_after_res_rules(struct stream *s); int http_replace_hdrs(struct stream* s, struct htx *htx, struct ist name, const char *str, struct my_regex *re, int full); int http_req_replace_stline(int action, const char *replace, int len, struct proxy *px, struct stream *s); diff --git a/include/proto/http_rules.h b/include/proto/http_rules.h index 608ca57607..3e57b9d4aa 100644 --- a/include/proto/http_rules.h +++ b/include/proto/http_rules.h @@ -29,9 +29,11 @@ extern struct action_kw_list http_req_keywords; extern struct action_kw_list http_res_keywords; +extern struct action_kw_list http_after_res_keywords; struct act_rule *parse_http_req_cond(const char **args, const char *file, int linenum, struct proxy *proxy); struct act_rule *parse_http_res_cond(const char **args, const char *file, int linenum, struct proxy *proxy); +struct act_rule *parse_http_after_res_cond(const char **args, const char *file, int linenum, struct proxy *proxy); struct redirect_rule *http_parse_redirect_rule(const char *file, int linenum, struct proxy *curproxy, const char **args, char **errmsg, int use_fmt, int dir); @@ -45,6 +47,11 @@ static inline void http_res_keywords_register(struct action_kw_list *kw_list) LIST_ADDQ(&http_res_keywords.list, &kw_list->list); } +static inline void http_after_res_keywords_register(struct action_kw_list *kw_list) +{ + LIST_ADDQ(&http_after_res_keywords.list, &kw_list->list); +} + #endif /* _PROTO_HTTP_RULES_H */ /* diff --git a/include/types/proxy.h b/include/types/proxy.h index c6b56aa5bc..f3b0e6bef6 100644 --- a/include/types/proxy.h +++ b/include/types/proxy.h @@ -315,6 +315,7 @@ struct proxy { struct list acl; /* ACL declared on this proxy */ struct list http_req_rules; /* HTTP request rules: allow/deny/... */ struct list http_res_rules; /* HTTP response rules: allow/deny/... */ + struct list http_after_res_rules; /* HTTP final response rules: set-header/del-header/... */ struct list redirect_rules; /* content redirecting rules (chained) */ struct list switching_rules; /* content switching rules (chained) */ struct list persist_rules; /* 'force-persist' and 'ignore-persist' rules (chained) */ diff --git a/src/cfgparse-listen.c b/src/cfgparse-listen.c index 3f16a2517f..70627c30a2 100644 --- a/src/cfgparse-listen.c +++ b/src/cfgparse-listen.c @@ -1413,6 +1413,36 @@ int cfg_parse_listen(const char *file, int linenum, char **args, int kwm) LIST_ADDQ(&curproxy->http_res_rules, &rule->list); } + else if (!strcmp(args[0], "http-after-response")) { + struct act_rule *rule; + + if (curproxy == &defproxy) { + ha_alert("parsing [%s:%d]: '%s' not allowed in 'defaults' section.\n", file, linenum, args[0]); + err_code |= ERR_ALERT | ERR_FATAL; + goto out; + } + + if (!LIST_ISEMPTY(&curproxy->http_after_res_rules) && + !LIST_PREV(&curproxy->http_after_res_rules, struct act_rule *, list)->cond && + (LIST_PREV(&curproxy->http_after_res_rules, struct act_rule *, list)->flags & ACT_FLAG_FINAL)) { + ha_warning("parsing [%s:%d]: previous '%s' action is final and has no condition attached, further entries are NOOP.\n", + file, linenum, args[0]); + err_code |= ERR_WARN; + } + + rule = parse_http_after_res_cond((const char **)args + 1, file, linenum, curproxy); + + if (!rule) { + err_code |= ERR_ALERT | ERR_ABORT; + goto out; + } + + err_code |= warnif_cond_conflicts(rule->cond, + (curproxy->cap & PR_CAP_BE) ? SMP_VAL_BE_HRS_HDR : SMP_VAL_FE_HRS_HDR, + file, linenum); + + LIST_ADDQ(&curproxy->http_after_res_rules, &rule->list); + } else if (!strcmp(args[0], "http-send-name-header")) { /* send server name in request header */ /* set the header name and length into the proxy structure */ if (warnifnotcap(curproxy, PR_CAP_BE, file, linenum, args[0], NULL)) diff --git a/src/cfgparse.c b/src/cfgparse.c index cd90d7e804..8ad64bcf9b 100644 --- a/src/cfgparse.c +++ b/src/cfgparse.c @@ -2841,6 +2841,16 @@ int check_config_validity() } } + /* check validity for 'http-after-response' layer 7 rules */ + list_for_each_entry(arule, &curproxy->http_after_res_rules, list) { + err = NULL; + if (arule->check_ptr && !arule->check_ptr(arule, curproxy, &err)) { + ha_alert("Proxy '%s': %s.\n", curproxy->id, err); + free(err); + cfgerr++; + } + } + if (curproxy->table && curproxy->table->peers.name) { struct peers *curpeers; diff --git a/src/haproxy.c b/src/haproxy.c index 5bca002125..f04ccea6ef 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -2440,6 +2440,7 @@ void deinit(void) deinit_act_rules(&p->tcp_req.l5_rules); deinit_act_rules(&p->http_req_rules); deinit_act_rules(&p->http_res_rules); + deinit_act_rules(&p->http_after_res_rules); deinit_stick_rules(&p->storersp_rules); deinit_stick_rules(&p->sticking_rules); diff --git a/src/http_act.c b/src/http_act.c index 3826ddf4e7..a74aeea0f8 100644 --- a/src/http_act.c +++ b/src/http_act.c @@ -1875,6 +1875,22 @@ static struct action_kw_list http_res_actions = { INITCALL1(STG_REGISTER, http_res_keywords_register, &http_res_actions); +static struct action_kw_list http_after_res_actions = { + .kw = { + { "add-header", parse_http_set_header, 0 }, + { "allow", parse_http_allow, 0 }, + { "del-header", parse_http_del_header, 0 }, + { "replace-header", parse_http_replace_header, 0 }, + { "replace-value", parse_http_replace_header, 0 }, + { "set-header", parse_http_set_header, 0 }, + { "set-status", parse_http_set_status, 0 }, + { "strict-mode", parse_http_strict_mode, 0 }, + { NULL, NULL } + } +}; + +INITCALL1(STG_REGISTER, http_after_res_keywords_register, &http_after_res_actions); + /* * Local variables: * c-indent-level: 8 diff --git a/src/http_ana.c b/src/http_ana.c index 5e0e24841a..2548a9ad56 100644 --- a/src/http_ana.c +++ b/src/http_ana.c @@ -1986,15 +1986,6 @@ int http_process_res_common(struct stream *s, struct channel *rep, int an_bit, s cur_proxy = sess->fe; } - /* After this point, this anayzer can't return yield, so we can - * remove the bit corresponding to this analyzer from the list. - * - * Note that the intermediate returns and goto found previously - * reset the analyzers. - */ - rep->analysers &= ~an_bit; - rep->analyse_exp = TICK_ETERNITY; - /* OK that's all we can do for 1xx responses */ if (unlikely(txn->status < 200 && txn->status != 101)) goto end; @@ -2116,6 +2107,14 @@ int http_process_res_common(struct stream *s, struct channel *rep, int an_bit, s } end: + /* + * Evaluate after-response rules before forwarding the response. rules + * from the backend are evaluated first, then one from the frontend if + * it differs. + */ + if (!http_eval_after_res_rules(s)) + goto return_int_err; + /* Always enter in the body analyzer */ rep->analysers &= ~AN_RES_FLT_XFER_DATA; rep->analysers |= AN_RES_HTTP_XFER_BODY; @@ -2130,10 +2129,9 @@ int http_process_res_common(struct stream *s, struct channel *rep, int an_bit, s s->do_log(s); s->logs.bytes_out = 0; } - DBG_TRACE_LEAVE(STRM_EV_STRM_ANA|STRM_EV_HTTP_ANA, s, txn); - return 1; done: + DBG_TRACE_LEAVE(STRM_EV_STRM_ANA|STRM_EV_HTTP_ANA, s, txn); rep->analysers &= ~an_bit; rep->analyse_exp = TICK_ETERNITY; return 1; @@ -3120,6 +3118,31 @@ resume_execution: return rule_ret; } +/* Executes backend and frontend http-after-response rules for the stream , + * in that order. it return 1 on success and 0 on error. It is the caller + * responsibility to catch error or ignore it. If it catches it, this function + * may be called a second time, for the internal error. + */ +int http_eval_after_res_rules(struct stream *s) +{ + struct session *sess = s->sess; + enum rule_result ret = HTTP_RULE_RES_CONT; + + /* prune the request variables if not already done and swap to the response variables. */ + if (s->vars_reqres.scope != SCOPE_RES) { + if (!LIST_ISEMPTY(&s->vars_reqres.head)) + vars_prune(&s->vars_reqres, s->sess, s); + vars_init(&s->vars_reqres, SCOPE_RES); + } + + ret = http_res_get_intercept_rule(s->be, &s->be->http_after_res_rules, s); + if ((ret == HTTP_RULE_RES_CONT || ret == HTTP_RULE_RES_STOP) && sess->fe != s->be) + ret = http_res_get_intercept_rule(sess->fe, &sess->fe->http_after_res_rules, s); + + /* All other codes than CONTINUE, STOP or DONE are forbidden */ + return (ret == HTTP_RULE_RES_CONT || ret == HTTP_RULE_RES_STOP || ret == HTTP_RULE_RES_DONE); +} + /* * Manage client-side cookie. It can impact performance by about 2% so it is * desirable to call it only when needed. This code is quite complex because @@ -4534,6 +4557,8 @@ int http_forward_proxy_resp(struct stream *s, int final) if (final) { htx->flags |= HTX_FL_PROXY_RESP; + if (!http_eval_after_res_rules(s)) + return 0; channel_auto_read(req); channel_abort(req); diff --git a/src/http_htx.c b/src/http_htx.c index accfa19e2b..587b9a548a 100644 --- a/src/http_htx.c +++ b/src/http_htx.c @@ -1308,7 +1308,7 @@ static int post_check_errors() if (htx_free_data_space(htx) < global.tune.maxrewrite) { ha_warning("config: errorfile '%s' runs over the buffer space" " reserved to headers rewritting. It may lead to internal errors if " - " http-final-response rules are evaluated on this message.\n", + " http-after-response rules are evaluated on this message.\n", (char *)node->key); err_code |= ERR_WARN; } diff --git a/src/http_rules.c b/src/http_rules.c index f1a187b162..2e58ec2e0b 100644 --- a/src/http_rules.c +++ b/src/http_rules.c @@ -47,6 +47,11 @@ struct action_kw_list http_res_keywords = { .list = LIST_HEAD_INIT(http_res_keywords.list) }; +/* List head of all known action keywords for "http-after-response" */ +struct action_kw_list http_after_res_keywords = { + .list = LIST_HEAD_INIT(http_after_res_keywords.list) +}; + /* * Return the struct http_req_action_kw associated to a keyword. */ @@ -63,6 +68,14 @@ static struct action_kw *action_http_res_custom(const char *kw) return action_lookup(&http_res_keywords.list, kw); } +/* + * Return the struct http_after_res_action_kw associated to a keyword. + */ +static struct action_kw *action_http_after_res_custom(const char *kw) +{ + return action_lookup(&http_after_res_keywords.list, kw); +} + /* parse an "http-request" rule */ struct act_rule *parse_http_req_cond(const char **args, const char *file, int linenum, struct proxy *proxy) { @@ -191,6 +204,71 @@ struct act_rule *parse_http_res_cond(const char **args, const char *file, int li return NULL; } + +/* parse an "http-after-response" rule */ +struct act_rule *parse_http_after_res_cond(const char **args, const char *file, int linenum, struct proxy *proxy) +{ + struct act_rule *rule; + struct action_kw *custom = NULL; + int cur_arg; + + rule = calloc(1, sizeof(*rule)); + if (!rule) { + ha_alert("parsing [%s:%d]: out of memory.\n", file, linenum); + goto out_err; + } + rule->from = ACT_F_HTTP_RES; + + if (((custom = action_http_after_res_custom(args[0])) != NULL)) { + char *errmsg = NULL; + + cur_arg = 1; + /* try in the module list */ + rule->kw = custom; + if (custom->parse(args, &cur_arg, proxy, rule, &errmsg) == ACT_RET_PRS_ERR) { + ha_alert("parsing [%s:%d] : error detected in %s '%s' while parsing 'http-after-response %s' rule : %s.\n", + file, linenum, proxy_type_str(proxy), proxy->id, args[0], errmsg); + free(errmsg); + goto out_err; + } + else if (errmsg) { + ha_warning("parsing [%s:%d] : %s.\n", file, linenum, errmsg); + free(errmsg); + } + } + else { + action_build_list(&http_after_res_keywords.list, &trash); + ha_alert("parsing [%s:%d]: 'http-after-response' expects %s%s, but got '%s'%s.\n", + file, linenum, *trash.area ? ", " : "", trash.area, + args[0], *args[0] ? "" : " (missing argument)"); + goto out_err; + } + + if (strcmp(args[cur_arg], "if") == 0 || strcmp(args[cur_arg], "unless") == 0) { + struct acl_cond *cond; + char *errmsg = NULL; + + if ((cond = build_acl_cond(file, linenum, &proxy->acl, proxy, args+cur_arg, &errmsg)) == NULL) { + ha_alert("parsing [%s:%d] : error detected while parsing an 'http-after-response %s' condition : %s.\n", + file, linenum, args[0], errmsg); + free(errmsg); + goto out_err; + } + rule->cond = cond; + } + else if (*args[cur_arg]) { + ha_alert("parsing [%s:%d]: 'http-after-response %s' expects" + " either 'if' or 'unless' followed by a condition but found '%s'.\n", + file, linenum, args[0], args[cur_arg]); + goto out_err; + } + + return rule; + out_err: + free(rule); + return NULL; +} + /* Parses a redirect rule. Returns the redirect rule on success or NULL on error, * with filled with the error message. If is not null, builds a * dynamic log-format rule instead of a static string. Parameter indicates diff --git a/src/proxy.c b/src/proxy.c index 2d6fe48fd1..8e325648f7 100644 --- a/src/proxy.c +++ b/src/proxy.c @@ -855,6 +855,7 @@ void init_new_proxy(struct proxy *p) LIST_INIT(&p->acl); LIST_INIT(&p->http_req_rules); LIST_INIT(&p->http_res_rules); + LIST_INIT(&p->http_after_res_rules); LIST_INIT(&p->redirect_rules); LIST_INIT(&p->mon_fail_cond); LIST_INIT(&p->switching_rules); diff --git a/src/vars.c b/src/vars.c index b2a2b7ed76..ff6baf5dcb 100644 --- a/src/vars.c +++ b/src/vars.c @@ -883,6 +883,14 @@ static struct action_kw_list http_res_kws = { { }, { INITCALL1(STG_REGISTER, http_res_keywords_register, &http_res_kws); +static struct action_kw_list http_after_res_kws = { { }, { + { "set-var", parse_store, 1 }, + { "unset-var", parse_store, 1 }, + { /* END */ } +}}; + +INITCALL1(STG_REGISTER, http_after_res_keywords_register, &http_after_res_kws); + static struct cfg_kw_list cfg_kws = {{ },{ { CFG_GLOBAL, "tune.vars.global-max-size", vars_max_size_global }, { CFG_GLOBAL, "tune.vars.proc-max-size", vars_max_size_proc },