MAJOR: http: update connection mode configuration

At the very beginning of haproxy, there was "option httpclose" to make
haproxy add a "Connection: close" header in both directions to invite
both sides to agree on closing the connection. It did not work with some
rare products, so "option forceclose" was added to do the same and actively
close the connection. Then client-side keep-alive was supported, so option
http-server-close was introduced. Now we have keep-alive with a fourth
option, not to mention the implicit tunnel mode.

The connection configuration has become a total mess because all the
options above may be combined together, despite almost everyone thinking
they cancel each other, as judging from the common problem reports on the
mailing list. Unfortunately, re-reading the doc shows that it's not clear
at all that options may be combined, and the opposite seems more obvious
since they're compared. The most common issue is options being set in the
defaults section that are not negated in other sections, but are just
combined when the user expects them to be overloaded. The migration to
keep-alive by default will only make things worse.

So let's start to address the first problem. A transaction can only work in
5 modes today :
  - tunnel : haproxy doesn't bother with what follows the first req/resp
  - passive close : option http-close
  - forced close : option forceclose
  - server close : option http-server-close with keep-alive on the client side
  - keep-alive   : option http-keep-alive, end to end

All 16 combination for each section fall into one of these cases. Same for
the 256 combinations resulting from frontend+backend different modes.

With this patch, we're doing something slightly different, which will not
change anything for users with valid configs, and will only change the
behaviour for users with unsafe configs. The principle is that these options
may not combined anymore, and that the latest one always overrides all the
other ones, including those inherited from the defaults section. The "no
option xxx" statement is still supported to cancel one option and fall back
to the default one. It is mainly needed to ignore defaults sections (eg:
force the tunnel mode). The frontend+backend combinations have not changed.

So for examplen the following configuration used to put the connection
into forceclose :

    defaults http
        mode http
        option httpclose

    frontend foo.
        option http-server-close

  => http-server-close+httpclose = forceclose before this patch! Now
     the frontend's config replaces the defaults config and results in
     the more expected http-server-close.

All 25 combinations of the 5 modes in (frontend,backend) have been
successfully tested.

In order to prepare for upcoming changes, a new "option http-tunnel" was
added. It currently only voids all other options, and has the lowest
precedence when mixed with another option in another frontend/backend.

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 141f12e975553ec010cf018ad10ce53be1f378dc..ae3023e52b8c3c3de3b68cb980f3c49198e8af40 100644 (file)
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -1202,6 +1202,7 @@ option http-keep-alive               (*)  X          X         X         X
 option http-no-delay                 (*)  X          X         X         X
 option http-pretend-keepalive        (*)  X          X         X         X
 option http-server-close             (*)  X          X         X         X
+option http-tunnel                   (*)  X          X         X         X
 option http-use-proxy-header         (*)  X          X         X         -
 option httpchk                            X          -         X         X
 option httpclose                     (*)  X          X         X         X
@@ -3710,15 +3711,16 @@ no option forceclose
 
   When this happens, it is possible to use "option forceclose". It will
   actively close the outgoing server channel as soon as the server has finished
-  to respond. This option implicitly enables the "httpclose" option. Note that
-  this option also enables the parsing of the full request and response, which
-  means we can close the connection to the server very quickly, releasing some
-  resources earlier than with httpclose.
+  to respond and release some resources earlier than with 'option httpclose'.
 
   This option may also be combined with "option http-pretend-keepalive", which
   will disable sending of the "Connection: close" header, but will still cause
   the connection to be closed once the whole response is received.
 
+  This option disables and replaces any previous 'option httpclose', 'option
+  http-server-close' or 'option http-keep-alive'. When frontend and backend
+  options differ, 'option forceclose' has precedence over all other options.
+
   If this option has been enabled in a "defaults" section, it can be disabled
   in a specific instance by prepending the "no" keyword before it.
 
@@ -3838,10 +3840,9 @@ no option http-keep-alive
   timeout defined by "timeout http-keep-alive" or "timeout http-request" if
   not set.
 
-  This option may be set both in a frontend and in a backend. It is enabled if
-  at least one of the frontend or backend holding a connection has it enabled.
-  It is worth noting that "option forceclose" and "option http-server-close"
-  have precedence over "option http-keep-alive".
+  This option disables and replaces any previous 'option httpclose', 'option
+  http-server-close' or 'option forceclose'. When frontend and backend options
+  differ, all of these 3 options have precedence over 'option http-keep-alive'.
 
   If this option has been enabled in a "defaults" section, it can be disabled
   in a specific instance by prepending the "no" keyword before it.
@@ -3956,9 +3957,11 @@ no option http-server-close
 
   This option may be set both in a frontend and in a backend. It is enabled if
   at least one of the frontend or backend holding a connection has it enabled.
-  It is worth noting that "option forceclose" has precedence over "option
-  http-server-close" and that combining "http-server-close" with "httpclose"
-  basically achieve the same result as "forceclose".
+  It disables and replaces any previous 'option httpclose', 'option forceclose'
+  or 'option http-keep-alive'. When frontend and backend options differ, both
+  'option forceclose' and 'option httpclose' have precedence over
+  'option http-server-close' and both result in the same setup as if only
+  'option forceclose' was set.
 
   If this option has been enabled in a "defaults" section, it can be disabled
   in a specific instance by prepending the "no" keyword before it.
@@ -3968,6 +3971,31 @@ no option http-server-close
              "1.1. The HTTP transaction model".
 
 
+option http-tunnel
+no option http-tunnel
+  Disable or enable HTTP connection processing after first transaction
+  May be used in sections :   defaults | frontend | listen | backend
+                                 yes   |    yes   |   yes  |   yes
+  Arguments : none
+
+  By default, when a client communicates with a server, HAProxy will only
+  analyze, log, and process the first request of each connection. Option
+  "http-tunnel" just does this and cancels any other option among
+  "option forceclose", "option httpclose", "option http-keep-alive",
+  and "option http-server-close". It is the mode with the lowest processing
+  overhead, which is normally not needed anymore unless in very specific
+  cases such as when using an in-house protocol that looks like HTTP but is
+  not compatible, or just to log one request per client in order to reduce
+  log size.
+
+  If this option has been enabled in a "defaults" section, it can be disabled
+  in a specific instance by prepending the "no" keyword before it.
+
+  See also : "option forceclose", "option http-server-close",
+             "option httpclose", "option http-keep-alive", and
+             "1.1. The HTTP transaction model".
+
+
 option http-use-proxy-header
 no option http-use-proxy-header
   Make use of non-standard Proxy-Connection header instead of Connection
@@ -4076,9 +4104,10 @@ no option httpclose
 
   This option may be set both in a frontend and in a backend. It is enabled if
   at least one of the frontend or backend holding a connection has it enabled.
-  If "option forceclose" is specified too, it has precedence over "httpclose".
-  If "option http-server-close" is enabled at the same time as "httpclose", it
-  basically achieves the same result as "option forceclose".
+  It disables and replaces any previous 'option http-server-close',
+  'option forceclose' or 'option http-keep-alive'. When frontend and backend
+  options differ, 'option httpclose' has precedence over all other options and
+  results in the same setup as if only 'option forceclose' was set.
 
   If this option has been enabled in a "defaults" section, it can be disabled
   in a specific instance by prepending the "no" keyword before it.

diff --git a/include/types/proxy.h b/include/types/proxy.h
index 6c6cc0b9d6dd96f61ed64ff879986983c82e7e62..68f1b72c77fd72e2c018ee9d0827e46a6ee9dd96 100644 (file)
--- a/include/types/proxy.h
+++ b/include/types/proxy.h
@@ -79,7 +79,7 @@ enum pr_mode {
 /* unused: 0x04, 0x08, 0x10 */
 #define PR_O_PREF_LAST  0x00000020      /* prefer last server */
 #define PR_O_DISPATCH   0x00000040      /* use dispatch mode */
-#define PR_O_KEEPALIVE  0x00000080      /* follow keep-alive sessions */
+/* unused: 0x00000080 */
 #define PR_O_FWDFOR     0x00000100      /* conditionally insert x-forwarded-for with client address */
 /* unused: 0x00000200 */
 #define PR_O_NULLNOLOG  0x00000400      /* a connect without request will not be logged */
@@ -87,17 +87,25 @@ enum pr_mode {
 #define PR_O_FF_ALWAYS  0x00002000      /* always set x-forwarded-for */
 #define PR_O_PERSIST    0x00004000      /* server persistence stays effective even when server is down */
 #define PR_O_LOGASAP    0x00008000      /* log as soon as possible, without waiting for the session to complete */
-#define PR_O_HTTP_CLOSE 0x00010000      /* force 'connection: close' in both directions */
+/* unused: 0x00010000 */
 #define PR_O_CHK_CACHE  0x00020000      /* require examination of cacheability of the 'set-cookie' field */
 #define PR_O_TCP_CLI_KA 0x00040000      /* enable TCP keep-alive on client-side sessions */
 #define PR_O_TCP_SRV_KA 0x00080000      /* enable TCP keep-alive on server-side sessions */
 #define PR_O_USE_ALL_BK 0x00100000      /* load-balance between backup servers */
-#define PR_O_FORCE_CLO  0x00200000      /* enforce the connection close immediately after server response */
+/* unused: 0x00020000 */
 #define PR_O_TCP_NOLING 0x00400000      /* disable lingering on client and server connections */
 #define PR_O_ABRT_CLOSE 0x00800000      /* immediately abort request when client closes */
 
-/* unused: 0x01000000, 0x02000000, 0x04000000 */
-#define PR_O_SERVER_CLO 0x08000000     /* option http-server-close */
+/* unused: 0x01000000, 0x02000000, 0x04000000, 0x08000000 */
+#define PR_O_HTTP_TUN   0x00000000      /* HTTP tunnel mode : no analysis past first request/response */
+#define PR_O_HTTP_PCL   0x01000000      /* HTTP passive close mode (httpclose) = tunnel with Connection: close */
+#define PR_O_HTTP_FCL   0x02000000      /* HTTP forced close mode (forceclose) */
+#define PR_O_HTTP_SCL   0x03000000      /* HTTP server close mode (http-server-close) */
+#define PR_O_HTTP_KAL   0x04000000      /* HTTP keep-alive mode (http-keep-alive) */
+/* unassigned values : 0x05000000, 0x06000000, 0x07000000 */
+#define PR_O_HTTP_MODE  0x07000000      /* MASK to retrieve the HTTP mode */
+
+/* unused: 0x08000000 */
 #define PR_O_CONTSTATS 0x10000000      /* continous counters */
 #define PR_O_HTTP_PROXY 0x20000000     /* Enable session to use HTTP proxy operations */
 #define PR_O_DISABLE404 0x40000000      /* Disable a server on a 404 response to a health-check */

diff --git a/src/cfgparse.c b/src/cfgparse.c
index c7a491ff8ae27ca393c48fee87b4b7eb414f40d3..4c52eed3bee9561ebd8a6221deda8ed73b5afbfc 100644 (file)
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -129,11 +129,7 @@ static const struct cfg_opt cfg_opts[] =
        { "clitcpka",     PR_O_TCP_CLI_KA, PR_CAP_FE, 0, 0 },
        { "contstats",    PR_O_CONTSTATS,  PR_CAP_FE, 0, 0 },
        { "dontlognull",  PR_O_NULLNOLOG,  PR_CAP_FE, 0, 0 },
-       { "forceclose",   PR_O_FORCE_CLO,  PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
        { "http_proxy",   PR_O_HTTP_PROXY, PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
-       { "httpclose",    PR_O_HTTP_CLOSE, PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
-       { "http-keep-alive",   PR_O_KEEPALIVE,   PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
-       { "http-server-close", PR_O_SERVER_CLO,  PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
        { "prefer-last-server", PR_O_PREF_LAST,  PR_CAP_BE, 0, PR_MODE_HTTP },
        { "logasap",      PR_O_LOGASAP,    PR_CAP_FE, 0, 0 },
        { "nolinger",     PR_O_TCP_NOLING, PR_CAP_FE | PR_CAP_BE, 0, 0 },
@@ -3505,6 +3501,72 @@ stats_error_parsing:
                        }
                }
 
+               /* HTTP options override each other. They can be cancelled using
+                * "no option xxx" which only switches to default mode if the mode
+                * was this one (useful for cancelling options set in defaults
+                * sections).
+                */
+               if (strcmp(args[1], "httpclose") == 0) {
+                       if (kwm == KWM_STD) {
+                               curproxy->options &= ~PR_O_HTTP_MODE;
+                               curproxy->options |= PR_O_HTTP_PCL;
+                               goto out;
+                       }
+                       else if (kwm == KWM_NO) {
+                               if ((curproxy->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL)
+                                       curproxy->options &= ~PR_O_HTTP_MODE;
+                               goto out;
+                       }
+               }
+               else if (strcmp(args[1], "forceclose") == 0) {
+                       if (kwm == KWM_STD) {
+                               curproxy->options &= ~PR_O_HTTP_MODE;
+                               curproxy->options |= PR_O_HTTP_FCL;
+                               goto out;
+                       }
+                       else if (kwm == KWM_NO) {
+                               if ((curproxy->options & PR_O_HTTP_MODE) == PR_O_HTTP_FCL)
+                                       curproxy->options &= ~PR_O_HTTP_MODE;
+                               goto out;
+                       }
+               }
+               else if (strcmp(args[1], "http-server-close") == 0) {
+                       if (kwm == KWM_STD) {
+                               curproxy->options &= ~PR_O_HTTP_MODE;
+                               curproxy->options |= PR_O_HTTP_SCL;
+                               goto out;
+                       }
+                       else if (kwm == KWM_NO) {
+                               if ((curproxy->options & PR_O_HTTP_MODE) == PR_O_HTTP_SCL)
+                                       curproxy->options &= ~PR_O_HTTP_MODE;
+                               goto out;
+                       }
+               }
+               else if (strcmp(args[1], "http-keep-alive") == 0) {
+                       if (kwm == KWM_STD) {
+                               curproxy->options &= ~PR_O_HTTP_MODE;
+                               curproxy->options |= PR_O_HTTP_KAL;
+                               goto out;
+                       }
+                       else if (kwm == KWM_NO) {
+                               if ((curproxy->options & PR_O_HTTP_MODE) == PR_O_HTTP_KAL)
+                                       curproxy->options &= ~PR_O_HTTP_MODE;
+                               goto out;
+                       }
+               }
+               else if (strcmp(args[1], "http-tunnel") == 0) {
+                       if (kwm == KWM_STD) {
+                               curproxy->options &= ~PR_O_HTTP_MODE;
+                               curproxy->options |= PR_O_HTTP_TUN;
+                               goto out;
+                       }
+                       else if (kwm == KWM_NO) {
+                               if ((curproxy->options & PR_O_HTTP_MODE) == PR_O_HTTP_TUN)
+                                       curproxy->options &= ~PR_O_HTTP_MODE;
+                               goto out;
+                       }
+               }
+
                if (kwm != KWM_STD) {
                        Alert("parsing [%s:%d]: negation/default is not supported for option '%s'.\n",
                                file, linenum, args[1]);

diff --git a/src/proto_http.c b/src/proto_http.c
index 7916c27d7de0a9543f42e222e6b556f2cb690d6f..164faac227c071ac00a220ab2881e54026b3d721 100644 (file)
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -3551,17 +3551,27 @@ int http_process_req_common(struct session *s, struct channel *req, int an_bit,
         */
 
        if ((!(txn->flags & TX_HDR_CONN_PRS) &&
-            (s->fe->options & (PR_O_KEEPALIVE|PR_O_SERVER_CLO|PR_O_HTTP_CLOSE|PR_O_FORCE_CLO))) ||
-           ((s->fe->options & (PR_O_KEEPALIVE|PR_O_SERVER_CLO|PR_O_HTTP_CLOSE|PR_O_FORCE_CLO)) !=
-            (s->be->options & (PR_O_KEEPALIVE|PR_O_SERVER_CLO|PR_O_HTTP_CLOSE|PR_O_FORCE_CLO)))) {
+            ((s->fe->options & PR_O_HTTP_MODE) != PR_O_HTTP_TUN)) ||
+           ((s->fe->options & PR_O_HTTP_MODE) != (s->be->options & PR_O_HTTP_MODE))) {
                int tmp = TX_CON_WANT_TUN;
 
-               if ((s->fe->options|s->be->options) & PR_O_KEEPALIVE ||
+               if ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_KAL ||
+                   (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_KAL ||
                    ((s->fe->options2|s->be->options2) & PR_O2_FAKE_KA))
                        tmp = TX_CON_WANT_KAL;
-               if ((s->fe->options|s->be->options) & PR_O_SERVER_CLO)
+
+               if ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_SCL ||
+                   (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_SCL)
                        tmp = TX_CON_WANT_SCL;
-               if ((s->fe->options|s->be->options) & PR_O_FORCE_CLO)
+
+               if ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_FCL ||
+                   (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_FCL)
+                       tmp = TX_CON_WANT_CLO;
+
+               /* option httpclose + anything other than tunnel => close */
+               if (tmp != TX_CON_WANT_TUN &&
+                   ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL ||
+                    (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL))
                        tmp = TX_CON_WANT_CLO;
 
                if ((txn->flags & TX_CON_WANT_MSK) < tmp)
@@ -3584,7 +3594,6 @@ int http_process_req_common(struct session *s, struct channel *req, int an_bit,
                     (txn->flags & TX_CON_WANT_MSK) == TX_CON_WANT_SCL) &&
                    ((txn->flags & TX_HDR_CONN_CLO) ||                         /* "connection: close" */
                     (!(msg->flags & HTTP_MSGF_VER_11) && !(txn->flags & TX_HDR_CONN_KAL)) || /* no "connection: k-a" in 1.0 */
-                    ((s->fe->options|s->be->options) & PR_O_HTTP_CLOSE) ||    /* httpclose+any = forceclose */
                     !(msg->flags & HTTP_MSGF_XFER_LEN) ||                     /* no length known => close */
                     s->fe->state == PR_STSTOPPED))                            /* frontend is stopping */
                    txn->flags = (txn->flags & ~TX_CON_WANT_MSK) | TX_CON_WANT_CLO;
@@ -3963,17 +3972,20 @@ int http_process_request(struct session *s, struct channel *req, int an_bit)
         */
        if (!(txn->flags & TX_HDR_CONN_UPG) &&
            (((txn->flags & TX_CON_WANT_MSK) != TX_CON_WANT_TUN) ||
-            ((s->fe->options|s->be->options) & PR_O_HTTP_CLOSE))) {
+            ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL ||
+             (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL))) {
                unsigned int want_flags = 0;
 
                if (msg->flags & HTTP_MSGF_VER_11) {
                        if (((txn->flags & TX_CON_WANT_MSK) >= TX_CON_WANT_SCL ||
-                           ((s->fe->options|s->be->options) & PR_O_HTTP_CLOSE)) &&
+                            ((s->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL ||
+                             (s->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL)) &&
                            !((s->fe->options2|s->be->options2) & PR_O2_FAKE_KA))
                                want_flags |= TX_CON_CLO_SET;
                } else {
                        if (((txn->flags & TX_CON_WANT_MSK) == TX_CON_WANT_KAL &&
-                            !((s->fe->options|s->be->options) & PR_O_HTTP_CLOSE)) ||
+                            ((s->fe->options & PR_O_HTTP_MODE) != PR_O_HTTP_PCL &&
+                             (s->be->options & PR_O_HTTP_MODE) != PR_O_HTTP_PCL)) ||
                            ((s->fe->options2|s->be->options2) & PR_O2_FAKE_KA))
                                want_flags |= TX_CON_KAL_SET;
                }
@@ -5608,7 +5620,8 @@ int http_process_res_common(struct session *t, struct channel *rep, int an_bit,
        }
        else if ((txn->status >= 200) && !(txn->flags & TX_HDR_CONN_PRS) &&
                 ((txn->flags & TX_CON_WANT_MSK) != TX_CON_WANT_TUN ||
-                 ((t->fe->options|t->be->options) & PR_O_HTTP_CLOSE))) {
+                 ((t->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL ||
+                  (t->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL))) {
                int to_del = 0;
 
                /* on unknown transfer length, we must close */
@@ -5874,7 +5887,8 @@ int http_process_res_common(struct session *t, struct channel *rep, int an_bit,
                 */
                if (!(txn->flags & TX_HDR_CONN_UPG) &&
                    (((txn->flags & TX_CON_WANT_MSK) != TX_CON_WANT_TUN) ||
-                    ((t->fe->options|t->be->options) & PR_O_HTTP_CLOSE))) {
+                    ((t->fe->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL ||
+                     (t->be->options & PR_O_HTTP_MODE) == PR_O_HTTP_PCL))) {
                        unsigned int want_flags = 0;
 
                        if ((txn->flags & TX_CON_WANT_MSK) == TX_CON_WANT_KAL ||