MINOR: proxy: add option idle-close-on-response

Avoid closing idle connections if a soft stop is in progress.

By default, idle connections will be closed during a soft stop. In some
environments, a client talking to the proxy may have prepared some idle
connections in order to send requests later. If there is no proper retry
on write errors, this can result in errors while haproxy is reloading.
Even though a proper implementation should retry on connection/write
errors, this option was introduced to support back compat with haproxy <
v2.4. Indeed before v2.4, we were waiting for a last request to be able
to add a "connection: close" header and advice the client to close the
connection.

In a real life example, this behavior was seen in AWS using the ALB in
front of a haproxy. The end result was ALB sending 502 during haproxy
reloads.
This patch was tested on haproxy v2.4, with a regular reload on the
process, and a constant trend of requests coming in. Before the patch,
we see regular 502 returned to the client; when activating the option,
the 502 disappear.

This patch should help fixing github issue #1506.
In order to unblock some v2.3 to v2.4 migraton, this patch should be
backported up to v2.4 branch.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
[wt: minor edits to the doc to mention other options to care about]
Signed-off-by: Willy Tarreau <w@1wt.eu>

diff --git a/doc/configuration.txt b/doc/configuration.txt
index cd8ab4b72e1e8f68bc6871fa19f7c22364ff1dff..2e43f387582c680d48b7948b05d9dbeb248e69a0 100644 (file)
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -3756,6 +3756,7 @@ option tcp-smart-connect             (*)  X          -         X         X
 option tcpka                              X          X         X         X
 option tcplog                             X          X         X         X
 option transparent                   (*)  X          -         X         X
+option idle-close-on-response        (*)  X          X         X         -
 external-check command                    X          -         X         X
 external-check path                       X          -         X         X
 persist rdp-cookie                        X          -         X         X
@@ -9015,6 +9016,35 @@ option external-check
   See also : "external-check", "external-check command", "external-check path"
 
 
+option idle-close-on-response
+no option idle-close-on-response
+  Avoid closing idle frontend connections if a soft stop is in progress
+  May be used in sections :   defaults | frontend | listen | backend
+                                 yes   |    yes   |   yes  |   no
+  Arguments : none
+
+  By default, idle connections will be closed during a soft stop. In some
+  environments, a client talking to the proxy may have prepared some idle
+  connections in order to send requests later. If there is no proper retry on
+  write errors, this can result in errors while haproxy is reloading. Even
+  though a proper implementation should retry on connection/write errors, this
+  option was introduced to support backwards compatibility with haproxy prior
+  to version 2.4. Indeed before v2.4, haproxy used to wait for a last request
+  and response to add a "connection: close" header before closing, thus
+  notifying the client that the connection would not be reusable.
+
+  In a real life example, this behavior was seen in AWS using the ALB in front
+  of a haproxy. The end result was ALB sending 502 during haproxy reloads.
+
+  Users are warned that using this option may increase the number of old
+  processes if connections remain idle for too long. Adjusting the client
+  timeouts and/or the "hard-stop-after" parameter accordingly might be
+  needed in case of frequent reloads.
+
+  See also: "timeout client", "timeout client-fin", "timeout http-request",
+            "hard-stop-after"
+
+
 option log-health-checks
 no option log-health-checks
   Enable or disable logging of health checks status updates

diff --git a/include/haproxy/proxy-t.h b/include/haproxy/proxy-t.h
index 8ca4e818d62f2bf30602a21e679469fc1dd39170..421f900e29c9c2b5954affe246a73950389ab29a 100644 (file)
--- a/include/haproxy/proxy-t.h
+++ b/include/haproxy/proxy-t.h
@@ -82,7 +82,7 @@ enum PR_SRV_STATE_FILE {
 #define PR_O_REUSE_ALWS 0x0000000C      /* always reuse a shared connection */
 #define PR_O_REUSE_MASK 0x0000000C      /* mask to retrieve shared connection preferences */
 
-/* unused: 0x10 */
+#define PR_O_IDLE_CLOSE_RESP 0x00000010 /* avoid closing idle connections during a soft stop */
 #define PR_O_PREF_LAST  0x00000020      /* prefer last server */
 #define PR_O_DISPATCH   0x00000040      /* use dispatch mode */
 #define PR_O_FORCED_ID  0x00000080      /* proxy's ID was forced in the configuration */

diff --git a/src/mux_h1.c b/src/mux_h1.c
index 7b6ab946d3d58bc5d73e9286ed99c58c98096fa3..1ec6cb77c9e835421f0e86afab8bcad75207f7aa 100644 (file)
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -2999,7 +2999,8 @@ static int h1_process(struct h1c * h1c)
         */
        if (!(h1c->flags & H1C_F_IS_BACK)) {
                if (unlikely(h1c->px->flags & (PR_FL_DISABLED|PR_FL_STOPPED))) {
-                       if (h1c->flags & H1C_F_WAIT_NEXT_REQ)
+                       if (!(h1c->px->options & PR_O_IDLE_CLOSE_RESP) &&
+                               h1c->flags & H1C_F_WAIT_NEXT_REQ)
                                goto release;
                }
        }

diff --git a/src/proxy.c b/src/proxy.c
index ff5e35e2cc73ba5d274460ceec53e0b6de63b833..e583e5104bac9a2d38602d474ad4cb4d64402bfb 100644 (file)
--- a/src/proxy.c
+++ b/src/proxy.c
@@ -70,6 +70,7 @@ const struct cfg_opt cfg_opts[] =
        { "dontlognull",  PR_O_NULLNOLOG,  PR_CAP_FE, 0, 0 },
        { "http-buffer-request", PR_O_WREQ_BODY,  PR_CAP_FE | PR_CAP_BE, 0, PR_MODE_HTTP },
        { "http-ignore-probes", PR_O_IGNORE_PRB, PR_CAP_FE, 0, PR_MODE_HTTP },
+       { "idle-close-on-response", PR_O_IDLE_CLOSE_RESP, PR_CAP_FE, 0, PR_MODE_HTTP },
        { "prefer-last-server", PR_O_PREF_LAST,  PR_CAP_BE, 0, PR_MODE_HTTP },
        { "logasap",      PR_O_LOGASAP,    PR_CAP_FE, 0, 0 },
        { "nolinger",     PR_O_TCP_NOLING, PR_CAP_FE | PR_CAP_BE, 0, 0 },