From: Amaury Denoyelle Date: Tue, 5 Aug 2025 12:02:44 +0000 (+0200) Subject: MINOR: quic: rename frontend sock-per-conn setting X-Git-Tag: v3.3-dev11~45 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=5bc659a4a27d0c29868ebcfea217e22cc6301246;p=thirdparty%2Fhaproxy.git MINOR: quic: rename frontend sock-per-conn setting On frontend side, a quic_conn can have a dedicated FD or use the listener one. These different modes can be activated via a global QUIC tune setting. This patch adjusts the option. First, it is renamed to the more meaningful name 'tune.quic.fe.sock-per-conn'. Also, arguments are now either 'default-on' or 'force-off'. The objective is to better highlight reliationship with 'quic-socket' bind option. The older option is deprecated and will be removed in 3.5. --- diff --git a/doc/configuration.txt b/doc/configuration.txt index 2ef28c8fd..5beab0579 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -1905,6 +1905,7 @@ The following keywords are supported in the "global" section : - tune.quic.fe.cc.reorder-ratio - tune.quic.fe.sec.glitches-threshold - tune.quic.fe.sec.retry-threshold + - tune.quic.fe.sock-per-conn - tune.quic.fe.tx.pacing - tune.quic.fe.tx.udp-gso - tune.quic.frontend.max-data-size @@ -1918,7 +1919,7 @@ The following keywords are supported in the "global" section : - tune.quic.mem.tx-max - tune.quic.reorder-ratio (deprecated) - tune.quic.retry-threshold (deprecated) - - tune.quic.socket-owner + - tune.quic.socket-owner (deprecated) - tune.quic.zero-copy-fwd-send - tune.renice.runtime - tune.renice.startup @@ -3101,24 +3102,24 @@ set-var-fmt setcap [,...] Sets a list of capabilities that must be preserved when starting and running - either as a non-root user (uid > 0), or when starting with uid 0 (root) - and switching then to a non-root. By default all permissions are - lost by the uid switch, but some are often needed when trying to connect to - a server from a foreign address during transparent proxying, or when binding - to a port below 1024, e.g. when using "tune.quic.socket-owner connection", - resulting in setups running entirely under uid 0. Setting capabilities - generally is a safer alternative, as only the required capabilities will be - preserved. The feature is OS-specific and only enabled on Linux when - USE_LINUX_CAP=1 is set at build time. The list of supported capabilities also - depends on the OS and is enumerated by the error message displayed when an - invalid capability name or an empty one is passed. Multiple capabilities may - be passed, delimited by commas. Among those commonly used, "cap_net_raw" - allows to transparently bind to a foreign address, and "cap_net_bind_service" - allows to bind to a privileged port and may be used by QUIC. If the process - is started and run under the same non-root user, needed capabilities should - be set on haproxy binary file with setcap along with this keyword. For more - details about setting capabilities on haproxy binary, please see chapter - 13.1 Linux capabilities support in the Management guide. + either as a non-root user (uid > 0), or when starting with uid 0 (root) and + switching then to a non-root. By default all permissions are lost by the uid + switch, but some are often needed when trying to connect to a server from a + foreign address during transparent proxying, or when binding to a port below + 1024, e.g. when using "tune.quic.fe.sock-per-conn default-on", resulting in + setups running entirely under uid 0. Setting capabilities generally is a + safer alternative, as only the required capabilities will be preserved. The + feature is OS-specific and only enabled on Linux when USE_LINUX_CAP=1 is set + at build time. The list of supported capabilities also depends on the OS and + is enumerated by the error message displayed when an invalid capability name + or an empty one is passed. Multiple capabilities may be passed, delimited by + commas. Among those commonly used, "cap_net_raw" allows to transparently bind + to a foreign address, and "cap_net_bind_service" allows to bind to a + privileged port and may be used by QUIC. If the process is started and run + under the same non-root user, needed capabilities should be set on haproxy + binary file with setcap along with this keyword. For more details about + setting capabilities on haproxy binary, please see chapter 13.1 Linux + capabilities support in the Management guide. Example: global @@ -4801,6 +4802,41 @@ tune.quic.retry-threshold (deprecated) part of the streamlining process apply on QUIC configuration. If used, this setting will only be applied on frontend connections. +tune.quic.fe.sock-per-conn { default-on | force-off } + Specifies globally how QUIC frontend connections will use socket for + receive/send operations. Connections can share listener socket or each + connection can allocate its own socket. + + The default value is "default-on". This is used to allocate a dedicated + socket for every QUIC connections. This option is the preferred one to + achieve the best performance with a large QUIC traffic. This is also the only + way to ensure soft-stop is conducted properly without data loss for QUIC + connections and cases of transient errors during sendto() operation are + handled efficiently. However, this relies on some advanced features from the + UDP network stack. If your platform is deemed not compatible, haproxy will + automatically switch to "force-off" mode on startup. Please note that QUIC + listeners running on privileged ports may require to run as uid 0, or some + OS-specific tuning to permit the target uid to bind such ports, such as + system capabilities. See also the "setcap" global directive. + + The "force-off" value indicates that QUIC transfers will occur on the shared + listener socket. This option can be a good compromise for small traffic as it + allows to reduce FD consumption. However, performance won't be optimal due to + a higher CPU usage if listeners are shared across a lot of threads or a large + number of QUIC connections can be used simultaneously. + + This setting is applied in conjunction with each "quic-socket" bind options. + If "default-on" mode is used on global tuning, it will be activated for each + listener, except for the ones with "quic-socket listener". However, if + "force-off" is used globally, it will be applied on every listener instance, + regardless of their individual configuration. + +tune.quic.socket-owner { connection | listener } (deprecated) + This keyword has been deprecated in 3.3 and will be removed in 3.5. It is + part of the streamlining process apply on QUIC configuration. The newer + option is named "tune.quic.fe.sock-per-conn", with legacy value "connection" + corresponding to "default-on" and "listener" to "force-off". + tune.quic.be.tx.pacing { on | off } tune.quic.fe.tx.pacing { on | off } Enables ('on') or disables ('off') pacing support for QUIC emission. By @@ -4829,6 +4865,7 @@ tune.quic.disable-udp-gso (deprecated) This keyword has been deprecated in 3.3 and will be removed in 3.5. It is part of the streamlining process apply on QUIC configuration. If used, this setting will only be applied on frontend connections. + tune.quic.frontend.max-data-size This setting is the hard limit for the number of data bytes in flight over a QUIC frontend connection. It is reused as the value for the initial_max_data @@ -4925,35 +4962,6 @@ tune.quic.frontend.max-tx-mem (deprecated) part of the streamlining process apply on QUIC configuration. If used, this setting will only be applied on frontend connections. -tune.quic.socket-owner { connection | listener } - Specifies globally how QUIC connections will use socket for receive/send - operations. Connections can share listener socket or each connection can - allocate its own socket. - - When default "connection" value is set, a dedicated socket will be allocated - by every QUIC connections. This option is the preferred one to achieve the - best performance with a large QUIC traffic. This is also the only way to - ensure soft-stop is conducted properly without data loss for QUIC connections - and cases of transient errors during sendto() operation are handled - efficiently. However, this relies on some advanced features from the UDP - network stack. If your platform is deemed not compatible, haproxy will - automatically switch to "listener" mode on startup. Please note that QUIC - listeners running on privileged ports may require to run as uid 0, or some - OS-specific tuning to permit the target uid to bind such ports, such as - system capabilities. See also the "setcap" global directive. - - The "listener" value indicates that QUIC transfers will occur on the shared - listener socket. This option can be a good compromise for small traffic as it - allows to reduce FD consumption. However, performance won't be optimal due to - a higher CPU usage if listeners are shared across a lot of threads or a - large number of QUIC connections can be used simultaneously. - - This setting is applied in conjunction with each "quic-socket" bind options. - If "connection" mode is used on global tuning, it will be activated for each - listener, unless its bind option is set to "listener". However, if "listener" - is used globally, it will be forced on every listener instance, regardless of - their individual configuration. - tune.quic.zero-copy-fwd-send { on | off } Enables ('on') of disabled ('off') the zero-copy sends of data for the QUIC multiplexer. It is enabled by default. @@ -6232,8 +6240,8 @@ bind / [, ...] [param*] was the FD of an accept(). Should be used carefully. - 'quic4@' -> address is resolved as IPv4 and protocol UDP is used. Note that to achieve the best performance with a - large traffic you should keep "tune.quic.socket-owner" on - connection. Else QUIC connections will be multiplexed + large traffic you should keep "tune.quic.fe.sock-per-conn + default-on". Else QUIC connections will be multiplexed over the listener socket. Another alternative would be to duplicate QUIC listener instances over several threads, for example using "shards" keyword to at least reduce @@ -17180,8 +17188,15 @@ quic-force-retry quic-socket [ connection | listener ] This QUIC specific setting allows to define the socket allocation mode for - the specific listeners. See "tune.quic.socket-owner" for a full description - of its usage. + the specific listeners. See "tune.quic.fe.sock-per-conn" for a full + description of the pros and cons of each mode. + + This setting is applied in conjunction with the global + "tune.quic.fe.sock-per-conn" option. If "default-on" mode is active on the + global tuning (this is the default value), each QUIC connection will use its + owned socket, except for listeners with "quic-socket listener". However, if + the global mode is set to "force-off", individual listener configuration will + be ignored. severity-output This setting is used with the stats sockets only to configure severity diff --git a/include/haproxy/quic_tune-t.h b/include/haproxy/quic_tune-t.h index 2db1e7e8c..0ccb8d50e 100644 --- a/include/haproxy/quic_tune-t.h +++ b/include/haproxy/quic_tune-t.h @@ -16,8 +16,7 @@ #define QUIC_TUNE_FE_LISTEN_OFF 0x00000001 - -#define QUIC_TUNE_SOCK_PER_CONN 0x00000004 +#define QUIC_TUNE_FE_SOCK_PER_CONN 0x00000002 #define QUIC_TUNE_FB_TX_PACING 0x00000001 #define QUIC_TUNE_FB_TX_UDP_GSO 0x00000002 @@ -43,7 +42,6 @@ struct quic_tune { } be; uint64_t mem_tx_max; - uint options; }; #endif /* USE_QUIC */ diff --git a/src/cfgparse-quic.c b/src/cfgparse-quic.c index 58fcf3c82..f16660f16 100644 --- a/src/cfgparse-quic.c +++ b/src/cfgparse-quic.c @@ -29,6 +29,7 @@ struct quic_tune quic_tune = { .cc_reorder_ratio = QUIC_DFLT_CC_REORDER_RATIO, .sec_retry_threshold = QUIC_DFLT_SEC_RETRY_THRESHOLD, .fb_opts = QUIC_TUNE_FB_TX_PACING|QUIC_TUNE_FB_TX_UDP_GSO, + .opts = QUIC_TUNE_FE_SOCK_PER_CONN, }, .be = { .cc_max_frame_loss = QUIC_DFLT_CC_MAX_FRAME_LOSS, @@ -216,6 +217,29 @@ static struct bind_kw_list bind_kws = { "QUIC", { }, { INITCALL1(STG_REGISTER, bind_register_keywords, &bind_kws); +/* parse "tune.quic.fe.sock-per-conn", accepts "default-on" or "force-off" */ +static int cfg_parse_quic_tune_sock_per_conn(char **args, int section_type, + struct proxy *curpx, + const struct proxy *defpx, + const char *file, int line, char **err) +{ + if (too_many_args(1, args, err, NULL)) + return -1; + + if (strcmp(args[1], "default-on") == 0) { + quic_tune.fe.opts |= QUIC_TUNE_FE_SOCK_PER_CONN; + } + else if (strcmp(args[1], "force-off") == 0) { + quic_tune.fe.opts &= ~QUIC_TUNE_FE_SOCK_PER_CONN; + } + else { + memprintf(err, "'%s' expects either 'default-on' or 'force-off' but got '%s'.", args[0], args[1]); + return -1; + } + + return 0; +} + /* parse "tune.quic.socket-owner", accepts "listener" or "connection" */ static int cfg_parse_quic_tune_socket_owner(char **args, int section_type, struct proxy *curpx, @@ -225,18 +249,22 @@ static int cfg_parse_quic_tune_socket_owner(char **args, int section_type, if (too_many_args(1, args, err, NULL)) return -1; + memprintf(err, "'%s' is deprecated in 3.3 and will be removed in 3.5. " + "Please use the newer keyword syntax 'tune.quic.fe.sock-per-conn'.", args[0]); + if (strcmp(args[1], "connection") == 0) { - quic_tune.options |= QUIC_TUNE_SOCK_PER_CONN; + quic_tune.fe.opts |= QUIC_TUNE_FE_SOCK_PER_CONN; } else if (strcmp(args[1], "listener") == 0) { - quic_tune.options &= ~QUIC_TUNE_SOCK_PER_CONN; + quic_tune.fe.opts &= ~QUIC_TUNE_FE_SOCK_PER_CONN; } else { memprintf(err, "'%s' expects either 'listener' or 'connection' but got '%s'.", args[0], args[1]); return -1; } - return 0; + /* Returns 1 to ensure deprecated warning is displayed. */ + return 1; } /* Must be used to parse tune.quic.* setting which requires a time @@ -552,7 +580,6 @@ static int cfg_parse_quic_tune_on_off(char **args, int section_type, struct prox static struct cfg_kw_list cfg_kws = {ILH, { { CFG_GLOBAL, "tune.quic.listen", cfg_parse_quic_tune_on_off }, { CFG_GLOBAL, "tune.quic.mem.tx-max", cfg_parse_quic_tune_setting }, - { CFG_GLOBAL, "tune.quic.socket-owner", cfg_parse_quic_tune_socket_owner }, { CFG_GLOBAL, "tune.quic.frontend.max-data-size", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.frontend.max-streams-bidi", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.frontend.max-idle-timeout", cfg_parse_quic_time }, @@ -566,6 +593,7 @@ static struct cfg_kw_list cfg_kws = {ILH, { { CFG_GLOBAL, "tune.quic.fe.cc.reorder-ratio", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.fe.sec.glitches-threshold", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.fe.sec.retry-threshold", cfg_parse_quic_tune_setting }, + { CFG_GLOBAL, "tune.quic.fe.sock-per-conn", cfg_parse_quic_tune_sock_per_conn }, { CFG_GLOBAL, "tune.quic.fe.tx.pacing", cfg_parse_quic_tune_on_off }, { CFG_GLOBAL, "tune.quic.fe.tx.udp-gso", cfg_parse_quic_tune_on_off }, @@ -587,6 +615,7 @@ static struct cfg_kw_list cfg_kws = {ILH, { { CFG_GLOBAL, "tune.quic.max-frame-loss", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.reorder-ratio", cfg_parse_quic_tune_setting }, { CFG_GLOBAL, "tune.quic.retry-threshold", cfg_parse_quic_tune_setting }, + { CFG_GLOBAL, "tune.quic.socket-owner", cfg_parse_quic_tune_socket_owner }, { 0, NULL, NULL } }}; diff --git a/src/haproxy.c b/src/haproxy.c index 5a0523487..da4f63228 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -1488,10 +1488,6 @@ static void init_args(int argc, char **argv) /* Use zero-copy forwarding by default */ global.tune.no_zero_copy_fwd = 0; -#ifdef USE_QUIC - quic_tune.options |= QUIC_TUNE_SOCK_PER_CONN; -#endif - /* keep a copy of original arguments for the master process */ old_argv = copy_argv(argc, argv); if (!old_argv) { diff --git a/src/proto_quic.c b/src/proto_quic.c index fe8faec4d..bbf5626f8 100644 --- a/src/proto_quic.c +++ b/src/proto_quic.c @@ -667,7 +667,7 @@ static int quic_deallocate_dghdlrs(void) } REGISTER_POST_DEINIT(quic_deallocate_dghdlrs); -/* Checks that connection socket-owner mode is supported. +/* Checks that connection sock-per-conn mode is supported. * Returns 1 if it is, 0 if not. A negative error code is used for an unknown * error which leaves support status as unknown. */ @@ -688,7 +688,7 @@ static int quic_test_conn_socket_owner(void) goto end; } - /* Connection socket-owner mode relies on several system features : + /* Connection sock-per-conn mode relies on several system features : * - IP_PKTINFO or equivalent, to retrieve peer address for connect() * - support for multiple UDP sockets bound on the same source address */ @@ -766,8 +766,8 @@ static int quic_test_socketopts(void) { int ret; - /* Check for connection socket-owner mode support. */ - if (quic_tune.options & QUIC_TUNE_SOCK_PER_CONN) { + /* Check for connection sock-per-conn mode support. */ + if (quic_tune.fe.opts & QUIC_TUNE_FE_SOCK_PER_CONN) { ret = quic_test_conn_socket_owner(); if (ret < 0) { goto err; @@ -775,7 +775,7 @@ static int quic_test_socketopts(void) else if (!ret) { ha_diag_warning("Your platform does not seem to support UDP source address retrieval through IP_PKTINFO or an alternative flag. " "QUIC connections will use listener socket.\n"); - quic_tune.options &= ~QUIC_TUNE_SOCK_PER_CONN; + quic_tune.fe.opts &= ~QUIC_TUNE_FE_SOCK_PER_CONN; } } @@ -808,7 +808,7 @@ static void quic_register_build_options(void) int ret; ret = quic_test_conn_socket_owner(); - memprintf(&ptr, "QUIC: connection socket-owner mode support : "); + memprintf(&ptr, "QUIC: connection sock-per-conn mode support : "); memprintf(&ptr, "%s%s\n", ptr, ret > 0 ? "yes" : !ret ? "no" : "unknown"); diff --git a/src/quic_conn.c b/src/quic_conn.c index f94a6b7dc..738936892 100644 --- a/src/quic_conn.c +++ b/src/quic_conn.c @@ -1266,8 +1266,8 @@ struct quic_conn *qc_new_conn(const struct quic_version *qv, int ipv4, eb64_insert(qc->cids, &conn_id->seq_num); } else if (HA_ATOMIC_LOAD(&l->rx.quic_mode) == QUIC_SOCK_MODE_CONN && - (quic_tune.options & QUIC_TUNE_SOCK_PER_CONN) && - is_addr(local_addr)) { + (quic_tune.fe.opts & QUIC_TUNE_FE_SOCK_PER_CONN) && + is_addr(local_addr)) { /* Listener only */ TRACE_USER("Allocate a socket for QUIC connection", QUIC_EV_CONN_INIT, qc); qc_alloc_fd(qc, local_addr, peer_addr);