From: Martin Schwenke Date: Tue, 22 Aug 2023 02:13:44 +0000 (+1000) Subject: ctdb-scripts: Add configuration variable CTDB_KILLTCP_USE_SS_KILL X-Git-Tag: tdb-1.4.13~623 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=4569c65288177969ca1e4d9bd6badec60552beb9;p=thirdparty%2Fsamba.git ctdb-scripts: Add configuration variable CTDB_KILLTCP_USE_SS_KILL This allows CTDB to be configured to use "ss -K" to reset TCP connections on "releaseip". This is only supported when the kernel is configured with CONFIG_INET_DIAG_DESTROY enabled. From the documentation: ss -K has been supported in ss since iproute 4.5 in March 2016 and in the Linux kernel since 4.4 in December 2015. However, the required kernel configuration item CONFIG_INET_DIAG_DESTROY is disabled by default. Although enabled in Debian kernels since ~2017 and in Ubuntu since at least 18.04,, this has only recently been enabled in distributions such as RHEL. There seems to be no way, including running ss -K, to determine if this is supported, so use of this feature needs to be configurable. When available, it should be the fastest, most reliable way of killing connections. For RHEL and derivatives, this was enabled as follows: * RHEL 8 via https://bugzilla.redhat.com/show_bug.cgi?id=2230213, arriving in version kernel-4.18.0-513.5.1.el8_9 * RHEL 9 via https://issues.redhat.com/browse/RHEL-212, arriving in kernel-5.14.0-360.el9 Enabling this option results in a small behaviour change because ss -K always does a 2-way kill (i.e. it also sends a RST to the client). Only a 1-way kill is done for SMB connections when ctdb_killtcp is used - the reasons for this are shrouded in history and the 2-way kill seems to work fine. For the summary that is logged, when CTDB_KILLTCP_USE_SS_KILL is "yes" or "try", always log the method used, even the fallback to ctdb_killtcp. However, when set to "no", maintain the existing output. The decision to use -K rather than --kill is because short options are trivial to implement in test stubs. Signed-off-by: Martin Schwenke Reviewed-by: Volker Lendecke Reviewed-by: Jerry Heyman Autobuild-User(master): Martin Schwenke Autobuild-Date(master): Thu Nov 7 00:12:34 UTC 2024 on atb-devel-224 --- diff --git a/ctdb/config/functions b/ctdb/config/functions index d1579e5156c..1ca3cebbbca 100755 --- a/ctdb/config/functions +++ b/ctdb/config/functions @@ -514,6 +514,7 @@ kill_tcp_summarise() { _mode="$1" _count="$2" + _method="$3" _connections=$(get_tcp_connections_for_ip "$_ip") if [ -z "$_connections" ]; then @@ -534,7 +535,11 @@ kill_tcp_summarise() esac _t="${_killed}/${_total}" - echo "Killed ${_t} TCP connections to released IP ${_ip}" + _m="" + if [ -n "$_method" ]; then + _m=", using ${_method}" + fi + echo "Killed ${_t} TCP connections to released IP ${_ip}${_m}" if [ -n "$_connections" ]; then echo "Remaining connections:" echo "$_connections" | sed -e 's|^| |' @@ -551,6 +556,16 @@ kill_tcp_connections() _oneway=true fi + case "$CTDB_KILLTCP_USE_SS_KILL" in + yes | try) + _killcount=$(ss -K -tnH state established src "$_ip" | wc -l) + kill_tcp_summarise "killed" "$_killcount" "ss -K" + if [ "$CTDB_KILLTCP_USE_SS_KILL" = "yes" ]; then + return + fi + ;; + esac + get_tcp_connections_for_ip "$_ip" | { _killcount=0 _connections="" @@ -588,7 +603,11 @@ kill_tcp_connections() return } - kill_tcp_summarise "total" "$_killcount" + _method="" + if [ "$CTDB_KILLTCP_USE_SS_KILL" = "try" ]; then + _method="ctdb_killtcp" + fi + kill_tcp_summarise "total" "$_killcount" "$_method" } } diff --git a/ctdb/doc/ctdb-script.options.5.xml b/ctdb/doc/ctdb-script.options.5.xml index a0122c0e77e..9298f9f3498 100644 --- a/ctdb/doc/ctdb-script.options.5.xml +++ b/ctdb/doc/ctdb-script.options.5.xml @@ -112,6 +112,95 @@ + + + CTDB_KILLTCP_USE_SS_KILL=yes|try|no + + + + Whether to use ss -K/--kill to reset + incoming TCP connections to public IP addresses during + releaseip. + + + + CTDB's standard method of resetting incoming TCP + connections during releaseip is via + its custom ctdb_killtcp command. + This uses network trickery to reset each connection: + send a "tickle ACK", capture the reply to extract the + TCP sequence number, send a reset (containing the + correct sequence number). + + + + ss -K has been supported in + ss since iproute 4.5 in March 2016 + and in the Linux kernel since 4.4 in December 2015. + However, the required kernel configuration item + CONFIG_INET_DIAG_DESTROY is disabled by + default. Although enabled in Debian kernels since ~2017 + and in Ubuntu since at least 18.04, this has only + recently been enabled in distributions such as RHEL. + There seems to be no way, including running ss + -K, to determine if this is supported, so use + of this feature needs to be configurable. When + available, it should be the fastest, most reliable way + of killing connections. + + + + Supported values are: + + + + yes + + + + Use ss -K and make no other + attempt to kill any remaining connections. This + is sane on modern Linux distributions that are + guaranteed to have + CONFIG_INET_DIAG_DESTROY enabled. + + + + + + + try + + + + Attempt to use ss -K and fall + back to ctdb_killtcp for any + remaining connections. This may be a good value + when ss supports the + -K option but it is uncertain + whether CONFIG_INET_DIAG_DESTROY is + enabled. + + + + + + + no + + + + Never attempt to use ss -K. + Rely only on ctdb_killtcp. + + + + + Default is "no". + + + + CTDB_PARTIALLY_ONLINE_INTERFACES=yes|no diff --git a/ctdb/tests/UNIT/eventscripts/10.interface.020.sh b/ctdb/tests/UNIT/eventscripts/10.interface.020.sh new file mode 100755 index 00000000000..89e6c9ae251 --- /dev/null +++ b/ctdb/tests/UNIT/eventscripts/10.interface.020.sh @@ -0,0 +1,27 @@ +#!/bin/sh + +. "${TEST_SCRIPTS_DIR}/unit.sh" + +define_test "Release 1 IP, 10 connections killed OK, use ss -K" + +setup + +setup_script_options <"$new" while read -r src dst; do if filter_socket "$srcs" "$sports" "$src"; then echo 0 0 "$src" "$dst" + else + echo "${src} ${dst}" >>"$new" fi done <"$FAKE_NETSTAT_TCP_ESTABLISHED_FILE" + if "$kill"; then + mv "$new" "$FAKE_NETSTAT_TCP_ESTABLISHED_FILE" + else + rm "$new" + fi } ############################################################ @@ -160,16 +171,18 @@ unix=false all=false listen=false header=true +kill=false orig="$*" -while getopts "txnalHh?" opt; do +while getopts "txnalHKh?" opt; do case "$opt" in t) tcp=true ;; x) unix=true ;; l) listen=true ;; a) all=true ;; H) header=false ;; + K) kill=true ;; n) : ;; \? | h) usage ;; esac