From: Remi Gacogne Date: Tue, 23 Jan 2024 13:54:29 +0000 (+0100) Subject: dnsdist: Update the XSK documentation for recent changes X-Git-Tag: dnsdist-1.9.0-rc1^2~5 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=ffbc306cefa96ba26055b27880bb28b37429300e;p=thirdparty%2Fpdns.git dnsdist: Update the XSK documentation for recent changes --- diff --git a/pdns/dnsdistdist/docs/advanced/tuning.rst b/pdns/dnsdistdist/docs/advanced/tuning.rst index e831485a79..ab5f59e12f 100644 --- a/pdns/dnsdistdist/docs/advanced/tuning.rst +++ b/pdns/dnsdistdist/docs/advanced/tuning.rst @@ -70,6 +70,11 @@ For DNS over HTTPS, every :func:`addDOHLocal` directive adds a new thread dealin When dealing with a large traffic load, it might happen that the internal pipe used to pass queries between the threads handling the incoming connections and the one getting a response from the backend become full too quickly, degrading performance and causing timeouts. This can be prevented by increasing the size of the internal pipe buffer, via the `internalPipeBufferSize` option of :func:`addDOHLocal`. Setting a value of `1048576` is known to yield good results on Linux. +`AF_XDP` / `XSK` +---------------- + +On recent versions of Linux (`>= 4.18`), DNSDist supports receiving UDP datagrams directly from the kernel, bypassing the usual network stack, via `AF_XDP`/`XSK`. This yields much better performance but comes with some limitations. Please see :doc:`xsk` for more information. + UDP buffer sizes ---------------- diff --git a/pdns/dnsdistdist/docs/advanced/xsk.rst b/pdns/dnsdistdist/docs/advanced/xsk.rst index c61f88e9b2..3badb7d6fd 100644 --- a/pdns/dnsdistdist/docs/advanced/xsk.rst +++ b/pdns/dnsdistdist/docs/advanced/xsk.rst @@ -10,9 +10,9 @@ Since 1.9.0, :program:`dnsdist` can use `AF_XDP `_ can be found in the ``contrib`` directory of the PowerDNS Git repository. The name of the network interface to use might have to be updated, though. Once it has been updated, the ``XDP`` program can be started:: +In addition to these, an ``eBPF`` ``XDP`` program needs to be loaded to decide which packets to distribute via the ``AF_XDP`` socket (and to which, as there are usually more than one). This program uses a ``BPF`` map of type ``XSKMAP`` (located at ``/sys/fs/bpf/dnsdist/xskmap`` by default) that is populated by :program:``dnsdist` at startup to locate the ``AF_XDP`` socket to use. :program:`dnsdist` also sets up two additional ``BPF`` maps (located at ``/sys/fs/bpf/dnsdist/xsk-destinations-v4`` and ``/sys/fs/bpf/dnsdist/xsk-destinations-v6``) to let the ``XDP`` program know which IP destinations are to be routed to the ``AF_XDP`` sockets and which are to be passed to the regular network stack (health-checks queries and responses, for example). A ready-to-use `XDP program `_ can be found in the ``contrib`` directory of the PowerDNS Git repository:: - $ python xdp.py + $ python xdp.py --xsk --interface eth0 Then :program:`dnsdist` needs to be configured to use ``AF_XDP``, first by creating a :class:`XskSocket` object that are tied to a specific queue of a specific network interface: @@ -52,17 +52,28 @@ The ``Combined`` lines tell us that the interface supports 8 queues, so we can d addLocal("192.0.2.1:53", {xskSocket=xsk, reusePort=true}) end -We can also instructs :program:`dnsdist` to use ``AF_XDP`` to send and receive UDP packets to a backend: +This will start one router thread per :class:`XskSocket` object, plus one worker thread per :func:`addLocal` using that :class:`XskSocket` object. + +We can instructs :program:`dnsdist` to use ``AF_XDP`` to send and receive UDP packets to a backend in addition to packets from clients: .. code-block:: lua - newServer("192.0.2.2:53", {xskSocket=xsk}) + local sockets = {} + for i=1,8 do + xsk = newXsk({ifName="enp1s0", NIC_queue_id=i-1, frameNums=65536, xskMapPath="/sys/fs/bpf/dnsdist/xskmap"}) + table.insert(sockets, xsk) + addLocal("192.0.2.1:53", {xskSocket=xsk, reusePort=true}) + end + + newServer("192.0.2.2:53", {xskSocket=sockets}) + +This will start one router thread per :class:`XskSocket` object, plus one worker thread per :func:`addLocal`/:func:`newServer` using that :class:`XskSocket` object. We are not passing the MAC address of the backend (or the gateway to reach it) directly, so :program:`dnsdist` will try to fetch it from the system MAC address cache. This may not work, in which case we might need to pass explicitly: .. code-block:: lua - newServer("192.0.2.2:53", {xskSocket=xsk, MACAddr='00:11:22:33:44:55'}) + newServer("192.0.2.2:53", {xskSocket=sockets, MACAddr='00:11:22:33:44:55'}) Performance @@ -70,7 +81,7 @@ Performance Using `kxdpgun `_, we can compare the performance of :program:`dnsdist` using the regular network stack and ``AF_XDP``. -This test was realized using two Intel E3-1270 with 4 cores (8 threads) running at 3.8 Ghz, using 10 Gbps network cards. On both the injector running ``kxdpgun`` and the box running :program:`dnsdist` there was no firewall, the governor was set to ``performance``, the UDP buffers were raised to ``16777216`` and the receive queue hash policy set to use the IP addresses and ports (see :doc:`tuning`). +This test was realized using two Intel E3-1270 with 4 cores (8 threads) running at 3.8 Ghz, using Intel 82599 10 Gbps network cards. On both the injector running ``kxdpgun`` and the box running :program:`dnsdist` there was no firewall, the governor was set to ``performance``, the UDP buffers were raised to ``16777216`` and the receive queue hash policy set to use the IP addresses and ports (see :doc:`tuning`). :program:`dnsdist` was configured to immediately respond to incoming queries with ``REFUSED``: diff --git a/pdns/dnsdistdist/docs/install.rst b/pdns/dnsdistdist/docs/install.rst index 13b672855a..34e3af1bfd 100644 --- a/pdns/dnsdistdist/docs/install.rst +++ b/pdns/dnsdistdist/docs/install.rst @@ -50,8 +50,9 @@ dnsdist depends on the following libraries: * `Editline (libedit) `_ * `libfstrm `_ (optional, dnstap support) * `GnuTLS `_ (optional, DoT and outgoing DoH support) -* `libh2o `_ (optional, incoming DoH support, deprecated in 1.9.0 in favor of ``nghttp2``) +* `libbpf `_ and `libxdp `_ (optional, `XSK`/`AF_XDP` support) * `libcap `_ (optional, capabilities support) +* `libh2o `_ (optional, incoming DoH support, deprecated in 1.9.0 in favor of ``nghttp2``) * `libsodium `_ (optional, DNSCrypt and console encryption support) * `LMDB `_ (optional, LMDB support) * `net-snmp `_ (optional, SNMP support) diff --git a/pdns/dnsdistdist/docs/reference/config.rst b/pdns/dnsdistdist/docs/reference/config.rst index 7311fc81f5..b3e2f5c2db 100644 --- a/pdns/dnsdistdist/docs/reference/config.rst +++ b/pdns/dnsdistdist/docs/reference/config.rst @@ -631,7 +631,7 @@ Servers Added ``autoUpgrade``, ``autoUpgradeDoHKey``, ``autoUpgradeInterval``, ``autoUpgradeKeep``, ``autoUpgradePool``, ``maxConcurrentTCPConnections``, ``subjectAddr``, ``lazyHealthCheckSampleSize``, ``lazyHealthCheckMinSampleCount``, ``lazyHealthCheckThreshold``, ``lazyHealthCheckFailedInterval``, ``lazyHealthCheckMode``, ``lazyHealthCheckUseExponentialBackOff``, ``lazyHealthCheckMaxBackOff``, ``lazyHealthCheckWhenUpgraded``, ``healthCheckMode`` and ``ktls`` to server_table. .. versionchanged:: 1.9.0 - Added ``MACAddr``, ``proxyProtocolAdvertiseTLS`` and ``xskSocket`` to server_table. + Added ``MACAddr``, ``proxyProtocolAdvertiseTLS`` and ``xskSockets`` to server_table. :param str server_string: A simple IP:PORT string. :param table server_table: A table with at least an ``address`` key @@ -720,7 +720,7 @@ Servers ``lazyHealthCheckWhenUpgraded`` ``bool`` "Whether the auto-upgraded version of this backend (see ``autoUpgrade``) should use the lazy health-checking mode. Default is false, which means it will use the regular health-checking mode." ``ktls`` ``bool`` "Whether to enable the experimental kernel TLS support on Linux, if both the kernel and the OpenSSL library support it. Default is false. Currently both DoT and DoH backend support this option." ``proxyProtocolAdvertiseTLS`` ``bool`` "Whether to set the SSL Proxy Protocol TLV in the proxy protocol payload sent to the backend if the query was received over an encrypted channel (DNSCrypt, DoQ, DoH or DoT). Requires ``useProxyProtocol=true``. Default is false." - ``xskSocket`` :class:`XskSocket` "A socket to enable ``XSK`` / ``AF_XDP`` support for this backend. See :doc:`../advanced/xsk` for more information." + ``xskSockets`` ``array`` "An array of :class:`XskSocket` objects to enable ``XSK`` / ``AF_XDP`` support for this backend. See :doc:`../advanced/xsk` for more information." ``MACAddr`` ``str`` "When the ``xskSocket`` option is set, this parameter can be used to specify the destination MAC address to use to reach the backend. If this options is not specified, dnsdist will try to get it from the IP of the backend by looking into the system's MAC address table, but it will fail if the corresponding MAC address is not present." .. function:: getServer(index) -> Server