From: Daniel Salzman Date: Thu, 16 Apr 2020 09:28:16 +0000 (+0200) Subject: doc: update XDP-related information X-Git-Tag: embedded_lmdb~35^2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=a8ae110b1bbe8cc537cbb2bfcb8723de3c8ea239;p=thirdparty%2Fknot-dns.git doc: update XDP-related information --- diff --git a/doc/man/knot.conf.5in b/doc/man/knot.conf.5in index dfabb5b3bd..c3a59e152c 100644 --- a/doc/man/knot.conf.5in +++ b/doc/man/knot.conf.5in @@ -196,7 +196,7 @@ server: edns\-client\-subnet: BOOL answer\-rotation: BOOL listen: ADDR[@INT] ... - listen\-xdp: STR ... + listen\-xdp: STR[@INT] | ADDR[@INT] ... .ft P .fi .UNINDENT @@ -387,21 +387,25 @@ Change of this parameter requires restart of the Knot server to take effect. \fIDefault:\fP not set .SS listen\-xdp .sp -One or more specifications in the form \fBeth_dev@port\fP, e.g. \fBeth1@53\fP\&. -Alternatively, an IP address can be used instead of a device name, but Knot -will still listen on all addresses belonging to the same interface! -.sp -If specified, Knot will create additional XDP workers, listening -on specified interface(s) and port(s) on UDP protocol, with enhanced -performance. The number of XDP workers is equal to the sum of the -interfaces\(aq RX queues. -.sp -It is strongly recommended to also \fI\%listen\fP on all those -addresses, at least to fulfill the requirement of working TCP fallback. +One or more network device names (e.g. \fBens786f0\fP) on which the Mode XDP +is enabled. Alternatively, an IP address can be used instead of a device name, +but the server will still listen on all addresses belonging to the same interface! +Optional port specification (default is 53) can be appended to each device name +or address using \fB@\fP separator. .sp Change of this parameter requires restart of the Knot server to take effect. .sp \fIDefault:\fP not set +.sp +\fBCAUTION:\fP +.INDENT 0.0 +.INDENT 3.5 +Since XDP workers only process regular DNS traffic over UDP, it is strongly +recommended to also \fI\%listen\fP on the addresses which are +intended to offer the DNS service, at least to fulfil the DNS requirement for +working TCP. +.UNINDENT +.UNINDENT .SH KEY SECTION .sp Shared TSIG keys used to authenticate communication with the server. diff --git a/doc/operation.rst b/doc/operation.rst index 9018aabc36..7f0ac544e6 100644 --- a/doc/operation.rst +++ b/doc/operation.rst @@ -947,45 +947,60 @@ which could, for example, publish the data in the JSON format via HTTP(S) or upload the data to a more efficient time series database. Take a look into the python folder of the project for these scripts. -.. _eXpress Data Path: +.. _Mode XDP: -eXpress Data Path -================= +Mode XDP +======== + +Thanks to recent Linux kernel capabilities, namely eXpress Data Path and AF_XDP +address family, Knot DNS offers a high-performance DNS over UDP packet processing +mode. The basic idea is to filter DNS messages close to the network device and +efectively forwarding them to the nameserver without touching the network stack +of the operating system. Other messages (including DNS over TCP) are processed +as usual. -This is an optional method to significantly improve the server's performance -(queries per second) by bypassing Linux network stack. It works only with simple -UDP queries, the rest is processed as usual by UDP and TCP workers. +If :ref:`listen-xdp ` is configured, the server creates +additional XDP workers, listening on specified interface(s) and port(s) for DNS +over UDP queries. Each XDP worker handles one RX and TX network queue pair. Pre-requisites -------------- -Linux kernel 5.x+, libbpf. - -Recommended new ixgbe or i40e network card drivers. - -Start knotd as root (it may drop to unpriv user after init). - -Notes ------ +* Linux kernel 4.18+ (5.x+ is recommended for optimal performance). +* A network card with native XDP support is highly recommended (successfully + tested cards are Intel series 500 and 700). +* If the `knotd` service is not directly executed in the privileged mode, some + additional Linux capabilities have to be set: -Don't attempt settings like:: + Execute command:: - # ip link set mtu $mtu dev $dev + systemctl edit knot -Troubleshooting ---------------- + And insert these lines:: -"Can't create socket" ... probably another instance is already using XDP on the same interface. + [Service] + CapabilityBoundingSet=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_SYS_RESOURCE + AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_SYS_RESOURCE -"Invalid parameter" ... the network card driver is in an incosistent state. Try removing -and inserting the kernel modules (ixgbe...) or rebooting the whole server. +Optimizations +------------- -Re-building BPF program ------------------------ +Some helpful commands:: -The BPF program bytecode is (as a binary blob) part of Knot DNS source codes, in file -``src/libknot/xdp/bpf-kernel-obj.c``. It may be re-generated with:: + ethtool -N rx-flow-hash udp4 sdfn + ethtool -N rx-flow-hash udp6 sdfn + ethtool -L combined + ethtool -G rx tx + renice -n 19 -p $(pgrep '^ksoftirqd/[0-9]*$') - $ make -C src/libknot/xdp +Limitations +----------- -Pre-requisites: clang, llc, kernel-headers. +* VLAN segmentation is not supported. +* Dynamic DNS over XDP is not supported. +* MTU higher than 1792 bytes is not supported. +* Symmetrical routing is required (query source address and reply destination address are the same). +* Systems with big-endian byte ordering require special recompilation of the nameserver. +* IPv4 header and UDP checksums are not verified on received DNS messages. +* DNS over XDP traffic is not visible to common system tools (e.g. firewall, tcpdump etc.). +* BPF filter is not automatically unloaded from the network device. diff --git a/doc/reference.rst b/doc/reference.rst index e5ca159b6f..a4373bc057 100644 --- a/doc/reference.rst +++ b/doc/reference.rst @@ -147,7 +147,7 @@ General options related to the server. edns-client-subnet: BOOL answer-rotation: BOOL listen: ADDR[@INT] ... - listen-xdp: STR ... + listen-xdp: STR[@INT] | ADDR[@INT] ... .. CAUTION:: When you change configuration parameters dynamically or via configuration file @@ -410,22 +410,22 @@ Change of this parameter requires restart of the Knot server to take effect. listen-xdp ---------- -One or more specifications in the form ``eth_dev@port``, e.g. ``eth1@53``. -Alternatively, an IP address can be used instead of a device name, but Knot -will still listen on all addresses belonging to the same interface! - -If specified, Knot will create additional XDP workers, listening -on specified interface(s) and port(s) on UDP protocol, with enhanced -performance. The number of XDP workers is equal to the sum of the -interfaces' RX queues. - -It is strongly recommended to also :ref:`server_listen` on all those -addresses, at least to fulfill the requirement of working TCP fallback. +One or more network device names (e.g. ``ens786f0``) on which the :ref:`Mode XDP` +is enabled. Alternatively, an IP address can be used instead of a device name, +but the server will still listen on all addresses belonging to the same interface! +Optional port specification (default is 53) can be appended to each device name +or address using ``@`` separator. Change of this parameter requires restart of the Knot server to take effect. *Default:* not set +.. CAUTION:: + Since XDP workers only process regular DNS traffic over UDP, it is strongly + recommended to also :ref:`listen ` on the addresses which are + intended to offer the DNS service, at least to fulfil the DNS requirement for + working TCP. + .. _Key section: Key section