Note that the source address and port used to contact a given backend is set at startup, for performance reasons, and then only changes on reconnect. There might be more than one socket, and thus several ports, if the ``sockets`` parameter was set to a higher value than 1 on the :func:`newServer` directive.
-Note that, since 1.7.0, UDP queries can be passed to the backend over TCP if the backend is TCP-only, or configured for DNS over TLS.
+Note that, since 1.7, UDP queries can be passed to the backend over TCP if the backend is TCP-only, or configured for DNS over TLS. This is done by passing the incoming query to a TCP worker over a pipe, as was already done for incoming TCP queries.
+
+.. figure:: ../imgs/DNSDistUDPDoT.png
+ :align: center
+ :alt: DNSDist UDP design for TCP-only, DoT backends
+
+In that case the response will be sent back, directly by the TCP worker, over UDP, instead of being passed back to the UDP responder thread.
TCP / DoT design
----------------
.. figure:: ../imgs/DNSDistDoH.png
:align: center
- :alt: DNSDist DoH design
+ :alt: DNSDist DoH design before 1.7
For DoH, two threads are created for each :func:`addDOHLocal` directive, one handling the TLS and HTTP layers, then passing the queries to the second one over a pipe. The second thread does DNS processing, applying rules and forwarding the query to the backend if needed, over UDP.
Note that even if the query does not need to be passed to a backend (cache-hit, self-generated answer), the response will be passed back to the first thread via a pipe, since only that thread deals with the client.
If the response comes from a backend, it will be picked up by the regular UDP listener for that backend, the corresponding *IDState* object located, and the response sent to the first thread over a pipe.
+
+Since 1.7, if the UDP response coming from the backend has been truncated (TC bit is set), dnsdist will retry over TCP by passing the query to a TCP worker over a pipe, as was already done for incoming TCP queries. The response will then be passed back to the DoH worker thread over the same pipe that for UDP queries. That also happens if the backend is marked TCP-only, or configured for DNS over TLS, in which case the query is obviously not sent over UDP first but immediately sent to a TCP worker thread.
+
+.. figure:: ../imgs/DNSDistDoH17.png
+ :align: center
+ :alt: DNSDist DoH design since 1.7
One of the most costly TLS operation is the negotiation of a new session, since both the client and the server need to generate and agree on cryptographic materials. In order to reduce that cost, TLS implements what is called session resumption, where a client opening a new connection to a server can reuse the cryptographic materials negotiated for a previous TLS session.
-The following figures show that, with the same number of established connections and queries per second, the ratio of new TLS sessions and resumed sessions has a huge impact on CPU usage:
+The following figures show that, with the same number of established incoming connections and queries per second, the ratio of new TLS sessions and resumed sessions has a huge impact on CPU usage:
.. figure:: ../imgs/tls_resumptions.png
:align: center
Knowing the STEK is all the information needed to be able to decrypt a live TLS session, but also a recorded one, so it is very important to keep that key well-protected. It should never be exchanged in clear-text, and ideally should not be written to persistent storage but be kept in a tmpfs with no swap configured. It should also be regularly rotated to preserve TLS' forward secrecy properties.
-Keys management in dnsdist
---------------------------
+Keys management for incoming connections in dnsdist
+---------------------------------------------------
-dnsdist supports both server's side (sessions) and client's side (tickets) resumption.
+dnsdist supports both server's side (sessions) and client's side (tickets) resumption for incoming connections (client to dnsdist).
Since server-side sessions cannot be shared between several instances, and pretty much all clients support tickets anyway, we do recommend disabling the sessions by passing ``numberOfStoredSessions=0`` to the :func:`addDOHLocal` (for DNS over HTTPS) and :func:`addTLSLocal` (for DNS over TLS) functions.
- a 16 bytes binary key identifier
- a 32 bytes AES 256 key
- a 16 bytes HMAC SHA-1 key
+
+Sessions management for outgoing connections
+--------------------------------------------
+
+Since 1.7, dnsdist supports securing the connection toward backends using DNS over TLS. For these connections, it keeps a cache of TLS tickets to be able to resume a TLS session quickly. By default that cache contains up to 20 TLS tickets per-backend, is cleaned up every every 60s, and TLS tickets expire if they have not been used after 600 seconds.
+These values can be set at configuration time via:
+
+ * :func:`setOutgoingTLSSessionsCacheMaxTicketsPerBackend`
+ * :func:`setOutgoingTLSSessionsCacheCleanupDelay`
+ * :func:`setOutgoingTLSSessionsCacheMaxTicketValidity`
Using a single connected UDP socket to contact a backend, and thus a single (source address, source port, destination address, destination port) tuple, might not play well with some load-balancing mechanisms present in front of the backend. Linux's ``reuseport``, for example, does not balance the incoming datagrams to several threads in that case. That can be worked around by using the ``sockets`` option of the :func:`newServer` directive to open several sockets instead of one. You may want to set that number to a value somewhat higher than the number of worker threads configured in the backend. dnsdist will then select a socket using round-robin to forward a query to the backend, and use event multiplexing on the receiving side.
-.. figure:: ../imgs/DNSDistDoH.png
+Note that, since 1.7, dnsdist supports marking a backend as "TCP only", as well as enabling DNS over TLS communication between dnsdist and that backend. That leads to a different model where UDP queries are instead passed to a TCP worker:
+
+.. figure:: ../imgs/DNSDistUDPDoT.png
:align: center
- :alt: DNSDist DoH design
+ :alt: DNSDist UDP design for TCP-only, DoT backends
For DNS over HTTPS, every :func:`addDOHLocal` directive adds a new thread dealing with incoming connections, so it might be useful to add more than one directive, as indicated above.
+.. figure:: ../imgs/DNSDistDoH17.png
+ :align: center
+ :alt: DNSDist DoH design
+
When dealing with a large traffic load, it might happen that the internal pipe used to pass queries between the threads handling the incoming connections and the one getting a response from the backend become full too quickly, degrading performance and causing timeouts. This can be prevented by increasing the size of the internal pipe buffer, via the `internalPipeBufferSize` option of :func:`addDOHLocal`. Setting a value of `1048576` is known to yield good results on Linux.
TCP and DNS over TLS
Internal design
---------------
-The internal design used for DoH handling uses two threads per :func:`addDOHLocal` directive. The first thread will handle the HTTP/2 communication with the client and pass the received DNS queries to a second thread which will apply the rules and pass the query to a backend, over **UDP** (except if the backend is TCP-only, or uses DNS over TLS). The response will be received by the regular UDP response handler for that backend and passed back to the first thread. That allows the first thread to be low-latency dealing with TLS and HTTP/2 only and never blocking.
+The internal design used for DoH handling uses two threads per :func:`addDOHLocal` directive. The first thread will handle the HTTP/2 communication with the client and pass the received DNS queries to a second thread which will apply the rules and pass the query to a backend, over **UDP** (except if the backend is TCP-only, or uses DNS over TLS, see the second schema below). The response will be received by the regular UDP response handler for that backend and passed back to the first thread. That allows the first thread to be low-latency dealing with TLS and HTTP/2 only and never blocking.
.. figure:: ../imgs/DNSDistDoH.png
:align: center
- :alt: DNSDist DoH design
+ :alt: DNSDist DoH design before 1.7
The fact that the queries are forwarded over UDP means that a large UDP payload size should be configured between dnsdist and the backend to avoid most truncation issues, and dnsdist will advise a 4096-byte UDP Payload Buffer size. UDP datagrams can still be larger than the MTU as long as fragmented datagrams are not dropped on the path between dnsdist and the backend.
-Since 1.7.0, truncated answers received over UDP for a DoH query will lead to a retry over TCP.
+Since 1.7.0, truncated answers received over UDP for a DoH query will lead to a retry over TCP, passing the query to a TCP worker, as illustrated below.
+
+.. figure:: ../imgs/DNSDistDoH17.png
+ :align: center
+ :alt: DNSDist DoH design since 1.7
Investigating issues
--------------------
DNS-over-TLS
============
-Since version 1.3.0, :program:`dnsdist` supports experimental DNS-over-TLS support.
+Incoming
+--------
+
+Since version 1.3.0, :program:`dnsdist` supports DNS-over-TLS for incoming queries.
To see if the installation supports this, run ``dnsdist --version``.
If the output shows ``dns-over-tls`` with one or more SSL libraries in brackets, DNS-over-TLS is supported.
More information about sessions management can also be found in :doc:`../advanced/tls-sessions-management`.
+Outgoing
+--------
+
+Support for securing the exchanges between dnsdist and the backend will be implemented in 1.7.0, and will lead to all queries, regardless of whether they were initially received by dnsdist over UDP, TCP, DoT or DoH, being forwarded over a secure DNS over TLS channel.
+That support can be enabled via the ``tls`` parameter of the :func:`newServer` command. Additional parameters control the validation of the certificate presented by the backend (``caStore``, ``validateCertificates``), the actual TLS ciphers used (``ciphers``, ``ciphersTLS13``) and the SNI value sent (``subjectName``).
+
Investigating issues
--------------------
- an IPv4 or IPv6 address followed by '@' then an interface name
Please note that specifying the interface name is only supported on system having `IP_PKTINFO`.
+
+Securing the channel
+--------------------
+
+Support for securing the exchanges between dnsdist and the backend will be implemented in 1.7.0, and will lead to all queries, regardless of whether they were initially received by dnsdist over UDP, TCP, DoT or DoH, being forwarded over a secure DNS over TLS channel.
+That support can be enabled via the ``tls`` parameter of the :func:`newServer` command. Additional parameters control the validation of the certificate presented by the backend (``caStore``, ``validateCertificates``), the actual TLS ciphers used (``ciphers``, ``ciphersTLS13``) and the SNI value sent (``subjectName``).
:param DNSName name: The name to test against the set.
+Outgoing TLS tickets cache management
+-------------------------------------
+
+Since 1.7, dnsdist supports securing the connection toward backends using DNS over TLS. For these connections, it keeps a cache of TLS tickets to be able to resume a TLS session quickly. By default that cache contains up to 20 TLS tickets per-backend, is cleaned up every every 60s, and TLS tickets expire if they have not been used after 600 seconds.
+These values can be set at configuration time via:
+
+.. function:: setOutgoingTLSSessionsCacheMaxTicketsPerBackend(num)
+
+ .. versionadded: 1.7.0
+
+ Set the maximum number of TLS tickets to keep, per-backend, to be able to quickly resume outgoing TLS connections to a backend. Keeping more tickets might provide a better TLS session resumption rate if there is a sudden peak of outgoing connections, at the cost of using a bit more memory.
+
+ :param int num: The number of TLS tickets to keep, per-backend. The default is 20.
+
+.. function:: setOutgoingTLSSessionsCacheCleanupDelay(delay)
+
+ .. versionadded: 1.7.0
+
+ Set the number of seconds between two scans of the TLS sessions cache, removing expired tickets and freeing up memory. Decreasing that value will lead to more scans, freeing up memory more quickly but using a bit more CPU doing so.
+
+ :param int delay: The number of seconds between two scans of the cache. The default is 60.
+
+.. function:: setOutgoingTLSSessionsCacheMaxTicketValidity(validity)
+
+ .. versionadded: 1.7.0
+
+ Set the number of seconds that a given TLS ticket can be kept inactive in the TLS sessions cache. After that delay the ticket will be removed during the next cleanup of the cache. Increasing that value might increase the TLS resumption rate if new connections are not often created, but it might also lead to trying to reuse a ticket that the server will consider too old and refuse.
+
+ :param int validity: The number of seconds a ticket is considered valid. The default is 600, which matches the default lifetime of TLS tickets set by OpenSSL.
+
Other functions
---------------
Understanding how queries are forwarded to backends
---------------------------------------------------
-Initially dnsdist tried to forward a query to the backend using the same protocol than the client used to contact dnsdist: queries received over UDP were forwarded over UDP, and the same for TCP. When incoming DNSCrypt and DNS over TLS support were added, the same logic was applied, so DoT queries are forwarded over TCP. For DNS over HTTPS, UDP was selected instead for performance reason, breaking with the existing logic:
-
-+--------------+----------+
-| Incoming | Outgoing |
-+==============+==========+
-| UDP | UDP |
-+--------------+----------+
-| TCP | TCP |
-+--------------+----------+
-| DNSCrypt UDP | UDP |
-+--------------+----------+
-| DNSCrypt TCP | TCP |
-+--------------+----------+
-| DoT | TCP |
-+--------------+----------+
-| DoH | **UDP** |
-+--------------+----------+
+Initially dnsdist tried to forward a query to the backend using the same protocol than the client used to contact dnsdist: queries received over UDP were forwarded over UDP, and the same for TCP. When incoming DNSCrypt and DNS over TLS support were added, the same logic was applied, so DoT queries are forwarded over TCP. For DNS over HTTPS, UDP was selected instead for performance reason, breaking with the existing logic.
Before 1.7.0, which introduced TCP fallback, that meant that there was a potential issue with very large answers and DNS over HTTPS, requiring careful configuration of the path between dnsdist and the backend. More information about that is available in the :doc:`DNS over HTTPS section <guides/dns-over-https>`.
+
+In addition to TCP fallback for DoH, 1.7.0 introduced two new notions:
+
+ * TCP-only backends, for which queries will always forwarded over a TCP connection (see the `tcpOnly` parameter of :func:`newServer`)
+ * and DNS over TLS backends, for which queries are forwarded over a DNS over TLS connection (see the `tls` parameter of :func:`newServer`)
+
+To sum it up:
+
++--------------+--------------------+---------------------------+----------------------+
+| Incoming | Outgoing (regular) | Outgoing (TCP-only, 1.7+) | Outgoing (TLS, 1.7+) |
++==============+====================+===========================+======================+
+| UDP | UDP | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
+| TCP | TCP | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
+| DNSCrypt UDP | UDP | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
+| DNSCrypt TCP | TCP | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
+| DoT | TCP | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
+| DoH | **UDP** | TCP | TLS |
++--------------+--------------------+---------------------------+----------------------+
projects / thirdparty / pdns.git / commitdiff
