dnsdist: Add more documentation about structured logging

Signed-off-by: Remi Gacogne <remi.gacogne@powerdns.com>

diff --git a/pdns/dnsdistdist/dnsdist-settings-definitions.yml b/pdns/dnsdistdist/dnsdist-settings-definitions.yml
index dde42e4cbb5985fdacca585d5fa02023cd9d560a..503ddf870309955e0cfcbf2eba44652ccc49113a 100644 (file)
--- a/pdns/dnsdistdist/dnsdist-settings-definitions.yml
+++ b/pdns/dnsdistdist/dnsdist-settings-definitions.yml
@@ -1802,11 +1802,11 @@ structured_logging:
       default: ""
       version_added: "2.1.0"
       description: |
-                   The backend used for structured logging output. Available backends are:
+                   The backend used for structured logging output. See :doc:`../advanced/structured-logging-dictionary` for more details. Available backends are:
 
-                   - default: use the traditional logging system to output structured logging information.
-                   - systemd-journal: use systemd-journal. When using this backend, provide -o verbose or simular output option to journalctl to view the full information.
-                   - json: JSON objects are written to the standard error stream.
+                   * ``default``: use the traditional logging system to output structured logging information.
+                   * ``systemd-journal``: use ``systemd-journal``. When using this backend, provide ``-o verbose`` or simular output option to ``journalctl`` to view the full information.
+                   * ``json``: JSON objects are written to the standard error stream.
 
 logging:
   description: "Logging settings"

diff --git a/pdns/dnsdistdist/docs/advanced/index.rst b/pdns/dnsdistdist/docs/advanced/index.rst
index e247c98b8e43be7ad7421238b4628deff9d8523d..36ce57aabd26422523a67d85cc132b34686fff07 100644 (file)
--- a/pdns/dnsdistdist/docs/advanced/index.rst
+++ b/pdns/dnsdistdist/docs/advanced/index.rst
@@ -23,5 +23,6 @@ These chapters contain information on the advanced features of dnsdist
    tls-sessions-management
    internal-design
    asynchronous-processing
+   structured-logging-dictionary
    xsk
    zero-scope

diff --git a/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst b/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst
new file mode 100644 (file)
index 0000000..294ea79
--- /dev/null
+++ b/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst
@@ -0,0 +1,156 @@
+Structured logging dictionary
+=============================
+
+This page describes the common entries of the Structured Logging component. Currently these backends are supported:
+
+* The ``default`` text based backend
+* The ``systemd-journal`` backend
+* The ``json`` backend
+
+The ``default`` backend
+-----------------------
+
+The default backend uses a text representation of the key-value pairs.
+A line is constructed by appending all key-value pairs as ``key="value"``, separated by spaces.
+The output is written by passing the resulting text line to the standard error stream and also to ``syslog`` if ``--disable-syslog`` was not passed on the command-line.
+
+An example line looks like this::
+
+  msg="Raised send buffer size" subsystem="setup" level="0" prio="Info" ts="2025-12-29T17:16:17+0100" frontend.address="127.0.0.1:8053" network.send_buffer_size="4194304
+
+- Key names are not quoted.
+- Values are quoted with double quotes.
+- If a value contains a double quote, it is escaped with a backslash.
+- Backslashes in the value are escaped by prepending a backslash.
+
+The following keys are always present:
+
++-------------+------------------+--------------------------------------+---------------------------------------+
+| **Key**     | **Type**         | **Example**                          | **Remarks**                           |
++-------------+------------------+--------------------------------------+---------------------------------------+
+|``msg``      |``string``        | ``"Raised send buffer size"``        |Value is the same for all instances of |
+|             |                  |                                      |this log entry, together with          |
+|             |                  |                                      |``subsystem`` it uniquely identifies   |
+|             |                  |                                      |the log message.                       |
++-------------+------------------+--------------------------------------+---------------------------------------+
+|``subsystem``|``string``        |``"setup"``                           |Uniquely identifies the log            |
+|             |                  |                                      |entry together with the value of       |
+|             |                  |                                      |``msg``.                               |
++-------------+------------------+--------------------------------------+---------------------------------------+
+| ``level``   |``number``        |``"0"``                               |The detail level of the log entry.     |
+|             |                  |                                      |Not actively used currently.           |
++-------------+------------------+--------------------------------------+---------------------------------------+
+| ``prio``    |``enum``          |``"Notice"``                          |One of ``Alert=1``, ``Critical=2``,    |
+|             |                  |                                      |``Error=3``, ``Warning=4``,            |
+|             |                  |                                      |``Notice=5``, ``Info=6``,              |
+|             |                  |                                      |``Debug=7``.                           |
++-------------+------------------+--------------------------------------+---------------------------------------+
+| ``ts``      |``number``        |``"1697614303.039"``                  |Number of seconds since the Unix epoch,|
+|             |                  |                                      |including fractional part.             |
++-------------+------------------+--------------------------------------+---------------------------------------+
+
+A log entry can also have zero or more additional key-value pairs. We tried to adhere to OpenTelemetry's
+`general attributes <https://opentelemetry.io/docs/specs/semconv/general/attributes/>`_ when possible, or more specific
+ones like for example `http attributes <https://opentelemetry.io/docs/specs/semconv/registry/attributes/http/>`_ for DNS over HTTPS.
+
+Common keys are:
+
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| **Key**                        | **Type**            |**Example**                           | **Remarks**                                                                                                                                                                               |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``error``                       |``string``           |``"No such file or directory"``       |An error cause.                                                                                                                                                                            |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``address``                     |``ip address:port``  |``"[::]:5301"``                       |An IP: port combination.                                                                                                                                                                   |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``addresses``                   |``list of subnets``  |``"127.0.0.0/8 ::ffff:0:0/96"``       |A list of subnets, space separated.                                                                                                                                                        |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``client.address``              |``ip address:port``  |``"[2001:db8::1]:1234"``              |Address of the DNS client.                                                                                                                                                                 |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``backend.address``             |``ip address:port``  |``"[2001:db8::2]:53"``                |Address of the downstream backend.                                                                                                                                                         |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``backend.name``                |``string``           |``"my-backend"``                      |Name of the downstream backend, if any.                                                                                                                                                    |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``backend.protocol``            |``string``           |``"DoT"``                             |Protocol used to send DNS queries to the downstream backend.                                                                                                                               |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``destination.address``         |``ip address:port``  |``"192.0.2.42:53"``                   |Destination address of a packet (may be more specific than ``frontend.address``, or be the next hop and differ from ``client.address`` when the proxy protocol is being used, for example).|
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.class``          |``DNS class``        |``"IN"``                              |The DNS query class.                                                                                                                                                                       |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.id``             |``unsigned integer`` |``42``                                |The DNS query ID.                                                                                                                                                                          |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.name``           |``DNS name``         |``"example.com"``                     |The DNS query name.                                                                                                                                                                        |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.real_time_sec``  |``integer``          |``1767095038``                        |The time this query was received, as the number of seconds since EPOCH.                                                                                                                    |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.real_time_nsec`` |``integer``          |``23``                                |Additional nano-seconds for ``dns.question.real_time_sec``, for extra precision.                                                                                                           |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.size``           |``unsigned integer`` |``42``                                |Size of the DNS query packet, in bytes.                                                                                                                                                    |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.question.type``           |``DNS type``         |``"AAAA"``                            |The DNS query type.                                                                                                                                                                        |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.class``          |``DNS class``        |``"IN"``                              |The DNS response class.                                                                                                                                                                    |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.id``             |``unsigned integer`` |``42``                                |The DNS response ID.                                                                                                                                                                       |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.latency_us``     |``float``            |``180.63``                            |The latency between the query and the response, in microseconds.                                                                                                                           |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.name``           |``DNS name``         |``"example.com"``                     |The DNS response name.                                                                                                                                                                     |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.rcode``          |``DNS Response Code``|``"No Error"``                        |DNS response code                                                                                                                                                                          |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.size``           |``unsigned integer`` |``42``                                |Size of the DNS response packet, in bytes.                                                                                                                                                 |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``dns.response.type``           |``DNS type``         |``"AAAA"``                            |The DNS response type.                                                                                                                                                                     |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``frontend.address``            |``ip address:port``  |``"[::]:53"``                         |Address this frontend is listening on.                                                                                                                                                     |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``frontend.protocol``           |``string``           |``"DoQ"``                             |Protocol this frontend is accepting DNS queries over.                                                                                                                                      |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``http.method``                 |``string``           |``"GET"``                             |HTTP/2 query method for DoH.                                                                                                                                                               |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``http.path``                   |``string``           |``"/dns-query"``                      |HTTP/2 query path for DoH.                                                                                                                                                                 |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``http.response.status_code``   |``HTTP status code`` |``"200"``                             |HTTP/2 response code for DoH.                                                                                                                                                              |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``http.stream_id``              |``HTTP stream ID``   |``"4242"``                            |HTTP/2 stream identifier for DoH.                                                                                                                                                          |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``network.local.address``       |``string``           |``"192.0.2.2:53"``                    |Local address and port DNSdist is listening on (webserver, console, ...).                                                                                                                  |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``network.peer.address``        |``string``           |``"192.0.2.2:53"``                    |Peer address of the network connection (may differ from ``client.address`` when the proxy protocol is used, for example).                                                                  |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``path``                        |``filesystem path``  |``"tmp/api-dir/apizones"``            |                                                                                                                                                                                           |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``pool``                        |``string``           |``"my-pool"``                         |Downstream backends pool name.                                                                                                                                                             |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|``protocol``                    |``string``           |``"udp"``                             |                                                                                                                                                                                           |
++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+The ``systemd-journal`` backend
+-------------------------------
+
+The ``systemd-journal`` structured logging backend uses mostly the same keys and values as the default backend, with the exceptions:
+
+- keys are capitalized as required for ``systemd-journal``.
+- ``msg`` is translated to ``MESSAGE``.
+- ``prio`` is translated to ``PRIORITY``.
+- ``ts`` is translated to ``TIMESTAMP``.
+- If the original key is in a list of keys special to ``systemd-journal``, it is capitalized and prepended by ``PDNS_``.
+  The list of special keys is: message, message_id, priority, code_file, code_line, code_func, errno, invocation_id, user_invocation_id, syslog_facility, syslog_identifier, syslog_pid, syslog_timestamp, syslog_raw, documentation, tid, unit, user_unit, object_pid.
+
+To use this logging backend, add the ``--structured-logging-backend systemd-journal`` to the command line in the systemd unit file.
+
+To query the log, use a command similar to::
+
+  # journalctl -r -n 1 -o json-pretty -u dnsdist.service
+
+The ``json`` backend
+--------------------
+
+The ``json`` structured logging backend uses the same keys and values as the default backend.
+An example of a log object::
+
+  {"level": "0", "msg": "Adding TCP worker thread to handle TCP connections from clients", "priority": "6", "ts": "2025-12-29T17:20:20+0100"}
+
+All values are represented as strings.
+
+The JSON log objects are written to the standard error stream.

diff --git a/pdns/dnsdistdist/docs/reference/config.rst b/pdns/dnsdistdist/docs/reference/config.rst
index 2c9ab633760cb149347e9ac3eae329a448f395e6..cf04952aafb908d4bb282925bb7c68d8ba8d1bf7 100644 (file)
--- a/pdns/dnsdistdist/docs/reference/config.rst
+++ b/pdns/dnsdistdist/docs/reference/config.rst
@@ -1403,9 +1403,11 @@ Status, Statistics and More
   .. versionadded:: 1.9.0
 
   .. versionchanged:: 2.1.0
-    The ``level_prefix`` option has no longer any effect because it was confusing. The log level is now always logged as ``level`` and the syslog priority, if any, as ``priority`` in all backends except the default one where it is named ``prio``
+    The ``backend`` option has been added.
+    The ``levelPrefix`` option has no longer any effect because it was confusing. The log level is now always logged as ``level`` and the syslog priority, if any, as ``priority`` in all backends except the default one where it is named ``prio``
+
+  Set whether log messages should be in structured-logging format. This is turned off by default. See :doc:`../advanced/structured-logging-dictionary` for more details.
 
-  Set whether log messages should be in a structured-logging-like format. This is turned off by default.
   The resulting format looks like this (when timestamps are enabled via ``--log-timestamps`` and ``timeFormat="ISO8601"``)::
 
     ts="2023-11-06T12:04:58+0100" level="Info" msg="Added downstream server 127.0.0.1:53"
@@ -1419,8 +1421,15 @@ Status, Statistics and More
 
   Options:
 
-  * ``levelPrefix=prefix``: string - Set the prefix for the log level. Default is ``prio``. Not supported since 2.1.0.
+  * ``backend``: string - The backend used for structured logging output, see below. Added in 2.1.0.
   * ``timeFormat=format``: string - Set the time format. Supported values are ``ISO8601`` and ``numeric``. Default is ``numeric``.
+  * ``levelPrefix=prefix``: string - Set the prefix for the log level. Default is ``prio``. No longer supported as of 2.1.0.
+
+ Available backends:
+
+ * ``default``: use the traditional logging system to output structured logging information.
+ * ``systemd-journal``: use ``systemd-journal``. When using this backend, provide ``-o verbose`` or simular output option to ``journalctl`` to view the full information.
+ * ``json``: JSON objects are written to the standard error stream.
 
 .. function:: setOpenTelemetryTracing(value)