From: Remi Gacogne Date: Mon, 29 Dec 2025 16:21:40 +0000 (+0100) Subject: dnsdist: Add more documentation about structured logging X-Git-Tag: rec-5.4.0-beta1~33^2~14 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=516b57b375dd8f877ee1bc993e8a3c11a0eaa590;p=thirdparty%2Fpdns.git dnsdist: Add more documentation about structured logging Signed-off-by: Remi Gacogne --- diff --git a/pdns/dnsdistdist/dnsdist-settings-definitions.yml b/pdns/dnsdistdist/dnsdist-settings-definitions.yml index dde42e4cbb..503ddf8703 100644 --- a/pdns/dnsdistdist/dnsdist-settings-definitions.yml +++ b/pdns/dnsdistdist/dnsdist-settings-definitions.yml @@ -1802,11 +1802,11 @@ structured_logging: default: "" version_added: "2.1.0" description: | - The backend used for structured logging output. Available backends are: + The backend used for structured logging output. See :doc:`../advanced/structured-logging-dictionary` for more details. Available backends are: - - default: use the traditional logging system to output structured logging information. - - systemd-journal: use systemd-journal. When using this backend, provide -o verbose or simular output option to journalctl to view the full information. - - json: JSON objects are written to the standard error stream. + * ``default``: use the traditional logging system to output structured logging information. + * ``systemd-journal``: use ``systemd-journal``. When using this backend, provide ``-o verbose`` or simular output option to ``journalctl`` to view the full information. + * ``json``: JSON objects are written to the standard error stream. logging: description: "Logging settings" diff --git a/pdns/dnsdistdist/docs/advanced/index.rst b/pdns/dnsdistdist/docs/advanced/index.rst index e247c98b8e..36ce57aabd 100644 --- a/pdns/dnsdistdist/docs/advanced/index.rst +++ b/pdns/dnsdistdist/docs/advanced/index.rst @@ -23,5 +23,6 @@ These chapters contain information on the advanced features of dnsdist tls-sessions-management internal-design asynchronous-processing + structured-logging-dictionary xsk zero-scope diff --git a/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst b/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst new file mode 100644 index 0000000000..294ea79ff9 --- /dev/null +++ b/pdns/dnsdistdist/docs/advanced/structured-logging-dictionary.rst @@ -0,0 +1,156 @@ +Structured logging dictionary +============================= + +This page describes the common entries of the Structured Logging component. Currently these backends are supported: + +* The ``default`` text based backend +* The ``systemd-journal`` backend +* The ``json`` backend + +The ``default`` backend +----------------------- + +The default backend uses a text representation of the key-value pairs. +A line is constructed by appending all key-value pairs as ``key="value"``, separated by spaces. +The output is written by passing the resulting text line to the standard error stream and also to ``syslog`` if ``--disable-syslog`` was not passed on the command-line. + +An example line looks like this:: + + msg="Raised send buffer size" subsystem="setup" level="0" prio="Info" ts="2025-12-29T17:16:17+0100" frontend.address="127.0.0.1:8053" network.send_buffer_size="4194304 + +- Key names are not quoted. +- Values are quoted with double quotes. +- If a value contains a double quote, it is escaped with a backslash. +- Backslashes in the value are escaped by prepending a backslash. + +The following keys are always present: + ++-------------+------------------+--------------------------------------+---------------------------------------+ +| **Key** | **Type** | **Example** | **Remarks** | ++-------------+------------------+--------------------------------------+---------------------------------------+ +|``msg`` |``string`` | ``"Raised send buffer size"`` |Value is the same for all instances of | +| | | |this log entry, together with | +| | | |``subsystem`` it uniquely identifies | +| | | |the log message. | ++-------------+------------------+--------------------------------------+---------------------------------------+ +|``subsystem``|``string`` |``"setup"`` |Uniquely identifies the log | +| | | |entry together with the value of | +| | | |``msg``. | ++-------------+------------------+--------------------------------------+---------------------------------------+ +| ``level`` |``number`` |``"0"`` |The detail level of the log entry. | +| | | |Not actively used currently. | ++-------------+------------------+--------------------------------------+---------------------------------------+ +| ``prio`` |``enum`` |``"Notice"`` |One of ``Alert=1``, ``Critical=2``, | +| | | |``Error=3``, ``Warning=4``, | +| | | |``Notice=5``, ``Info=6``, | +| | | |``Debug=7``. | ++-------------+------------------+--------------------------------------+---------------------------------------+ +| ``ts`` |``number`` |``"1697614303.039"`` |Number of seconds since the Unix epoch,| +| | | |including fractional part. | ++-------------+------------------+--------------------------------------+---------------------------------------+ + +A log entry can also have zero or more additional key-value pairs. We tried to adhere to OpenTelemetry's +`general attributes `_ when possible, or more specific +ones like for example `http attributes `_ for DNS over HTTPS. + +Common keys are: + ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Key** | **Type** |**Example** | **Remarks** | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``error`` |``string`` |``"No such file or directory"`` |An error cause. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``address`` |``ip address:port`` |``"[::]:5301"`` |An IP: port combination. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``addresses`` |``list of subnets`` |``"127.0.0.0/8 ::ffff:0:0/96"`` |A list of subnets, space separated. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``client.address`` |``ip address:port`` |``"[2001:db8::1]:1234"`` |Address of the DNS client. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``backend.address`` |``ip address:port`` |``"[2001:db8::2]:53"`` |Address of the downstream backend. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``backend.name`` |``string`` |``"my-backend"`` |Name of the downstream backend, if any. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``backend.protocol`` |``string`` |``"DoT"`` |Protocol used to send DNS queries to the downstream backend. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``destination.address`` |``ip address:port`` |``"192.0.2.42:53"`` |Destination address of a packet (may be more specific than ``frontend.address``, or be the next hop and differ from ``client.address`` when the proxy protocol is being used, for example).| ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.class`` |``DNS class`` |``"IN"`` |The DNS query class. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.id`` |``unsigned integer`` |``42`` |The DNS query ID. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.name`` |``DNS name`` |``"example.com"`` |The DNS query name. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.real_time_sec`` |``integer`` |``1767095038`` |The time this query was received, as the number of seconds since EPOCH. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.real_time_nsec`` |``integer`` |``23`` |Additional nano-seconds for ``dns.question.real_time_sec``, for extra precision. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.size`` |``unsigned integer`` |``42`` |Size of the DNS query packet, in bytes. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.question.type`` |``DNS type`` |``"AAAA"`` |The DNS query type. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.class`` |``DNS class`` |``"IN"`` |The DNS response class. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.id`` |``unsigned integer`` |``42`` |The DNS response ID. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.latency_us`` |``float`` |``180.63`` |The latency between the query and the response, in microseconds. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.name`` |``DNS name`` |``"example.com"`` |The DNS response name. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.rcode`` |``DNS Response Code``|``"No Error"`` |DNS response code | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.size`` |``unsigned integer`` |``42`` |Size of the DNS response packet, in bytes. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``dns.response.type`` |``DNS type`` |``"AAAA"`` |The DNS response type. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``frontend.address`` |``ip address:port`` |``"[::]:53"`` |Address this frontend is listening on. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``frontend.protocol`` |``string`` |``"DoQ"`` |Protocol this frontend is accepting DNS queries over. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``http.method`` |``string`` |``"GET"`` |HTTP/2 query method for DoH. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``http.path`` |``string`` |``"/dns-query"`` |HTTP/2 query path for DoH. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``http.response.status_code`` |``HTTP status code`` |``"200"`` |HTTP/2 response code for DoH. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``http.stream_id`` |``HTTP stream ID`` |``"4242"`` |HTTP/2 stream identifier for DoH. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``network.local.address`` |``string`` |``"192.0.2.2:53"`` |Local address and port DNSdist is listening on (webserver, console, ...). | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``network.peer.address`` |``string`` |``"192.0.2.2:53"`` |Peer address of the network connection (may differ from ``client.address`` when the proxy protocol is used, for example). | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``path`` |``filesystem path`` |``"tmp/api-dir/apizones"`` | | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``pool`` |``string`` |``"my-pool"`` |Downstream backends pool name. | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +|``protocol`` |``string`` |``"udp"`` | | ++--------------------------------+---------------------+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +The ``systemd-journal`` backend +------------------------------- + +The ``systemd-journal`` structured logging backend uses mostly the same keys and values as the default backend, with the exceptions: + +- keys are capitalized as required for ``systemd-journal``. +- ``msg`` is translated to ``MESSAGE``. +- ``prio`` is translated to ``PRIORITY``. +- ``ts`` is translated to ``TIMESTAMP``. +- If the original key is in a list of keys special to ``systemd-journal``, it is capitalized and prepended by ``PDNS_``. + The list of special keys is: message, message_id, priority, code_file, code_line, code_func, errno, invocation_id, user_invocation_id, syslog_facility, syslog_identifier, syslog_pid, syslog_timestamp, syslog_raw, documentation, tid, unit, user_unit, object_pid. + +To use this logging backend, add the ``--structured-logging-backend systemd-journal`` to the command line in the systemd unit file. + +To query the log, use a command similar to:: + + # journalctl -r -n 1 -o json-pretty -u dnsdist.service + +The ``json`` backend +-------------------- + +The ``json`` structured logging backend uses the same keys and values as the default backend. +An example of a log object:: + + {"level": "0", "msg": "Adding TCP worker thread to handle TCP connections from clients", "priority": "6", "ts": "2025-12-29T17:20:20+0100"} + +All values are represented as strings. + +The JSON log objects are written to the standard error stream. diff --git a/pdns/dnsdistdist/docs/reference/config.rst b/pdns/dnsdistdist/docs/reference/config.rst index 2c9ab63376..cf04952aaf 100644 --- a/pdns/dnsdistdist/docs/reference/config.rst +++ b/pdns/dnsdistdist/docs/reference/config.rst @@ -1403,9 +1403,11 @@ Status, Statistics and More .. versionadded:: 1.9.0 .. versionchanged:: 2.1.0 - The ``level_prefix`` option has no longer any effect because it was confusing. The log level is now always logged as ``level`` and the syslog priority, if any, as ``priority`` in all backends except the default one where it is named ``prio`` + The ``backend`` option has been added. + The ``levelPrefix`` option has no longer any effect because it was confusing. The log level is now always logged as ``level`` and the syslog priority, if any, as ``priority`` in all backends except the default one where it is named ``prio`` + + Set whether log messages should be in structured-logging format. This is turned off by default. See :doc:`../advanced/structured-logging-dictionary` for more details. - Set whether log messages should be in a structured-logging-like format. This is turned off by default. The resulting format looks like this (when timestamps are enabled via ``--log-timestamps`` and ``timeFormat="ISO8601"``):: ts="2023-11-06T12:04:58+0100" level="Info" msg="Added downstream server 127.0.0.1:53" @@ -1419,8 +1421,15 @@ Status, Statistics and More Options: - * ``levelPrefix=prefix``: string - Set the prefix for the log level. Default is ``prio``. Not supported since 2.1.0. + * ``backend``: string - The backend used for structured logging output, see below. Added in 2.1.0. * ``timeFormat=format``: string - Set the time format. Supported values are ``ISO8601`` and ``numeric``. Default is ``numeric``. + * ``levelPrefix=prefix``: string - Set the prefix for the log level. Default is ``prio``. No longer supported as of 2.1.0. + + Available backends: + + * ``default``: use the traditional logging system to output structured logging information. + * ``systemd-journal``: use ``systemd-journal``. When using this backend, provide ``-o verbose`` or simular output option to ``journalctl`` to view the full information. + * ``json``: JSON objects are written to the standard error stream. .. function:: setOpenTelemetryTracing(value)