]> git.ipfire.org Git - thirdparty/pdns.git/blob - pdns/recursordist/docs/upgrade.rst
rec: Update new b-root-server.net addresses in built-in hints.
[thirdparty/pdns.git] / pdns / recursordist / docs / upgrade.rst
1 Upgrade Guide
2 =============
3
4 Before upgrading, it is advised to read the :doc:`changelog/index`.
5 When upgrading several versions, please read **all** notes applying to the upgrade.
6
7 4.9.0 to 5.0.0 and master
8 --------------------------
9
10 YAML settings
11 ^^^^^^^^^^^^^
12 Starting with version 5.0.0-alpha1 the settings file(s) can be specified using YAML syntax.
13 The old-style settings files are still accepted but will be unsupported in a future release.
14 When a ``recursor.yml`` settings file is encountered it will be processed instead of a ``recursor.conf`` file.
15 Refer to :doc:`yamlsettings` for details and the :doc:`appendices/yamlconversion` guide for how to convert old-style settings to the new YAML format.
16
17 Rust
18 ^^^^
19 Some parts of the Recursor code are now written in Rust.
20 This has impact if you do local builds or are third-package maintainer.
21 According to `cargo msrv` the minimum version to compile the Rust code and its dependencies is 1.64.
22 Some distributions ship with an older Rust compiler, see `Rustup <https://rustup.rs/>`__ for a way to install a more recent one.
23 For our package builds, we install a Rust compiler from the ``Standalone`` section of `Other Rust Installation Methods <https://forge.rust-lang.org/infra/other-installation-methods.html>`__.
24
25 New settings
26 ^^^^^^^^^^^^
27 - The :ref:`setting-bypass-server-throttling-probability` setting has been introduced to try throttled servers once in a while.
28 - The :ref:`setting-tcp-threads` setting has been introduced to set the number of threads dedicated to processing incoming queries over TCP.
29 Previously either the distributor thread(s) or the general worker threads would process TCP queries.
30 - The :ref:`setting-qname-max-minimize-count` and :ref:`setting-qname-minimize-one-label` have been introduced to allow tuning of the parameters specified in :rfc:`9156`.
31
32 Changed settings
33 ^^^^^^^^^^^^^^^^
34 - The :ref:`setting-loglevel` can now be set to a level below 3 (error).
35
36 4.8.0 to 4.9.0
37 --------------
38
39 Metrics
40 ^^^^^^^
41 The way metrics are collected has been changed to increase performance, especially when many thread are used.
42 This allows for solving a long standing issue that some statistics were not updated on packet cache hits.
43 This is now resolved, but has the consequence that some metrics (in particular response related ones) changed behaviour as they now also reflect packet cache hits, while they did not before.
44 This affects the results shown by ``rec_control get-qtypelist`` and the ``response-by-qtype``, ``response-sizes`` and ``response-by-rcode`` items returned by the ``/api/v1/servers/localhost/statistics`` API endpoint.
45 Additionally, most ``RCodes`` and ``QTypes`` that are marked ``Unassigned``, ``Reserved`` or ``Obsolete`` by IANA are not accounted, to reduce the memory consumed by these metrics.
46
47 New settings
48 ^^^^^^^^^^^^
49 - The :ref:`setting-packetcache-negative-ttl` settings to control the TTL of negative (NxDomain or NoData) answers in the packet cache has been introduced.
50 - The :ref:`setting-stack-cache-size` setting to control the number of allocated mthread stacks has been introduced.
51 - The :ref:`setting-packetcache-shards` settings to control the number of shards in the packet cache has been introduced.
52 - The :ref:`setting-aggressive-cache-min-nsec3-hit-ratio` setting to control which NSEC3 records are stored in the aggressive NSEC cache has been introduced.
53 This setting can be used to switch off aggressive caching for NSEC3 only.
54 - The :ref:`setting-dnssec-disabled-algorithms` has been introduced to not use DNSSEC algorithms disabled by the platform's security policy.
55 This applies specifically to Red Hat Enterprise Linux 9 and derivatives.
56 The default value (automatically determine the algorithms that are disabled) should work for many cases.
57 - The setting ``includeSOA`` was added to the :func:`rpzPrimary` and :func:`rpzFile` Lua functions to include the SOA of the RPZ the responses modified by the RPZ.
58
59 Changed settings
60 ^^^^^^^^^^^^^^^^
61 The first two settings below have effect on the way the recursor distributes queries over threads.
62 In some cases, this can lead to imbalance of the number of queries process per thread.
63 See :doc:`performance`, in particular the :ref:`worker_imbalance` section.
64
65 - The :ref:`setting-pdns-distributes-queries` default has been changed to ``no``.
66 - The :ref:`setting-reuseport` default has been changed to ``yes``.
67 - The :ref:`setting-packetcache-ttl` default has been changed to 24 hours.
68 - The :ref:`setting-max-recursion-depth` default has been changed to 16. Before it was, 40, but effectively the CNAME length chain limit (fixed at 16) took precedence.
69 If you increase :ref:`setting-max-recursion-depth`, you also have to increase :ref:`setting-stack-size`.
70 A starting point of 5k per recursion depth is suggested. Add some extra safety margin to avoid running out of stack.
71 - The :ref:`setting-hint-file` setting gained a new special value to disable refreshing of root hints completely. See :ref:`handling-of-root-hints`.
72
73 :program:`rec_control`
74 ^^^^^^^^^^^^^^^^^^^^^^
75 The ``trace_regex`` subcommand has been changed to take a file argument.
76 Refer to :doc:`rec_control trace-regex <manpages/rec_control.1>` and :ref:`tracing` for details and example use.
77
78 4.8.1 to 4.8.2
79 --------------
80
81 Cache eviction policy
82 ^^^^^^^^^^^^^^^^^^^^^
83 The cache eviction policy for the record and the negative caches has been improved to reduce imbalance between shards.
84 The maximum size of the negative cache is now 1/8th of the size of the record cache and its number of shards is 1/8th of the :ref:`setting-record-cache-shards` setting.
85 Previously the size was 1/10th of the record cache size and the number of shards was equal to the
86 number of shards of the record cache.
87 The ``rec_control dump-cache`` command now prints more information about shards.
88
89
90 4.7.0 to 4.8.0
91 --------------
92
93 Structured logging
94 ^^^^^^^^^^^^^^^^^^
95 All logging (except query tracing) has been converted to structured logging.
96 Switch to old style logging by setting the :ref:`setting-structured-logging` setting to ``no``.
97 When using ``systemd``, structured logging information will be sent to ``journald`` using formatted text strings that list the key-value pairs and are human readable.
98 Switch to native key-value pair logging (more suitable for automated log processing) by setting :ref:`setting-structured-logging-backend` on the command line to ``systemd-journal``.
99
100 New settings
101 ^^^^^^^^^^^^
102 - The :ref:`setting-max-ns-per-resolve` setting to limit the number of NS records processed to resolve a name has been introduced.
103 - The :ref:`setting-serve-stale-extensions` setting to control the new ``Serve Stale`` feature has been introduced.
104 - The :ref:`setting-record-cache-locked-ttl-perc` setting to control locking of record sets in the record cache has been introduced.
105 - The :ref:`setting-edns-padding-out` setting to control EDNS padding for outgoing DoT has been introduced.
106 - The :ref:`setting-structured-logging-backend` setting to control the type of structured logging to ``journald`` has been introduced.
107
108 :program:`pdns_recursor` changes
109 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
110 THe ``--config`` command line option now implements the ``check``, ``default`` and ``diff`` keywords.
111
112 :program:`rec_control` changes
113 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
114 The ``dump-throttle`` and ``dump-edns`` subcommands no longer produces a table per thread, as the corresponding tables are now shared by all threads.
115 Additionally, the ``dump-edns`` command now only lists IPs that have a not OK status.
116 The ``dump-nsspeeds`` command has changed format to make it more readable and lists the last round trip time recorded for each address.
117 The ``get-proxymapping-stats`` and ``get-remotelogger-stats`` subcommands have been added.
118
119 4.7.2 to 4.7.3
120 --------------
121
122 New settings
123 ^^^^^^^^^^^^
124 - The :ref:`setting-max-ns-per-resolve` setting to limit the number of NS records processed to resolve a name has been introduced.
125
126 4.6.2 to 4.7.0
127 ---------------
128
129 Zone to Cache Changes
130 ^^^^^^^^^^^^^^^^^^^^^
131 The :ref:`ztc` feature now validates ``ZONEMD`` records. This means that zones containing invalid ``ZONEMD`` records will
132 be rejected by default, while previously the ``ZONEMD`` records would be ignored. For more detail, refer to :ref:`ztc`.
133
134 Asynchronous retrieval of ``AAAA`` records for nameservers
135 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
136 If ``IPv6`` is enabled for outgoing queries using :ref:`setting-query-local-address`, the :program:`Recursor` will schedule an asynchronous task to resolve ``IPv6`` addresses of nameservers it did not otherwise learn.
137 These addresses will then be used (in addition to ``IPv4`` addresses) for future queries to authoritative nameservers.
138 This has the consequence that authoritative nameservers will be contacted over ``IPv6`` in more case than before.
139
140 New Lua Configuration Functions
141 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
142 - The :func:`addAllowedAdditionalQType` ``Lua`` configuration function was added to make the :program:`Recursor` add additional records to answers for specific query types.
143 - The :func:`addProxyMapping` ``Lua`` configuration function was added to map source addresses to alternative addresses.
144
145 Post Resolve FFI Function
146 ^^^^^^^^^^^^^^^^^^^^^^^^^
147 A new :func:`postresolve_ffi` Lua callback function has been introduced.
148
149 New settings
150 ^^^^^^^^^^^^
151 - The :ref:`setting-save-parent-ns-set` setting has been introduced, enabling fallback cases if the parent ``NS`` set contains names not in the child ``NS`` set.
152 - The :ref:`setting-max-busy-dot-probes` settings has been introduced, enabling the :program:`Recursor` probe for ``DoT`` support of authoritative servers.
153 This is an experimental function, use with care.
154
155 :program:`rec_control` changes
156 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
157 The ``dump-nsspeeds``, ``dump-failedservers`` and ``dump-non-resolving`` subcommands no longer produce a table per thread, as the corresponding tables are now shared by all threads.
158 They also use a better readable and sortable timestamp format.
159
160 4.6.3 to 4.6.4
161 --------------
162
163 New settings
164 ^^^^^^^^^^^^
165 - The :ref:`setting-max-ns-per-resolve` setting to limit the number of NS records processed to resolve a name has been introduced.
166
167 4.6.1 to 4.6.2
168 --------------
169
170 Deprecated and changed settings
171 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
172 - The :ref:`setting-hint-file` gained a special value ``no`` to indicate that no hint file should be processed. The hint processing code is also made less verbose.
173
174 4.5.x to 4.6.1
175 --------------
176
177 Offensive language
178 ^^^^^^^^^^^^^^^^^^
179 Using the settings mentioned in :ref:`upgrade-offensive` now generates a warning. Please start using the new names.
180
181 File descriptor usage
182 ^^^^^^^^^^^^^^^^^^^^^
183 The number of file descriptors used by the Recursor has increased because the Recursor now keeps idle outgoing TCP/DoT connections open for a while.
184 The extra file descriptors used in comparison to previous versions of the Recursor is :ref:`setting-tcp-out-max-idle-per-thread` times the number of worker threads (:ref:`setting-threads`).
185
186 New settings
187 ^^^^^^^^^^^^
188 - The :ref:`setting-dot-to-auth-names` setting to list nameservers that should be contacted over DoT has been introduced.
189 - The :ref:`setting-dot-to-port-853` setting to specify that nameservers or forwarders using port 853 should be contacted over DoT has been introduced.
190 - The :ref:`setting-ignore-unknown-settings` setting has been introduced to make it easier to switch between recursor versions supporting different settings.
191 - The :ref:`setting-webserver-hash-plaintext-credentials` has been introduced to avoid keeping cleartext sensitive information in memory.
192 - The :ref:`setting-tcp-out-max-idle-ms`, :ref:`setting-tcp-out-max-idle-per-auth`, :ref:`setting-tcp-out-max-queries` and :ref:`setting-tcp-out-max-idle-per-thread` settings have been introduced to control the new TCP/DoT outgoing connections pooling. This mechanism keeps connections to authoritative servers or forwarders open for later re-use.
193 - The :ref:`setting-structured-logging` setting has been introduced to prefer structured logging (the default) when both an old style and a structured log messages is available.
194 - The :ref:`setting-max-include-depth` setting has been introduced to limit the number of nested ``$include`` directives while processing a zone file.
195 - The :ref:`setting-allow-notify-for`, :ref:`setting-allow-notify-for-file`, :ref:`setting-allow-notify-from` and :ref:`setting-allow-notify-from-file` settings have been introduced, allowing incoming notify queries to clear cache entries.
196
197 Deprecated and changed settings
198 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
199 - The :ref:`setting-api-key` and :ref:`setting-webserver-password` settings now accept a hashed and salted version (if the support is available in the openssl library used).
200
201 Privileged port binding in Docker
202 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
203
204 In our Docker image, our binaries are no longer granted the ``net_bind_service`` capability, as this is unnecessary in many deployments.
205 For more information, see the section `"Privileged ports" in Docker-README <https://github.com/PowerDNS/pdns/blob/master/Docker-README.md#privileged-ports>`__.
206
207 4.5.10 to 4.5.11
208 ----------------
209
210 New settings
211 ^^^^^^^^^^^^
212 - The :ref:`setting-max-ns-per-resolve` setting to limit the number of NS records processed to resolve a name has been introduced.
213
214 4.5.1 to 4.5.2
215 --------------
216
217 Deprecated and changed settings
218 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
219 - The :ref:`setting-nsec3-max-iterations` default value has been changed from 2500 to 150.
220
221 4.4.x to 4.5.1
222 --------------
223
224 .. _upgrade-offensive:
225
226 Offensive language
227 ^^^^^^^^^^^^^^^^^^
228 Synonyms for various settings names containing ``master``, ``slave``,
229 ``whitelist`` and ``blacklist`` have been introduced.
230
231 - For :ref:`setting-stats-api-blacklist` use :ref:`setting-stats-api-disabled-list`.
232 - For :ref:`setting-stats-carbon-blacklist` use :ref:`setting-stats-carbon-disabled-list`.
233 - For :ref:`setting-stats-rec-control-blacklist` use :ref:`setting-stats-rec-control-disabled-list`.
234 - For :ref:`setting-stats-snmp-blacklist` use :ref:`setting-stats-snmp-disabled-list`.
235 - For :ref:`setting-edns-subnet-whitelist` use :ref:`setting-edns-subnet-allow-list`.
236 - For :ref:`setting-new-domain-whitelist` use :ref:`setting-new-domain-ignore-list`.
237 - For :ref:`setting-snmp-master-socket` use :ref:`setting-snmp-daemon-socket`.
238 - For the LUA config function :func:`rpzMaster` use :func:`rpzPrimary`.
239
240 Currently, the older setting names are also accepted and used.
241 The next release will start deprecating them.
242 Users are advised to start using the new names to avoid future
243 trouble.
244
245 Special domains
246 ^^^^^^^^^^^^^^^
247 Queries for all names in the ``.localhost`` domain will answer in accordance with :rfc:`6761` section 6.3 point 4.
248 That means that they will be answered with ``127.0.0.1``, ``::1`` or a negative response.
249
250 :program:`rec_control` command writing to a file
251 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
252 For the commands that write to a file, the file to be dumped to is now opened by the :program:`rec_control` command itself using the credentials and the current working directory of the user running :program:`rec_control`.
253 A single minus *-* can be used as a filename to write the data to the standard output stream.
254 Previously the file was opened by the recursor, possibly in its chroot environment.
255
256 New settings
257 ^^^^^^^^^^^^
258 - The :ref:`setting-extended-resolution-errors` setting has been added, enabling adding EDNS Extended Errors to responses.
259 - The :ref:`setting-refresh-on-ttl-perc` setting has been added, enabling an automatic cache-refresh mechanism.
260 - The :ref:`setting-ecs-ipv4-never-cache` and :ref:`setting-ecs-ipv6-never-cache` settings have been added, allowing an overrule of the existing decision whether to cache EDNS responses carrying subnet information.
261 - The :ref:`setting-aggressive-nsec-cache-size` setting has been added, enabling the functionality described in :rfc:`8198`.
262 - The :ref:`setting-x-dnssec-names` setting has been added, allowing DNSSEC metrics to be recorded in a different set of counter for given domains.
263 - The :ref:`setting-non-resolving-ns-max-fails` and :ref:`setting-non-resolving-ns-throttle-time` settings have been added, allowing the control of the cache of nameservers failing to resolve.
264 - The :ref:`setting-edns-padding-from` and :ref:`setting-edns-padding-mode` and :ref:`setting-edns-padding-tag` settings have been added, to control how padding is applied to answers sent to clients.
265 - The :ref:`setting-tcp-fast-open-connect` setting has been added, it enables TCP Fast Connect for outgoing connections. Please read :ref:`tcp-fast-open-support` before enabling this feature.
266
267 Deprecated and changed settings
268 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
269 - The :ref:`setting-minimum-ttl-override` and :ref:`setting-ecs-minimum-ttl-override` defaults have ben changed from 0 to 1.
270 - The :ref:`setting-spoof-nearmiss-max` default has been changed from 20 to 1.
271 - The :ref:`setting-dnssec` default has changed from ``process-no-validate`` to ``process``.
272 - The meaning of the :ref:`setting-max-packetcache-entries` has changed: previously there was one packet cache instance per worker thread.
273 Since queries incoming over TCP are now also using the packet cache, there is now also one packet cache instance per distributor thread.
274 Each cache instance has a size of :ref:`setting-max-packetcache-entries` divided by (:ref:`setting-threads` + :ref:`setting-distributor-threads`).
275
276 Removed settings
277 ^^^^^^^^^^^^^^^^
278 - The ``query-local-address6`` setting has been removed. It already was deprecated.
279
280 4.3.x to 4.4.0
281 --------------
282
283 Response Policy Zones (RPZ)
284 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
285
286 To conform better to the standard, RPZ processing has been modified.
287 This has consequences for the points in the resolving process where matches are checked and callbacks are called.
288 See :ref:`rpz` for details. Additionally a new type of callback has been introduced: :func:`policyEventFilter`.
289
290 Dropping queries from Lua callbacks
291 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
292 The method to drop a query from a Lua callback has been changed.
293 Previously, you could set `rcode` to `pdns.DROP`. See :ref:`hook-semantics` for the new method.
294
295 Parsing of unknown record types
296 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
297 The parsing (from zone files) of unknown records types (of the form
298 ``\# <length> <hex data>``) has been made more strict. Previously, invalid formatted records could produce
299 inconsistent results.
300
301 Deprecated and changed settings
302 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
303 - The :ref:`setting-query-local-address` setting has been modified to be able to include both IPv4 and IPv6 addresses.
304 - The ``query-local-address6`` setting is now deprecated.
305
306 New settings
307 ^^^^^^^^^^^^
308 - The :ref:`setting-dns64-prefix` setting has been added, enabling common cases of DNS64 handling without having to write Lua code.
309 - The :ref:`setting-proxy-protocol-from` and :ref:`setting-proxy-protocol-maximum-size` settings have been added to allow for passing of Proxy Protocol Version 2 headers between a client and the recursor.
310 - The :ref:`setting-record-cache-shards` setting has been added, enabling the administrator to change the number of shards in the records cache. The value of the metric ``record-cache-contended`` divided by ``record-cache-acquired`` indicates if the record cache locks are contended. If so, increasing the number of shards can help reducing the contention.
311
312 4.2.x to 4.3.0
313 ------------------------
314
315 Lua Netmask class methods changed
316 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
317 - Netmask class methods ``isIpv4`` and ``isIpv6`` have been deprecated in Lua, use :func:`Netmask.isIPv4` and :func:`Netmask.isIPv6` instead. In C++ API these methods have been removed.
318
319 ``socket-dir`` changed
320 ^^^^^^^^^^^^^^^^^^^^^^
321 The default :ref:`setting-socket-dir` has changed to include ``pdns-recursor`` in the path.
322 For non-chrooted setups, it is now whatever is passed to ``--with-socketdir`` during configure (``/var/run`` by default) plus ``pdns-recursor``.
323 The systemd unit-file is updated to reflect this change and systemd will automatically create the directory with the proper permissions.
324 The packaged sysV init-script also creates this directory.
325 For other operating systems, update your init-scripts accordingly.
326
327 Systemd service and permissions
328 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
329 The systemd service-file that is installed no longer uses the ``root`` user to start.
330 It uses the user and group set with the ``--with-service-user`` and ``--with-service-group`` switches during
331 configuration, "pdns" on Debian and "pdns-recursor" on CentOS by default.
332 This could mean that PowerDNS Recursor cannot read its configuration, lua scripts, auth-zones or other data.
333 It is recommended to recursively ``chown`` directories used by PowerDNS Recursor::
334
335 # For Debian-based systems
336 chown -R root:pdns /etc/powerdns
337
338 # For CentOS and RHEL based systems
339 chown -R root:pdns-recursor /etc/pdns-recursor
340
341 Packages provided on `the PowerDNS Repository <https://repo.powerdns.com>`__ will ``chown`` directories created by them accordingly in the post-installation steps.
342
343 New settings
344 ^^^^^^^^^^^^
345 - The :ref:`setting-allow-trust-anchor-query` setting has been added. This setting controls if negative trust anchors can be queried. The default is `no`.
346 - The :ref:`setting-max-concurrent-requests-per-tcp-connection` has been added. This setting controls how many requests are handled concurrently per incoming TCP connection. The default is 10.
347 - The :ref:`setting-max-generate-steps` setting has been added. This sets the maximum number of steps that will be performed when loading a BIND zone with the ``$GENERATE`` directive. The default is 0, which is unlimited.
348 - The :ref:`setting-nothing-below-nxdomain` setting has been added. This setting controls the way cached NXDOMAIN replies imply non-existence of a whole subtree. The default is `dnssec` which means that only DNSSEC validated NXDOMAINS results are used.
349 - The :ref:`setting-qname-minimization` setting has been added. This options controls if QName Minimization is used. The default is `yes`.
350
351 4.1.x to 4.2.0
352 --------------
353
354 Two new settings have been added:
355
356 - ``xpf-allow-from`` can contain a list of IP addresses ranges from which `XPF (X-Proxied-For) <https://datatracker.ietf.org/doc/draft-bellis-dnsop-xpf/>`_ records will be trusted.
357 - ``setting-xpf-rr-code`` should list the number of the XPF record to use (in lieu of an assigned code).
358
359 4.0.x to 4.1.0
360 --------------
361
362 :ref:`setting-loglevel` defaulted to 4 but was always overridden to 6 during
363 the startup. The issue has been fixed and the default value set to 6 to keep the behavior
364 consistent.
365
366 The ``--with-libsodium`` configure flag has changed from 'no' to 'auto'.
367 This means that if libsodium and its development header are installed, it will be linked in.
368
369 4.0.3 to 4.0.4
370 --------------
371
372 One setting has been added to limit the risk of overflowing the stack:
373
374 - :ref:`setting-max-recursion-depth`: defaults to 40 and was unlimited before
375
376 4.0.0 to 4.0.1
377 --------------
378
379 Two settings have changed defaults, these new defaults decrease CPU usage:
380
381 - :ref:`setting-root-nx-trust` changed from "no" to "yes"
382 - :ref:`setting-log-common-errors` changed from "yes" to "no"