]>
git.ipfire.org Git - thirdparty/pdns.git/log
Otto Moerbeek [Mon, 19 Dec 2022 15:06:33 +0000 (16:06 +0100)]
Merge pull request #12260 from omoerbeek/webserver-json-utf8
auth: Properly encode json strings containing binary data
Peter van Dijk [Mon, 19 Dec 2022 13:08:32 +0000 (14:08 +0100)]
Merge pull request #12284 from jsoref/github-output
Switch from set-output to GITHUB_OUTPUT
Otto Moerbeek [Mon, 19 Dec 2022 10:27:53 +0000 (11:27 +0100)]
Merge pull request #12337 from omoerbeek/rec-tcounter-test-tweaks
rec: tcounter test tweaks
Otto Moerbeek [Mon, 19 Dec 2022 09:55:57 +0000 (10:55 +0100)]
Update pdns/recursordist/testrunner.cc
Co-authored-by: Remi Gacogne <github@coredump.fr>
Otto Moerbeek [Mon, 19 Dec 2022 08:08:14 +0000 (09:08 +0100)]
Some systems have a low-resolution nanosleep(2), calling it will
sleep for at least a few ms. Compensate for that by running fewer
loops with longer sleeps.
Also use dns_random and make sure it is initlized properly for all tests.
Otto Moerbeek [Sat, 17 Dec 2022 07:46:34 +0000 (08:46 +0100)]
Merge pull request #12309 from omoerbeek/test-anybind
Test ANY bind UDP handling for the v4 case
Otto Moerbeek [Fri, 16 Dec 2022 10:16:05 +0000 (11:16 +0100)]
Merge pull request #12333 from omoerbeek/rec-sockbufsize-log
Setting socket buf size: not decreasing is not an error
Peter van Dijk [Fri, 16 Dec 2022 10:01:14 +0000 (11:01 +0100)]
Merge pull request #12322 from mind04/auth-lmdb-tsig
auth: lmdb, fix TSIG key removal
Peter van Dijk [Fri, 16 Dec 2022 09:53:41 +0000 (10:53 +0100)]
Merge pull request #12325 from Habbie/remove-toysdig
remove toysdig
Peter van Dijk [Fri, 16 Dec 2022 08:36:52 +0000 (09:36 +0100)]
pdnsutil and testrunner need validate.hh, which the previous commit removed from dist
Otto Moerbeek [Fri, 16 Dec 2022 08:24:44 +0000 (09:24 +0100)]
For setting socket buf size not decreasing is not an error
Otto Moerbeek [Fri, 16 Dec 2022 07:52:30 +0000 (08:52 +0100)]
Merge pull request #12323 from omoerbeek/rec-tcounter-responsestats
rec: make response stats a tcounter object
Remi Gacogne [Thu, 15 Dec 2022 08:39:28 +0000 (09:39 +0100)]
Merge pull request #12327 from rgacogne/ddist-fix-tcp-only-checktimeout-ms
dnsdist: Fix the health-check timeout computation for DoH backend
Remi Gacogne [Wed, 14 Dec 2022 16:03:20 +0000 (17:03 +0100)]
Merge pull request #12328 from rgacogne/ddist-stop-responders-faster
dnsdist: Stop the responders more quickly during the tests
Remi Gacogne [Wed, 14 Dec 2022 15:19:49 +0000 (16:19 +0100)]
dnsdist: Stop the responders more quickly during the tests
We use `SO_REUSEPORT` in these tests so if the old responder is
still around when the next test starts, it is quite likely that
it might get one of the new queries. This is usually fine because
responders with a different behaviour listen on different ports,
but if a query is queued to an old responder socket right during
the time that responder is checking whether it should stop and
the actual exit, the query will be lost.
Remi Gacogne [Wed, 14 Dec 2022 15:10:49 +0000 (16:10 +0100)]
dnsdist: Fix the health-check timeout computation for DoH backend
The remaining milliseconds after handling the full seconds was not
properly converted to microseconds.
Remi Gacogne [Wed, 14 Dec 2022 14:45:18 +0000 (15:45 +0100)]
Merge pull request #12326 from omoerbeek/dnsdist-regr-wait-responders
dnsdist: Wait for TCP responder to become active before starting dnsdist
Peter van Dijk [Wed, 14 Dec 2022 10:29:26 +0000 (11:29 +0100)]
remove toysdig
Otto Moerbeek [Wed, 14 Dec 2022 09:55:28 +0000 (10:55 +0100)]
Wait for TCP responder to become active before starting dnsdist
Otto Moerbeek [Tue, 13 Dec 2022 11:25:12 +0000 (12:25 +0100)]
rec: make response stats a tcounter object
This allows for the packet cache hit path to record response stats without performance impact.
The qtype and rcode counters are capped, as i ran into trouble with
the thread stack sizes on macOS and OpenBSD. See the source comment
for explanation.
Closes #11534
Kees Monshouwer [Tue, 13 Dec 2022 22:25:27 +0000 (23:25 +0100)]
auth: lmdb, fix TSIG key removal
Otto Moerbeek [Tue, 13 Dec 2022 11:25:59 +0000 (12:25 +0100)]
Merge pull request #12319 from omoerbeek/dnsdist-rlim_t
dnsdist: Use rlim_t for limit
Otto Moerbeek [Tue, 13 Dec 2022 10:14:08 +0000 (11:14 +0100)]
Use rlim_t for limit
Otto Moerbeek [Tue, 13 Dec 2022 10:12:11 +0000 (11:12 +0100)]
Merge pull request #12318 from omoerbeek/rec-reorg
Rec reorg: move recursor specific files to recursordist
Otto Moerbeek [Tue, 13 Dec 2022 08:30:46 +0000 (09:30 +0100)]
Step 3: reformat moved files previously not formatted
Otto Moerbeek [Tue, 13 Dec 2022 08:22:42 +0000 (09:22 +0100)]
Step 2: mv rec specific files to recursordist
Otto Moerbeek [Tue, 13 Dec 2022 08:21:40 +0000 (09:21 +0100)]
Step one: remove symlinks to rec-specific files
Otto Moerbeek [Tue, 13 Dec 2022 06:36:12 +0000 (07:36 +0100)]
Merge pull request #12193 from omoerbeek/rec-tcounters
Introducing TCounters
Otto Moerbeek [Tue, 13 Dec 2022 06:29:32 +0000 (07:29 +0100)]
Merge pull request #12317 from omoerbeek/rec-fix-freebsd
rec: Fix compilation on FreeBSD
Otto Moerbeek [Mon, 12 Dec 2022 19:03:04 +0000 (20:03 +0100)]
Fix compilation on FreeBSD. reported by HellSpawn
Otto Moerbeek [Mon, 12 Dec 2022 18:59:40 +0000 (19:59 +0100)]
Two Bucket fields can be const if we take care in the asssignment op.
Otto Moerbeek [Mon, 12 Dec 2022 17:10:33 +0000 (18:10 +0100)]
Typo in comment
Co-authored-by: Remi Gacogne <github@coredump.fr>
Otto Moerbeek [Mon, 12 Dec 2022 17:09:48 +0000 (18:09 +0100)]
Apply suggestions from code review
Co-authored-by: Remi Gacogne <github@coredump.fr>
Remi Gacogne [Mon, 12 Dec 2022 15:26:11 +0000 (16:26 +0100)]
Merge pull request #12316 from rgacogne/ddist-disable-tsan-send-wrappers
dnsdist: Disable the send wrappers in our CI
Remi Gacogne [Mon, 12 Dec 2022 14:42:57 +0000 (15:42 +0100)]
dnsdist: Disable the send wrappers in our CI
The way the send wrappers are implemented, reading the data _after_
it has been sent, cause them to report a data race that does not
exist with existing implementations:
- we call `send()` from thread 1 to send a query to a backend, never
touching the data or associated metadata again from that thread
- we get a response from the backend in a different thread, thread 2,
which will then access the metadata and sometimes (truncated UDP
answers following a DoH query) even modify the data itself
- ASAN and TSAN complain because the wrapper might still be reading
the data after the UDP datagram has been sent, which is effectively
a race, but it does not really make any sense for an actual
implementation of `send()` to do that.
We work around that by disabling the `send()` wrappers in our CI,
for the dnsdist regression tests only, via `intercept_send=0`.
Otto Moerbeek [Mon, 12 Dec 2022 12:39:38 +0000 (13:39 +0100)]
Merge pull request #12308 from omoerbeek/rec-prep-4.8.0
Prep for rec-4.8.0 final release
Otto Moerbeek [Mon, 12 Dec 2022 11:52:34 +0000 (12:52 +0100)]
Apply suggestions from code review
Co-authored-by: Peter van Dijk <peter.van.dijk@powerdns.com>
Otto Moerbeek [Tue, 6 Dec 2022 10:18:56 +0000 (11:18 +0100)]
Close a race: some test immediately query metrics when a query result is received.
to avoid ordering issues, update metrics snap before answers are sent out.
Otto Moerbeek [Tue, 6 Dec 2022 09:09:41 +0000 (10:09 +0100)]
Since we now register stats in the main startup code, no need to do
it per-thread
Otto Moerbeek [Tue, 8 Nov 2022 10:10:54 +0000 (11:10 +0100)]
Introducing TCounters
This is a mostly lockless (and not using atomics) way to keep track of
counters and other metrics.
Atomic value are more expensive than you would think (especially if
your platform has no native atomic support for your data type), and
using locking all the time for often updated counters is very
expensive as well.
The idea for `TCounters` is based on
https://github.com/ahupowerdns/mcounter
But addresses the issues raised in
https://github.com/ahupowerdns/mcounter/issues/3
Templates are used, the application has to provide a specific class to
hold the values and enums to index these values. The application
specific class also has to provide a `merge()` method to merge two
instances of the application specific data. For counters that is
simple: just add them. Averages (or histogrfam) requires a bit more
work. This is demonstrated in `rec-tcounters.{cc,hh}`
At the end of a body of work the application's threads should call the
`updateSnap()` function. If a certain amount of time has passed since
the last time, a thread local snapshot of the thread local data will
be created in a thread-safe way.
The class that collects the aggregated values reads (also in a thread
safe way) from the snapshot values in each thread.
Updates of individual counters are done on thread-local data,
potentially many times per second. The snaps contain a consistent set
of the values and are taken by default once per 100ms, so reletively
seldom.
By using the snap mnechanism the aggragate values computed are based
on internally consistent counter values (as long as related counters
are updated from the same thread). A (small) drawback is that the
values computed might be a bit out of date.
The snapshot approach was suggested by @wojas.
This PR de demonstrates `TCounters` for a few Recursor metrics: simple
counters and double typed average values. For the latter weights are
kept, so that the average of averages can be computed in a proper way.
Otto Moerbeek [Mon, 12 Dec 2022 08:18:29 +0000 (09:18 +0100)]
Mention dnstapNODFrameStreamServer
Remi Gacogne [Mon, 12 Dec 2022 09:16:39 +0000 (10:16 +0100)]
Merge pull request #12315 from rgacogne/spell-check-fixes
spell-check: Make the spell-checker happy
Remi Gacogne [Mon, 12 Dec 2022 09:11:43 +0000 (10:11 +0100)]
spell-check: Make the spell-checker happy
Remi Gacogne [Fri, 9 Dec 2022 15:30:43 +0000 (16:30 +0100)]
Merge pull request #12311 from omoerbeek/dnsdist-docs-top
dnsdist: List default of top argument of topX functions
Otto Moerbeek [Fri, 9 Dec 2022 15:26:09 +0000 (16:26 +0100)]
List default of top argument of topX functions
Remi Gacogne [Fri, 9 Dec 2022 14:43:33 +0000 (15:43 +0100)]
Merge pull request #12280 from rgacogne/ddist-cache-inserted-rules
dnsdist: Add a new chain of rules triggered after cache insertion
Otto Moerbeek [Fri, 9 Dec 2022 12:44:52 +0000 (13:44 +0100)]
Test ANY bind UDP handling for the v4 case
v6 is more work, as the test client code assumes v4
Tested by neutering IsAnyAddress handling of auth and rec and verifying the tests fail.
Fixes #3965 (at least th v4 part).
Otto Moerbeek [Fri, 9 Dec 2022 11:03:59 +0000 (12:03 +0100)]
Merge pull request #12307 from omoerbeek/update-builder
Update builder to work better on MacOS
Otto Moerbeek [Fri, 9 Dec 2022 10:57:28 +0000 (11:57 +0100)]
Upgrade guide additions for 4.8.0
Otto Moerbeek [Thu, 8 Dec 2022 10:02:20 +0000 (11:02 +0100)]
Prep for rec-4.8.0 final release
Peter van Dijk [Fri, 9 Dec 2022 10:09:57 +0000 (11:09 +0100)]
Merge pull request #12305 from Habbie/auth-docs-4.5.5-4.6.4-4.7.3
auth 4.5..5 / 4.6.4 / 4.7.3: changelog & secpoll
Remi Gacogne [Fri, 9 Dec 2022 09:19:17 +0000 (10:19 +0100)]
Merge pull request #12306 from rgacogne/ddist-docs-checkclass-typo
dnsdist: Fix a typo in the Healthcheck configuration guide
Otto Moerbeek [Fri, 9 Dec 2022 09:10:59 +0000 (10:10 +0100)]
Update builder to work better on MacOS
Remi Gacogne [Fri, 9 Dec 2022 08:47:27 +0000 (09:47 +0100)]
dnsdist: Fix a typo in the Healthcheck configuration guide
As reported by Kai Stian Olstad (thanks!).
Peter van Dijk [Fri, 9 Dec 2022 08:33:06 +0000 (09:33 +0100)]
Merge pull request #12297 from chbruyand/auth-ifurlup-byteslimit
minicurl: fix missing CURLOPT_XFERINFOFUNCTION on old curl versions
Peter van Dijk [Fri, 9 Dec 2022 08:27:42 +0000 (09:27 +0100)]
auth 4.5..5 / 4.6.4 / 4.7.3: changelog & secpoll
Otto Moerbeek [Thu, 8 Dec 2022 14:25:35 +0000 (15:25 +0100)]
Merge pull request #12302 from omoerbeek/dnsdist-docs-implicit
dnsdist: Document what happens to a packet not handled by any action
Otto Moerbeek [Thu, 8 Dec 2022 13:45:29 +0000 (14:45 +0100)]
dnsdist: Document what happens to a packet not handled by any action
Otto Moerbeek [Thu, 8 Dec 2022 12:59:41 +0000 (13:59 +0100)]
Merge pull request #12301 from omoerbeek/rec-docs-preoutquery
rec: Document preoutquery limitations
Otto Moerbeek [Thu, 8 Dec 2022 12:14:03 +0000 (13:14 +0100)]
rec: Document preoutquery limitations
Fixes #10247
Charles-Henri Bruyand [Wed, 7 Dec 2022 14:26:15 +0000 (15:26 +0100)]
minicurl: fix missing CURLOPT_XFERINFOFUNCTION on old curl versions
Otto Moerbeek [Thu, 8 Dec 2022 09:42:38 +0000 (10:42 +0100)]
Merge pull request #12203 from sspans/patch-3
rec: Allow both A and AAAA when importing /etc/hosts
Peter van Dijk [Thu, 8 Dec 2022 09:12:20 +0000 (10:12 +0100)]
Merge pull request #12282 from Habbie/lmdb-notify
auth lmdb: make outgoing notifications work
Otto Moerbeek [Fri, 2 Dec 2022 08:16:55 +0000 (09:16 +0100)]
Properly encode json string containing binary data
The existing code assumes the strings are alreayd valid UTF8 and contain potential out-of-bound accesses.
Also urlEncode path in log lines, as it trips pytest.xml:
Running tests...
$ 'pytest' '--junitxml=pytest.xml' '-v'
==STDOUT===
==STDERRR===
File "/home/otto/pdns/regression-tests.api/runtests.py", line 304, in <module>
print(serverproc.stderr.read())
File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 4304: invalid continuation byte
There might be more places where this is needed.
Otto Moerbeek [Thu, 8 Dec 2022 08:55:34 +0000 (09:55 +0100)]
Merge pull request #11554 from yog-singh/yog-singh/ddist-ebpf-memlock-limit
dnsdist: Raise RLIMIT_MEMLOCK automatically when eBPF is requested
Otto Moerbeek [Wed, 7 Dec 2022 09:51:04 +0000 (10:51 +0100)]
Take searchsuffix into acount when generating targets of PTR records.
Fix tests for that and also add a testcode for boths v4 and v6 localhost
Otto Moerbeek [Wed, 30 Nov 2022 09:45:33 +0000 (10:45 +0100)]
Avoid too many lookups by using iterator
Sten Spans [Wed, 16 Nov 2022 21:46:21 +0000 (22:46 +0100)]
Allow multiple records with same name for etc/hosts processing
This seems to do something right at least
Remi Gacogne [Thu, 8 Dec 2022 08:43:42 +0000 (09:43 +0100)]
dnsdist: Add a few missing bindings and docs for the cache-inserted rules
Remi Gacogne [Thu, 8 Dec 2022 08:33:10 +0000 (09:33 +0100)]
Merge pull request #12281 from rgacogne/ddist-optim-cross
dnsdist: Get rid of TCPCrossProtocolQuerySender
Peter van Dijk [Tue, 6 Dec 2022 17:56:58 +0000 (18:56 +0100)]
lmdb tests: actually use lmdb config
Peter van Dijk [Tue, 6 Dec 2022 16:42:27 +0000 (17:42 +0100)]
auth lmdb: make outgoing notifications work
Peter van Dijk [Wed, 7 Dec 2022 18:34:31 +0000 (19:34 +0100)]
Merge pull request #12298 from Habbie/tinydns-data-fix2
actually fix tinydns data
Remi Gacogne [Wed, 7 Dec 2022 15:37:55 +0000 (16:37 +0100)]
Merge pull request #12274 from rgacogne/ddist-debug-snmp
dnsdist: Add logs to investigate the SNMP regression tests failure
Peter van Dijk [Wed, 7 Dec 2022 15:00:02 +0000 (16:00 +0100)]
actually fix tinydns data
Otto Moerbeek [Wed, 7 Dec 2022 14:31:52 +0000 (15:31 +0100)]
Merge pull request #12290 from omoerbeek/mincurl-coverity
Coverity
1501408 : Uninitialized scalar field
Otto Moerbeek [Wed, 7 Dec 2022 13:33:56 +0000 (14:33 +0100)]
Merge pull request #12289 from omoerbeek/rec-unsupported-qtype
rec: refactor unsuppored qtype code and make sure we ServFail on all unsupported qtypes
Peter van Dijk [Wed, 7 Dec 2022 13:15:44 +0000 (14:15 +0100)]
Merge pull request #12285 from mind04/auth-api-consumer
auth: api, do not create SOA and NS records for consumer zones
Otto Moerbeek [Wed, 7 Dec 2022 13:09:56 +0000 (14:09 +0100)]
Only raise the limit to 1M if the current is lower than 1M
Otto Moerbeek [Wed, 7 Dec 2022 12:37:13 +0000 (13:37 +0100)]
Elaborate on NSEC/NSEC3 diffference in comment
Otto Moerbeek [Wed, 7 Dec 2022 12:30:19 +0000 (13:30 +0100)]
Apply suggestions from code review
Co-authored-by: Peter van Dijk <peter.van.dijk@powerdns.com>
Remi Gacogne [Wed, 7 Dec 2022 12:15:54 +0000 (13:15 +0100)]
Merge pull request #12288 from rgacogne/fix-spelling
Fix a typo and allow 'byteslimit'
Remi Gacogne [Wed, 7 Dec 2022 12:14:32 +0000 (13:14 +0100)]
Merge pull request #11065 from pieterlexis/sd-protectproc
service files: Add more sandboxing options
Peter van Dijk [Wed, 7 Dec 2022 11:05:17 +0000 (12:05 +0100)]
Merge pull request #12185 from PenelopeFudd/master
Enhancing dnsupdate documentation
Otto Moerbeek [Wed, 7 Dec 2022 10:54:01 +0000 (11:54 +0100)]
Coverity
1501408 : Uninitialized scalar field
Peter van Dijk [Wed, 7 Dec 2022 10:21:56 +0000 (11:21 +0100)]
Merge pull request #12216 from kpfleming/catalog-zone-doc-clarifications
Small clarifications to docs for API usage with catalog zones.
Otto Moerbeek [Wed, 7 Dec 2022 09:09:25 +0000 (10:09 +0100)]
Incorporate comments from @rgacogne
Otto Moerbeek [Wed, 7 Dec 2022 09:54:49 +0000 (10:54 +0100)]
rec: refactor unsuppored qtype code and make sure we ServFail on all unsupported qtypes
This fixes #12251
Also I'd like to know why we ServFail on NSEC3 but not on NSEC: we should either fix that or add a comment explaining this.
Remi Gacogne [Wed, 7 Dec 2022 09:05:20 +0000 (10:05 +0100)]
rec: Fix a typo in the doc
Remi Gacogne [Wed, 7 Dec 2022 09:04:50 +0000 (10:04 +0100)]
spell-check: Allow 'byteslimit' (name of a parameter to a Lua function)
Peter van Dijk [Wed, 7 Dec 2022 09:03:23 +0000 (10:03 +0100)]
Merge pull request #12287 from Habbie/fix-tinydns-data
auth: fix tinydns data, missed this spot in #12279
Remi Gacogne [Tue, 29 Nov 2022 15:10:57 +0000 (16:10 +0100)]
systemd service: Only enable MemoryDenyWriteExecute for ixfrdist
Because it does not play well with LuaJIT, which all other products
use.
Pieter Lexis [Fri, 3 Dec 2021 12:08:09 +0000 (13:08 +0100)]
systemd service: disallow access to devices (except, zero, full, null, random, urandom)
Pieter Lexis [Fri, 3 Dec 2021 12:04:57 +0000 (13:04 +0100)]
systemd service: lock down IPC
Pieter Lexis [Fri, 3 Dec 2021 10:01:00 +0000 (11:01 +0100)]
service files: Add MemoryDenyWriteExecute
This disallows the services to write executable memory.
Pieter Lexis [Fri, 3 Dec 2021 09:37:46 +0000 (10:37 +0100)]
service files: Add ProtectProc
Another sandboxing option,
[ProtectProc](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#ProtectProc=)
hides all /proc/<pid> that are not owned by the service user and hides
some kernel things from /proc as well.
Remi Gacogne [Wed, 7 Dec 2022 08:56:37 +0000 (09:56 +0100)]
dnsdist: Rename the field containing the TCP worker thread ID
Remi Gacogne [Wed, 7 Dec 2022 08:42:25 +0000 (09:42 +0100)]
Merge pull request #12248 from kpfleming/issue-11153
systemd: Add "After" dependency on time-sync.target
Remi Gacogne [Wed, 7 Dec 2022 08:33:31 +0000 (09:33 +0100)]
Merge pull request #12237 from rgacogne/ddist-unscrew-resumption-ossl3
dnsdist: Ignore unclean TLS session shutdown
Remi Gacogne [Wed, 7 Dec 2022 08:33:23 +0000 (09:33 +0100)]
Merge pull request #12283 from rgacogne/ddist-fix-long-double-warning
dnsdist: Fix a warning about long to double conversion