There is a data race when QP is reclaiming chunks on the call_rcu
threads and it tries to log the number of reclaimed chunks while the
server is shuttingdown. Workaround this by adding rcu_barrier() before
shuttingdown the global log context.
This required couple of internal changes to the isc_mem_debugging.
The isc_mem_debugging is now internal to isc_mem unit and there are
three new functions:
1. isc_mem_setdebugging() can change the debugging setting for an
individual memory context. This is need for the memory contexts used
for OpenSSL, libxml and libuv accounting as recording and tracing
memory is broken there.
2. isc_mem_debugon() / isc_mem_debugoff() can be used to change default
memory debugging flags as well as debugging flags for isc_g_mctx.
Additionally, the memory debugging is inconsistent across the code-base.
For now, we are keeping the existing flags, but three new environment
variables have been added 'ISC_MEM_DEBUGRECORD', 'ISC_MEM_DEBUGTRACE'
and 'ISC_MEM_DEBUGUSAGE' to set the global debugging flags at any
program using the memory contexts.
Add and use global memory context called isc_g_mctx
Instead of having individual memory contexts scattered across different
files and called different names, add a single memory context called
isc_g_mctx that replaces named_g_mctx and various other global memory
contexts in various utilities and tests.
Mark Andrews [Wed, 17 Nov 2021 02:09:03 +0000 (13:09 +1100)]
validator.c:check_signer now clones val->sigrdataset
Spurious validation failures were traced back to check_signer looping
over val->sigrdataset directly. Cloning val->sigrdataset prevents
check_signer from interacting with callers that are also looping
over val->sigrdataset.
Most of the shell-based tests in the `dnssec` system test have been converted to python. The only exceptions are the test cases that exercised the `dnssec-*` command line tools, and did not interact with a name server; those have been relocated into a new `dnssectools` system test.
Merge branch 'each-convert-dnssec-test' into 'main'
If nsX.reconfigure() is used in a way that might affect other tests
within the same module, it's best to split up the tests which need the
reconfig to a separate module. This ensures the reconfigure() won't
interfere with test results in case the tests are executed separately,
or in a different order.
many of the zones in the dnssec system test were identical or
had only trivial differences, and it would be easier to keep track
of them if they were sourced from template files.
also, the extra_artifacts have been simplified and restored to
the test files.
the shell tests that queried servers to check correct signing
behavior (using dnssec-signzone, dnssec-policy and nsupdate),
as well as "rndc signing", private-type records, rndc zonestatus,
offline keys, etc, have been moved to tests_signing.py.
the minimal update test in the dnssec_update_test.pl script
was also moved here and the perl script has been removed.
Evan Hunt [Fri, 27 Jun 2025 03:29:24 +0000 (20:29 -0700)]
convert dnssec validation tests to python
begin converting DNSSEC validation tests from shell to python,
and simplify the name servers used in the test.
ns4, the name server used for validation tests, is now configured
using jinja2 templates.
ns8, which was previously used for testing unsupported, disabled
and revoked keys and trust anchors, has been removed. we now
use a jinja2 configuration in ns5 for this purpose.
the configurations in ns7 and ns6 didn't conflict with one another,
so the two servers have been merged into one.
Evan Hunt [Fri, 27 Jun 2025 22:35:11 +0000 (15:35 -0700)]
move some test cases to different system tests
- dnssec tool tests (i.e., dnssec-signzone, dnssec-keygen, etc) that
don't require interaction with a running server have been moved
to a new 'dnssectools' system test directory.
- a dig formatting test has been moved to digdelv.
- a named-rrchecker test has been moved to rrchecker
Evan Hunt [Thu, 26 Jun 2025 20:05:40 +0000 (13:05 -0700)]
move NTA test cases to a separate python test
the tests of "rndc nta" behavior have been moved out of the
dnssec shell test, into bin/tests/system/nta, and converted
to python. features of the dnssec test framework that were
needed for NTA testing have been moved to the nta test, and
dnssec has been correspondingly simplified.
Evan Hunt [Thu, 26 Jun 2025 02:00:22 +0000 (19:00 -0700)]
start converting dnssec system test to python/jinja2
- changed named.conf.in to named.conf.j2 in all server directories that
don't currently need to use copy_setports() during the test.
- converted the tests that use ns5 to python, and replaced
named1.conf.in and named2.conf.in with a jinja2 template instead.
the only remaining server that still needs copy_setports() is ns4.
- removed ns4/named5.conf.in, and moved its functions to ns5 (which
is supposed to be for servers with broken trust-anchor configurations,
so it should have been there in the first place). converted the tests
that used that ns4 configuration to use ns5 with jinja instead.
- revised the remaining ns4 configurations (named[1-4].conf.in) to
minimize the differences between them. this will make it easier to
convert it into a jinja2 template later.
Aydın Mercan [Tue, 29 Jul 2025 12:48:46 +0000 (12:48 +0000)]
reintroduce cross version config tests
The `cross-version-config-tests` CI job had to be removed due to radical
differences between meson and autoconf. With the release of 9.21.11
this job now can be reintroduced by comparing two meson built BIND
source trees.
Michał Kępień [Wed, 30 Jul 2025 15:41:13 +0000 (17:41 +0200)]
chg: dev: Check meson.build formatting in CI
Add a new CI job that checks whether all meson.build files in the
repository are formatted in the exact same way as "muon fmt" would
format them. This enforces formatting consistency across all
meson.build files in the repository and enables updating their contents
using dedicated tools, e.g. "meson rewrite".
See #5379
Merge branch '5379-check-meson.build-formatting-in-ci' into 'main'
Michał Kępień [Wed, 30 Jul 2025 15:37:38 +0000 (17:37 +0200)]
Check meson.build formatting in CI
Add a new CI job that checks whether all meson.build files in the
repository are formatted in the exact same way as "muon fmt" would
format them. This enforces formatting consistency across all
meson.build files in the repository and enables updating their contents
using dedicated tools, e.g. "meson rewrite".
Colin Vidal [Wed, 30 Jul 2025 10:11:46 +0000 (12:11 +0200)]
fix: test: fix macOS build for plugin unit test
MR !10753 breaks macOS build for plugin unit test as its linker doesn't
supports `--wrap` option, which is used in in order to mock the function
`isc_file_exits()`.
To work around the problem, a mocked `isc_file_exits()` is implemented
inside the plugin test as a static function before inlining the file
using it, which effectively links to this version rather than the isclib
one.
Closes #5455
Merge branch '5455-fix-macos-build-plugintest' into 'main'
Colin Vidal [Wed, 30 Jul 2025 08:35:26 +0000 (10:35 +0200)]
fix ns_plugin_expandpath tests with no extension
Parts of ns_plugin_expandpath() test expected the plugin extension to be
appened automatically (the plugin name/path is provided without the
extension), this enable to test the logic which adds the correct
extension based on the platfrom.
But the expected expanded paths from the test were hard coded with the
`.so` extension, so the test can't pass on macOS platform. This fixes
the test by using the macro providing the current-platform extension.
Colin Vidal [Wed, 30 Jul 2025 07:54:05 +0000 (09:54 +0200)]
fix macOS build for plugin unit test
MR !10753 breaks macOS build for plugin unit test as its linker doesn't
supports `--wrap` option, which is used in in order to mock the function
`isc_file_exits()`.
To work around the problem, a mocked `isc_file_exits()` is implemented
inside the plugin test as a static function before inlining the file
using it, which effectively links to this version rather than the isclib
one.
added some helper functions in isctest to reduce code repetition
in dnssec-related tests:
- isctest.check.adflag() - checks that a response contains AD=1
- isctest.check.noadflag() - checks that a response contains AD=0
- isctest.check.rdflag() - checks that a response contains RD=1
- isctest.check.nordflag() - checks that a response contains RD=0
- isctest.check.raflag() - checks that a response contains RA=1
- isctest.check.noraflag() - checks that a response contains RA=0
- isctest.check.rr_count_eq() - checks the number of RRsset in a section
- isctest.check.same_data() - checks that two message have the
same rcode and data
- isctest.check.same_answer() - checks that two message have the same
rcode and answer
- isctest.query.create() - a wrapper for dns.message.make_query() that
creates a query message similar to dig +dnssec
Rather than using the dnspython's facilities and defaults to create the
queries, use the isctest.query.create function in all the cases that
don't require special handling to have consistent defaults.
Use a common function to count the number of RRs in any section of the
DNS message. For the ADDITIONAL section, stick with the dnspython
convention of not including OPT and TSIG.
Evan Hunt [Thu, 26 Jun 2025 22:19:45 +0000 (15:19 -0700)]
add helper functions to isctest
added some helper functions in isctest to reduce code repetition
in dnssec-related tests:
- isctest.check.adflag() - checks that a response contains AD=1
- isctest.check.noadflag() - checks that a response contains AD=0
- isctest.check.rdflag() - checks that a response contains RD=1
- isctest.check.nordflag() - checks that a response contains RD=0
- isctest.check.answer_count_eq() - checks the answer count is correct
- isctest.check.additional_count_eq() - same for authority count
- isctest.check.authority_count_eq() - same for additional count
- isctest.check.same_data() - check that two message have the
same rcode and data
- isctest.check.same_answer() - check that two message have the same
rcode and answer
- isctest.dnssec.msg() - a wrapper for dns.message.make_query() that
creates a query message similar to dig +dnssec:
use_edns=True, want_dnssec=True,
and flags are set to (RD|AD) by default, but
options exist to disable AD or enable CD.
(to generate non-DNSSEC queries, use
message.make_query() directly.)
fix: test: Improve key collision detection in ksr system test
MR !10238 added key collision detection in the ksr system test but it was flawed because for every "collide" in the output we also log
"Generating an new key" and for each "Generating" we add the counter by one, nullifying the subtract by one.
Fix by splitting the output on ':' rather than on the default whitespace. Also make the substring matching more strict.
Improve key collision detection in ksr system test
MR !10238 added key collision detection in the ksr system test but it
was flawed because for every "collide" in the output we also log
"Generating an new key" and for each "Generating" we add the counter
by one, nullifying the subtract by one.
Use regular expressions to search in the output and make the string
expression more strict.
Michał Kępień [Fri, 25 Jul 2025 12:50:49 +0000 (14:50 +0200)]
Adjust type hints for the "nsec3-answer" test
Add missing type hints in the tests_nsec3.py module. Tweak the syntax
used for type hints for better consistency with other Python code in
bin/tests/system/.
Petr Špaček [Fri, 6 Jun 2025 15:10:42 +0000 (17:10 +0200)]
Add consistency checks to responses with NSEC3
Basic sanity checks - limited to responses from a single zone:
- NSEC3 type cannot be present in type bitmap:
By definition, the type bitmap describes state of the unhashed name
but NSEC3 RR is present at a different owner name. RFC 7129 section 5
- NSEC3 owner names cannot be duplicated:
Unless the response crosses zone boundary, parent zone has insecure
delegation for child, but child is signed ... don't do that.
- All parameters are consistent across all RRs present in answer:
RFC 5155 section 7.2, last paragraph - at least when we don't cross
zone boundary.
Petr Špaček [Thu, 5 Jun 2025 13:15:08 +0000 (15:15 +0200)]
Split NXDOMAIN/NOERROR/NODATA test cases
Untangling individual cases allows for clearer documentation and makes
it easier to build similar but slightly different test cases. Wildcard
NODATA answer was added.
Petr Špaček [Fri, 23 May 2025 07:07:02 +0000 (09:07 +0200)]
Generate comprehensive tests for ZoneAnalyzer utility class
Test all combinations of wildcard, ENT, DNAME, NS, and ordinary
TXT records.
Test zone and expected outputs are generated by another script which
encodes node content into node name. This encoding removes 'node
content' level of indirection and thus enables simpler implementation of
same logic which needs to be in ZoneAnalyzer itself.
For humans the generated zone file also lists expected 'categories' a
name belongs to as dot-separated list on right hand side of a generated
RR.
Petr Špaček [Wed, 21 May 2025 15:58:17 +0000 (17:58 +0200)]
Test ZoneAnalyzer utility class
I've considered writing hypothesis test for this but I would have to
reimplement the same thing, which would probably have the same logic
bugs, so I will leave it as an exercise for someone else.
Check the correctness of NSEC3 hash generation by generating random
combinations of name, salt, and iterations and comparing the outputs
of the nsec3hash tool against the dnspython nsec3_hash function
for the same inputs.
Add a property based test for NSEC3 proofs for non-existent QNAMEs
For any given NSEC3 signed zone, when doing queries for non-existent
names, the response must contain:
- NSEC3 RR that matches the closest encloser,
- NSEC3 RR that covers the next closer name,
- NSEC3 RR that covers the wildcard.
Colin Vidal [Mon, 28 Jul 2025 21:43:11 +0000 (23:43 +0200)]
chg: usr: plugin extension in plugin path is now optional
Plugin configuration no longer requires the library file extension, so it is now possible to invoke a plugin using the syntax `plugin query "library"` instead of `plugin query "libary.so"`.
Closes #5377
Merge branch '5377-implicit-plugin-ext' into 'main'
Colin Vidal [Mon, 21 Jul 2025 13:58:31 +0000 (15:58 +0200)]
ns_plugin_expandpath() auto-extension unit-tests
Update existing ns_plugin_expandpath() unit test to cover the logic
appending the plugin extension if missing.
Because ns_plugin_expandpath() now relies on isc_file_exists() API, a
mocked version has been added in tests/ns/plugin_test.c and relies on the
linker --wrap mechanism.
Colin Vidal [Mon, 21 Jul 2025 13:06:11 +0000 (15:06 +0200)]
plugin expand path automatically adds extension
If a plugin is configured without the extension,
`ns_plugin_expandpath()` automatically take cares of appending the
suffix to the path. The way it works is by checking if a file exists at
the expanded path. If it doesn't, it assumes the plugin path (or name)
doesn't have the extension and append the extension (which is
platform-specific) to the actual path.
Colin Vidal [Mon, 21 Jul 2025 12:38:30 +0000 (14:38 +0200)]
Export plugin extension in config.h
Dynamically loadable libraries all use the `.so` extension on
BIND9-supported platforms, except for macOS. Export the dynamic library
extension of the current build platform in the generated `config.h`
file, in order to let the plugin code building plugin path based on a
simple plugin name. (which then would be platform-independent)
Fix one-definition-rule violation in the loop unit test
Locally, clang reported following odr-violation:
=================================================================
==1132009==ERROR: AddressSanitizer: odr-violation (0x555555589280):
[1] size=8 'isc__loopmgr' ../lib/isc/loop.c:52:16 in /home/ondrej/Projects/bind9/build/tests/isc/loop
[2] size=8 'isc__loopmgr' ../lib/isc/loop.c:52:16 in /home/ondrej/Projects/bind9/build/tests/isc/../../libisc.so
These globals were registered at these points:
[1]:
#0 0x7ffff785306f in __asan_register_globals ../../../../src/libsanitizer/asan/asan_globals.cpp:350
#1 0x55555556abce in _sub_I_00099_1 (/home/ondrej/Projects/bind9/build/tests/isc/loop+0x16bce) (BuildId: e7c586e966e6986532a3da40df41223ae16e55c9)
#2 0x7ffff702a303 in call_init ../csu/libc-start.c:145
#3 0x7ffff702a303 in __libc_start_main_impl ../csu/libc-start.c:347
#4 0x5555555622e4 in _start (/home/ondrej/Projects/bind9/build/tests/isc/loop+0xe2e4) (BuildId: e7c586e966e6986532a3da40df41223ae16e55c9)
[2]:
#0 0x7ffff785306f in __asan_register_globals ../../../../src/libsanitizer/asan/asan_globals.cpp:350
#1 0x7ffff75335b9 in _sub_I_00099_1 (/home/ondrej/Projects/bind9/build/tests/isc/../../libisc.so+0x1335b9) (BuildId: 33ab72bc676e9ef9111b3db1fc4347595069cd29)
#2 0x7ffff7fca71e in call_init elf/dl-init.c:74
#3 0x7ffff7fca823 in call_init elf/dl-init.c:120
#4 0x7ffff7fca823 in _dl_init elf/dl-init.c:121
#5 0x7ffff7fe459f (/lib64/ld-linux-x86-64.so.2+0x1f59f) (BuildId: 281ac1521b4102509b1c7ac7004db7c1efb81796)
==1132009==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'isc__loopmgr' at ../lib/isc/loop.c:52:16 in /home/ondrej/Projects/bind9/build/tests/isc/loop
==1132009==ABORTING
Aborted (core dumped)
Rename isc__loopmgr when including the loop.c into loop_test.c to
prevent odr-violation over isc__loopmgr.
Michał Kępień [Wed, 23 Jul 2025 10:16:25 +0000 (12:16 +0200)]
Account for idle timeouts in the "dispatch" test
When the tests-connreset.py module was initially implemented in commit 5c17919019ef0af8226e5bb61214b805bb3e2451, the dispatch code did not
properly apply the idle timeout to TCP connections. This allowed the
check in that test module to reset the TCP connection after 5 seconds as
named did not attempt to tear the connection down earlier than that.
However, as the dispatch code was improved, the idle timeout started
being enforced for TCP dispatches; the exact value it is set to in the
current code depends on a given server's SRTT, but it defaults to about
1.2 seconds for responsive servers. This means that the code paths
triggered by the "dispatch" system test are now different than the ones
it was originally supposed to trigger because it is now named itself
that shuts the TCP connection down cleanly before the ans3 server gets a
chance to reset it.
Account for the above by lowering the amount of time after which the
ans3 server in the "dispatch" system test resets TCP connections to just
1 second, so that the test actually does what its name implies.
Michał Kępień [Wed, 23 Jul 2025 10:16:25 +0000 (12:16 +0200)]
Enable resetting TCP connections
Add a TCP connection handler, ConnectionReset, which enables closing TCP
connections without emptying the client socket buffer, causing the
kernel to send an RST segment to the client. This relies on a horrible
asyncio hack that can break at any point in the future due to abusing
implementation details in the Python Standard Library. Despite the eye
bleeding this code may cause, the approach it takes was still deemed
preferable to implementing an asyncio transport from scratch just to
enable triggering connection resets.
Michał Kępień [Tue, 15 Jul 2025 10:16:32 +0000 (12:16 +0200)]
Enable requesting TCP connections to be closed
In response to client queries, AsyncDnsServer users can currently only
make the server either send a reply or silently ignore the query. In
the case of TCP queries, neither of these actions causes the client's
connection to be closed - the onus of doing that is on the client.
However, in some cases the server may be required to close the
connection on its own, so AsyncDnsServer users need to have some way of
requesting such an action.
Add a new ResponseAction subclass, ResponseDropAndCloseConnection, which
enables AsyncDnsServer users to conveniently request TCP connections to
be closed. Instead of returning the response to send,
ResponseDropAndCloseConnection raises a custom exception that
AsyncDnsServer._handle_tcp() handles accordingly.
=================================================================
==588371==ERROR: AddressSanitizer: odr-violation (0x55555556a060):
[1] size=256 'client_addrs' ../tests/ns/netmgr_wrap.c:36:18 in /home/ondrej/Projects/bind9/build/tests/ns/query
[2] size=256 'client_addrs' ../tests/ns/netmgr_wrap.c:36:18 in /home/ondrej/Projects/bind9/build/tests/ns/../libbindtest.so
These globals were registered at these points:
[1]:
#0 0x7ffff785306f in __asan_register_globals ../../../../src/libsanitizer/asan/asan_globals.cpp:350
#1 0x7ffff6a2a303 in call_init ../csu/libc-start.c:145
#2 0x7ffff6a2a303 in __libc_start_main_impl ../csu/libc-start.c:347
#3 0x55555555a084 in _start (/home/ondrej/Projects/bind9/build/tests/ns/query+0x6084) (BuildId: fbe4a3fcf1a249c7d7da69ee8b255a1dbb610c7a)
[2]:
#0 0x7ffff785306f in __asan_register_globals ../../../../src/libsanitizer/asan/asan_globals.cpp:350
#1 0x7ffff7fca71e in call_init elf/dl-init.c:74
#2 0x7ffff7fca823 in call_init elf/dl-init.c:120
#3 0x7ffff7fca823 in _dl_init elf/dl-init.c:121
#4 0x7ffff7fe459f (/lib64/ld-linux-x86-64.so.2+0x1f59f) (BuildId: 281ac1521b4102509b1c7ac7004db7c1efb81796)
==588371==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'client_addrs' at ../tests/ns/netmgr_wrap.c:36:18 in /home/ondrej/Projects/bind9/build/tests/ns/query
==588371==ABORTING
Move the client_addrs and client_refs to libtest to prevent this.
There is only a single network manager running on top of the loop
manager (except for tests). Refactor the network manager to be a
singleton (a single instance) and change the unit tests, so that the
shorter read timeouts apply only to a specific handle, not the whole
extra 'connect_nm' network manager instance.
All the applications built on top of the loop manager were required to
create a single instance of the loop manager. Refactor the loop
manager not to expose this instance to the callers, and keep the loop
manager object internal to the `isc_loop` compilation unit.
This significantly simplifies a number of data structures and calls to
the `isc_loop` API.
Merge branch 'ondrej/refactor-isc_loopmgr-to-be-singleton' into 'main'
All the applications built on top of the loop manager were required to
create just a single instance of the loop manager. Refactor the loop
manager to not expose this instance to the callers and keep the loop
manager object internal to the isc_loop compilation unit.
This significantly simplifies a number of data structures and calls to
the isc_loop API.
chg: usr: Reword the 'shut down hung fetch while resolving' message
The log message 'shut down hung fetch while resolving' may be confusing
because no detection of hung fetches actually takes place, but rather
the timer on the fetch context expires and the resolver gives up.
Change the log message to actually say that instead of the original
cryptic message about hung fetch.
Closes #3148
Merge branch '3148-rename-shut-down-hung-fetch' into 'main'
Reword the 'shut down hung fetch while resolving' message
The log message 'shut down hung fetch while resolving' may be confusing
because no detection of hung fetches actually takes place, but rather
the timer on the fetch context expires and the resolver gives up.
Change the log message to actually say that instead of the original
cryptic message about hung fetch.
chg: test: Use isctest.asyncserver in the "zero" test
The original `ans.pl` server was a copy of the one in `fetchlimit`, so
there are some changes:
- The server now only responds with A replies (which is the only thing
needed).
- The incrementing of the IP address goes beyond the least significant
octet (so, after 192.0.2.255 it will yield 192.0.3.0).
Merge branch 'stepan/zero-asyncserver' into 'main'
Štěpán Balážik [Wed, 18 Jun 2025 10:13:37 +0000 (12:13 +0200)]
Use isctest.asyncserver in the "zero" test
The original `ans.pl` server was based on a copy of the one in
`fetchlimit`, so there are some changes:
- The server now only responds with A replies (which is the only thing
needed).
- The incrementing of the IP address goes beyond the least significant
octet (so, after 192.0.2.255 it will yield 192.0.3.0).
fix: usr: Stale RRsets in a CNAME chain were not always refreshed
With serve-stale enabled, a CNAME chain that contains a stale RRset, the refresh query doesn't always properly refresh the stale RRsets. This has been fixed.
Closes #5243
Merge branch '5243-stale-refresh-as-prefetch' into 'main'