Alessio Podda [Thu, 5 Jun 2025 09:51:29 +0000 (11:51 +0200)]
Decouple database and node lifetimes by adding node-specific vtables
All databases in the codebase follow the same structure: a database is
an associative container from DNS names to nodes, and each node is an
associative container from RR types to RR data.
Each database implementation (qpzone, qpcache, sdlz, builtin, dyndb) has
its own corresponding node type (qpznode, qpcnode, etc). However, some
code needs to work with nodes generically regardless of their specific
type - for example, to acquire locks, manage references, or
register/unregister slabs from the heap.
Currently, these generic node operations are implemented as methods in
the database vtable, which creates problematic coupling between database
and node lifetimes. If a node outlives its parent database, the node
destructor will destroy all RR data, and each RR data destructor will
try to unregister from heaps by calling a virtual function from the
database vtable. Since the database was already freed, this causes a
crash.
This commit breaks the coupling by standardizing the layout of all
database nodes, adding a dedicated vtable for node operations, and
moving node-specific methods from the database vtable to the node
vtable.
Refactor sdlz to use name instead of pointer to name
Right now dns_sdlzlookup has a slight difference from other dbnode
implementations in that it stores a pointer to a dns name instead of
the dns name itself.
This commit harmonizes dns_sdlzlookup with other dbnode
implementations, facilitating further refactoring.
Each run of `meson test` overwrites the default log file testlog.txt,
this means we lose the backtraces of previous run. This commit assigns
a different log file for each run.
Nicki Křížek [Wed, 6 Aug 2025 10:38:32 +0000 (12:38 +0200)]
fix: ci: Allow unit tests to fail on AlmaLinux 8
The doh unit test has been timing out recently and we don't have a fix
for it yet. Mark it as warning rather than a hard failure, since it's a
known issue.
Related #5448
Merge branch '5448-allow-failure-unit-almalinux8-doh' into 'main'
Nicki Křížek [Tue, 5 Aug 2025 15:28:52 +0000 (17:28 +0200)]
Allow unit tests to fail on AlmaLinux 8
The doh unit test has been timing out recently and we don't have a fix
for it yet. Mark it as warning rather than a hard failure, since it's a
known issue.
Nicki Křížek [Mon, 4 Aug 2025 14:30:41 +0000 (16:30 +0200)]
Use full path for shared test code imports in rollover tests
Previously, symlinks and relative directory imports were used in test
modules. This caused a name clash when a shared code module "common.py"
was introduced for a different test. To avoid the issue, use full paths
in imports.
cut down the number of identical lines in the filter-aaaa test:
- replace identical test cases with small check functions
(check_aaaa_only, check_any, check_nodata, etc).
- group those together into large check functions (check_filter,
check_filter_other_family) that have options for recursive and
break_dnssec, then run those for each combination of options
on servers connfigured with filter-aaaa-on-v4 and filter-aaaa-on-v6.
Ondřej Surý [Tue, 5 Aug 2025 11:27:59 +0000 (13:27 +0200)]
fix: dev: Refactor resolver cache_name() and validated() functions
These functions were excessive in length and complexity, with McCabe complexity values of 110 and 105 respectively, and also included some dead code. They have been cleaned up and split into smaller functions, with a maximum complexity of 27. A few minor coding errors were discovered and fixed along the way.
Merge branch 'each-refactor-cache-name' into 'main'
Evan Hunt [Wed, 26 Feb 2025 21:59:19 +0000 (13:59 -0800)]
refactor validated()
- there was special-case code in validated() to handle the results
of a validator started by a CD=1 query. since that never happens,
the code has been removed.
- the section of code that handles opportunistic caching of
validated SOA, NS and NSEC data has been split out to a separate
function.
- the number of goto statements has been reduced considerably.
Evan Hunt [Sun, 2 Mar 2025 09:24:08 +0000 (01:24 -0800)]
split out helper functions
- fctx_setresult() sets the event result in a fetch response
according to the rdataset being returned - DNS_R_NCACHENXDOMAIN or
DNS_R_NXRRSET for negative responses, ISC_R_SUCCESS, DNS_R_CNAME,
or DNS_R_DNAME for positive ones.
- cache_rrset() looks up a node and adds an rdataset.
- delete_rrset() looks up a node and removes rdatasets of a specified
type and, optionally, the associated signatures.
- gettrust() returns the trust level of an rdataset, or dns_trust_none
if the rdataset is NULL or not associated.
- getrrsig() scans the rdatasets associated with a name for the
RRSIG covering a given type.
Evan Hunt [Sun, 2 Mar 2025 06:15:11 +0000 (22:15 -0800)]
further subdivide caching functions
rctx_cacherdataset() has been split into two functions:
- rctx_cache_secure() starts validation for rdatasets
that need it; they are then cached by the validator
completion callback validated()
- rctx_cache_insecure() caches rdatasets immediately; it
is called when validation is disabled or the data
to be cached is glue.
Evan Hunt [Sun, 2 Mar 2025 04:04:18 +0000 (20:04 -0800)]
rename and refactor cache_name() and related functions
- renamed cache_message() to rctx_cachemessage()
- renamed cache_name() to rctx_cachename()
- merged ncache_message() into rctx_ncache()
- split out a new function, rctx_cacherdataset(), which is
called by rctx_cachename() in a loop to process each of
the rdatasets associated with the name.
Evan Hunt [Sun, 2 Mar 2025 05:38:34 +0000 (21:38 -0800)]
reduce code duplication around findnoqname()
every call to findnoqname() was followed by a call to
dns_rdataset_addnoqname(). we can move that call into
findnoqname() itself, and simplify the calling functions
a bit.
Evan Hunt [Sat, 1 Mar 2025 23:40:07 +0000 (15:40 -0800)]
set ANSWERSIG flag when processing ANY responses
previously, rctx_answer_any() set the ANSWER flag for all
rdatasets in the answer section; it now sets ANSWERSIG for
RRSIG/SIG rdatasets and ANSWER for everything else. this
error didn't cause any harm in the current code, but it
could have led to unexpected behavior in the future.
Evan Hunt [Thu, 27 Feb 2025 22:28:37 +0000 (14:28 -0800)]
split out some functionality in cache_name()
there are now separate functions to check the cacheability of
an rdataset or to normalize TTLs, and the code to determine
whether validation is necessary has been simplified.
Evan Hunt [Fri, 28 Feb 2025 01:10:21 +0000 (17:10 -0800)]
add functions to match rdataset types
- dns_rdataset_issigtype() returns true if the rdataset is
of type RRSIG and covers a specified type
- dns_rdataset_matchestype() returns true if the rdataset
is of the specified type *or* the RRSIG covering it.
Evan Hunt [Thu, 27 Feb 2025 20:43:52 +0000 (12:43 -0800)]
reduce steps for negative caching
whenever ncache_adderesult() was called, some preparatory code
was run first; this has now been moved into a single function
negcache() to reduce code duplication.
Evan Hunt [Thu, 27 Feb 2025 06:06:40 +0000 (22:06 -0800)]
change issecuredomain() functions to bool
dns_keytable_issecuredomain() and dns_view_issecuredomain()
previously returned a result code to inform the caller of
unexpected database failures when looking up names in the
keytable and/or NTA table. such failures are not actually
possible. both functions now return a simple bool.
also, dns_view_issecuredomain() now returns false if
view->enablevalidation is false, so the caller no longer
has to check for that.
Ondřej Surý [Tue, 5 Aug 2025 09:24:35 +0000 (11:24 +0200)]
fix: test: Add support for small stack size for threads
When running the isc_quota unit test with less than
usual amount of RAM (e.g. in a CI for architectures
with 32 bits of address space), the pthread_create()
function fails with the "Resource temporarily unavailable
(11)" error code.
Add functions to get and set the thread stack size (if requested),
and use these to set the thread stack size to smaller value in the
isc_quota unit test.
Merge branch 'aram/isc-thread-stack-size-small' into 'main'
Ondřej Surý [Tue, 5 Aug 2025 05:34:15 +0000 (07:34 +0200)]
Document the current default stack sizes on different systems
The default stack sizes varies between operating systems and between
different system libc libraries from 128kB (Alpine Linux with MUSL) to
8M (Linux with glibc). Document the different values used to justify
the value of THREAD_MINSTACKSIZE (currently set to 1MB).
Ondřej Surý [Mon, 4 Aug 2025 15:03:42 +0000 (17:03 +0200)]
Add support for setting thread stack size
When running the isc_quota unit test with less than usual amount of
RAM (e.g. in a CI for architectures with 32 bits of address space),
the pthread_create() function fails with the "Resource temporarily
unavailable (11):" error code.
Add functions to get and set the thread stack size (if requested),
and use these to set the thread stack size to smaller value in the
isc_quota unit test.
fix: usr: Add RPZ extended DNS error for zones with a CNAME override policy configured
When the zone is configured with a CNAME override policy, or the response policy zone contains a wildcard CNAME, the extended DNS error code was not added. This has been fixed.
Closes #5342
Merge branch '5342-rpz-cname-override-ede-not-added' into 'main'
Ondřej Surý [Mon, 4 Aug 2025 10:11:01 +0000 (12:11 +0200)]
chg: dev: Add and use global memory context called isc_g_mctx
Instead of having individual memory contexts scattered across different
files and called different names, add a single memory context called
isc_g_mctx that replaces named_g_mctx and various other global memory
contexts in various utilities and tests.
Merge branch 'ondrej/add-global-isc_g_mctx-instance' into 'main'
There is a data race when QP is reclaiming chunks on the call_rcu
threads and it tries to log the number of reclaimed chunks while the
server is shuttingdown. Workaround this by adding rcu_barrier() before
shuttingdown the global log context.
This required couple of internal changes to the isc_mem_debugging.
The isc_mem_debugging is now internal to isc_mem unit and there are
three new functions:
1. isc_mem_setdebugging() can change the debugging setting for an
individual memory context. This is need for the memory contexts used
for OpenSSL, libxml and libuv accounting as recording and tracing
memory is broken there.
2. isc_mem_debugon() / isc_mem_debugoff() can be used to change default
memory debugging flags as well as debugging flags for isc_g_mctx.
Additionally, the memory debugging is inconsistent across the code-base.
For now, we are keeping the existing flags, but three new environment
variables have been added 'ISC_MEM_DEBUGRECORD', 'ISC_MEM_DEBUGTRACE'
and 'ISC_MEM_DEBUGUSAGE' to set the global debugging flags at any
program using the memory contexts.
Add and use global memory context called isc_g_mctx
Instead of having individual memory contexts scattered across different
files and called different names, add a single memory context called
isc_g_mctx that replaces named_g_mctx and various other global memory
contexts in various utilities and tests.
Mark Andrews [Wed, 17 Nov 2021 02:09:03 +0000 (13:09 +1100)]
validator.c:check_signer now clones val->sigrdataset
Spurious validation failures were traced back to check_signer looping
over val->sigrdataset directly. Cloning val->sigrdataset prevents
check_signer from interacting with callers that are also looping
over val->sigrdataset.
Most of the shell-based tests in the `dnssec` system test have been converted to python. The only exceptions are the test cases that exercised the `dnssec-*` command line tools, and did not interact with a name server; those have been relocated into a new `dnssectools` system test.
Merge branch 'each-convert-dnssec-test' into 'main'
If nsX.reconfigure() is used in a way that might affect other tests
within the same module, it's best to split up the tests which need the
reconfig to a separate module. This ensures the reconfigure() won't
interfere with test results in case the tests are executed separately,
or in a different order.
many of the zones in the dnssec system test were identical or
had only trivial differences, and it would be easier to keep track
of them if they were sourced from template files.
also, the extra_artifacts have been simplified and restored to
the test files.
the shell tests that queried servers to check correct signing
behavior (using dnssec-signzone, dnssec-policy and nsupdate),
as well as "rndc signing", private-type records, rndc zonestatus,
offline keys, etc, have been moved to tests_signing.py.
the minimal update test in the dnssec_update_test.pl script
was also moved here and the perl script has been removed.
Evan Hunt [Fri, 27 Jun 2025 03:29:24 +0000 (20:29 -0700)]
convert dnssec validation tests to python
begin converting DNSSEC validation tests from shell to python,
and simplify the name servers used in the test.
ns4, the name server used for validation tests, is now configured
using jinja2 templates.
ns8, which was previously used for testing unsupported, disabled
and revoked keys and trust anchors, has been removed. we now
use a jinja2 configuration in ns5 for this purpose.
the configurations in ns7 and ns6 didn't conflict with one another,
so the two servers have been merged into one.
Evan Hunt [Fri, 27 Jun 2025 22:35:11 +0000 (15:35 -0700)]
move some test cases to different system tests
- dnssec tool tests (i.e., dnssec-signzone, dnssec-keygen, etc) that
don't require interaction with a running server have been moved
to a new 'dnssectools' system test directory.
- a dig formatting test has been moved to digdelv.
- a named-rrchecker test has been moved to rrchecker
Evan Hunt [Thu, 26 Jun 2025 20:05:40 +0000 (13:05 -0700)]
move NTA test cases to a separate python test
the tests of "rndc nta" behavior have been moved out of the
dnssec shell test, into bin/tests/system/nta, and converted
to python. features of the dnssec test framework that were
needed for NTA testing have been moved to the nta test, and
dnssec has been correspondingly simplified.
Evan Hunt [Thu, 26 Jun 2025 02:00:22 +0000 (19:00 -0700)]
start converting dnssec system test to python/jinja2
- changed named.conf.in to named.conf.j2 in all server directories that
don't currently need to use copy_setports() during the test.
- converted the tests that use ns5 to python, and replaced
named1.conf.in and named2.conf.in with a jinja2 template instead.
the only remaining server that still needs copy_setports() is ns4.
- removed ns4/named5.conf.in, and moved its functions to ns5 (which
is supposed to be for servers with broken trust-anchor configurations,
so it should have been there in the first place). converted the tests
that used that ns4 configuration to use ns5 with jinja instead.
- revised the remaining ns4 configurations (named[1-4].conf.in) to
minimize the differences between them. this will make it easier to
convert it into a jinja2 template later.
Aydın Mercan [Tue, 29 Jul 2025 12:48:46 +0000 (12:48 +0000)]
reintroduce cross version config tests
The `cross-version-config-tests` CI job had to be removed due to radical
differences between meson and autoconf. With the release of 9.21.11
this job now can be reintroduced by comparing two meson built BIND
source trees.
Michał Kępień [Wed, 30 Jul 2025 15:41:13 +0000 (17:41 +0200)]
chg: dev: Check meson.build formatting in CI
Add a new CI job that checks whether all meson.build files in the
repository are formatted in the exact same way as "muon fmt" would
format them. This enforces formatting consistency across all
meson.build files in the repository and enables updating their contents
using dedicated tools, e.g. "meson rewrite".
See #5379
Merge branch '5379-check-meson.build-formatting-in-ci' into 'main'
Michał Kępień [Wed, 30 Jul 2025 15:37:38 +0000 (17:37 +0200)]
Check meson.build formatting in CI
Add a new CI job that checks whether all meson.build files in the
repository are formatted in the exact same way as "muon fmt" would
format them. This enforces formatting consistency across all
meson.build files in the repository and enables updating their contents
using dedicated tools, e.g. "meson rewrite".
Colin Vidal [Wed, 30 Jul 2025 10:11:46 +0000 (12:11 +0200)]
fix: test: fix macOS build for plugin unit test
MR !10753 breaks macOS build for plugin unit test as its linker doesn't
supports `--wrap` option, which is used in in order to mock the function
`isc_file_exits()`.
To work around the problem, a mocked `isc_file_exits()` is implemented
inside the plugin test as a static function before inlining the file
using it, which effectively links to this version rather than the isclib
one.
Closes #5455
Merge branch '5455-fix-macos-build-plugintest' into 'main'
Colin Vidal [Wed, 30 Jul 2025 08:35:26 +0000 (10:35 +0200)]
fix ns_plugin_expandpath tests with no extension
Parts of ns_plugin_expandpath() test expected the plugin extension to be
appened automatically (the plugin name/path is provided without the
extension), this enable to test the logic which adds the correct
extension based on the platfrom.
But the expected expanded paths from the test were hard coded with the
`.so` extension, so the test can't pass on macOS platform. This fixes
the test by using the macro providing the current-platform extension.
Colin Vidal [Wed, 30 Jul 2025 07:54:05 +0000 (09:54 +0200)]
fix macOS build for plugin unit test
MR !10753 breaks macOS build for plugin unit test as its linker doesn't
supports `--wrap` option, which is used in in order to mock the function
`isc_file_exits()`.
To work around the problem, a mocked `isc_file_exits()` is implemented
inside the plugin test as a static function before inlining the file
using it, which effectively links to this version rather than the isclib
one.
added some helper functions in isctest to reduce code repetition
in dnssec-related tests:
- isctest.check.adflag() - checks that a response contains AD=1
- isctest.check.noadflag() - checks that a response contains AD=0
- isctest.check.rdflag() - checks that a response contains RD=1
- isctest.check.nordflag() - checks that a response contains RD=0
- isctest.check.raflag() - checks that a response contains RA=1
- isctest.check.noraflag() - checks that a response contains RA=0
- isctest.check.rr_count_eq() - checks the number of RRsset in a section
- isctest.check.same_data() - checks that two message have the
same rcode and data
- isctest.check.same_answer() - checks that two message have the same
rcode and answer
- isctest.query.create() - a wrapper for dns.message.make_query() that
creates a query message similar to dig +dnssec
Rather than using the dnspython's facilities and defaults to create the
queries, use the isctest.query.create function in all the cases that
don't require special handling to have consistent defaults.
Use a common function to count the number of RRs in any section of the
DNS message. For the ADDITIONAL section, stick with the dnspython
convention of not including OPT and TSIG.
Evan Hunt [Thu, 26 Jun 2025 22:19:45 +0000 (15:19 -0700)]
add helper functions to isctest
added some helper functions in isctest to reduce code repetition
in dnssec-related tests:
- isctest.check.adflag() - checks that a response contains AD=1
- isctest.check.noadflag() - checks that a response contains AD=0
- isctest.check.rdflag() - checks that a response contains RD=1
- isctest.check.nordflag() - checks that a response contains RD=0
- isctest.check.answer_count_eq() - checks the answer count is correct
- isctest.check.additional_count_eq() - same for authority count
- isctest.check.authority_count_eq() - same for additional count
- isctest.check.same_data() - check that two message have the
same rcode and data
- isctest.check.same_answer() - check that two message have the same
rcode and answer
- isctest.dnssec.msg() - a wrapper for dns.message.make_query() that
creates a query message similar to dig +dnssec:
use_edns=True, want_dnssec=True,
and flags are set to (RD|AD) by default, but
options exist to disable AD or enable CD.
(to generate non-DNSSEC queries, use
message.make_query() directly.)
fix: test: Improve key collision detection in ksr system test
MR !10238 added key collision detection in the ksr system test but it was flawed because for every "collide" in the output we also log
"Generating an new key" and for each "Generating" we add the counter by one, nullifying the subtract by one.
Fix by splitting the output on ':' rather than on the default whitespace. Also make the substring matching more strict.
Improve key collision detection in ksr system test
MR !10238 added key collision detection in the ksr system test but it
was flawed because for every "collide" in the output we also log
"Generating an new key" and for each "Generating" we add the counter
by one, nullifying the subtract by one.
Use regular expressions to search in the output and make the string
expression more strict.
Michał Kępień [Fri, 25 Jul 2025 12:50:49 +0000 (14:50 +0200)]
Adjust type hints for the "nsec3-answer" test
Add missing type hints in the tests_nsec3.py module. Tweak the syntax
used for type hints for better consistency with other Python code in
bin/tests/system/.