Add assertion failure when adding to hashmap when iterating
When iterating the table, we can't add new nodes to the hashmap because
we can't assure that we are not adding the new node before the iterator.
This also applies to rehashing - which might be triggered by both
isc_hashmap_add() and isc_hashmap_delete(), but not
isc_hashmap_iter_delcurrent_next().
Mark Andrews [Tue, 19 Sep 2023 01:42:03 +0000 (11:42 +1000)]
Fix hashmap iteration
When isc_hashmap_iter_delcurrent_next calls hashmap_delete_node
nodes from the front of the table could be added to the end of
the table resulting in them being returned twice. Detect when
this is happening and prevent those nodes being returned twice
buy reducing the effective size of the table by one each time
it happens.
Replace the linked list of TCP dispatches with hash table
Reusing TCP connections with dns_dispatch_gettcp() used linear linked
list to lookup existing outgoing TCP connections that could be reused.
Replace the linked list with per-loop cds_lfht hashtable to speedup the
lookups. We use cds_lfht because it allows non-unique node insertion
that we need to check for dispatches in different connection states.
Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the
dispatches bound to threads and make dns_dispatchset_t create a dispatch
for each thread (event loop).
This required couple of other changes:
1. The dns_dispatch_createudp() must be called on loop, so the isc_tid()
is already initialized - changes to nsupdate and mdig were required.
2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead
of using single dispatch, use dns_dispatchset_t for each protocol -
this is same as dns_resolver.
Looking up unique message ID in the dns_dispatch has been using custom
hash tables. Rewrite the custom hashtable to use cds_lfht API, removing
one extra lock in the cold-cache resolver hot path.
Ondřej Surý [Mon, 28 Aug 2023 08:08:59 +0000 (10:08 +0200)]
Refactor isc_hashmap to accept custom match function
Refactor isc_hashmap to allow custom matching functions. This allows us
to have better tailored keys that don't require fixed uint8_t arrays,
but can be composed of more fields from the stored data structure.
Don't warn about subject line length for the fixup commits
The fixup commits' subject line has a prefix which has its own
length, so warning about the exceeding length is not accurate.
Given that the fixup commits can not be merged, because they
cause a danger failure, it's safe to ignore the length check
for them.
Mark Andrews [Tue, 6 Aug 2019 07:28:02 +0000 (17:28 +1000)]
Test resolver-use-dns64
Test resolver-use-dns64 by simulating a connection to an IPv4-only
server through a NAT64.
This test uses EXTRAPORT1 rather than PORT for DNS traffic exchanged
between ns3 and ns4. Both servers also listen on PORT on their IPv4
addresses to support server startup testing in start.pl.
Mark Andrews [Sun, 21 Jul 2019 16:26:39 +0000 (02:26 +1000)]
Add the ability to use DNS64 internally
Add a configuration option, resolver-use-dns64, which when true
will cause named to map IPv4 address to IPv6 addresses using the
view's DNS64 mapping rules when making iterative queries.
Implement incremental version of SipHash 2-4 and HalfSipHash 2-4
When inserting items into hashtables (hashmaps), we might have a
fragmented key (as an example we might want to hash DNS name + class +
type). We either need to construct continuous key in the memory and
then hash it en bloc, or incremental hashing is required.
This incremental version of SipHash 2-4 algorithm is the first building
block.
As SipHash 2-4 is often used in the hot paths, I've turned the
implementation into header-only version in the process.
1. Replace the "high-performance" macros that were only used if
DNS_NAME_USEINLINE was defined before including <dns/name.h> with
inline header functions with assertion checks and thus use them
everywhere.
2. Replace the old struct initializers with C99 designated initializers
for better understanding what is happening in these macros.
Mark Andrews [Fri, 1 Sep 2023 00:17:00 +0000 (10:17 +1000)]
Adjust level of log messages when transferring in a zone
This raises the log level of messages treated as FORMERR to NOTICE
when transfering in a zone. This also adds a missing log message
for TYPE0 and meta types received during a zone transfer.
Mark Andrews [Mon, 4 Sep 2023 23:46:05 +0000 (09:46 +1000)]
Restore dns_validator_destroy and fetchctx_detach call order
7a78a85b moved the destruction of the validator from near the start
validated to the end. This reversed the order of dns_validator_destroy
and fetchctx_detach. Restore the order so that val->name remains
valid for the lifetime of the validator.
Synchronize used library versions reporting functions
There are libraries which are reported in printversion(), but not
reported in setup(). Synchronize the functions, so that the log
file could have the same information as reported by the 'named -V'
command execution.
Print the used jemalloc version in autoconf and named -V output
The autoconf and named -V now prints used version of jemalloc. This
doesn't work with system supplied jemalloc, so in it prints `system`
instead in the autoconf and nothing in named -V output.
We now depend on explicitly creating memory arenas and disabling tcache
on those, and these features are not available with jemalloc < 4.
Instead of working around these issues, make the jemalloc >= 4.0.0 hard
requirement by looking for sdallocx() symbol that's only available from
that version.
The jemalloc < 4 was only used by RHEL 7 which is not supported since
BIND 9.19+.
Artem Boldariev [Thu, 10 Aug 2023 20:08:25 +0000 (23:08 +0300)]
Allocate DNS send buffers using dedicated per-worker memory arenas
This commit ensures that memory allocations related to DNS send
buffers are routed through dedicated per-worker memory arenas in order
to decrease memory usage on high load caused by TCP-based DNS
transports.
We do that by following jemalloc developers suggestions:
Artem Boldariev [Thu, 10 Aug 2023 14:02:43 +0000 (17:02 +0300)]
Make it possible to create memory contexts backed by jemalloc arenas
This commit extends the internal memory management middleware code in
BIND so that memory contexts backed by dedicated jemalloc arenas can
be created. A new function (isc_mem_create_arena()) is added for that.
Moreover, it extends the existing code so that specialised memory
contexts can be created easily, should we need that functionality for
other future purposes. We have achieved that by passing the flags to
the underlying jemalloc-related calls. See the above
isc_mem_create_arena(), which can serve as an example of this.
Having this opens up possibilities for creating memory contexts tuned
for specific needs.
Michał Kępień [Mon, 4 Sep 2023 09:54:57 +0000 (11:54 +0200)]
Move security-related information to SECURITY.md
To follow current best practices, create a short SECURITY.md file in the
root of the repository that contains information about the project's
security policy and guidelines for reporting potential security issues.
Replace the relevant bits of text in other files with references to the
new SECURITY.md file, so that the relevant information only needs to be
maintained in one place.
Replace all occurrences of the generic security-officer@isc.org email
with a dedicated address for reporting BIND 9 security issues,
bind-security@isc.org.
Evan Hunt [Thu, 24 Aug 2023 20:47:19 +0000 (13:47 -0700)]
unconditionally create view and resolver nametrees
instead of allowing a NULL nametree in dns_nametree_covered(),
require nametree to exist, and ensure that the nametrees defined
for view and resolver objects are always created.
Evan Hunt [Thu, 17 Aug 2023 06:26:50 +0000 (23:26 -0700)]
add semantics to name trees to support counters
name trees can now also hold trees of counters. each time a name
dns_nametree_add() is called with a given name, the counter for that
name is incremented; the name is not deleted until dns_nametree_delete()
is called the same number of times.
this is meant to be used for synth-from-dnssec, which is incremented for
each key defined at a name, and decremented when a key is removed, the
name must continue to exist until the number of keys has reached zero.
Evan Hunt [Thu, 17 Aug 2023 02:59:50 +0000 (19:59 -0700)]
add semantics to dns_nametree to support bitfields
name trees can now hold either boolean values or bit fields. the
type is selected when the name tree is created.
the behavior of dns_nametree_add() differs slightly beteween the types:
in a boolean tree adding an existing name will return ISC_R_EXISTS,
but in a bitfield tree it simply sets the specified bit in the bitfield
and returns ISC_R_SUCCESS.
Evan Hunt [Tue, 15 Aug 2023 22:30:12 +0000 (15:30 -0700)]
refactor disable_algorithm and disable_ds_digest to use one data structure
the functions for disabling DNSSEC signing algorithms and DS digest
algorithms in resolver.c had a lot of duplicated code.
this commit adds functions to implement a "bitfield tree", which is
(currently) an RBT in which the node data contains arbitrary-sized
bitfields to indicate whether a value has been added at the given
node or not. (it can be changed to a QP trie later.)
it also replaces the functions dns_resolver_disable_algorithm(),
dns_resolver_algorithm_supported(), dns_resolver_disable_ds_digest()
and dns_resolver_ds_digest_supported() with simple wrappers that call
the new functions.
Evan Hunt [Wed, 16 Aug 2023 20:28:36 +0000 (13:28 -0700)]
use dns_nametree in place of RBTs
replace the use of RBTs for deny-answer-aliases, the exclude
lists for deny-answer-aliases and deny-answer-addresses, and
dnssec-must-be-secure, with name trees.
Evan Hunt [Wed, 16 Aug 2023 19:08:53 +0000 (12:08 -0700)]
add dns_nametree structure for policy match lookups
this is a QP trie of boolean values to indicate whether a name is
included in or excluded from some policy. this can be used for
synth-from-dnssec, deny-answer-aliases, etc.
Evan Hunt [Wed, 16 Aug 2023 04:04:54 +0000 (21:04 -0700)]
replace RBTs with hashmaps in dns_transport
as dns_transport_find() is only concerned with finding an exact
match on the specified name it doesn't need to use a tree data
structure internally, we can replace the RBTs with hash tables.
projects / thirdparty / bind9.git / log
