Tony Finch [Fri, 22 Apr 2022 13:35:36 +0000 (14:35 +0100)]
Move random number re-seeding out of the hot path
Instead of checking if we need to re-seed for every isc_random call,
seed the random number generator in the libisc global initializer
and the per-thread initializer.
Disable stringop-overread with gcc-11+ Address Sanitizer
When Address Sanitizer is enabled in gcc-11+, number of false positives
might appear like this:
netmgr/udp.c: In function 'isc__nm_udp_send':
netmgr/udp.c:729:13: warning: 'uv_udp_send' reading 16 bytes from a region of size 8 [-Wstringop-overread]
729 | r = uv_udp_send(&uvreq->uv_req.udp_send, &sock->uv_handle.udp,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
730 | &uvreq->uvbuf, 1, sa, udp_send_cb);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
netmgr/udp.c:729:13: note: referencing argument 3 of type 'const uv_buf_t[0]'
In file included from ./include/isc/uv.h:17,
from ./include/isc/barrier.h:31,
from netmgr/udp.c:17:
/usr/include/uv.h:711:15: note: in a call to function 'uv_udp_send'
711 | UV_EXTERN int uv_udp_send(uv_udp_send_t* req,
| ^~~~~~~~~~~
Disable the warning globally in the autoconf, instead of just locally in
a single CI job, as it might affect people outside our GitLab CI.
Ondřej Surý [Mon, 29 Aug 2022 11:42:14 +0000 (13:42 +0200)]
Add missing isc_refcount_destroy() for isc__nmsocket_t
The destructor for the isc__nmsocket_t was missing call to the
isc_refcount_destroy() on the reference counter, which might lead to
spurious ThreadSanitizer data race warnings if we ever change the
acquire-release memory order in the isc_refcount_decrement().
Ondřej Surý [Mon, 29 Aug 2022 10:11:37 +0000 (12:11 +0200)]
Reorder the uv_close() calls to close the socket immediately
Simplify the closing code - during the loopmgr implementation, it was
discovered that the various lists used by the uv_loop_t aren't FIFO, but
LIFO. See doc/dev/libuv.md for more details.
With this knowledge, we can close the protocol handles (uv_udp_t and
uv_tcp_t) and uv_timer_t at the same time by reordering the uv_close()
calls, and thus making sure that after calling the
isc__nm_stoplistening(), the code will not issue any additional callback
calls (accept, read) on the socket that stopped listening.
This might help with the TLS and DoH shutting down sequence as described
in the [GL #3509] as we now stop the reading, stop the timer and call
the uv_close() as earliest as possible.
Improve the udp_shutdown_read and udp_cancel_read tests
In the udp_shutdown_read unit test, delay the isc_loopmgr_shutdown() to
the send callback, and in the udp_cancel_read test wait for a single
timed out test, then read again, send an UDP packet and cancel the read
from the send callback.
The network manager UDP code was misinterpreting when the libuv called
the udp_recv_cb with nrecv == 0 and addr == NULL -> this doesn't really
mean that the "stream" has ended, but the libuv indicates that the
receive buffer can be freed. This could lead to assertion failure in
the code that calls isc_nm_read() from the network manager read callback
due to the extra spurious callbacks.
Properly handle the extra callback calls from the libuv in the client
read callback, and refactor the UDP isc_nm_read() implementation to be
synchronous, so no datagram is lost between the time that we stop the
reading from the UDP socket and we restart it again in the asychronous
udpread event.
Add a unit test that tests the isc_nm_read() call from the read
callback to receive two datagrams.
def _net_read(sock, count, expiration):
"""Read the specified number of bytes from sock. Keep trying until we
either get the desired amount, or we hit EOF.
A Timeout exception will be raised if the operation is not completed
by the expiration time.
"""
s = b''
while count > 0:
try:
> n = sock.recv(count)
E socket.timeout: timed out
Ondřej Surý [Wed, 20 Oct 2021 16:14:49 +0000 (18:14 +0200)]
Add support for reporting status via sd_notify()
sd_notify() may be called by a service to notify the service manager
about state changes. It can be used to send arbitrary information,
encoded in an environment-block-like string. Most importantly, it can be
used for start-up completion notification.
Add libsystemd check to autoconf script and when the library is detected
add calls to sd_notify() around the server->reload_status changes.
Mark Andrews [Wed, 7 Sep 2022 23:48:27 +0000 (09:48 +1000)]
Emit key algorithm + key id in dnssec signing statsistics
If there was a collision of key id across algorithms it was not
possible to determine where counter applies to which algorithm for
xml statistics while for json only one of the values was emitted.
The key names are now "<algorithm-number>+<id>" (e.g. "8+54274").
the 'resolve' binary was added for testing dns_client as part of
the export library. the export libraries are no longer supported,
and tests using 'delv' provide the same coverage, so 'resolve' can
be removed now.
merge dns_request_createvia() into dns_request_create()
dns_request_create() was a front-end to dns_request_createvia() that
was only used by test binaries. dns_request_createvia() has been
renamed to dns_request_create(), and the test programs that formerly
used dns_request_create() have been updated to use the new parameters.
Tony Finch [Tue, 13 Sep 2022 16:17:55 +0000 (17:17 +0100)]
Fix dig idna test on Debian 10 "buster"
The test expected `xn--ah-` to be treated as a syntax error (punycode
requires letters after the last hyphen) but libidn2 on buster
converted the label to `ah` instead. To avoid this bug, change the
invalid label to `xn--0000h` which translates to an out-of-range
unicode codepoint (beyond the maximum value) which is corectly
trated as invalid in older libidn2.
Tony Finch [Tue, 13 Sep 2022 10:35:08 +0000 (11:35 +0100)]
Fix out-of-tree tests
The change to `testsock.pl` in commit 258a896a broke the system
tests in out-of-tree builds because `ifconfig.sh.in` is not
copied to the worktree. Use `ifconfig.sh` instead.
In rndc_recvdone(), if 'sends' was not 0, then 'recvs' was not
decremented, in which case isc_loopmgr_shutdown() was never reached,
which could cause a hang. (This has not been observed to happen, but
the code was incorrect on examination.)
Tony Finch [Thu, 30 Jun 2022 15:31:15 +0000 (16:31 +0100)]
CHANGES note for [GL !6516]
[cleanup] Move the duplicated ASCII case conversion tables to
isc_ascii where they can be shared, and replace the
various hot-path tolower() loops with calls to new
isc_ascii implementations.
Tony Finch [Mon, 27 Jun 2022 11:57:28 +0000 (12:57 +0100)]
General-purpose unrolled ASCII tolower() loops
When converting a string to lower case, the compiler is able to
autovectorize nicely, so a nice simple implementation is also very
fast, comparable to memcpy().
Comparisons are more difficult for the compiler, so we convert eight
bytes at a time using "SIMD within a register" tricks. Experiments
indicate it's best to stick to simple loops for shorter strings and
the remainder of long strings.
Tony Finch [Fri, 24 Jun 2022 21:11:02 +0000 (22:11 +0100)]
Consolidate some ASCII tables in `isc/ascii` and `isc/hex`
There were a number of places that had copies of various ASCII
tables (case conversion, hex and decimal conversion) that are intended
to be faster than the ctype.h macros, or avoid locale pollution.
Move them into libisc, and wrap the lookup tables with macros that
avoid the ctype.h gotchas.
Tony Finch [Wed, 31 Aug 2022 20:09:06 +0000 (21:09 +0100)]
The system tests are using another IP address
Reduce the number of places that know about the number of IP addresses
required by the system tests, by changing `testsock.pl` to read the
`max` from `ifconfig.sh.in`. This should make the test runner fail
early with a clear message when the interfaces have been set up by an
obsolete script.
Add comments to cross-reference `ifconfig.sh.in`, `testsock.pl`, and
`org.isc.bind.system` to make it easier to remember what needs
updating when an IP address is added.
Tony Finch [Mon, 5 Sep 2022 14:49:49 +0000 (15:49 +0100)]
More lenient IDNA processing in dig
If there are any problems with IDN processing, DiG will now quietly
handle the name as if IDN were disabled. This means that international
query names are rendered verbatim on the wire, and ACE names are
printed raw without conversion to UTF8.
If you want to check the syntax of international domain names,
use the `idn2` utility.
Tony Finch [Fri, 9 Sep 2022 07:21:10 +0000 (08:21 +0100)]
Ensure that named_server_t is properly initialized
There was a ubsan error reporting an invalid value for interface_auto
(a boolean value cannot be 190) because it was not initialized. To
avoid this problem happening again, ensure the whole of the server
structure is initialized to zero before setting the (relatively few)
non-zero elements.
Michał Kępień [Fri, 9 Sep 2022 18:25:47 +0000 (20:25 +0200)]
Fix error reporting for POSIX Threads functions
Commit 3608abc8fa6a33046e1d34a0789cf7c9547f09ad inadvertently carried
over a mistake in logging pthread_cond_init() errors to the
ERRNO_CHECK() preprocessor macro: instead of passing the value returned
by a given pthread_*() function to strerror_r(), ERRNO_CHECK() passes
the errno variable to strerror_r(). This causes bogus error reports
because POSIX Threads API functions do not set the errno variable.
Fix by passing the value returned by a given pthread_*() function
instead of the errno variable to strerror_r(). Since this change makes
the name of the affected macro (ERRNO_CHECK()) confusing, rename the
latter to PTHREADS_RUNTIME_CHECK(). Also log the integer error value
returned by a given pthread_*() function verbatim to rule out any
further confusion in runtime error reporting.
Fix RRL responses-per-second bypass using wildcard names
It is possible to bypass Response Rate Limiting (RRL)
`responses-per-second` limitation using specially crafted wildcard
names, because the current implementation, when encountering a found
DNS name generated from a wildcard record, just strips the leftmost
label of the name before making a key for the bucket.
While that technique helps with limiting random requests like
<random>.example.com (because all those requests will be accounted
as belonging to a bucket constructed from "example.com" name), it does
not help with random names like subdomain.<random>.example.com.
The best solution would have been to strip not just the leftmost
label, but as many labels as necessary until reaching the suffix part
of the wildcard record from which the found name is generated, however,
we do not have that information readily available in the context of RRL
processing code.
Fix the issue by interpreting all valid wildcard domain names as
the zone's origin name concatenated to the "*" name, so they all will
be put into the same bucket.
Matthijs Mekking [Tue, 30 Aug 2022 08:04:16 +0000 (10:04 +0200)]
Update inline system test, zone 'retransfer3.'
The zone 'retransfer3.' tests whether zones that 'rndc signing
-nsec3param' requests are queued even if the zone is not loaded.
The test assumes that if 'rndc signing -list' shows that the zone is
done signing with two keys, and there are no NSEC3 chains pending, the
zone is done handling the '-nsec3param' queued requests. However, it
is possible that the 'rndc signing -list' command is received before
the corresponding privatetype records are added to the zone (the records
that are used to retrieve the signing status with 'rndc signing').
This is what happens in test failure
https://gitlab.isc.org/isc-projects/bind9/-/jobs/2722752.
The 'rndc signing -list retransfer3' is thus an unreliable check.
It is simpler to just remove the check and wait for a certain amount
of time and check whether ns3 has re-signed the zone using NSEC3.
Michał Kępień [Wed, 7 Sep 2022 10:50:08 +0000 (12:50 +0200)]
Fix building with --disable-doh
Commit b69e783164cd50e3306364668558e460617ee8fc inadvertently caused
builds using the --disable-doh switch to fail, by putting the
declaration of the isc__nm_async_settlsctx() function inside an #ifdef
block that is only evaluated when DNS-over-HTTPS support is enabled.
This results in the following compilation errors being triggered:
netmgr/netmgr.c:2657:1: error: no previous prototype for 'isc__nm_async_settlsctx' [-Werror=missing-prototypes]
2657 | isc__nm_async_settlsctx(isc__networker_t *worker, isc__netievent_t *ev0) {
| ^~~~~~~~~~~~~~~~~~~~~~~
Fix by making the declaration of the isc__nm_async_settlsctx() function
in lib/isc/netmgr/netmgr-int.h visible regardless of whether
DNS-over-HTTPS support is enabled or not.
Mark Andrews [Mon, 18 Jul 2022 07:21:25 +0000 (17:21 +1000)]
Silence REVERSE_INULL
Remove unnecessary != NULL checks
*** CID 352809: Null pointer dereferences (REVERSE_INULL) /lib/dns/message.c: 4654 in dns_message_buildopt()
4648 if (rdata != NULL) {
4649 dns_message_puttemprdata(message, &rdata);
4650 }
4651 if (rdataset != NULL) {
4652 dns_message_puttemprdataset(message, &rdataset);
4653 }
>>> CID 352809: Null pointer dereferences (REVERSE_INULL)
>>> Null-checking "rdatalist" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
4654 if (rdatalist != NULL) {
4655 dns_message_puttemprdatalist(message, &rdatalist);
4656 }
4657 return (result);
4658 }
4659
The usage of xmlInitThreads() and xmlCleanupThreads() functions in
libxml2 is now marked as deprecated, and these functions will be made
private in the future.
Use xmlInitParser() and xmlCleanupParser() instead of them.
The isc_nm_listentlsdns() function erroneously calls
isc__nm_tcpdns_stoplistening() instead of isc__nm_tlsdns_stoplistening()
when something goes wrong, which can cause an assertion failure.
Ondřej Surý [Fri, 26 Aug 2022 10:24:07 +0000 (12:24 +0200)]
Allow fallback to IDNA2003 processing
In several cases where IDNA2008 mappings do not exist whereas IDNA2003
mappings do, dig was failing to process the suplied domain name. Take a
backwards compatible approach, and convert the domain to IDNA2008 form,
and if that fails try the IDNA2003 conversion.
projects / thirdparty / bind9.git / log
