From: Remi Gacogne Date: Wed, 16 Aug 2023 13:16:33 +0000 (+0200) Subject: Fix building our fuzzing targets from a dist tarball X-Git-Tag: rec-5.0.0-alpha1~20^2~2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=cabda036928ed338184b4a7a677d7f51b35c21b2;p=thirdparty%2Fpdns.git Fix building our fuzzing targets from a dist tarball Until now all of our fuzzing targets where built with the authoritative server, even though one of them is specific to dnsdist. This made it easy to build all of them at once, especially for OSS-Fuzz and CI-Fuzz, but had the unfortunate drawback of pulling several dnsdist-specific files into the main pdns/ directory for no good reason. It also prevented building the fuzzing targets from a dist tarball/directory. This commit moves the dnsdist-specific fuzzing target to the dnsdist build process, and ensure that the standalone_fuzz_target_runner.cc file is part of the dist tarball, making it possible to build the fuzzing targets from the dist. It does not move the dnsdist-specific files to the pdns/dnsdistdist/ directory yet because this would conflict with existing PRs. --- diff --git a/fuzzing/README.md b/fuzzing/README.md index c2637d4338..f8ec89d190 100644 --- a/fuzzing/README.md +++ b/fuzzing/README.md @@ -5,21 +5,24 @@ This repository contains several fuzzing targets that can be used with generic fuzzing engines like AFL and libFuzzer. These targets are built by passing the --enable-fuzz-targets option to the -configure, then building as usual. You can also build only these targets -by going into the pdns/ directory and issuing a 'make fuzz_targets' command. +configure of the authoritative server and dnsdist, then building them as usual. +You can also build only these targets manually by going into the pdns/ directory +and issuing a 'make fuzz_targets' command for the authoritative server, +or going into the pdns/dnsdistdist and issuing a 'make fuzz_targets' command for +dnsdist. The current targets cover: -- the auth, dnsdist and rec packet caches (fuzz_target_packetcache and - fuzz_target_dnsdistcache) ; +- the auth and rec packet cache (fuzz_target_packetcache) ; - MOADNSParser (fuzz_target_moadnsparser) ; - the Proxy Protocol parser (fuzz_target_proxyprotocol) ; - the HTTP parser we use (YaHTTP, fuzz_target_yahttp) ; - ZoneParserTNG (fuzz_target_zoneparsertng). - Parts of the ragel-generated parser (parseRFC1035CharString in - fuzz_target_dnslabeltext) + fuzz_target_dnslabeltext) ; +- the dnsdist packet cache (fuzz_target_dnsdistcache). By default the targets are linked against a standalone target, -pdns/standalone_fuzz_target_runner.cc, which does no fuzzing but makes it easy +standalone_fuzz_target_runner.cc, which does no fuzzing but makes it easy to check a given test file, or just that the fuzzing targets can be built properly. This behaviour can be changed via the LIB_FUZZING_ENGINE variable, for example @@ -59,7 +62,7 @@ in the fuzzing/corpus/zones/ directory. Quickly getting started (using clang 11) ---------------------------------------- -First, confgure: +First, configure the authoritative server: ``` LIB_FUZZING_ENGINE="/usr/lib/clang/11.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a" \ @@ -70,6 +73,12 @@ LIB_FUZZING_ENGINE="/usr/lib/clang/11.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a" ./configure --without-dynmodules --with-modules= --disable-lua-records --disable-ixfrdist --enable-fuzz-targets --disable-dependency-tracking --disable-silent-rules --enable-asan --enable-ubsan ``` +If you build the fuzzing targets only, you will need to issue the following commands first: +``` +make -j2 -C ext/arc4random/ +make -j2 -C ext/yahttp/ +``` + Then build: ``` diff --git a/pdns/Makefile.am b/pdns/Makefile.am index 625bc1e044..502a2e33a1 100644 --- a/pdns/Makefile.am +++ b/pdns/Makefile.am @@ -63,6 +63,7 @@ EXTRA_DIST = \ lua-record.cc \ minicurl.cc \ minicurl.hh \ + standalone_fuzz_target_runner.cc \ api-swagger.yaml \ api-swagger.json \ requirements.txt \ @@ -1527,7 +1528,6 @@ LIB_FUZZING_ENGINE ?= standalone_fuzz_target_runner.o standalone_fuzz_target_runner.o: standalone_fuzz_target_runner.cc fuzz_targets_programs = \ - fuzz_target_dnsdistcache \ fuzz_target_moadnsparser \ fuzz_target_packetcache \ fuzz_target_proxyprotocol \ @@ -1601,31 +1601,6 @@ fuzz_target_proxyprotocol_DEPENDENCIES = $(fuzz_targets_deps) fuzz_target_proxyprotocol_LDFLAGS = $(fuzz_targets_ldflags) fuzz_target_proxyprotocol_LDADD = $(fuzz_targets_libs) -fuzz_target_dnsdistcache_SOURCES = \ - channel.hh channel.cc \ - dns.cc dns.hh \ - dnsdist-cache.cc dnsdist-cache.hh \ - dnsdist-ecs.cc dnsdist-ecs.hh \ - dnsdist-idstate.hh \ - dnsdist-protocols.cc dnsdist-protocols.hh \ - dnslabeltext.cc \ - dnsname.cc dnsname.hh \ - dnsparser.cc dnsparser.hh \ - dnswriter.cc dnswriter.hh \ - doh.hh \ - ednsoptions.cc ednsoptions.hh \ - ednssubnet.cc ednssubnet.hh \ - fuzz_dnsdistcache.cc \ - iputils.cc iputils.hh \ - misc.cc misc.hh \ - packetcache.hh \ - qtype.cc qtype.hh \ - svc-records.cc svc-records.hh - -fuzz_target_dnsdistcache_DEPENDENCIES = $(fuzz_targets_deps) -fuzz_target_dnsdistcache_LDFLAGS = $(fuzz_targets_ldflags) -fuzz_target_dnsdistcache_LDADD = $(fuzz_targets_libs) - fuzz_target_yahttp_SOURCES = \ fuzz_yahttp.cc diff --git a/pdns/dnsdistdist/Makefile.am b/pdns/dnsdistdist/Makefile.am index 9b951a5866..57ecaaa26e 100644 --- a/pdns/dnsdistdist/Makefile.am +++ b/pdns/dnsdistdist/Makefile.am @@ -106,6 +106,7 @@ EXTRA_DIST=COPYING \ kqueuemplexer.cc \ portsmplexer.cc \ cdb.cc cdb.hh \ + standalone_fuzz_target_runner.cc \ ext/lmdb-safe/lmdb-safe.cc ext/lmdb-safe/lmdb-safe.hh \ ext/protozero/include/* \ builder-support/gen-version @@ -451,6 +452,60 @@ testrunner_SOURCES += \ portsmplexer.cc endif +if FUZZ_TARGETS + +LIB_FUZZING_ENGINE ?= standalone_fuzz_target_runner.o + +standalone_fuzz_target_runner.o: standalone_fuzz_target_runner.cc + +fuzz_targets_programs = \ + fuzz_target_dnsdistcache + +fuzz_targets: $(ARC4RANDOM_LIBS) $(fuzz_targets_programs) + +bin_PROGRAMS += \ + $(fuzz_targets_programs) + +fuzz_targets_libs = \ + $(LIBCRYPTO_LIBS) \ + $(LIB_FUZZING_ENGINE) + +fuzz_targets_ldflags = \ + $(AM_LDFLAGS) \ + $(DYNLINKFLAGS) \ + $(LIBCRYPTO_LDFLAGS) \ + $(FUZZING_LDFLAGS) + +# we need the mockup runner to be built, but not linked if a real fuzzing engine is used +fuzz_targets_deps = standalone_fuzz_target_runner.o + +fuzz_target_dnsdistcache_SOURCES = \ + channel.hh channel.cc \ + dns.cc dns.hh \ + dnsdist-cache.cc dnsdist-cache.hh \ + dnsdist-ecs.cc dnsdist-ecs.hh \ + dnsdist-idstate.hh \ + dnsdist-protocols.cc dnsdist-protocols.hh \ + dnslabeltext.cc \ + dnsname.cc dnsname.hh \ + dnsparser.cc dnsparser.hh \ + dnswriter.cc dnswriter.hh \ + doh.hh \ + ednsoptions.cc ednsoptions.hh \ + ednssubnet.cc ednssubnet.hh \ + fuzz_dnsdistcache.cc \ + iputils.cc iputils.hh \ + misc.cc misc.hh \ + packetcache.hh \ + qtype.cc qtype.hh \ + svc-records.cc svc-records.hh + +fuzz_target_dnsdistcache_DEPENDENCIES = $(fuzz_targets_deps) +fuzz_target_dnsdistcache_LDFLAGS = $(fuzz_targets_ldflags) +fuzz_target_dnsdistcache_LDADD = $(fuzz_targets_libs) + +endif + MANPAGES=dnsdist.1 dist_man_MANS=$(MANPAGES) diff --git a/pdns/dnsdistdist/configure.ac b/pdns/dnsdistdist/configure.ac index aa7ffbf7a4..57856ee8e4 100644 --- a/pdns/dnsdistdist/configure.ac +++ b/pdns/dnsdistdist/configure.ac @@ -35,6 +35,7 @@ AC_FUNC_STRERROR_R BOOST_REQUIRE([1.42]) PDNS_ENABLE_UNIT_TESTS +PDNS_ENABLE_FUZZ_TARGETS PDNS_WITH_RE2 DNSDIST_ENABLE_DNSCRYPT PDNS_WITH_EBPF diff --git a/pdns/fuzz_dnsdistcache.cc b/pdns/dnsdistdist/fuzz_dnsdistcache.cc similarity index 100% rename from pdns/fuzz_dnsdistcache.cc rename to pdns/dnsdistdist/fuzz_dnsdistcache.cc diff --git a/pdns/dnsdistdist/fuzz_target_dnsdistcache b/pdns/dnsdistdist/fuzz_target_dnsdistcache new file mode 100755 index 0000000000..8eb51b815d Binary files /dev/null and b/pdns/dnsdistdist/fuzz_target_dnsdistcache differ diff --git a/pdns/dnsdistdist/m4/pdns_enable_fuzz_targets.m4 b/pdns/dnsdistdist/m4/pdns_enable_fuzz_targets.m4 new file mode 120000 index 0000000000..7bec31c434 --- /dev/null +++ b/pdns/dnsdistdist/m4/pdns_enable_fuzz_targets.m4 @@ -0,0 +1 @@ +../../../m4/pdns_enable_fuzz_targets.m4 \ No newline at end of file diff --git a/pdns/dnsdistdist/standalone_fuzz_target_runner.cc b/pdns/dnsdistdist/standalone_fuzz_target_runner.cc new file mode 120000 index 0000000000..61ca1e384c --- /dev/null +++ b/pdns/dnsdistdist/standalone_fuzz_target_runner.cc @@ -0,0 +1 @@ +../standalone_fuzz_target_runner.cc \ No newline at end of file diff --git a/tasks.py b/tasks.py index e78e0a5521..0b326ef26c 100644 --- a/tasks.py +++ b/tasks.py @@ -510,6 +510,7 @@ def ci_dnsdist_configure(c, features): -DDISABLE_FALSE_SHARING_PADDING \ -DDISABLE_NPN' unittests = ' --enable-unit-tests' if os.getenv('UNIT_TESTS') == 'yes' else '' + fuzztargets = '--enable-fuzz-targets' if os.getenv('FUZZING_TARGETS') == 'yes' else '' sanitizers = ' '.join('--enable-'+x for x in os.getenv('SANITIZERS').split('+')) if os.getenv('SANITIZERS') != '' else '' cflags = '-O1 -Werror=vla -Werror=shadow -Wformat=2 -Werror=format-security -Werror=string-plus-int' cxxflags = cflags + ' -Wp,-D_GLIBCXX_ASSERTIONS ' + additional_flags @@ -524,7 +525,7 @@ def ci_dnsdist_configure(c, features): --enable-fortify-source=auto \ --enable-auto-var-init=pattern \ --enable-lto=thin \ - --prefix=/opt/dnsdist %s %s %s''' % (cflags, cxxflags, features_set, sanitizers, unittests), warn=True) + --prefix=/opt/dnsdist %s %s %s %s''' % (cflags, cxxflags, features_set, sanitizers, unittests, fuzztargets), warn=True) if res.exited != 0: c.run('cat config.log') raise UnexpectedExit(res)