Re-Running the Tests
---
If there is a requirement to re-run a test (or the entire test suite), the
-files produced by the tests should be deleted first. Normally, these files are
-deleted if the test succeeds but are retained on error. The legacy.run.sh
-script automatically calls a given test's clean.sh script before invoking its
-setup.sh script.
-
-Deletion of the files produced by the set of tests (e.g. after the execution
-of "runall.sh") can be carried out using the command:
-
- sh cleanall.sh
-
-or
-
- make testclean
-
-(Note that the Makefile has two other targets for cleaning up files: "clean"
-will delete all the files produced by the tests, as well as the object and
-executable files used by the tests. "distclean" does all the work of "clean"
-as well as deleting configuration files produced by "configure".)
+files produced by the tests should be deleted first. This is no longer
+supported by the legacy runner and the pytest runner should be used instead.
Developer Notes
tests_sh_xyz.py A glue file for the pytest runner for executing shell tests.
-clean.sh Run at the end to clean up temporary files, but only if the test
- was completed successfully and its running was not inhibited by the
- "-n" switch being passed to "legacy.run.sh". Otherwise the
- temporary files are left in place for inspection.
-
ns<N> These subdirectories contain test name servers that can be queried
or can interact with each other. The value of N indicates the
address the server listens on: for example, ns2 listens on
(cd mytest ; sh prereq.sh -D xyz)
(cd mytest ; sh setup.sh -D xyz)
(cd mytest ; sh tests.sh -D xyz)
- (cd mytest ; sh clean.sh -D xyz)
No arguments will be passed to the test scripts if the test is run as part of
a run of the full test suite (e.g. the tests are started with "runall.sh").
def test_xyz(run_tests_sh):
run_tests_sh()
-clean.sh
----
-The inverse of "setup.sh", this is invoked by the framework to clean up the
-test directory. It should delete all files that have been created by the test
-during its run.
-
Starting Nameservers
---
test is written to bin/tests/system/. This file is only produced when the test
is run as part of the entire test suite (e.g. via "runall.sh").
-If the test fails, all these files are retained. But if the test succeeds,
-they are cleaned up at different times:
-
-1. Files generated by the test itself are cleaned up by the test's own
-"clean.sh", which is called from "legacy.run.sh".
-
-2. Files that may not be cleaned up if named exits abnormally can be removed
-using the "cleanall.sh" script.
-
-3. "test.output.*" files are deleted when the test suite ends. At this point,
-the file "testsummary.sh" is called which concatenates all the "test.output.*"
-files into a single "systests.output" file before deleting them.
+If the test fails, all these files are retained.
. ../conf.sh
-$SHELL clean.sh
$SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 2 3 >ns2/example.db
$SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 2 3 >ns2/tsigzone.db
copy_setports ns2/named1.conf.in ns2/named.conf
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named1.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named1.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
set -e
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
# has been appended since the last time we read it.
#
# Calling some of these functions causes temporary *.prev files to be
-# created that need to be cleaned up manually (usually by a given system
-# test's clean.sh script).
+# created.
#
# Note that unlike other nextpart*() functions, nextpartread() is not
# meant to be directly used in system tests; its sole purpose is to
This fixture is responsible for creating (and potentially removing) a
copy of the system test directory which is used as a temporary
directory for the test execution.
-
- FUTURE: This removes the need to have clean.sh scripts.
"""
def get_test_result():
set -e
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
set -e
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
test -r $RANDFILE || $GENRANDOM 800 $RANDFILE
copy_setports ns1/named.conf.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named1.conf.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named1.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns2/named1.conf.in ns2/named.conf
for i in 1 2 3 4 5 6 7 other bogus; do
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named.conf.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns2/named.conf.in ns2/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
copy_setports ns4/named.conf.in ns4/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
cp ns1/generic.db.in ns1/changed.db
cp ns1/changed.ver1.jnl.saved ns1/changed.db.jnl
set -e
-$SHELL clean.sh
-
mkdir keys
mkdir ns3/keys
. ../conf.sh
-$SHELL clean.sh
-
(cd ns6 && $SHELL -e sign.sh)
(cd ns7 && $SHELL -e sign.sh)
-
-$SHELL clean.sh
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.plain.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
# shellcheck source=conf.sh
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
set -e
-$SHELL clean.sh
-
copy_setports ns2/named.conf.in ns2/named.conf
(
cd ns2
. ../conf.sh
-$SHELL clean.sh
-
$SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 1 >ns1/example.db
copy_setports ns1/named.conf.in ns1/named.conf
. ../conf.sh
-#
-# jnl and database files MUST be removed before we start
-#
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL ./clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
for dir in ns*; do
touch $dir/named.run
nextpart $dir/named.run >/dev/null
. ../conf.sh
-$SHELL clean.sh
-
$PERL testgen.pl
copy_setports ns1/named.conf.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns2/named1.conf.in ns2/named.conf
copy_setports ns2/named-alt1.conf.in ns2/named-alt1.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named1.conf.in ns1/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
copy_setports ns4/named.conf.in ns4/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
copy_setports ns3/named.conf.in ns3/named.conf
# shellcheck disable=SC1091
. ../conf.sh
-$SHELL clean.sh
-
$SHELL "${TOP_SRCDIR}"/bin/tests/system/genzone.sh 2 >ns1/example.db
copy_setports ns1/named.conf.in ns1/named.conf
# shellcheck disable=SC1091
. ../conf.sh
-$SHELL clean.sh
-
$SHELL "${TOP_SRCDIR}"/bin/tests/system/genzone.sh 2 >ns1/example.db
copy_setports ns1/named.conf.in ns1/named.conf
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
if $FEATURETEST --md5; then
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
key=$($KEYGEN -Cq -K ns1 -a $DEFAULT_ALGORITHM -b $DEFAULT_BITS -n HOST -T KEY key.example.nil.)
. ../conf.sh
-$SHELL clean.sh
-
copy_setports ns1/named.conf.in ns1/named.conf
copy_setports ns2/named.conf.in ns2/named.conf
- `tests.sh`: runs all the test cases. A non-zero return value results in R:FAIL
-- `clean.sh`: run at the end to clean up temporary files, but only if the
- test was completed successfully; otherwise the temporary files are left
- in place for inspection.
-
- `ns[X]`: these subdirectories contain test name servers that can be
queried or can interact with each other. (For example, `ns1` might be
running as a root server, `ns2` as a TLD server, and `ns3` as a recursive