===
The tests can be run individually using the following command:
- sh run.sh [flags] <test-name>
+ sh run.sh [flags] <test-name> [<test-arguments>]
e.g.
Optional flags are:
- -p <number> Sets the range of ports used by the test. If not
- specified, the test will use ports 5300 to 5309.
+ -p <number> Sets the range of ports used by the test. A block of 10
+ ports is available for each test, the number given to the
+ "-p" switch being the number of the start of that block
+ (e.g. "-p 7900" will mean that the test is able to use
+ ports 7900 through 7909). If not specified, the test will
+ use ports 5300 to 5309.
-n Noclean - do not remove the output files if the test
completes successfully. By default, files created by the
test are deleted if it passes; they are not deleted if the
Python or Perl) that feature test-specific behavior. The
servers are automatically started before the test is run
and stopped after it ends. This flag leaves them running
- at the end of the test.
+ at the end of the test, so that additional queries can be
+ sent by hand. To stop the servers afterwards, use the
+ command "sh stop.sh <test-name>".
-d <arg> Arguments to the "date" command used to produce the
start and end time of the tests. For example, the
switch
-d "+%Y-%m-%d:%h:%M:%s"
- would cause the "S" and "E" messages (see below) to have the
- date looking like "2017-11-23:16:06:32" instead of the
+ would cause the "S" and "E" messages (see below) to have
+ the date looking like "2017-11-23:16:06:32" instead of the
default "Thu, 23 Nov 2017 16:06:32 +0000".
+Arguments are:
+
+ test-name Mandatory. The name of the test, which is the name of the
+ subdirectory in bin/tests/system holding the test files.
+
+ test-arguments Optional arguments that are passed to each of the test's
+ scripts.
+
To run all the system tests, type either:
sh runall.sh [numproc]
(in the bin/tests/system) directory.
The "numproc" option specifies the maximum number of tests that can run
-simultaneously. The default is 1, which means that all the test run
+simultaneously. The default is 1, which means that all of the tests run
sequentially. If greater than 1, up to "numproc" tests will run simultaneously.
(Each will use a unique set of ports, so there is no danger of them interfering
with one another.)
Deletion of files produced by an individual test can be done with the
command:
- (cd testname ; sh clean.sh)
+ sh clean.sh <test-name>
Deletion of the files produced by the set of tests (e.g. after the execution
of "runall.sh") can be deleted by the command:
Port Usage
---
In order for the tests to run in parallel, each test requires a unique set of
-ports. These are specified by the "-p" option passed to "run.sh". This option
-is then passed to each of the test control scripts listed above.
+ports. These are specified by the "-p" option passed to "run.sh", which sets
+environment variables that the scripts listed above can reference.
The convention used in the system tests is that the number passed is the start
of a range of 10 ports. The test is free to use the ports as required,
although present usage is that the lowest port is used as the query port and
-the highest is used as the control port. This is reinforced by the script
-getopts.sh: if used to parse the "-p" option (see below), the script sets the
-following shell variables:
+the highest is used as the control port. This is reinforced by the environment
+variables set by run.sh. These are:
+
+ PORT Number to be used for the query port.
+ CONTROLPORT Number to be used as the RNDC control port.
+ EXTRAPORT1 - EXTRAPORT8 Eight port numbers that can be use as needed.
+
+Two other environment variables are defined:
- port Number to be used for the query port.
- controlport Number to be used as the RNDC control port.
- aport1 - aport8 Eight port numbers that the test can use as needed.
+ LOWPORT The lowest port number in the range.
+ HIGHPORT The highest port number in the range.
When running tests in paralel (i.e. giving a value of "numproc" greater than 1
in the "make" or "runall.sh" commands listed above), it is guaranteed that each
test will get a set of unique port numbers.
-In addition, the "getopts.sh" script also defines the following symbols:
-
- portlow Lowest port number in the range.
- porthigh Highest port number in the range.
-
Writing a Test
---
General
---
-Each of the four scripts will be invoked with the command
+1. Each of the four scripts will be invoked with the command
+
+ sh <script> [<arguments>]
- sh <script> -p <baseport> -- <arguments>
+Thhe optional arguments are test-specific. They are
Each script should start with the following lines:
SYSTEMTESTTOP=..
. $SYSTEMTESTTOP/conf.sh
- . $SYSTEMTESTTOP/getopts.sh
"conf.sh" defines a series of environment variables together with functions
-useful for the test scripts. "getopts.sh" parses the "-p" option and sets the
-shell variables listed above. (They are not combined into one script because,
-in certain instances - notably in "run.sh" - some processing is required
-between the setting of the environment variables and the parsing of the port).
-
-The "--" between the "-p <baseport>" and any other arguments is required:
-without it, any other switches passed to the script would be parsed by
-getopts.sh, which would return an error because it would not recognise them.
-getopts.sh removes the "-p <port> --" from the argument list, leaving the
-script free to do its own parsing of any additional arguments.
-
-For example, if "test.sh" is invoked as:
-
- sh tests.sh -p 12340 -- -D 1
-
-... and it includes the three lines listed above, after the execution of the
-code in getopts.sh, the following variables would be defined (with their
-associated values):
-
- port 12340
- aport1 12341
- aport2 12342
- : :
- aport8 12348
- controlport 12349
- portlow 12340
- porthigh 12349
-
- $1 -D
- $2 1
-
-Should a script need to invoke another, it should pass the base port with the
-"-p" switch and add any additional arguments after the "--", i.e. using the
-same format as listed above in the example for invoking "tests.sh"
+useful for the test scripts.
prereq.sh
software needed to run the test is available and/or whether BIND has been
configured with the appropriate options required.
- * If the software required to run the test is present and the BIND configure
- options are correct, prereq.sh should return with a status code of 0.
+ * If the software required to run the test is present and the BIND
+ configure options are correct, prereq.sh should return with a status code
+ of 0.
* If the software required to run the test is not available and/or BIND
has not been configured with the appropriate options, prereq.sh should
port numbers.
To do this, configuration files should be supplied in the form of templates
-containing tokens identifying ports. The tokens have the same name as the shell
-variables listed above, but in upper-case and prefixed and suffixed by the "@"
-symbol. For example, a fragment of a configuration file template might look
-like:
+containing tokens identifying ports. The tokens have the same name as the
+shell variables listed above, but in upper-case and prefixed and suffixed by
+the "@" symbol. For example, a fragment of a configuration file template might
+look like:
controls {
inet 10.53.0.1 port @CONTROLPORT@ allow { any; } keys { rndc_key; };
};
setup.sh should copy the template to the desired filename using the
-"copy_setports" shell function defined in "getopts.sh", i.e.
+"copy_setports" shell function defined in "conf.sh", i.e.
copy_setports ns1/named.conf.in ns1/named.conf
-This replaces the tokens @PORT@, @CONTROLPORT@, @APORT1@ through @APORT8@ with
-the contents of the shell variables listed above. setup.sh should do this for
-all configuration files required when the test starts.
-
-A second important responsibility of setup.sh is to create a file called
-named.port in each "named" nameserver directory that defines the query port for
-that server, e.g.
-
- echo $port > ns1/named.port
-
-The file is used by the framework to determine the port it should use to query
-the nameserver. In most cases, all nameservers will use the same port number
-for queries. Some tests may require a different port to be used for queries,
-which is why this task has been delegated to test-specific setup rather than be
-part of the framework.
-
+This replaces the tokens @PORT@, @CONTROLPORT@, @EXTRAPORT1@ through
+@EXTRAPORT8@ with the contents of the shell variables listed above. setup.sh
+should do this for all configuration files required when the test starts.
tests.sh
---
-This is the main test file and the contents depend on the test. The contents are
-completely up to the developer, although most test scripts have a form similar
-to the following for each test:
+This is the main test file and the contents depend on the test. The contents
+are completely up to the developer, although most test scripts have a form
+similar to the following for each test:
1. n=`expr $n + 1`
2. echo_i "prime cache nodata.example ($n)"
+++ /dev/null
-#!/bin/sh
-# Copyright (C) 2017 Internet Systems Consortium, Inc. ("ISC")
-#
-# This Source Code Form is subject to the terms of the Mozilla Public
-# License, v. 2.0. If a copy of the MPL was not distributed with this
-# file, You can obtain one at http://mozilla.org/MPL/2.0/.
-
-# Shell script snippet, must be sourced.
-#
-# Most system tests require use of at least two ports: the nameserver query
-# port and a port for RNDC access. In addition, some tests require additional
-# ports (e.g. for tests of transfers between nameservers).
-#
-# To allow tests to run in parallel, each test must be allocated a unique set
-# ports to use.
-#
-# This script is used during testing to parse the "-p" option on the command
-# line invoking scripts used during the test. The option sets the base of
-# a block of 10 ports used by the test. A shell symbol is set for each port:
-#
-# port Port used for queries (default to 5300)
-# aport1 First additional port (set to $port + 1)
-# :
-# aport8 Eighth additional port (set to $port + 8)
-# controlport Port used for RNDC (set to $port + 9)
-#
-# The fiule also defines a simple shell function to
-
-port=5300
-
-while getopts ":p:" flag; do
- case "$flag" in
- p) port=$OPTARG ;;
- -) break ;;
- *) exit 1 ;;
- esac
-done
-shift $(($OPTIND - 1))
-OPTIND=1
-
-# Ensure port is numeric, above 1024 (limit of privileged port) and that
-# the upper of the 10 ports notionally assigned does not exceed 65535.
-
-if [ "$((${port}+0))" != "${port}" ] || [ "${port}" -le 1024 ] || [ "${port}" -gt 65520 ]; then
- echo "Base of port range ($port) must be numeric and in the range 1025 to 65520" >&2
- exit 1
-fi
-
-aport1=$(($port + 1))
-aport2=$(($port + 2))
-aport3=$(($port + 3))
-aport4=$(($port + 4))
-aport5=$(($port + 5))
-aport6=$(($port + 6))
-aport7=$(($port + 7))
-aport8=$(($port + 8))
-controlport=$(($port + 9))
-
-# Two more symbols that denote the limits of the range.
-
-portlow=$port
-porthigh=$controlport
-
-
-# copy_setports - Copy Configuration File and Replace Ports
-#
-# Convenience function to copy a configuration file, replacing the symbols
-# PORT, CONTROLPORT and APORT[1-8] with the port numbers set by the "-p"
-# option passed to the script.
-#
-# Usage:
-# copy_setports infile outfile
-
-copy_setports() {
- sed -e "s/@PORT@/${port}/g" \
- -e "s/@APORT1@/${aport1}/g" \
- -e "s/@APORT2@/${aport1}/g" \
- -e "s/@APORT3@/${aport1}/g" \
- -e "s/@APORT4@/${aport1}/g" \
- -e "s/@APORT5@/${aport1}/g" \
- -e "s/@APORT6@/${aport1}/g" \
- -e "s/@APORT7@/${aport1}/g" \
- -e "s/@APORT8@/${aport1}/g" \
- -e "s/@CONTROLPORT@/${controlport}/g" < $1 > $2
-}
test -d $test || { echofail "$0: $test: no such test" >&2; exit 1; }
-# Validate the port number and obtain other port numbers.
-. $SYSTEMTESTTOP/getopts.sh -p "$baseport"
+# Define the number of ports allocated for this test, and the lowest and
+# highest valid values for the "-p" option. The lowest valid value is one more
+# than the highest privileged port (1024). As the number specifies the lowest
+# port number in a block of ports, the highest valid value is such that the
+# highest port number in that block is 65535.
+#
+# N.B. It is assumed that the number of ports is >= 10.
+numport=10
+minvalid=`expr 1024 + 1`
+maxvalid=`expr 65535 - $numport + 1`
+
+test "$baseport" -eq "$baseport" > /dev/null 2>&1
+if [ $? -ne 0 ]; then
+ echofail "Must specify a numeric value for the port"
+ exit 1
+elif [ $baseport -lt $minvalid -o $baseport -gt $maxvalid ]; then
+ echofail "The port must be in the range $minvalid to $maxvalid" >&2
+ exit 1
+fi
+
+# Name the first 10 ports in the set: the query port, the control port and
+# eight extra ports. Since the lowest numbered port (specified in the command
+# line) will usually be a multiple of 10, the names are chosen so that the
+# number of EXTRAPORTn is "n".
+export PORT=$baseport
+export EXTRAPORT1=`expr $baseport + 1`
+export EXTRAPORT2=`expr $baseport + 2`
+export EXTRAPORT3=`expr $baseport + 3`
+export EXTRAPORT4=`expr $baseport + 4`
+export EXTRAPORT5=`expr $baseport + 5`
+export EXTRAPORT6=`expr $baseport + 6`
+export EXTRAPORT7=`expr $baseport + 7`
+export EXTRAPORT8=`expr $baseport + 8`
+export CONTROLPORT=`expr $baseport + 9`
+
+export LOWPORT=$baseport
+export HIGHPORT=`expr $baseport + $numport - 1`
+
echoinfo "S:$test:`date $dateargs`" >&2
echoinfo "T:$test:1:A" >&2
echoinfo "A:$test:System test $test" >&2
-echoinfo "I:$test:PORTRANGE:${portlow} - ${porthigh}"
+echoinfo "I:$test:PORTRANGE:${LOWPORT} - ${HIGHPORT}"
if [ x${PERL:+set} = x ]
then
fi
# Check for test-specific prerequisites.
-test ! -f $test/prereq.sh || ( cd $test && $SHELL prereq.sh -p "$port" -- "$@" )
+test ! -f $test/prereq.sh || ( cd $test && $SHELL prereq.sh "$@" )
result=$?
if [ $result -eq 0 ]; then
fi
# Test sockets after the prerequisites has been setup
-$PERL testsock.pl -p "${port}" || {
+$PERL testsock.pl -p $PORT || {
echowarn "I:$test:Network interface aliases not set up. Skipping test." >&2;
echowarn "R:$test:UNTESTED" >&2;
echoinfo "E:$test:`date $dateargs`" >&2;
# Set up any dynamically generated test data
if test -f $test/setup.sh
then
- ( cd $test && $SHELL setup.sh -p "$port" -- "$@" )
+ ( cd $test && $SHELL setup.sh "$@" )
fi
# Start name servers running
-$PERL start.pl --port $port $test || { echofail "R:$test:FAIL"; echoinfo "E:$test:`date $dateargs`"; exit 1; }
+$PERL start.pl --port $PORT $test || { echofail "R:$test:FAIL"; echoinfo "E:$test:`date $dateargs`"; exit 1; }
# Run the tests
-( cd $test ; $SHELL tests.sh -p "$port" -- "$@" )
+( cd $test ; $SHELL tests.sh "$@" )
status=$?
if $stopservers
rm -f $SYSTEMTESTTOP/random.data
if test -f $test/clean.sh
then
- ( cd $test && $SHELL clean.sh "-p" "$port" -- "$@" )
+ ( cd $test && $SHELL clean.sh "$@" )
fi
if test -d ../../../.git
then
projects / thirdparty / bind9.git / commitdiff
