From: Michał Kępień Date: Fri, 6 Dec 2019 13:11:01 +0000 (+0100) Subject: Remove the -r switch from system test scripts X-Git-Tag: v9.15.7~30^2~2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=b4d37878f6ab6f0026c4a36763148be527e04a38;p=thirdparty%2Fbind9.git Remove the -r switch from system test scripts Since files containing system test output are no longer stored in test subdirectories, bin/tests/system/clean.sh no longer needs to take care of removing the test.output file for a given test as testsummary.sh already takes care of that and even if a test suite terminates abnormally and another one is started, tee invoked without the -a command line switch overwrites the destination file if it exists, so leftover test.output.* files from previous test suite runs are not a concern. Remove the -r command line switch and the code associated with it from the relevant scripts. --- diff --git a/bin/tests/system/Makefile.in b/bin/tests/system/Makefile.in index 44465014381..6db2c86396b 100644 --- a/bin/tests/system/Makefile.in +++ b/bin/tests/system/Makefile.in @@ -64,7 +64,7 @@ parallel.mk: test: parallel.mk subdirs @$(MAKE) -f parallel.mk check - @$(SHELL) ./runsequential.sh -r + @$(SHELL) ./runsequential.sh @$(SHELL) ./testsummary.sh check: test diff --git a/bin/tests/system/README b/bin/tests/system/README index 552344ba87a..5190f4dd3f7 100644 --- a/bin/tests/system/README +++ b/bin/tests/system/README @@ -98,19 +98,6 @@ Optional flags are: ports 7900 through 7999). If not specified, the test will have ports 5000 to 5099 available to it. - -r The "runall" flag. This is related to cleaning up after - the tests (see "Maintenance Notes" below). If specified, - it prevents a copy of the test's output listing from being - deleted when the directory is cleaned up after the test - completes. (The test's output listing comprises messages - produced by the test during its execution; it does not - include the output files produced by utilities such as - "dig" or "rndc", nor any logging output from named itself.) - It is usually only used when "run.sh" is being called - during a run of the entire test suite. Note that if "-n" - is specified on the "run.sh" command line, the test output - is retained even if this option is omitted. - Arguments are: test-name Mandatory. The name of the test, which is the name of the @@ -257,15 +244,7 @@ deleted if the test succeeds but are retained on error. Deletion of files produced by an individual test can be done with the command: - sh clean.sh [-r] - -The optional flag is: - - -r The "runall" flag. This is related to cleaning up after - the tests (see "Maintenance Notes" below). If specified, - it prevents a copy of the test's output listing from being - deleted when the directory is cleaned after the test - completes. + sh clean.sh Deletion of the files produced by the set of tests (e.g. after the execution of "runall.sh") can be deleted by the command: @@ -709,7 +688,7 @@ the ports are assigned when the tests are run. This is achieved by having the when "make check" is run, and contains a target for each test of the form: : - @$(SHELL) run.sh -r -p + @$(SHELL) run.sh -p The is unique and the values of for each test are separated by at least 100 ports. @@ -742,27 +721,3 @@ by the system's "clean.sh". 3. "test.output.*" files are deleted when the test suite ends. At this point, the file "testsummary.sh" is called which concatenates all the "test.output.*" files into a single "systests.output" file before deleting them. - -A complication arises with the "test.output" file however: - -1. "clean.sh" is called by "run.sh" if the test ends successfully. For -this reason, "clean.sh" cannot delete "test.output" as, if the test is -being run as part of a test suite, the file must be retained. - -2. If the deletion of "test.output" were to be solely the responsibility of -"testsummary.sh", should a test suite terminate abnormally, cleaning up a test -directory with "sh clean.sh " would leave the file present. - -3. An additional step could be added to "cleanall.sh" (which calls the -system's "clean.sh" for each test) to remove the "test.output" file. However, -although the file would be deleted should all test directories be cleaned, -it would still mean that running "clean.sh" on a particular test directory -could leave the file present. - -To get round this, the system's "clean.sh" script takes an optional flag, "-r" -(the "runall" flag). When the test suite is run, each invocation of "run.sh" -is passed the runall flag. In turn, "run.sh" passes the flag to "clean.sh", -which causes that script not to delete the "tests.output" file. In other -words, when the system's "clean.sh" is invoked standalone on a test directory -(or as part of a run of "cleanall.sh"), it will delete the "test.output" if it -is present. When invoked during a run of the entire test suite, it won't. diff --git a/bin/tests/system/clean.sh b/bin/tests/system/clean.sh index 2af06fb82db..f75a98a2964 100644 --- a/bin/tests/system/clean.sh +++ b/bin/tests/system/clean.sh @@ -18,33 +18,14 @@ SYSTEMTESTTOP="$(cd -P -- "$(dirname -- "$0")" && pwd -P)" export SYSTEMTESTTOP -# See if the "-r" flag is present. This will usually be set when all the tests -# are run (e.g. from "runall.sh") and tells the script not to delete the -# test.output file created by run.sh. This is because the script running all -# the tests will call "testsummary.sh", which will concatenate all test output -# files into a single systests.output. - -runall=0 - -while getopts "r" flag; do - case $flag in - r) runall=1 ;; - esac -done -shift `expr $OPTIND - 1` - if [ $# -eq 0 ]; then - echo "usage: $0 [-r] test-directory" >&2 + echo "usage: $0 test-directory" >&2 exit 1 fi systest=$1 shift -if [ $runall -eq 0 ]; then - rm -f $systest/test.output -fi - if [ -f $systest/clean.sh ]; then ( cd $systest && $SHELL clean.sh "$@" ) else diff --git a/bin/tests/system/parallel.sh b/bin/tests/system/parallel.sh index 2eab88c444e..e59b4d585f0 100644 --- a/bin/tests/system/parallel.sh +++ b/bin/tests/system/parallel.sh @@ -29,6 +29,6 @@ port=${STARTPORT:-5000} for directory in $PARALLELDIRS ; do echo echo "test-`echo $directory | tr _ -`: check_interfaces" - echo " @${SHELL} ./run.sh -r -p $port $directory 2>&1 | tee test.output.$directory" + echo " @${SHELL} ./run.sh -p $port $directory 2>&1 | tee test.output.$directory" port=`expr $port + 100` done diff --git a/bin/tests/system/run.sh b/bin/tests/system/run.sh index 502a2a5caf1..7718f93e087 100755 --- a/bin/tests/system/run.sh +++ b/bin/tests/system/run.sh @@ -37,13 +37,12 @@ while getopts "knp:r-:" flag; do k) stopservers=false ;; n) clean=false ;; p) baseport=$OPTARG ;; - r) runall="-r" ;; esac done shift `expr $OPTIND - 1` if [ $# -eq 0 ]; then - echofail "Usage: $0 [-k] [-n] [-p ] [-r] test-directory [test-options]" >&2; + echofail "Usage: $0 [-k] [-n] [-p ] test-directory [test-options]" >&2; exit 1 fi diff --git a/bin/tests/system/runall.sh b/bin/tests/system/runall.sh index 3246457919c..93df8dab498 100755 --- a/bin/tests/system/runall.sh +++ b/bin/tests/system/runall.sh @@ -80,7 +80,7 @@ if [ "$NOPARALLEL" = "" ]; then # of parallel execution of system tests, and use that. $SHELL parallel.sh > parallel.mk make -f parallel.mk -j $numproc check - $SHELL ./runsequential.sh -r + $SHELL ./runsequential.sh $SHELL ./testsummary.sh || status=1 fi else