From: Michal Nowak <mnowak@isc.org>
Date: Wed, 30 Dec 2020 12:23:18 +0000 (+0100)
Subject: Record skipped test as skipped in testsuite summary
X-Git-Tag: v9.17.11~54^2~4
X-Git-Url: http://git.ipfire.org/gitweb/?a=commitdiff_plain;h=fbdd464038d9890c56ebdd848d6473e156bd17a7;p=thirdparty%2Fbind9.git

Record skipped test as skipped in testsuite summary

When system test execution was ported to Automake, SKIPPED and UNTESTED
system test result were not made to match Automake expectations,
therefore a skipped test is recorded by Automake as "PASS":

    $ make check TESTS=cpu V=1
    I:cpu:cpu test only runs on Linux, skipping test
    I:cpu:Prerequisites missing, skipping test.
    R:cpu:SKIPPED
    E:cpu:2020-12-16T11:36:58+0000
    PASS: cpu
    ====================================================================
    Testsuite summary for BIND 9.17.7
    ====================================================================
    # TOTAL: 1
    # PASS:  1

For a test to be recorded by Automake as skipped, the test, or it's test
driver, needs to exit with code 77:

    $ make check TESTS=cpu V=1
    I:cpu:cpu test only runs on Linux, skipping test
    I:cpu:Prerequisites missing, skipping test.
    R:cpu:SKIPPED
    E:cpu:2020-12-16T11:39:10+0000
    SKIP: cpu
    ====================================================================
    Testsuite summary for BIND 9.17.7
    ====================================================================
    # TOTAL: 1
    # PASS:  0
    # SKIP:  1
---

diff --git a/bin/tests/system/run.sh.in b/bin/tests/system/run.sh.in
index 7027ab21924..3a71afef08e 100644
--- a/bin/tests/system/run.sh.in
+++ b/bin/tests/system/run.sh.in
@@ -172,7 +172,7 @@ else
         echowarn "R:$systest:UNTESTED"
     fi
     echoend "E:$systest:$(date_with_args)"
-    exit 0
+    exit 77
 fi
 
 # Check for PKCS#11 support