]> git.ipfire.org Git - thirdparty/automake.git/commit
parallel-tests: allow each test to have multiple results
authorStefano Lattarini <stefano.lattarini@gmail.com>
Fri, 3 Jun 2011 20:50:56 +0000 (22:50 +0200)
committerStefano Lattarini <stefano.lattarini@gmail.com>
Tue, 21 Jun 2011 20:27:26 +0000 (22:27 +0200)
commitc96b881f41b180f6176064cc00fe7af620c24c63
tree74b17a63af0e0f1afb799d3f85add8c5e12760a7
parent79922343073c66c3f8d4692efb1f31fe47c16fa9
parallel-tests: allow each test to have multiple results

With this change, we improve the code creating the `test-suite.log'
global log and the console testsuite summary to make it able to
grasp multiple results per test script.  This is required in order
to introduce the planned support for test protocols, like TAP and
SubUnit, which can indeed run multiple tests per test script, each
with its individual result.

The implementation makes use of a custom reStructuredText field
`:test-result:'.

Note that no new documentation is added by this change; that is
be left for follow-up changes.

* lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
recognize a report of a test's result only if it is declared with
the custom `:test-result:' reStructuredText field placed at the
beginning of a line.  Extend and add explanatory comments.
(recheck, recheck-html): Add explanatory comments.
* lib/test-driver: Write an appropriate reStructuredText field
`:test-result:' in the generated log file.  Use a reStructuredText
transition to better separate the test outcome report from the
test script's registered output.  Improve comments.
* tests/test-driver-custom-xfail-tests.test: Adjust.
* tests/parallel-tests7.test: Adjust.
* tests/parallel-tests-empty-testlogs.test: New test.
* tests/parallel-tests-recheck-override.test: Likewise.
* tests/parallel-tests2.test: Extend and keep more in-sync with ...
* tests/test-driver-custom-html.test: ... this new related test.
* tests/test-driver-custom-no-html.test: New test.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/trivial-test-driver: New file, used by the last four tests
above.
* tests/Makefile.am (TESTS): Update.
(EXTRA_DIST): Distribute `trivial-test-driver'.
(test-driver-custom-multitest.log): Depend on `trivial-test-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.
17 files changed:
ChangeLog
lib/Automake/tests/Makefile.in
lib/am/check.am
lib/test-driver
tests/Makefile.am
tests/Makefile.in
tests/parallel-tests-empty-testlogs.test [new file with mode: 0755]
tests/parallel-tests2.test
tests/parallel-tests7.test
tests/test-driver-custom-html.test [new file with mode: 0755]
tests/test-driver-custom-multitest-recheck.test [new file with mode: 0755]
tests/test-driver-custom-multitest-recheck2.test [new file with mode: 0755]
tests/test-driver-custom-multitest.test [new file with mode: 0755]
tests/test-driver-custom-no-extra-driver.test
tests/test-driver-custom-no-html.test [new file with mode: 0755]
tests/test-driver-custom-xfail-tests.test
tests/trivial-test-driver [new file with mode: 0644]