parallel-tests: allow each test to have multiple results
With this change, we improve the code creating the `test-suite.log'
global log and the console testsuite summary to make it able to
grasp multiple results per test script. This is required in order
to introduce the planned support for test protocols, like TAP and
SubUnit, which can indeed run multiple tests per test script, each
with its individual result.
The implementation makes use of a custom reStructuredText field
`:test-result:'.
Note that no new documentation is added by this change; that is
be left for follow-up changes.
* lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
recognize a report of a test's result only if it is declared with
the custom `:test-result:' reStructuredText field placed at the
beginning of a line. Extend and add explanatory comments.
(recheck, recheck-html): Add explanatory comments.
* lib/test-driver: Write an appropriate reStructuredText field
`:test-result:' in the generated log file. Use a reStructuredText
transition to better separate the test outcome report from the
test script's registered output. Improve comments.
* tests/test-driver-custom-xfail-tests.test: Adjust.
* tests/parallel-tests7.test: Adjust.
* tests/parallel-tests-empty-testlogs.test: New test.
* tests/parallel-tests-recheck-override.test: Likewise.
* tests/parallel-tests2.test: Extend and keep more in-sync with ...
* tests/test-driver-custom-html.test: ... this new related test.
* tests/test-driver-custom-no-html.test: New test.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/trivial-test-driver: New file, used by the last four tests
above.
* tests/Makefile.am (TESTS): Update.
(EXTRA_DIST): Distribute `trivial-test-driver'.
(test-driver-custom-multitest.log): Depend on `trivial-test-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.