]> git.ipfire.org Git - thirdparty/automake.git/commit
parallel-tests: make parsing of test results safer
authorStefano Lattarini <stefano.lattarini@gmail.com>
Wed, 29 Jun 2011 14:02:52 +0000 (16:02 +0200)
committerStefano Lattarini <stefano.lattarini@gmail.com>
Wed, 6 Jul 2011 20:43:51 +0000 (22:43 +0200)
commit61b2bb0d647ea5f192e49f5f5885009ea83968be
treebc55f9c892aaa71348562394fe7121b3b7a920ef
parent1402741313cf6c2fe8611eb294f9a8111c994a32
parallel-tests: make parsing of test results safer

The new code for parsing the testsuite-generated `.log' files,
as introduced in commit `v1.11-872-gc96b881', considers each
`:test-result:' field anywhere in a `.log' file as a declaration
of a test result, and accounts for it as such in the testsuite
summary.  Unfortunately this could easily cause spurious test
failures being reported in the testsuite summary.  This happened
in practice with the Automake's own testsuite; for example:

  $ make check TESTS='check12-p.test'; echo exit: $?
  ...
  PASS: check12-p.test
  =====================================
  4 of 5 tests failed
  See tests/test-suite.log
  Please report to bug-automake@gnu.org
  =====================================
  make[2]: *** [test-suite.log] Error 1
  make: *** [check-am] Error 2
  exit: 2

This change introduces a new special `:test-result:' "END", that,
when seen, prevents the rest of the log file from being parsed.

For more information, refer to the thread:
<http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>

* lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
file as soon as the special ":test-result:END" directive is seen.
Related changes and enhancements.
* lib/test-driver: Protect the rest of the log after the result
lined with a ":test-result:END" directive.
* tests/parallel-tests-no-spurious-summary.test: New test.
* tests/test-driver-end-test-results.test: Likewise.
* tests/Makefile.am (TESTS): Update.
ChangeLog
doc/automake.texi
lib/Automake/tests/Makefile.in
lib/am/check.am
lib/test-driver
tests/Makefile.am
tests/Makefile.in
tests/parallel-tests-no-spurious-summary.test [new file with mode: 0755]
tests/test-driver-end-test-results.test [new file with mode: 0755]