]> git.ipfire.org Git - thirdparty/automake.git/commit
[ng] tests: do not run very expensive tests by default
authorStefano Lattarini <stefano.lattarini@gmail.com>
Sat, 28 Jul 2012 09:54:27 +0000 (11:54 +0200)
committerStefano Lattarini <stefano.lattarini@gmail.com>
Sat, 28 Jul 2012 11:27:20 +0000 (13:27 +0200)
commitb300a4cb19fc524d4d640f62754a2b803d8018ba
treece856fb0c2dd80da1c78d2e6190c28a045ca4ec5
parent86da267360d2b32941f17ccad07204f94bbd043d
[ng] tests: do not run very expensive tests by default

Some tests in the Automake testsuite (especially those testing for bugs
w.r.t. command-line length limits) require a *long* time to run.  For
example, the test 'parallel-tests-many.sh' (the most extreme example of
this issue) takes ~ 11 minutes to execute on a modern, blazingly fast
multi-core ppc64 system (64 cores at 3.5 GHz each).

This slow-down is unacceptable during routine runs of the testsuite,
that I do every bunch of commits (or every commit, for more tricky
changes).  And is even less acceptable for the casual user that just
run the testsuite before installing Automake, maybe on an aging and
slower system (or maybe on Cygwin *shudder*).

So, let's follow the route of GNU coreutils here: declare some tests as
"expensive", and let them run only if called if the RUN_EXPENSIVE_TESTS
environment variable is set to "yes".

* test-lib.sh (expensive_): New function; used in a test, cause it to be
skipped unless the RUN_EXPENSIVE_TESTS variable is defined to "yes".
* t/parallel-tests-many.sh, t/testsuite-summary-count-many.sh,
t/dist-many.sh: Call 'expensive_'.
* runtest.in: Export RUN_EXPENSIVE_TESTS to "yes": if a test is called
directly through './runtest', it should be run even it it's expensive.
* Makefile.am (check-expensive, installcheck-expensive): New convenience
targets that run the testsuite with RUN_EXPENSIVE_TESTS exported to
"yes", so that the expensive tests are not skipped.
* t/README: Update.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Makefile.am
runtest.in
t/README
t/ax/test-lib.sh
t/dist-many.sh
t/parallel-tests-many.sh
t/testsuite-summary-count-many.sh