+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: add auxiliary script 'test-driver', refactor
+ This refactoring should cause no API of functionality change,
+ and is meant only to simplify the future implementation of TAP
+ and SubUnit testsuite drivers. More precisely, our roadmap is
+ to move most of the "testsuite driving" features out of the
+ Automake-generated Makefiles, and into external scripts with
+ well-defined interfaces. This will allow the user to define
+ its own personalized testsuite drivers, and will also offer us
+ a framework upon which to implement our new TAP and SubUnit
+ drivers, all in a very unobtrusive way and retaining an high
+ degree of code reuse and backward-compatibility.
+ * lib/test-driver: New auxiliary script.
+ * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
+ * automake.in (handle_tests): Require the new auxiliary script
+ `test-driver', and define a new internal makefile variable
+ `$(am__test_driver)', used to call it. Perform new substitution
+ on `DRIVER' when processing the `check2.am' file.
+ * lib/check.am (am__tty_colors): Define new shell variable
+ `$am__color_tests'.
+ (am__rst_section): Removed, its role taken over by the new
+ `test-driver' script.
+ (am__test_driver_flags): New variable, contains the command
+ line options passed to `test-driver'.
+ (am__check_pre): Do not deal with temporary files and exit
+ traps anymore, as the `test-driver' script takes care of that
+ now. Define shell variable `$am__enable_hard_errors', used by
+ `$(am__test_driver_flags)'. Reorder so that we don't need to
+ save and restore the value of the `TERM' environment variable
+ anymore.
+ Other related adjustments.
+ (am__check_post): Remove, as its role has been completely taken
+ over by the `test-driver' script.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Call the test script through the Automake
+ substituted `%DRIVER%', and honor the command-line options
+ in `$(am__test_driver_flags)'. Do not call the obsoleted
+ `$(am__check_post)' anymore.
+ * doc/automake.texi (Auxiliary Programs): Mention the new
+ `test-driver' script.
+ (Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR.
+ Since we are at it, break the list of auxiliary scripts by
+ placing one per line, to simplify potential future additions
+ of new scripts.
+ * tests/check.test: Adjust.
+ * tests/check2.test : Likewise.
+ * tests/check3.test : Likewise.
+ * tests/check4.test : Likewise.
+ * tests/check10.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/comment9.test: Likewise.
+ * tests/dejagnu.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/maken3.test: Likewise.
+ * tests/maken4.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/posixsubst-tests.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/check-no-test-driver.test: New test.
+ * tests/parallel-test-driver-install.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+ * NEWS: Update.
+
2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
maintcheck: extend 'sc_tests_plain_*' checks
- New `cscope' target to build a cscope database for the source tree.
+* Changes to Automake-generated testsuite harnesses:
+
+ - The parallel-tests driver is now implemented (partly at least) with
+ the help of automake-provided auxiliary scripts (e.g., `test-driver'),
+ instead of relying entirely on code in the generated Makefile.in.
+ This has two noteworthy implications. The first one is that projects
+ using the `parallel-tests' option should now either run automake with
+ the `--add-missing' option, or manually copy the `test-driver' script
+ into their tree. The second, and more important, implication is that
+ now, when the `parallel-tests' option is in use, TESTS_ENVIRONMENT can
+ not be used anymore to define a test runner, and the command specified
+ in LOG_COMPILER (and <ext>_LOG_COMPILER) must be a *real* executable
+ program or script. For example, this is still a valid usage (albeit
+ a little contorted):
+
+ TESTS_ENVIRONMENT = \
+ if test -n '$(STRICT_TESTS)'; then \
+ maybe_errexit='-e'; \
+ else \
+ maybe_errexit=''; \
+ fi;
+ LOG_COMPILER = $(SHELL) $$maybe_errexit
+
+ while this is not anymore:
+
+ TESTS_ENVIRONMENT = \
+ $(SHELL) `test -n '$(STRICT_TESTS_CHECKING)' && echo ' -e'`
+
+ neither is this:
+
+ TESTS_ENVIRONMENT = \
+ run_with_perl_or_shell () \
+ { \
+ if grep -q '^#!.*perl' $$1; then
+ $(PERL) $$1; \
+ else \
+ $(SHELL) $$1; \
+ fi; \
+ }
+ LOG_COMPILER = run_with_per_or_shell
+
* WARNING: Future backward-incompatibilities!
- The Automake support for automatic de-ANSI-fication will be removed in
append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
if (var ('XFAIL_TESTS'));
- if (option 'parallel-tests')
+ if (my $parallel_tests = option 'parallel-tests')
{
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
+ 'test-driver');
+ define_variable ('am__test_driver',
+ "\$(SHELL) $am_config_aux_dir/test-driver",
+ INTERNAL);
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
define_variable ('TEST_SUITE_HTML', '$(TEST_SUITE_LOG:.log=.html)', INTERNAL);
my $suff = '.test';
GENERIC => 0,
OBJ => $obj,
SOURCE => $val,
+ DRIVER => '$(am__test_driver)',
COMPILE =>'$(' . $compile . ')',
EXT => '',
am__EXEEXT => 'FALSE');
GENERIC => 1,
OBJ => '',
SOURCE => '$<',
+ DRIVER => '$(am__test_driver)',
COMPILE => '$(' . $compile . ')',
EXT => $test_suffix,
am__EXEEXT => $am_exeext);
multilibs (@pxref{Multilibs}). This file is maintained in the GCC
tree at @url{http://gcc.gnu.org/svn.html}.
+@item test-driver
+This implements the default testsuite driver offered by the
+@option{parallel-tests} testsuite harness.
+
@item texinfo.tex
Not a program, this file is required for @samp{make dvi}, @samp{make
ps} and @samp{make pdf} to work when Texinfo sources are in the
@item AC_CONFIG_AUX_DIR
Automake will look for various helper scripts, such as
@file{install-sh}, in the directory named in this macro invocation.
-@c This list is accurate relative to version 1.8
-(The full list of scripts is: @file{ar-lib}, @file{config.guess},
-@file{config.sub}, @file{depcomp}, @file{elisp-comp}, @file{compile},
-@file{install-sh}, @file{ltmain.sh}, @file{mdate-sh}, @file{missing},
-@file{mkinstalldirs}, @file{py-compile}, @file{texinfo.tex}, and
-@file{ylwrap}.) Not all scripts are always searched for; some scripts
+@c This list is accurate relative to version 1.11
+(The full list of scripts is:
+@file{ar-lib},
+@file{config.guess},
+@file{config.sub},
+@file{depcomp},
+@file{elisp-comp},
+@file{compile},
+@file{install-sh},
+@file{ltmain.sh},
+@file{mdate-sh},
+@file{missing},
+@file{mkinstalldirs},
+@file{py-compile},
+@file{test-driver},
+@file{texinfo.tex},
+@file{ylwrap}.)
+Not all scripts are always searched for; some scripts
will only be sought if the generated @file{Makefile.in} requires them.
If @code{AC_CONFIG_AUX_DIR} is not given, the scripts are looked for in
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
# Prefix each line by 'col' and terminate each with 'std', for coloring.
# Multi line coloring is problematic with "less -R", so we really need
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
+am__test_driver = $(SHELL) $(top_srcdir)/lib/test-driver
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
am__test_logs1 = $(TESTS:=.log)
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS) TEST_LOGS="'"$$list"'"'
.pl.log:
- @p='$<'; $(am__check_pre) $(PL_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(am__test_driver) $(am__test_driver_flags) -- \
+ $(PL_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib test-driver
EXTRA_DIST = gnupload
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib test-driver
EXTRA_DIST = gnupload
all: all-recursive
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
std='\e[m'; \
}
else !%?COLOR%
-am__tty_colors = \
-red=; grn=; lgn=; blu=; std=
+am__tty_colors = red= grn= lgn= blu= std= am__color_tests=no
endif !%?COLOR%
.PHONY: check-TESTS
## test is XFAIL or not. You can disable this feature by setting the
## variable DISABLE_HARD_ERRORS to a nonempty value.
-# Restructured Text title and section.
-am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
+# Restructured Text title.
+am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
# Put stdin (possibly several lines separated by ". ") in a box.
# Prefix each line by 'col' and terminate each with 'std', for coloring.
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
+
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
+## The use of $dir below is required to account for VPATH
+## rewriting done by Sun make.
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
$(TEST_SUITE_LOG): $(TEST_LOGS)
@$(am__sh_e_setup); \
## automake - create Makefile.in from Makefile.am
-## Copyright (C) 2008, 2009 Free Software Foundation, Inc.
+## Copyright (C) 2008, 2009, 2011 Free Software Foundation, Inc.
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## From a test file to a log file.
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %DRIVER% $(am__test_driver_flags) -- \
+ %COMPILE% "$$tst"
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
## conflict with the previous one.
if %am__EXEEXT%
?GENERIC?%EXT%$(EXEEXT).log:
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %DRIVER% $(am__test_driver_flags) -- \
+ %COMPILE% "$$tst"
endif %am__EXEEXT%
--- /dev/null
+#! /bin/sh
+# test-driver - basic driver script for the `parallel-tests' mode.
+
+scriptversion=2011-06-21.19; # UTC
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <bug-automake@gnu.org> or send patches to
+# <automake-patches@gnu.org>.
+
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
+
+fatal ()
+{
+ echo "$0: fatal: $*" >&2
+ exit 1
+}
+
+usage_error ()
+{
+ echo "$0: $*" >&2
+ print_usage >&2
+ exit 2
+}
+
+print_usage ()
+{
+ cat <<END
+Usage:
+ test-driver [--help|--version] --test-name=NAME --log-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
+The \`--test-name' and \`--log-file' options are mandatory.
+END
+}
+
+# Restructured Text section.
+rst_section () { sed 'p;s/./=/g;p;g'; }
+
+# TODO: better error handling in option parsing (in particular, ensure
+# TODO: $logfile and $test_name are defined).
+test_name= # Used for reporting.
+logfile= # Where to save the result and output of the test script.
+expect_failure=no
+color_tests=no
+enable_hard_errors=yes
+while test $# -gt 0; do
+ case $1 in
+ --help) print_usage; exit $?;;
+ --version) echo "test-driver $scriptversion"; exit $?;;
+ --test-name) test_name=$2; shift;;
+ --log-file) logfile=$2; shift;;
+ --color-tests) color_tests=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --enable-hard-errors) enable_hard_errors=$2; shift;;
+ --) shift; break;;
+ -*) usage_error "invalid option: '$1'";;
+ esac
+ shift
+done
+
+if test $color_tests = yes; then
+ red='\e[0;31m' # Red.
+ grn='\e[0;32m' # Green.
+ lgn='\e[1;32m' # Light green.
+ blu='\e[1;34m' # Blue.
+ std='\e[m' # No color.
+else
+ red= grn= lgn= blu= std=
+fi
+
+tmpfile=$logfile-t
+do_exit='rm -f $tmpfile; (exit $st); exit $st'
+trap "st=129; $do_exit" 1
+trap "st=130; $do_exit" 2
+trap "st=141; $do_exit" 13
+trap "st=143; $do_exit" 15
+rm -f $tmpfile
+
+# Test script is run here.
+"$@" >$tmpfile 2>&1
+estatus=$?
+if test $enable_hard_errors = no && test $estatus -eq 99; then
+ estatus=1
+fi
+
+case $estatus:$expect_failure in
+ 0:yes) col=$red; res=XPASS;;
+ 0:*) col=$grn; res=PASS ;;
+ 77:*) col=$blu; res=SKIP ;;
+ 99:*) col=$red; res=FAIL ;;
+ *:yes) col=$lgn; res=XFAIL;;
+ *:*) col=$red; res=FAIL ;;
+esac
+echo "${col}${res}${std}: $test_name"
+echo "$res: $test_name (exit: $estatus)" | rst_section > $logfile
+cat $tmpfile >> $logfile
+rm -f $tmpfile
+
+# Local Variables:
+# mode: shell-script
+# sh-indentation: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-test-driver.test \
checkall.test \
clean.test \
clean2.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-test-driver-install.test \
parse.test \
percent.test \
percent2.test \
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
# Prefix each line by 'col' and terminate each with 'std', for coloring.
# Multi line coloring is problematic with "less -R", so we really need
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
+am__test_driver = $(SHELL) $(top_srcdir)/lib/test-driver
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
TEST_EXTENSIONS = .test
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-test-driver.test \
checkall.test \
clean.test \
clean2.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-test-driver-install.test \
parse.test \
percent.test \
percent2.test \
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS) TEST_LOGS="'"$$list"'"'
.test.log:
- @p='$<'; $(am__check_pre) $(TEST_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(am__test_driver) $(am__test_driver_flags) -- \
+ $(TEST_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'test-driver' doesn't get needlessly
+# installed or referenced when the 'parallel-tests' option is not
+# used.
+
+parallel_tests=no
+. ./defs || Exit 1
+
+echo 'TESTS = foo.test' > Makefile.am
+
+$ACLOCAL
+
+for opts in '' '-a' '--add-missing --copy'; do
+ $AUTOMAKE $opts
+ $FGREP 'test-driver' Makefile.in && Exit 1
+ find . | $FGREP 'test-driver' && Exit 1
+done
+
+:
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > test-driver
+
: > frob.test
$ACLOCAL
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
(
CLEANFILES = echo.sh
END
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir"/lib/test-driver .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure --prefix "`pwd`/inst"
$MAKE check >stdout
cat stdout
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing
+
./configure --prefix "`pwd`/inst"
+
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
grep 'FAIL: fail.sh' stdout
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
$AUTOCONF
+$AUTOMAKE --add-missing
+
./configure
test_color ()
AM_COLOR_TESTS=always $MAKE -e check >stdout && { cat stdout; Exit 1; }
cat stdout
test_color
+
+:
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
+$AUTOMAKE -a
$AUTOCONF
./configure
|| { cat stdout; Exit 1; }
cat stdout
test_no_color
+
+:
7.test
EOF
+: > test-driver
+
$ACLOCAL
$AUTOMAKE
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > test-driver
+
$ACLOCAL
$AUTOMAKE
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing --copy
./configure
$MAKE print-bin > output
cat output
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'test-driver' gets automatically installed
+# in the correct directory by 'parallel-tests' option.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+: Try first with parallel-tests defined in AM_INIT_AUTOMAKE.
+
+mkdir am-init-automake
+cd am-init-automake
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([my_aux_dir])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([Makefile sub/Makefile])
+AC_OUTPUT
+END
+
+mkdir sub my_aux_dir
+
+cat > Makefile.am <<END
+SUBDIRS = sub
+TESTS = foo.test
+END
+
+cat > sub/Makefile.am <<END
+TESTS = bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE -a 2>stderr || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . sub my_aux_dir # For debugging.
+test -f my_aux_dir/test-driver
+test ! -r test-driver
+test ! -r sub/test-driver
+
+grep '^configure\.in:3:.*installing.*my_aux_dir/test-driver' stderr
+
+cd ..
+
+: Now try with parallel-tests defined in AUTOMAKE_OPTIONS.
+
+mkdir automake-options
+cd automake-options
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([dir/GNUmakefile])
+AC_OUTPUT
+END
+
+mkdir build-aux dir
+
+cat > dir/GNUmakefile.am <<END
+TESTS = foo.test
+AUTOMAKE_OPTIONS = parallel-tests
+TESTS += bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE --add-missing --copy dir/GNUmakefile 2>stderr \
+ || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . dir build-aux # For debugging.
+test -f build-aux/test-driver
+test ! -r test-driver
+test ! -r dir/test-driver
+
+grep '^dir/GNUmakefile\.am:2:.*installing.*build-aux/test-driver' stderr
+
+:
cat > Makefile.am << 'END'
TESTS = foo.test
-## Ugly, but required by foo.test. See below.
-TEST_LOG_COMPILER = echo $$$$ > pid && exec 9>&2 && $(SHELL) -x
+## Provide more debugging info.
+TEST_LOG_COMPILER = $(SHELL) -x
+## Rut required by foo.test; see below.
+AM_TESTS_ENVIRONMENT = 9>&2
END
# This is hacky and ugly, but has the great advantage of avoiding us a lot
# of pain with background processes and related synchronization issues.
+
+cat - "$top_testsrcdir"/lib/test-driver > test-driver <<'END'
+#!/bin/sh
+echo $$ > pid
+END
+
cat > foo.test << 'END'
#!/bin/sh
exec 2>&9
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE
./configure
rm -f pid fail *.log *.log-t
env signum=$signum $MAKE check && { ls -l; Exit 1; }
ls -l
+ # These files shouldn't exist, but in case they do, their content might
+ # provide helpful information about the causes of the failure(s).
cat foo.log-t || :
cat foo.log || :
cat test-suite.log || :
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
$MAKE check >out 2>&1 && { cat out; Exit 1; }
}
END
-cp $testsrcdir/../lib/compile .
+cp "$top_testsrcdir"/lib/compile "$top_testsrcdir"/lib/test-driver .
$ACLOCAL
$AUTOMAKE --foreign --foreign -Wall 2>stderr || { cat stderr >&2; Exit 1; }
projects / thirdparty / automake.git / commitdiff
