+2011-04-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: new developer-reserved AM_TESTS_SETUP variable
+ For reference, see the discussion at:
+ <http://lists.gnu.org/archive/html/automake-patches/2011-01/msg00213.html>
+ * lib/am/check.am [%?PARALLEL_TESTS%] (am__check_pre): Pass also
+ $(AM_TESTS_SETUP). Comments updated, and some typos fixed.
+ * doc/automake.texi (Simple Tests using parallel-tests): Document
+ AM_TESTS_SETUP. Reorder some of the existing documentation a bit.
+ * tests/parallel-tests-am_tests_setup.test: New test.
+ From a suggestion by Ralf Wildenhues.
+
2011-04-17 Stefano Lattarini <stefano.lattarini@gmail.com>
depcomp tests: don't reject slower dependency extractors
have not completed in a prior run, summary and verbose output in
@samp{RST} (reStructuredText) and @samp{HTML} format, and hard errors
for exceptional failures. Similar to the simple test driver,
-@code{TESTS_ENVIRONMENT}, @code{AM_COLOR_TESTS}, @code{XFAIL_TESTS}, and
+@code{AM_COLOR_TESTS}, @code{XFAIL_TESTS}, and
the @code{check_*} variables are honored, and the environment variable
-@env{srcdir} is set during test execution.
+@env{srcdir} is set during test execution. Also, @code{TESTS_ENVIRONMENT}
+is still honored, but is complemented by a new developer-reserved variable
+@code{AM_TESTS_SETUP} (described below).
This test driver is still experimental and may undergo changes in order
to satisfy additional portability requirements.
@code{TESTS}, which in turn contain all output produced by the
corresponding tests.
+@vindex VERBOSE
+As with the simple driver above, by default one status line is printed
+per completed test, and a short summary after the suite has completed.
+However, standard output and standard error of the test are redirected
+to a per-test log file, so that parallel execution does not produce
+intermingled output. The output from failed tests is collected in the
+@file{test-suite.log} file. If the variable @samp{VERBOSE} is set, this
+file is output after the summary. For best results, the tests should be
+verbose by default now.
+
@vindex TEST_EXTENSIONS
@vindex TEST_LOGS
Each log file is created when the corresponding test has completed.
@samp{TESTS_ENVIRONMENT} variable is still expanded before the driver,
but should be reserved for the user.
-@vindex VERBOSE
-As with the simple driver above, by default one status line is printed
-per completed test, and a short summary after the suite has completed.
-However, standard output and standard error of the test are redirected
-to a per-test log file, so that parallel execution does not produce
-intermingled output. The output from failed tests is collected in the
-@file{test-suite.log} file. If the variable @samp{VERBOSE} is set, this
-file is output after the summary. For best results, the tests should be
-verbose by default now.
+@vindex AM_TESTS_SETUP
+The @code{AM_TESTS_SETUP} variable can be used to run initialization code
+and set environment variables for the tests' runs. The user can still
+employ the @code{TESTS_ENVIRONMENT} variable to override settings from
+@code{AM_TESTS_SETUP}. Note that, for implementation reasons, if the
+@code{AM_TESTS_SETUP} variable is set, its contents @emph{must} be
+terminated by a semicolon.
+
+@example
+# The tests below are expected to use the file descriptor passed
+# in the environment variable 'warn_fileno' to print warnings
+# (e.g., about skipped and failed tests). If this variable were
+# to be set to `2' (i.e. default stderr), the warnings would be
+# redirected by the automake parallel-tests driver into the .log
+# files. But the AM_TESTS_SETUP definition below will cause the
+# reasons for skip/failure to be printed to the console instead.
+# The user can still override this by setting TESTS_ENVIRONMENT
+# to e.g. `warn_fileno=2' at make runtime, which will cause the
+# warnings to be sent to the .log files again.
+TESTS = test1.sh test2.sh ...
+AM_TESTS_SETUP = exec 9>&2; warn_fileno=9; export warn_fileno;
+@end example
@trindex mostlyclean
@trindex check-html
am__sh_e_setup = case $$- in *e*) set +e;; esac
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
-# containing $f, in $tst the test, in $log the log, and passes
-# TESTS_ENVIRONMENT. Save and restore TERM around use of
-# TESTS_ENVIRONMENT, in case that unsets it.
+# containing $f, in $tst the test, in $log the log. Executes the
+# developer- defined test setup AM_TESTS_SETUP (if any), and passes
+# TESTS_ENVIRONMENT. Saves and restores TERM around uses of
+# TESTS_ENVIRONMENT and AM_TESTS_SETUP, in case any of them unsets
+# it.
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
+$(AM_TESTS_SETUP) \
$(TESTS_ENVIRONMENT)
# To be appended to the command running the test. Handle the stdout
# and stderr redirection, and catch the exit status.
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
-# containing $f, in $tst the test, in $log the log, and passes
-# TESTS_ENVIRONMENT. Save and restore TERM around use of
-# TESTS_ENVIRONMENT, in case that unsets it.
+# containing $f, in $tst the test, in $log the log. Executes the
+# developer- defined test setup AM_TESTS_SETUP (if any), and passes
+# TESTS_ENVIRONMENT. Saves and restores TERM around uses of
+# TESTS_ENVIRONMENT and AM_TESTS_SETUP, in case any of them unsets
+# it.
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
+$(AM_TESTS_SETUP) \
$(TESTS_ENVIRONMENT)
# To be appended to the command running the test. Handle the stdout
parallel-tests8.test \
parallel-tests9.test \
parallel-tests10.test \
+parallel-tests-am_tests_setup.test \
parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parse.test \
am__sh_e_setup = case $$- in *e*) set +e;; esac
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
-# containing $f, in $tst the test, in $log the log, and passes
-# TESTS_ENVIRONMENT. Save and restore TERM around use of
-# TESTS_ENVIRONMENT, in case that unsets it.
+# containing $f, in $tst the test, in $log the log. Executes the
+# developer- defined test setup AM_TESTS_SETUP (if any), and passes
+# TESTS_ENVIRONMENT. Saves and restores TERM around uses of
+# TESTS_ENVIRONMENT and AM_TESTS_SETUP, in case any of them unsets
+# it.
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
+$(AM_TESTS_SETUP) \
$(TESTS_ENVIRONMENT)
# To be appended to the command running the test. Handle the stdout
# and stderr redirection, and catch the exit status.
parallel-tests8.test \
parallel-tests9.test \
parallel-tests10.test \
+parallel-tests-am_tests_setup.test \
parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parse.test \
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests: check AM_TESTS_SETUP support, and its interaction
+# with TESTS_ENVIRONMENT.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_PROG_MKDIR_P
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .sh .test
+TESTS = foo.test bar.sh
+SH_LOG_COMPILER = sh
+AM_TESTS_SETUP = $(MKDIR_P) quux.dir; exec 9>&2; fd=9; export fd;
+EXTRA_DIST = $(TESTS)
+END
+
+cat > foo.test << 'END'
+#! /bin/sh
+echo "FOO='$FOO'"
+echo "BAR='$BAR'"
+test x"$FOO" = x"$BAR"
+END
+chmod a+x foo.test
+
+cat > bar.sh << 'END'
+: ${fd=2}
+eval "echo 'diagnostic message from test bar' >&${fd}"
+END
+
+minicheck ()
+{
+ cat foo.log
+ cat bar.log
+ test -d quux.dir
+}
+
+miniclean ()
+{
+ rmdir quux.dir
+ rm -f foo.log bar.log test-suite.log
+}
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+FOO=1; export FOO
+
+TESTS_ENVIRONMENT='BAR=1' $MAKE check || { cat test-suite.log; Exit 1; }
+minicheck
+miniclean
+TESTS_ENVIRONMENT='BAR=2' $MAKE check && { cat test-suite.log; Exit 1; }
+minicheck
+miniclean
+
+BAR=$FOO; export BAR
+
+$MAKE check 2>stderr || { cat stderr >&2; Exit1; }
+cat stderr >&2
+minicheck
+grep 'diagnostic message from test bar' stderr
+grep 'diagnostic message from test bar' bar.log && Exit 1
+miniclean
+
+TESTS_ENVIRONMENT='fd=2' $MAKE check 2>stderr || { cat stderr >&2; Exit1; }
+cat stderr >&2
+minicheck
+grep 'diagnostic message from test bar' stderr && Exit 1
+grep 'diagnostic message from test bar' bar.log
+miniclean
+
+$MAKE distcheck
+
+:
projects / thirdparty / automake.git / commitdiff
