features.
For a bug report, a feature request or other suggestions, please send
-email to @email{@value{PACKAGE_BUGREPORT}}. This will then open a new
-bug in the @uref{http://debbugs.gnu.org/@/automake, bug tracker}. Be
-sure to include the versions of Autoconf and Automake that you use.
+email to @email{@value{PACKAGE_BUGREPORT}}.
+@c FIXME: we'll need a bug tracker for Automake-NG, sooner or later ...
+@c This will then open a new bug in the
+@c @uref{http://debbugs.gnu.org/@/automake, bug tracker}.
+Be sure to include the versions of Autoconf and Automake-NG that you use.
Ideally, post a minimal @file{Makefile.am} and @file{configure.ac} that
reproduces the problem you encounter. If you have encountered test
- suite failures, please attach the @file{tests/test-suite.log} file.
+ suite failures, please attach the @file{test-suite.log} file.
@c ========================================================== Appendices
set-up of a test scenario, you can use the 'framework_fail_' function
instead.
- For tests that use the 'parallel-tests' Automake-NG option, set the
- shell variable 'am_parallel_tests' to "yes" before including ./defs.
- For tests that are *not* meant to work with the 'parallel-tests'
- Automake-NG option (these should be very very few), set the shell
- variable 'am_parallel_tests' to "no" before including ./defs.
+ For those tests checking the Automake-provided test harnesses that are
+ expected to work also when the 'serial-tests' Automake option is used
+ (thus causing the serial testsuite harness to be used in the generated
+ Makefile), place a line containing "try-with-serial-tests" somewhere
+ in the file. That will ensure that the 'gen-testsuite-part' script
+ generates a sibling of that test which uses the serial harness instead
+ of the parallel one. For those tests that are *not* meant to work with
+ the parallel testsuite harness at all (these should be very very few),
+ set the shell variable 'am_serial_tests' to "yes" before including
+ ./defs.
- Some tests in the Automake testsuite are auto-generated; those tests
+ Some tests in the Automake-NG testsuite are auto-generated; those tests
might have custom extensions, but their basename (that is, with such
extension stripped) is expected to end with "-w" string, optionally
followed by decimal digits. For example, the name of a valid
./configure
-# Some make implementations don't grok the '-j' option.
-$MAKE -j1 || Exit 77
-
for j in '' -j1 -j2; do
$MAKE $j check && Exit 1
- TESTS=foo.test $MAKE $j -e check && Exit 1
+ $MAKE $j TESTS=foo.test check && Exit 1
- if test x"$am_parallel_tests" = x"yes"; then
- $MAKE $j recheck && Exit 1
- $MAKE $j TESTS=foo check && Exit 1
- rm -f test-suite.log
- $MAKE $j test-suite.log && Exit 1
- test -f test-suite.log || Exit 1
- else
- : # For shells with buggy 'set -e'.
- fi
+ $MAKE $j recheck && Exit 1
- TEST_LOGS=foo.log $MAKE $j -e check && Exit 1
++ $MAKE $j TESTS=foo check && Exit 1
+ rm -f test-suite.log
+ $MAKE $j test-suite.log && Exit 1
+ test -f test-suite.log || Exit 1
done
:
./configure
-env TESTS=skip $MAKE -e check >stdout || { cat stdout; Exit 1; }
+$MAKE check TESTS=skip >stdout || { cat stdout; Exit 1; }
cat stdout
- if test x"$am_parallel_tests" = x"yes"; then
- count_test_results total=1 pass=0 fail=0 skip=1 xfail=0 xpass=0 error=0
- else
+ if test x"$am_serial_tests" = x"yes"; then
grep '1.*passed' stdout && Exit 1
: For shells with buggy 'set -e'.
+ else
+ count_test_results total=1 pass=0 fail=0 skip=1 xfail=0 xpass=0 error=0
fi
-env TESTS="skip skip2" $MAKE -e check >stdout || { cat stdout; Exit 1; }
+$MAKE check TESTS="skip skip2" >stdout || { cat stdout; Exit 1; }
cat stdout
- if test x"$am_parallel_tests" = x"yes"; then
- count_test_results total=2 pass=0 fail=0 skip=2 xfail=0 xpass=0 error=0
- else
+ if test x"$am_serial_tests" = x"yes"; then
grep '2.*passed' stdout && Exit 1
: For shells with buggy 'set -e'.
+ else
+ count_test_results total=2 pass=0 fail=0 skip=2 xfail=0 xpass=0 error=0
fi
:
./configure
$MAKE check
-EXEEXT=.bin $MAKE -e print-xfail-tests >stdout || { cat stdout; Exit 1; }
-cat stdout
-$FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
+
- if test x"$am_parallel_tests" != x"yes"; then
++if test x"$am_serial_tests" = x"yes"; then
+ $MAKE EXEEXT=.bin print-xfail-tests >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ $FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
+fi
$MAKE distcheck
# Make sure $(EXEEXT) is appended to programs and to tests that are
# programs, but not to @substitutions@.
- am_parallel_tests=no
-# For gen-testsuite-part: ==> try-with-serial-tests <==
++am_serial_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
# This test exercises the GCS-mandated targets (except for dist)
# as well as tags, TAGS.
+ # For gen-testsuite-part: ==> try-with-serial-tests <==
. ./defs || Exit 1
-# Does $MAKE support the '.MAKE' special target?
-have_dotmake=false
-if using_gmake; then
- have_dotmake=: # GNU make must support it.
-else
- unindent > mk.tmp << 'END'
- targ.tmp:
- : > $@
- .MAKE: targ.tmp
-END
- if $MAKE -n -f mk.tmp targ.tmp && test -f targ.tmp; then
- have_dotmake=:
- fi
-fi
-
mkdir sub sub2
cat >> configure.ac << 'END'
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests driver should be able to cope with test scripts
+# whose names end with several concatenated suffixes.
+
+. ./defs || Exit 1
+
+cat >> configure.ac << 'END'
+AC_OUTPUT
+END
+
+tests='foo.sh foo.t.sh foo.sh.t foo.x.x foo.x.t.sh foo.t.x.sh foo.sh.t.x'
+
+for t in $tests; do
+ (echo '#!/bin/sh' && echo 'echo == /$0 ==') > $t
+ chmod a+x $t
+done
+
+cat > Makefile.am <<END
+TEST_EXTENSIONS = .t .sh .x
+TESTS = $tests
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+for j in '' -j4; do
+
+ # Use append mode here to avoid dropping output. See automake bug#11413.
+ # Also, use 'echo' here to "nullify" the previous contents of 'stdout',
+ # since Solaris 10 /bin/sh would try to optimize a ':' away after the
+ # first iteration, even if it is redirected.
+ echo " " >stdout
+ $MAKE $j check >>stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=7 pass=7 fail=0 skip=0 xfail=0 xpass=0 error=0
+ for t in $tests; do grep "^PASS: $t *$" stdout; done
+
+ grep '== .*/foo\.sh ==' foo.log
+ grep '== .*/foo\.t\.sh ==' foo.t.log
+ grep '== .*/foo\.sh\.t ==' foo.sh.log
+ grep '== .*/foo\.x\.x ==' foo.x.log
+ grep '== .*/foo\.x\.t\.sh ==' foo.x.t.log
+ grep '== .*/foo\.t\.x\.sh ==' foo.t.x.log
+ grep '== .*/foo\.sh\.t\.x ==' foo.sh.t.log
+
+ $MAKE $j clean
+ test ! -f foo.log
+ test ! -f foo.t.log
+ test ! -f foo.sh.log
+ test ! -f foo.x.log
+ test ! -f foo.x.t.log
+ test ! -f foo.t.x.log
+ test ! -f foo.sh.t.log
+
+done
+
+:
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that dynamic content for $(TESTS) is supported, both when set from
+# inside the Makefile.am and when overriddend from the command line.
+
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+cat > ok <<'END'
+#!/bin/sh
+exit 0
+END
+
+cat > ko <<'END'
+#!/bin/sh
+exit 1
+END
+
+cat > er << 'END'
+#!/bin/sh
+echo $0 should not be run >&2
+exit 99
+END
+
+chmod a+x ko ok
+
+mkdir t
+cp ok t/nosuffix
+
+cp ok g1.sh
+cp ok g2.sh
+cp ok g3.sh
+cp ok g4.sh
+cp er g5.sh
+
+cp ok t00-foo.sh
+cp ok t02.sh
+cp ok t57_mu.sh
+cp ok t7311.sh
+cp ko t98S.sh
+cp ko t99.sh
+cp er t1.sh
+cp er t9.sh
+cp er tx98.sh
+
+cat > get-tests-list <<END
+#!/bin/sh
+echo "g1.sh ${tab}g2.sh "
+if :; then echo ' g3.sh'; fi
+echo
+echo g4.sh
+END
+chmod a+x get-tests-list
+
+cat > Makefile.am << 'END'
+my_add_dirprefix = $(strip $(1))/$(strip $(2))
+EXTRA_DIST = $(TESTS) get-tests-list
+TEST_EXTENSIONS = .sh
+TESTS = $(wildcard $(srcdir)/t[0-9][0-9]*.sh)
+TESTS += $(shell $(srcdir)/get-tests-list)
+TESTS += $(call my_add_dirprefix, t, nosuffix)
+XFAIL_TESTS = $(wildcard $(srcdir)/t9[0-9]*.sh)
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE check > stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=11 pass=9 fail=0 xpass=0 xfail=2 skip=0 error=0
+
+grep '^PASS: t/nosuffix$' stdout
+grep '^PASS: g1\.sh$' stdout
+grep '^PASS: g2\.sh$' stdout
+grep '^PASS: g3\.sh$' stdout
+grep '^PASS: g4\.sh$' stdout
+grep '^PASS: t00-foo\.sh' stdout
+grep '^PASS: t02\.sh' stdout
+grep '^PASS: t57_mu\.sh' stdout
+grep '^PASS: t7311\.sh' stdout
+grep '^XFAIL: t98S\.sh' stdout
+grep '^XFAIL: t99\.sh' stdout
+
+$MAKE mostlyclean
+test "`find . -name *.log`" = ./config.log
+
+$MAKE distcheck > stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=11 pass=9 fail=0 xpass=0 xfail=2 skip=0 error=0
+
+$MAKE check tests1='$(wildcard t00*.sh t98?.sh)' \
+ tests2='$(shell ./get-tests-list | sed 1d)' \
+ TESTS='$(tests1) $(tests2)' \
+ > stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=3 fail=0 xpass=0 xfail=1 skip=0 error=0
+
+grep '^PASS: g3\.sh$' stdout
+grep '^PASS: g4\.sh$' stdout
+grep '^PASS: t00-foo\.sh' stdout
+grep '^XFAIL: t98S\.sh' stdout
+
+$MAKE mostlyclean
+test "`find . -name *.log`" = ./config.log
+
+$MAKE check TESTS='$(shell echo t00 | sed "s/$$/-foo/") t99'
+test -f t00-foo.log
+test -f t99.log
+
+$MAKE check \
+ foo='E9E9E' \
+ a='t00.err' \
+ b='${a:.err=-foo}' \
+ TESTS='$(b) t$(subst E,,$(foo)) $(call my_add_dirprefix,t,nosuffix)' \
+ > stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=2 fail=0 xpass=0 xfail=1 skip=0 error=0
+grep '^PASS: t/nosuffix' stdout
+grep '^PASS: t00-foo\.sh' stdout
+grep '^XFAIL: t99\.sh' stdout
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features:
-# - empty TESTS
-# - empty TEST_LOGS
+# Check parallel-tests features: empty TESTS
- am_parallel_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test the example of usage of generic and extension-specific
-# LOG_COMPILER and LOG_FLAGS given in the manual.
+# LOG_COMPILER, LOG_FLAGS and LOG_DEPENDNECIES given in the manual.
- am_parallel_tests=yes
required=python
. ./defs || Exit 1
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests driver must prefer tests with an extension to
+# extension-less tests. This is required to allow the user to have
+# a, say, 'all.test' test case even in the face of the 'all' target.
+
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+cat > foo <<'END'
+#!/bin/sh
+echo "foo without suffix run" >&2
+exit 99
+END
+
+cat > foo.test <<'END'
+#!/bin/sh
+echo "$0 has been run"
+END
+chmod a+x foo.test
+
+cp foo.test all.test
+cp foo.test dist.test
+cp foo.test install.test
+cp foo.test bad-target.test
+
+cat > Makefile.am << 'END'
+bad-target:
+ @echo $@ has been run >&2; exit 1
+install-data-local:
+ @echo $@ has been run >&2; exit 1
+TESTS = foo.test all.test install.test dist.test bad-target.test
+EXTRA_DIST = oops-this-does-not-exist
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure --prefix="`pwd`/inst"
+
+$MAKE check
+ls -l # For debugging.
+test ! -d inst
+for t in foo all install dist bad-target; do
+ grep "$t\.test has been run" $t.log
+done
+
+:
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests harness do not cause the same test to be
+# uselessly run multiple times.
+
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+echo TESTS = foo.test > Makefile.am
+
+cat > foo.test <<'END'
+#! /bin/sh
+ls -l && mkdir bar
+END
+chmod a+x foo.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE -j1 check || { cat test-suite.log; Exit 1; }
+rmdir bar
+$MAKE -j2 check || { cat test-suite.log; Exit 1; }
+rmdir bar
+$MAKE -j4 check || { cat test-suite.log; Exit 1; }
+
+:
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The user should be able to easily specify extra dependencies for
+# the test cases, depending on their extension (or lack thereof).
+# We do so with the help of "${prefix}LOG_DEPENDENCIES" variables.
+# See the last wishlist in automake bug#11287.
+
+. ./defs || Exit 1
+
+cat >> configure.ac <<'END'
+AC_SUBST([EXEEXT], [.bin])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .test .sh
+TESTS = foo.test foo2.test bar.sh baz zard.oz quux.bin mu.test.bin
+
+TEST_LOG_DEPENDENCIES = test-dep
+SH_LOG_DEPENDENCIES = sh-dep1 sh-dep2
+LOG_DEPENDENCIES = dep
+
+DEPS = test-dep sh-dep1 sh-dep2 dep new-test-dep
+$(DEPS):
+ echo dummy > $@
+CLEANFILES = $(DEPS)
+
+.PHONY: setup
+setup:
+ chmod a+x $(TESTS)
+EXTRA_DIST = $(TESTS)
+END
+
+cat > foo.test <<'END'
+#! /bin/sh
+test -f test-dep || test -f new-test-dep
+END
+
+cat > foo2.test <<'END'
+#! /bin/sh
+test -f test-dep
+END
+
+cp foo2.test mu.test.bin
+
+cat > bar.sh <<'END'
+#! /bin/sh
+test -f sh-dep1 && test -f sh-dep2
+END
+
+cat > baz <<'END'
+#! /bin/sh
+test -f dep
+END
+
+cp baz quux.bin
+
+cat > zard.oz <<'END'
+#! /bin/sh
+test -f dep
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE setup
+
+cleanup () { rm -f test-dep sh-dep1 sh-dep2 dep; }
+
+$MAKE check -j4
+test ! -f new-test-dep
+test -f test-dep
+test -f sh-dep1
+test -f sh-dep2
+test -f dep
+test -f quux.log # Sanity check.
+test -f mu.log # Likewise.
+
+cleanup
+
+$MAKE check TESTS=foo.test
+test -f test-dep
+test ! -f sh-dep1
+test ! -f sh-dep2
+test ! -f dep
+
+cleanup
+rm -f bar.log
+$MAKE check TESTS=bar.sh AM_LAZY_CHECK=yes
+test ! -f test-dep
+test -f sh-dep1
+test -f sh-dep2
+test ! -f dep
+
+cleanup
+$MAKE check TESTS=baz
+test ! -f test-dep
+test ! -f sh-dep1
+test ! -f sh-dep2
+test -f dep
+
+cleanup
+$MAKE check TESTS='foo bar'
+test -f test-dep
+test -f sh-dep1
+test -f sh-dep2
+test ! -f dep
+
+cleanup
+$MAKE check TESTS=zard.oz
+test ! -f test-dep
+test ! -f sh-dep1
+test ! -f sh-dep2
+test -f dep
+
+cleanup
+$MAKE check TESTS=mu.test.bin
+test -f test-dep
+test ! -f sh-dep1
+test ! -f sh-dep2
+test ! -f dep
+
+cleanup
+$MAKE check TESTS='quux.bin bar.sh'
+test ! -f test-dep
+test -f sh-dep1
+test -f sh-dep2
+test -f dep
+
+cleanup
+$MAKE check TESTS=foo TEST_LOG_DEPENDENCIES=new-test-dep
+test -f new-test-dep
+test ! -f test-dep
+test ! -f sh-dep1
+test ! -f sh-dep2
+test ! -f dep
+
+cleanup
+$MAKE check TESTS=baz XFAIL_TESTS=baz LOG_DEPENDENCIES=
+test ! -f dep
+grep ':test-result: XFAIL' baz.trs
+
+$MAKE distcheck
+
+:
--- /dev/null
- am_parallel_tests=yes
+#! /bin/sh
+# Copyright (C) 2009-2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - listing $(srcdir)/ or $(top_srcdir)/ in TESTS doesn't work ATM,
+# and is thus diagnosed.
+
+# TODO: this test should also ensure that the 'make' implementation
+# properly adheres to rules in all cases. See the Autoconf
+# manual for the ugliness in this area, when VPATH comes into
+# play. :-/
+
+. ./defs || Exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+cat > Makefile.am << 'END'
+TESTS = \
+ $(srcdir)/foo \
+ @srcdir@/foo2 \
+ @srcdir@/bar.test \
+ ${srcdir}/sub/baz.test \
+ built.test
+
+XFAIL_TESTS = $(srcdir)/bar.test foo2
+
+built.test:
+ (echo '#!/bin/sh' && echo 'exit 77') >$@-t
+ chmod a-w,a+x $@-t && mv -f $@-t $@
+END
+
+cat > foo <<'END'
+#!/bin/sh
+exit 0
+END
+chmod a+x foo
+
+cat > foo2 <<'END'
+#!/bin/sh
+exit 1
+END
+chmod a+x foo2
+
+cp foo2 bar.test
+
+mkdir sub
+cp foo sub/baz.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+../configure
+$MAKE check
+
+ls -l . .. # For debugging.
+
+test -f built.log
+test -f foo.log
+test -f bar.log
+test -f sub/baz.log
+test -f test-suite.log
+
+test ! -f ../built.log
+test ! -f ../foo.log
+test ! -f ../bar.log
+test ! -f ../sub/baz.log
+test ! -f ../test-suite.log
+
+:
# - log files, and what goes in 'test-suite.log'
# - make clean
# - dependencies between tests
-# - TESTS redefinition at runtime
-# - TEST_LOGS redefinition at runtime
-# - RECHECK_LOGS redefinition at runtime
+# - TESTS redefinition at runtime (with and without test suffixes)
+# - AM_LAZY_CHECK
- am_parallel_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
# Actually, this test doesn't ensure that things happen concurrently.
# It merely serves as demonstration. :-)
- am_parallel_tests=yes
-required='cc native GNUmake'
+required='cc native'
. ./defs || Exit 1
cat >> configure.ac << 'END'
# Check parallel-tests features:
# - empty TESTS
-# BSD make will expand '$(TESTS:=.log)' to '.log' unless overridden.
# See parallel-tests10.test for a similar issue.
- am_parallel_tests=yes
. ./defs || Exit 1
+
cat >> configure.ac << 'END'
AC_OUTPUT
END
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features:
-# - generated distributed tests.
-# - listing $(srcdir)/ or $(top_srcdir)/ in TESTS doesn't work ATM,
-# and is thus diagnosed.
-
-# TODO: this test should also ensure that the 'make' implementation
-# properly adheres to rules in all cases. See the Autoconf
-# manual for the ugliness in this area, when VPATH comes into
-# play. :-/
+# Check parallel-tests features: generated distributed tests.
- am_parallel_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
# Auxiliary test to set up common data used by many tests on TAP support.
- am_parallel_tests=yes
. ./defs || Exit 1
-cat >> configure.ac << END
+cat >> configure.ac << 'END'
+# FIXME: must define this otherwise automake will require the presence
+# FIXME: of the 'test-driver' script. This issue should be documented
+# FIXME: in the manual ...
+AC_SUBST([LOG_DRIVER],
+ ['$(error LOG_DRIVER should be never used) false'])
AC_OUTPUT
END
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# TAP support:
-# - RECHECK_LOGS
+# TAP support: AM_LAZY_CHECK
- am_parallel_tests=yes
. ./defs || Exit 1
cat > Makefile.am << 'END'
# Custom test drivers: try the "recheck" functionality with test protocols
# that allow multiple testcases in a single test script. In particular,
-# check that this still works when we override $(TESTS) and $(TEST_LOGS)
-# at make runtime.
-# See also related tests 'test-driver-custom-multitest-recheck.test' and
-# 'parallel-tests-recheck-override.test'.
+# check that this still works when we override $(TESTS) at make runtime.
+# See also related tests 'test-driver-custom-multitest-recheck.test'.
- am_parallel_tests=yes
. ./defs || Exit 1
cp "$am_testauxdir"/trivial-test-driver . \
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# parallel-tests:
-# - non-existent scripts listed in TESTS get diagnosed
-# See also related test 'test-missing2.test'.
+# parallel-tests: non-existent scripts listed in TESTS get diagnosed.
- am_parallel_tests=yes
. ./defs || Exit 1
cat >> configure.ac << 'END'
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# parallel-tests support: the following are registered with '.SUFFIXES':
-# - .log
-# - .trs (used by files that store test results and metadata)
-# - .test if $(TEST_EXTENSIONS) is not defined
-# - stuff in $(TEST_EXTENSIONS) otherwise
+# Check parallel harness features:
+# - recovery from deleted '.log' and '.trs' files, with parallel make
- am_parallel_tests=yes
. ./defs || Exit 1
-: > Makefile.am
+all= log= trs=
+for i in 1 2 3 4 5 6 7 8 9 10 11 12; do
+ all="$all $i" log="$log $i" trs="$trs $i"
+done
-cat > 1.am << 'END'
-TESTS =
-END
+echo AC_OUTPUT >> configure.ac
+echo TESTS = > Makefile.am
-cat > 2.am << 'END'
-TEST_EXTENSIONS = .SH .abcdef
-TESTS =
-END
+for i in $all; do
+ echo TESTS += $i.test >> Makefile.am
+ (echo "#!/bin/sh" && echo "mkdir $i.d") > $i.test
+ chmod a+x $i.test
+done
-: > test-driver
+ls -l # For debugging.
$ACLOCAL
-
-$AUTOMAKE 1
-$AUTOMAKE 2
-
-sed -e 's/$/ /' 1.in > mk.1
-sed -e 's/$/ /' 2.in > mk.2
-
-grep '^\.SUFFIXES:' mk.1
-grep '^\.SUFFIXES:' mk.2
-
-for suf in test log trs; do
- grep "^\\.SUFFIXES:.* \\.$suf " mk.1
-done
-
-for suf in SH abcdef log trs; do
- grep "^\\.SUFFIXES:.* \\.$suf " mk.2
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+: Create the required log files.
+$MAKE check
+
+for n in 1 2 5 7 12; do
+ for suf in log trs; do
+ rmdir *.d
+ rm -f *.$suf
+ $MAKE -j$n check
+ for f in $all; do
+ test -f $f.log
+ test -f $f.trs
+ done
+ done
done
:
projects / thirdparty / automake.git / commitdiff
