tests: add testcases sanity-checking the testsuite
Helper subroutines, variables and other pieces of code defined
in the `tests/defs' and used by many testcases are non-obvious,
and tricky to get to work portably; but until now, they weren't
tested at all in a clear and self-contained way.
This change should remedy to the situation.
* tests/self-check-cleanup.test: New test, check removal of
temporary test working directory by `./defs'.
* tests/self-check-dir.test: New test, check that tests using
`./defs' create a proper temporary directory, and run in it.
* tests/self-check-exit.test: New test, check that, in case of
failing commands, the correct exit status is passed to the exit
trap installed by the `./defs' script.
* tests/self-check-is_newest.test: New test, checking the
`is_newest' subroutine.
* tests/self-check-me.test: New test, checking that $me gets
defined automatically by `tests/defs' if not set, and that it
can be overridden from either the shell or the environment.
* tests/self-check-sanity.test: New test, check that the sanity
checks performed by the `tests/defs' script works correctly.
* tests/self-check-unindent.test: New test, checking the
`unindent' subroutine.
* tests/Makefile.am (TESTS): Update.
tests: tempdirs with restrictive permissions are cleaned correctly
Before this change, the removal of a temporary test directory
containing subdirectories with restrictive permissions (such as
'r--r--r--') could fail.
* tests/defs: Ensure that all the subdirectories of a temporary
test directory have the 'read', 'write' and 'execute' bits set,
before trying to remove it with `rm -rf'.
* tests/Makefile.am (clean-local-check): Likewise.
After the split of `instspc.test' into various generated tests,
the running time of the testsuite has noticeably increased, since
all these new generated tests must run aclocal, autoconf and
automake, whereas previously they were run only once (at the
beginning of `instspc.test'). But luckily, since the new tests
share the same input files for the autotools, this situation can
be easily worked around (at the expenses of a slight increase of
complexity for the testsuite scaffolding).
* tests/instspc-data.test: New helper test, properly calling
the `instspc-tests.sh' script to generate input data for the
others `instspc-*.test' tests.
* tests/Makefile.am (TESTS): Add `instspc-data.test'.
($(instspc_tests:.test=.log)): Depend on its log file.
(instspc-data.log): Depend on `instspc-tests.sh'.
* tests/instspc-tests.sh: Recognize new action `generate-data',
and use it to create hand-written and autotools-generated static
files shared by all the `instspc-*.test' tests.
When sourced by the `instspc-*.test' tests, use those previously
created files instead of recreating them from scratch.
(unindent, create_input_data): New subroutines.
Some other related changes and refactorings.
* tests/python-vars.test: Ignore cached values from config.site
by exporting CONFIG_SITE=/dev/null. Be laxer in matching the
expected values of output variables `pythondir' and `pyexecdir',
since they can change quite unpredictably among different python
installations. Also, avoid "hyping" debugging output, thus
offering smaller trace output and more informative diff.
* tests/python-virtualenv.test: Require python, since we call it
even after the virtualenv has been deactivated. Ignore cached
values from config.site by exporting CONFIG_SITE=/dev/null.
tests: tweak few tests on simple and parallel test drivers
* tests/check-exported-srcdir.test: Improve heading comments.
* tests/check-tests-in-builddir.test: Likewise. Also, unset the
`FOO_EXIT_STATUS' variable, so that any pre-existing value in the
environment won't risk to interfere with the test.
python: extend and improve tests, fix minor glitches
* m4/python.m4 (AM_PATH_PYTHON): Ensure the "checking ..." messages
from configure are always complete, even in case of failure. Tiny
cosmetic improvement in info/error messages.
* tests/python.test: Also check that automake complains if the
PYTHON primary is used but the `py-compile' script is not present.
Make grepping of generated Makefile.in laxer w.r.t. whitespace.
Add trailing `:' command.
* tests/python2.test: Remove repeated calls aclocal: they are
useless because configure.in is never modified. Make grepping
of automake stderr more comprehensive. Remove the pre-existing
`py-compile' file before trying to install it with `--add-missing'.
Add trailing `:' command.
* tests/python3.test: Add trailing `:' command.
* tests/python11.test: Likewise.
* tests/python4.test: Likewise. Also, try to pass PYTHON config
variable to configure from the environment, rather than only from
the command line.
* tests/python5.test: Ensure that the "checking ..." messages from
configure are always complete. Use proper m4 quoting. Add a
trailing `:' command.
* tests/python6.test: Simplify test logic, by checking for files
created by configure rather then grepping its output.
* tests/python7.test: Likewise.
* tests/python8.test: Also check that `$PYTHON' is meaningfully
set in the ACTION-IF-TRUE argument of AM_PATH_PYTHON.
* tests/python9.test: Likewise.
* tests/python10.test: Add trailing `:' command.
(Makefile.am): Declare `disttest' target as `.PHONY', and add
an `ls -l' to its recipe, for debugging.
* tests/nobase-python.test: In testing "make uninstall" and
"make install" results, prefer idioms that make verbose logs
more helpful. Remove a couple of lines of dead code. Add a
trailing `:' command.
* tests/python5b.test: New test, checking that configure performs
the check on the python version even when the choice of the python
interpreter is forced by the user.
* tests/python-dist.test: New test, checking the distribution of
*_PYTHON files.
* tests/python-vars.test: New test, checking that AM_PATH_PYTHON
correctly set all the output variables advertised in the manual.
* tests/python-virtualenv.test: New test, checking that python
support offered by automake works well with "virtual python
environments" created by the `virtualenv' program.
* tests/instdir-prog.test: Also check `$(pyexecdir)'. Existing
checks made slightly stricter.
* tests/instdir-prog.test: Also check `$(pyexecdir)'. Existing
* tests/instdir-ltlib.test: Likewise.
* tests/Makefile.am (TESTS): Update.
* doc/automake.texi (Simple Tests): Document that automatic
recognition of a capable terminal attached to stdout can fail
with some make implementation when running in parallel mode,
thus causing colored test output not to be automatically
activated when it should.
Ralf Wildenhues [Sun, 23 Jan 2011 10:26:50 +0000 (11:26 +0100)]
tests: avoid instspc.test failures due to make's whitespace trimming
* tests/instspc.test: Prepend './' when passing the test
characters, to avoid leading whitespace characters to be trimmed
from macros set from environment variables. Fixes testsuite
failures with HP-UX, IRIX, and Tru64/OSF make.
coverage: test semantics of "dummy" per-target flags
* tests/specflg-dummy.test: New test, ensuring that even "dummy"
per-target flags triggers the use of renamed objects.
* tests/Makefile.am (TESTS): Update.
* tests/defs.in ($IFS): Define to <space>, <tab>, <newline>.
($sp): New variable, holding a single whitespace character.
($tab): New variable, holding a tabulation character.
($nl): New variable, holding a newline character.
Ralf Wildenhues [Sun, 23 Jan 2011 09:57:21 +0000 (10:57 +0100)]
tests: avoid instspc* failures due to make's whitespace trimming
* tests/instspc-tests.sh: Prepend './' when passing the test
characters, to avoid leading whitespace characters to be trimmed
from macros set from environment variables. Fixes testsuite
failures with HP-UX and Tru64/OSF make.
Ralf Wildenhues [Sat, 22 Jan 2011 12:11:39 +0000 (13:11 +0100)]
tests: fix VPATH auto-expansion workarounds.
* tests/suffix10.test, tests/suffix11.test, tests/suffix12.test,
tests/suffix3.test, tests/suffix5.test, tests/suffix8.test:
Ensure $< is not surrounded by white space, to prevent Solaris
make from applying automatic VPATH text expansion.
Ralf Wildenhues [Sat, 22 Jan 2011 17:20:41 +0000 (18:20 +0100)]
tests: fix VPATH auto-expansion workarounds.
* tests/parallel-tests8.test, tests/suffix13.test:
Ensure $< is not surrounded by white space, to prevent Solaris
make from applying automatic VPATH text expansion.
Ralf Wildenhues [Sat, 22 Jan 2011 12:46:27 +0000 (13:46 +0100)]
tests: avoid fn99*.test failures due to buggy AIX 5.3 cp -R.
* tests/fn99.test, tests/fn99subdir.test: Skip if an initial
`cp -R' of the subdir tree already fails; AIX 5.3 cp messes
up its internal memory when copying this tree.
Ralf Wildenhues [Fri, 21 Jan 2011 18:54:35 +0000 (19:54 +0100)]
tests: avoid false failure in cygnus-dependency-tracking.test.
* tests/cygnus-dependency-tracking.test: Be less restrictive
when grepping the compiler error message, GCC 3.4.6 on FreeBSD
does not mention an undefined symbol.
Ralf Wildenhues [Wed, 19 Jan 2011 20:50:02 +0000 (21:50 +0100)]
install-sh: avoid Tru64 sh `test' operator precedence issues.
* lib/install-sh: Protect file names and directory components
that consist of `=', `(', `)', or `!'. Move protection as early
as possible, to avoid errors such as with Tru64 sh `test -z ='.
* tests/instsh2.test: Extend test to cover more possibilities.
Fixes 1.12 instspc-equal-install.test failure on Tru64/OSF 5.1.
Ralf Wildenhues [Tue, 18 Jan 2011 21:58:18 +0000 (22:58 +0100)]
tests: work around Tru64 sh -e issues for instspc*.test.
Apparently, Tru64 sh does not like turning off errexit mode,
and gets confused.
* tests/defs: Document 'errexit' cleanliness requirement.
* tests/defs-static.in: Likewise. Avoid error from command
substitution to abort instspc*.test with Tru64/OSF 5.1 sh.
* tests/instspc-tests.sh: Drop now-unneeded temporary errexit
dropping. Add strategic '|| Exit' to let tests work on Tru64.
Ralf Wildenhues [Wed, 19 Jan 2011 18:46:27 +0000 (19:46 +0100)]
Allow _AM_DEPENDENCIES to be used later in configure.
* m4/depend.m4 (_AM_DEPENDENCIES): Remove a previously existing
conftest.dir before recreating it.
Fixes bug#7864.
Report by Eric Blake, from report by Scott McCreary against M4.
Ralf Wildenhues [Sun, 16 Jan 2011 21:27:37 +0000 (22:27 +0100)]
tests: allow double-colon spy.test to work with HP-UX make.
* tests/spy.test: Fix comment typos. Ensure prerequisites we
do not want to depend on are strictly older than the target.
Also test with a target out of date wrt. more than one rule.
Ralf Wildenhues [Sun, 16 Jan 2011 12:48:17 +0000 (13:48 +0100)]
parallel-tests: work around Tru64/OSF 5.1 sh read bugs.
* lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html):
Test file readability before redirecting input from it, to avoid
exiting Tru64/OSF 5.1 sh which treats read as special builtin.
* tests/parallel-tests-unreadable-log.test: New test.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
Ralf Wildenhues [Sun, 16 Jan 2011 13:17:38 +0000 (14:17 +0100)]
tests: avoid failing command substitution in errexit mode.
* tests/vartypo2.test, tests/vartypos.test: Rewrite to not use
a command substitution with a nonzero exit status, that causes
IRIX and Tru64/OSF sh to fail the whole test.
* warns-win-over-strictness:
Update docs w.r.t. warning and strictness options.
More tests on warnings/strictness precedence.
Update NEWS about the warnings-over-strictness precedence.
Warnings win over strictness in AUTOMAKE_OPTIONS.
Change signature of 'Automake::Options::_process_option_list()'.
Add more tests about AUTOMAKE_OPTIONS.
Warnings win over strictness in AM_INIT_AUTOMAKE.
Warnings win over strictness on command line.
More tests on warnings and strictness.
New test on silent-rules mode and portability warnings.
Add new tests on strictness and warnings precedence and overriding.
Update docs w.r.t. warning and strictness options.
* doc/automake.texi (Strictness): Document that some warnings are
turned off by default in `foreign' strictness.
(Options): Divide into new sections "Options generalities" and
"List of Automake options". Fix typo (colon instead of full
stop). Document option precedence (AUTOMAKE_OPTIONS wins over
AM_INIT_AUTOMAKE which wins over command line). Also document
interactions between options specifying strictness and those
specifying warnings.
* tests/warning-groups-win-over-strictness.test: New test, similar
to `warnings-win-over-strictness.test', but checking the explicit
catch-all warning flags (like `-Wall' and `-Wnone').
* tests/Makefile.am (TESTS): Update.
Ensure that, for what concerns the options specified in
AUTOMAKE_OPTIONS, explicitly-defined warnings always take
precedence over implicit strictness-implied warnings.
This finally fixes Automake bug#7669 a.k.a. PR/547.
* automake.in (handle_options): Call 'process_option_list'
only once per set of options.
* lib/Automake/Options.pm (process_global_option_list,
process_option_list): Add sanity checks.
($_options_processed, $_global_options_processed): New
internal variables, used by the sanity checks above.
* tests/warnings-win-over-strictness.test: Extend.
Change signature of 'Automake::Options::_process_option_list()'.
This only modifies internal details in the automake implementation,
bearing no externally visible effect, but preparing the way for the
final fix of Automake bug#7669 a.k.a. PR/547.
* lib/Automake/Options.pm (_process_option_list): Now accepts as
arguments a list of hash references with keys 'option' and 'where',
where 'option' is an option as might occur in AUTOMAKE_OPTIONS or
M_INIT_AUTOMAKE, and 'where' is the location where that occurred.
(process_option_list, process_global_option_list): Update.
* automake.in (handle_options, scan_autoconf_traces): Update.
In view of soon-to-follow refactorings (still in the pursuit of a
fix for Automake bug#7669 a.k.a. PR/547), add some more tests on
AUTOMAKE_OPTIONS support, to prevent obvious regressions.
This change ensures that, for what concerns the options specified
in AM_INIT_AUTOMAKE, explicitly-defined warnings always take
precedence over implicit strictness-implied warnings. Related to
Automake bug#7669 a.k.a. PR/547.
* lib/Automake/Options.pm (_process_option_list): Parse explicit
warnings only after the strictness level has been set. Fix POD
documentation.
* tests/warnings-win-over-strictness.test: Extend.
This change ensures that, on the command line at least, explicitly
defined warnings always take precedence over implicit strictness
implied warnings. Related to Automake bug#7669 a.k.a. PR/547.
* automake.in (parse_arguments): Parse warnings only after the
strictness level has been processed.
* tests/gnuwarn.test: Update, plus miscellaneous improvements.
* tests/warnings-win-over-strictness.test: New test.
* tests/Makefile.am (TESTS): Update.
Add new tests on strictness and warnings precedence and overriding.
* tests/strictness-overriding.test: New test.
* tests/strictness-precedence.test: New test.
* tests/warnings-overriding.test: New test.
* tests/warnings-precedence.test: New test.
* tests/Makefile.am (TESTS): Update.
Ralf Wildenhues [Sat, 15 Jan 2011 13:54:51 +0000 (14:54 +0100)]
docs: ensure example are separated with empty lines in the input
* doc/automake.texi (Extending aclocal, Emacs Lisp, Rebuilding)
(API Versioning, Renamed Objects, Multiple Outputs): Add empty
lines before `@example' and after `@end example' lines, so info
output is rendered correctly, and a following @noindent honored.
Report by Stefano Lattarini.
* tests/txinfo.test ($required): Add 'makeinfo'.
* tests/txinfo8.test: Create a dummy 'textutils.info' file, so
that make won't try to run makeinfo (which could be unavailable)
to build it.
Found by NixOS Hydra, reported by Ralf Wildenhues.
Ralf Wildenhues [Thu, 13 Jan 2011 19:21:39 +0000 (20:21 +0100)]
Avoid testsuite failures due to Autoconf Fortran change.
Autoconf v2.68-21-g727ce95 causes AC_F77_LIBRARY_LDFLAGS to
require computing the canonical host name. Ensure config.guess
and config.sub files are present for respective checks.
* tests/compile_f_c_cxx.test: Add stub files.
* tests/flibs.test: Likewise.
* tests/fort4.test: Use $AUTOMAKE -a for installing files.
projects / thirdparty / automake.git / log
