tests/README: suggest how to run tests in cross-compile mode
* tests/README (Section "User interface" subsection "Running the
tests"): Briefly explain how to override 'host_alias' at runtime
to force the use of cross-compilers by the testsuite. Give an
example.
* configure.ac (AC_CANONICAL_HOST): New, probably not strictly
necessary, but useful to complements AC_CANONICAL_BUILD.
* tests/defs-static.in ($host_alias, $build_alias): New variables,
exported, user-overridable, defaults AC_SUBST'd by configure.
* tests/defs (cross_compiling): Consider cross-compilation mode
active only if `$host_alias' is not empty *and* it differs from
`$build_alias'. This is more faithful to the way configure
determines whether it is cross-compiling, but still allows the
user to easily force cross-compilation by overriding `host_alias'
and `build_alias' at runtime.
* master:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
* maint:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
testsuite: avoid re-running few tests with 'parallel-tests' option
Some tests in our testsuite use the 'simple-tests' driver only
marginally, or simply as a mean to conveniently check unrelated
invariants. It makes little sense to force these tests to also
run with the 'parallel-tests' Automake option active, as doing so
offers no real gain in coverage, while often causing a measurable
overhead in execution time (for an already too-slow testsuite).
* tests/pr401.test (parallel_tests): Define to "no", to prevent
the generation of a sibling test script using the 'parallel-tests'
driver.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
testsuite: allow tests to avoid the use of 'parallel-tests' option
* tests/gen-parallel-tests: Do not generate "siblings" for tests
that explicitly define the `parallel_tests', whether to "yes" or
to any other value. Extend heading comments to give a rationale
for this behaviour.
* tests/README: Update.
* tests/README (Section "Writing test cases" subsection "Do"):
Do not suggest to use the `*-p.test' pattern for the names of
hand-written tests which use the `parallel-tests' Automake option.
Not only is this not respected by the existing tests, but it is
more likely to cause conflicts with auto-generated tests.
So, suggest to *avoid* using the `*-p.test' pattern in names
of hand-written tests instead.
(Section "Writing test cases" subsection "Do not"): When
suggesting not to override Makefile variables using command
line arguments, do not use the badly outdated variables `U'
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.
tests: avoid spurious failures in cross-compile mode
* tests/depcomp2.test: Ensure verbose printing of captured stderr
from configure.
* tests/ansi3.test ($required): Add 'native', as the ansi2knr
functionality is not meant to work with a cross-compiler.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi8.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/check8.test ($required): Add 'native', as this test is
not meant to work with a cross-compiler.
* tests/gnits2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/check5.test: Likewise. Also, ensure verbose printing of
captured make output, and thrown in few cosmetic and consistency
improvements.
* tests/ltinit.test: Be laxer in grepping configure output, to
avoid spurious failures on systems which lack POSIX dynamic
linking (e.g., MinGW), or when cross-compiling for such systems.
* tests/ansi9.test: Skip parts of the test that would require a
native compiler.
* tests/backcompat6.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/lex3.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/autohdr4.test: Likewise. Also, extend the test to ensure
more coverage in cross-compile mode.
* tests/canon7.test: Likewise.
* tests/check9.test: Removed, split into two new tests ...
* tests/check-subst-prog.test: ... this one, which requires a
native C compiler ...
* tests/check-subst.test: ... and this one, which doesn't require
any compiler.
* tests/parallel-tests4.test: Removed, split into two new tests ...
* tests/parallel-tests-suffix-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-suffix.test: ... and this one, which
doesn't require any compiler.
* tests/parallel-tests7.test: Removed, split into two new tests ...
* tests/parallel-tests-ext-driver-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-ext-driver.test: ... and this one, which
doesn't require any compiler.
* tests/Makefile.am (TESTS): Update.
* tests/defs.in (cross_compiling): New subroutine.
(am__tool_prefix): New internal variable.
(gcc, g++, gcj): Force the use of the correct "tool prefix"
when cross compiling.
(gfortran, g77, non-cross): New requirements.
tests: fix portability issues in 'repeated-options.test'
* tests/repeated-options.test: Do not assume that object files
have `.o' suffix and executables have no default suffix; instead,
use `$(OBJEXT)' and `$(EXEEXT)'.
tests: fix spurious failure of extradep.test on FreeBSD
* tests/extradep.test: When using `ls -t' to determine whether a
file has been updated, make sure to use as reference a file whose
timestamp is expected to be *strictly* older that that of the file
being checked. This is required because at least FreeBSD `ls' do
not sort files with the same timestamp in alphabetical order when
using the `-t' option.
* tests/extradep2.test: Likewise.
tests: fix spurious failure of txinfo21.test on FreeBSD
* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
This is required because at least FreeBSD `ls' do not sort files
with the same timestamp in alphabetical order when using the `-t'
option.
* tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
newlines from `$all_depmodes'. The latter idiom with `tr' is
wrong since it gets run before `./defs' is sourced, and thus `$nl'
and `$sp' are undefined.
Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
empty strings as arguments.
tests: add excerpts from few original bug reports, for clarity.
* tests/comment6.test: Add excerpts and/or details from the
original bug report that prompted this tests to be added and/or
extended.
* tests/dejagnu4.test: Likewise.
* tests/installdir.test: Likewise.
With some edits to `tests/defs' and `ChangeLog', to reflect the
fact that in master we already had a `yacc' requirement, and no
more had an explicitly-recognized `bison' requirement.
tests defs: allow requirements for compilers (mostly dummy)
Most of the new requirements that are now accepted in `$required'
as consequence of this patch are still dummy. They are planned
to be implemented only in master (or in some derived branch), but
having them here (even just as no-op) will allow for an easier
integration/backporting of potential new testcases.
* tests/defs.in (cc, c++, fortran, fortran77): New requirements,
still dummy.
(flex): New requirement, picking LEX for configure.
(lex): New requirement, alias for `flex'. A more appropriate
implementation, looking for a generic `lex' program, will follow
in the future.
(yacc): New requirement, alias for `bison'. A more appropriate
implementation, looking for a generic `yacc' program, will follow
in the future.
tests: 'silent-many-gcc.test' improved and made more robust
* tests/silent-many-gcc.test: Also force "fast" gcc depmode
for C++ compilations. Add sanity checks verifying that the
cache variables we force are really used by configure. Fix
typo in comments.
* tests/lflags2.test ($required): Remove 'g++', as any working
C++ compiler should be enough, and ./configure will cause the
test to skip if no working C++ compiler is found.
* tests/yflags2.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/silentcxx.test: Likewise. Also, do not force depmodes
that could cause non-GNU C++ compilers to fail.
* tests/silentcxx-gcc.test: New test, like `silentcxx.test',
but forcing "fast" gcc depmode (and thus requiring the GNU C++
compiler).
* tests/specflg10.test: Add proper "fixme" comment telling that
we should make this test work with a generic C++ compiler.
* tests/Makefile.am (TESTS): Update.
tests defs: one more environment sanitization (corner case)
* tests/defs (am__using_gmake): Initialize. The `using_gmake'
subroutine was using this variable for caching, but wasn't
initializing it, which could cause problems in the (admittedly
very unlikely) case in which it was pre-existent in the
environment.
* tests/Makefile.am (TESTS): Added test scripts man[678].test,
which have been present on the filesystem and in the repository,
but which had erroneously been left out from $(TESTS).
* CheckListOfTests.am: New file.
(maintainer-check-list-of-tests): New target,
check for consistency between list of tests defined in the
including Makefile and list of tests on the filesystem.
(clean-maintcheck-testslist-tmp): New rule, to clean up the
temporary files that might be left around by the rules associated
with the previous target.
(clean-local): Depend on it.
* lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
(maintainer-check): Added dependency from the new target
`maintainer-check-list-of-tests'.
* tests/Makefile.am: Likewise.
* Makefile.am (maintainer-check-list-of-test): New target,
calling recursively into `tests/' and `lib/Automake/tests/',
using ...
(TEST_SUBDIRS): ... this new variable.
* tests/defs (GNUmake): Instead of just skipping the tests
requiring GNU make if $MAKE is not GNU make, try to look for
it and, if found, redefine $MAKE accordingly. This will help
to transparently increase coverage on non-GNU systems which
have GNU make available in PATH.
* tests/acloca22.test: Improve and extend the heading comments.
Add reference to related tests 'remake-deleted-m4-file.test' and
'remake-renamed-m4-macro-and-file.test'. Since we are at it,
add trailing `:' command, and few blank lines for readability.
* tests/remake-renamed-m4-macro-and-file.test: In the heading
comments, add reference to the related test 'acloca22.test'.
tests: few more checks on automatic remake support
* tests/remake-all-1.test: New test, check that the "all" target
triggers rebuilt of outdated Makefiles.
* tests/remake-all-2.test: Likewise, but for when the makefiles
are not named `Makefile'.
* tests/Makefile.am (TESTS): Update.
tests: check some remake features with non-GNU make too
* tests/acloca14.test ($required): Drop "GNUmake". This test
works as-is with non-GNU make implementations.
* tests/remake1a.test: Likewise.
* tests/aclocal4.test ($required): Drop "GNUmake".
Modify the test to have it work also with non-GNU make.
* tests/remake5.test: Likewise.
* tests/remake8a.test: Likewise.
* tests/remake8b.test: Likewise.
* tests/remake9a.test: Likewise.
* tests/remake9b.test: Likewise.
* tests/remake9c.test: Likewise.
* tests/remake9d.test: Likewise.
* tests/remake10a.test: Likewise.
* tests/remake10b.test: Likewise.
* tests/remake10c.test: Likewise.
* tests/remake12.test: ($required): Drop "GNUmake".
Adapt the test to make it work also with non-GNU make (if
it supports an "include" directive).
* tests/color2.test: If $MAKE contains command-line arguments (as
in e.g., "make -j2"), expect's directive "spawn $env(MAKE)" fails
spuriously, because it tries to run "$MAKE" as a single command.
Fix this with proper uses of the TCL `eval' builtin.
tests: extend tests on dependency tracking with VPATH
* tests/depmod-tests.sh: New script, fulfilling a threefold role:
1. it is called to generate a Makefile.am snippet, containing the
definition of proper lists of tests;
2. it is called to set up a directory containing some common data
files and autotools-generated files used by the aforementioned
tests (this is done for speed reasons only); and
3. it is called to properly run those tests, one at a time.
* tests/depcomp9.test: Delete, it's obsolete now.
* tests/depcomp10.test: Likewise.
* tests/Makefile.am ($(srcdir)/depmod-tests.am): Include this
snippet, which defines ...
(depmode_tests): ... this new macro, containing the list of the
newly generated `*.depmod' tests.
(TESTS_EXTENSIONS): Add `.depmod'.
(DEPMOD_LOG_COMPILER): Define. It calls `depmod-tests.sh', so that
the "depmode tests" will be executed by passing that driver script
a proper parameter.
($(depmod_tests)): New dependency declaration (dummy, but required
in order to have make actually produce expected log files from the
`.depmod.log' suffix rule).
(TESTS): Add $(depmod_tests).
(EXTRA_DIST): Distribute depmod-tests.sh.
Other minor cosmetic changes and reorderings.
* bootstrap: Generate depmod-tests.am.
* tests/.gitignore: Updated.
tests: avoid extra test generation (for `instspc' tests)
We don't really need to generate tons and tons of wrapper test
scripts (presently, ~ 90) just to run what is basically a single
test (specifically, `instspc-tests.sh') with different parameters.
In fact, the possibility of running a single driver script with
multiple files/configurations/parameters is one of the major
selling points of the 'parallel-tests' testsuite driver of
automake. So why not use ourselves more extensively the best
features we provide?
* tests/instspc-tests.sh: Update heading comments. Expect
to be executed, not sourced. Update handling of command-line
arguments. Adjust to reflect the renaming of "instspc tests"
from `instspc-*.test' to `*.instspc'.
* tests/instspc-data.test: Update heading comments. Execute
driver script `instspc-tests.sh', instead of sourcing it.
Give more informative messages in case of internal errors.
* tests/Makefile.am (TESTS_EXTENSIONS): Define explicitly (to
`.test' and `.instspc').
(TESTS): Add `$(instspc_tests)'.
(generated_tests): Remove `$(instspc_tests)'.
(INSTSPC_LOG_COMPILER): Define. Calls `instspc-tests.sh', so
that the "instspc tests" will be now executed by passing that
driver script a proper parameter.
($(instspc_tests)): Remove, we don't need anymore to generate
this tests.
($(instspc_tests:.test=.log)): Remove, substituted by ...
($(instspc_tests:.instspc=.log)): ... this.
($(instspc_tests)): New dependency declaration (dummy, but
required in order to have make actually produce expected log
files from the `.instspc.log' suffix rule).
(MAINTAINERCLEANFILES, generated_tests): Don't extend with
$(instspc_tests) anymore.
Update comments.
build: improve the definition of the list of testcases
* tests/Makefile.am (handwritten_tests): New variable.
(generated_tests): Likewise.
(TESTS): Redefine as the union of the above.
(EXTRA_DIST): Extend using $(handwritten_tests) and
$(generated_tests) rather than $(TESTS).
* tests/gen-parallel-tests: Update accordingly, and
make more robust.
* tests/defs: Use shorter messages when giving reasons for test
skipping; it turns out these shorter messages are also clearer.
If more info might be useful, send them to the log file only.
tests: fix self check spurious failure with older bash versions
* tests/self-check-cleanup.test: Add trailing `:' to the test code
passed to $SHELL, otherwise older versions of bash (at least 2.05
and 3.2.39) fail to correctly remove the temporary directory in
the exit trap.
tests: add sanity check to 'self-check-cleanup.test'
* tests/self-check-cleanup.test: Check that the "hacked" file
`defs-static' used by the test differs from the "vanilla" one
in builddir. This also offers a little more debugging output.
testsuite: display reasons for skips to the console
* tests/Makefile.am (AM_TESTS_ENVIRONMENT): Open file descriptor
`9' to the original stderr; define `stderr_fileno_' to `9', and
export it.
* tests/self-check-report.test: Prevent new spurious failures by
removing from the environment any definition of `stderr_fileno_'.
* tests/defs: Move the checks verifying that the variables `me',
`parallel_tests' and `required' aren't set in the environment ...
* tests/defs-static.in: ... in here, with some optimizations to
avoid useless forks. Also, do the same checks for the variables
`original_AUTOMAKE' and `original_ACLOCAL' too.
* tests/self-check-env-sanitize.test: Update.
* tests/Makefile.am (TESTS_ENVIRONMENT): Unset also variables
`original_AUTOMAKE' and `original_ACLOCAL'.
tests: fix spurious failure in a test on TESTS (VPATH-related)
* check-tests-in-builddir.test: When not using the parallel-tests
option, do not check that VPATH components are not present in the
displayed test name, since the simple-tests driver do not try to
strip them.
Jim Meyering [Sun, 16 Jan 2011 14:36:07 +0000 (15:36 +0100)]
test defs: new subroutines for test skipping/failing.
* tests/defs.in (Exit): Move definition of this function earlier.
(warn_, skip_, fail_, framework_failure_): New functions, inspired
to the homonyms in gnulib's tests/init.sh.
($stderr_fileno_): New global variable, used by the new functions
above.
* tests/README: Updated.
* tests/mclean.test: Remove, it's a weak grepping test completely
superseded by the much more complete `maintclean.test'.
* tests/Makefile.am (TESTS): Update.
* tests/defs.in: Sanity check: abort if any of `parallel_tests'
or `required' is in the environment.
($sed_unindent_prog): Initialize to empty, to avoid interferences
from the environment.
* tests/self-check-me-in-env.test: Renamed to ...
* tests/self-check-env-sanitize.test: ... this, and extended.
* tests/Makefile.am (TESTS): Update.
(TESTS_ENVIRONMENT): Unset variables `parallel_tests' and
`required'. Adjust comments.
tests: don't allow `$me' to be overridden from the environment
* tests/defs.in: Sanity check: abort if $me is in the environment.
* tests/self-check-me-in-env.test: New test.
* tests/Makefile.am (TESTS_ENVIRONMENT): Unset variable `me'.
(TESTS): Update.
The AM_TESTS_SETUP naming was not a good one after all. It may
be technically more correct than AM_TESTS_ENVIRONMENT, but the
latter is a better one simply because it is easier to remember,
and even if you've never heard of it and only know the semantics
of TESTS_ENVIRONMENT, you can have a straightforward way to
figure out how AM_TESTS_ENVIRONMENT would work.
* tests/check.am (am__check_pre): Update.
* doc/automake.in (Simple Tests using parallel-tests): Update.
* tests/parallel-tests-am_tests_setup.test: Renamed ...
* tests/parallel-tests-am_tests_environment.test: ... to this,
and updated.
* tests/Makefile.am (TESTS): Update.
check: new developer-reserved AM_TESTS_SETUP variable
For reference, see the discussion at:
<http://lists.gnu.org/archive/html/automake-patches/2011-01/msg00213.html>
* lib/am/check.am [%?PARALLEL_TESTS%] (am__check_pre): Pass also
$(AM_TESTS_SETUP). Comments updated, and some typos fixed.
* doc/automake.texi (Simple Tests using parallel-tests): Document
AM_TESTS_SETUP. Reorder some of the existing documentation a bit.
* tests/parallel-tests-am_tests_setup.test: New test.
* tests/depcomp8b.test: Add the `--enable-dependency-tracking'
option to the ./configure invocation, so that slower dependency
extractors are not rejects.
* tests/depcomp8b.test: Likewise. Also ...
(foo.c): ... since we are at it, fix spacing to be consistent
with GNU coding standards.
The following additional changes has been made:
* m4/amversion.in: Bump serial number from `1' to `8', to avoid
potential future conflicts with branch-1.11.
* m4/amversion.m4: Likewise.
* m4/gcj.m4: Likewise.
* m4/python.m4: Likewise.
* m4/install-sh.m4: Likewise.
* tests/self-check-cleanup.test: Be laxer when grepping output
from `ls -l', to account for ACLs and SELinux-only files.
* tests/self-check-dir.test: Source `defs-static' to read in the
correct definition for $SHELL.
* tests/self-check-me.test: Likewise, and extend a bit.
projects / thirdparty / automake.git / log
