* tests/seenc.test: Make grepping of automake stderr stricter.
Add a trailing `:' command.
* tests/symlink.test: Likewise. Also, define `$am_create_testdir'
to "empty" to avoid bringing in unused auxiliary files (only to
have to remove them right away), and use proper m4 quoting in
`configure.in'.
* tests/vpath.test: Make grepping of generated `Makefile.in'
slightly stricter. Prefer trailing `:' over trailing `Exit 0'.
* tests/suffix6c.test: Unset OBJEXT to avoid interferences from
the environment.
* tests/suffix12.test: Do not whitespace-indent `##' comments
when they are embedded in a makefile rule: having them indented
is not part of the Automake API, and might cause failures with
e.g., Tru64 make.
* tests/syntax.test: Simplify the `Makefile.am' to ensure that
automake doesn't fail for the wrong reasons. Make grepping of
automake stderr slightly stricter.
* tests/test-harness-vpath-rewrite.test: Remove useless variable
definition from `Makefile.am'.
* tests/test-driver-custom-multitest.test: Fix typo in comments.
* tests/self-check-me.tap: Fix minor typo in test description.
* tests/vars3.test: Make grepping of automake stderr stricter
and safer.
* tests/version6.test: Add sanity check, verifying that the
version number extracted from `automake --version' output seems
legit.
* tests/auxdir2.test: Renamed ...
* tests/auxdir-compauted.tap: ... to this, and converted to the
use of TAP.
* tests/auxdir4.test: Renamed ...
* tests/auxdir-unportable.tap: ... to this, and converted to the
use of TAP.
* tests/auxdir3.test: Renamed ...
* tests/auxdir-misplaced.test: ... to this.
* tests/auxdir5.test: Renamed ...
* tests/auxdir-nonexistent.test: ... to this.
* tests/auxdir9.test: Renamed ...
* tests/auxdir-autodetect.test: ... to this.
* tests/Makefile.am (TESTS): Update.
(XFAIL_TESTS): Remove `auxdir2.test'.
* test-protocols:
testsuite: fix some redundant autotools calls in tests on TAP
* THANKS (Daniel Richard G.): Update e-mail address.
java: avoid compilation errors when CLASSPATH is empty
parallel-tests: no more spurious successes for FreeBSD make
tap: improve granularity for tests on problematic TAP messages
tap: correctly handle string "0" in TAP messages
tap: a minor simplification in the perl TAP driver
parallel-tests: fix help screen for test driver scripts
tap: add a dummy TAP driver script implemented in shell + awk
testsuite: refactor tests on TAP support in view of future changes
fix: regenerate tests/Makefile.in
hacking: we now require autoconf 2.68
testsuite: fix some redundant autotools calls in tests on TAP
* tests/tap-deps.test: Do not call autotools and configure; there
is no need to, since this script sources `tap-setup.sh'.
* tests/tap-empty-diagnostic.test: Likewise.
* tests/tap-escape-directive.test: Likewise.
* tests/tap-fancy2.test: Likewise.
* tests/tap-no-spurious.test: Likewise.
* tests/tap-out-of-order.test: Likewise.
* master:
* THANKS (Daniel Richard G.): Update e-mail address.
java: avoid compilation errors when CLASSPATH is empty
parallel-tests: no more spurious successes for FreeBSD make
hacking: we now require autoconf 2.68
* maint:
* THANKS (Daniel Richard G.): Update e-mail address.
java: avoid compilation errors when CLASSPATH is empty
parallel-tests: no more spurious successes for FreeBSD make
hacking: we now require autoconf 2.68
Daniel Richard G [Tue, 16 Aug 2011 13:19:14 +0000 (15:19 +0200)]
java: avoid compilation errors when CLASSPATH is empty
* lib/am/java.am (CLASSPATH_ENV): When redefining `$CLASSPATH',
do not append an empty component in case the previous value of
CLASSPATH is empty or unset.
* tests/java-empty-classpath.test: New test.
* tests/Makefile.am (TESTS): Update.
parallel-tests: no more spurious successes for FreeBSD make
Work around a bug of FreeBSD make bug that was causing the
automake-generated "check" target to complete with success
even if some tests failed; this happened only when FreeBSD
make was run in concurrent mode (as in, e.g., "make -j2
check"). The bug is not present in NetBSD make.
This change fixes automake bug#9245:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9245>
See also FreeBSD PR bin/159730:
<http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>
* lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a
more "safe" (and apparently redundant) idiom to exit with error,
so that the non-zero exit status is picked up also by FreeBSD
make when it's running in concurrent mode.
* NEWS: Update.
* tests/check-concurrency-bug9245.test: New test.
* tests/Makefile.am (TESTS): Update.
tap: improve granularity for tests on problematic TAP messages
* tests/tap-message-0.test: Break up into ...
* tests/tap-msg0-result.test, tests/tap-msg0-directive.test,
tests/tap-msg0-planskip.test, tests/tap-msg0-misc.test,
tests/tap-msg0-bailout.test: ... these new tests, and extend
a little.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
* lib/tap-driver.pl (is_null_string): New function, can be used
to determine whether a given string variable is empty or undefined.
Useful to avoid pitfalls like:
if ($message) { print "$message\n"; }
which wouldn't print anything if $message is the literal "0".
(handle_tap_test, handle_tap_plan, handle_tap_bailout): Use it,
to avoid missing messages composed only by a literal "0" in TAP
result descriptions and in skip, todo and bailout explanations.
* tests/tap-message-0.test: Enhance.
* tests/Makefile.am (XFAIL_TESTS): Remove it, it passes now.
tap: add a dummy TAP driver script implemented in shell + awk
The user can also now decide which implementation of the TAP driver
to use in the testsuite by defining the `$am_tap_implementation'
variable to either "perl" or "shell". Future enhancements will
allow the testsuite to automatically run the test scripts on TAP
support with both the TAP driver implementations, to improve
coverage.
* tests/defs (fetch_tap_driver): Honor the `$am_tap_implementation'
variable to decide which implementation of the TAP driver to fetch.
($am_tap_implementation): Default to "perl".
* tests/tap-common-setup.test: Do not fetch the TAP driver, the
code in tap-setup.sh does that already (and respecting runtime
overriding of `$am_tap_implementation').
* lib/tap-driver: Renamed ...
* lib/tap-driver.pl: ... to this, and ...
($ME): ... adjusted this.
* doc/automake.texi: Adjust to the renaming.
* tests/Makefile.am (TAP_LOG_DRIVER): Likewise.
* tests/tap-doc2.test: Likewise.
* lib/tap-driver.sh: New script, still mostly dummy.
* lib/Makefile.am (dist_script_DATA): Update, and since we are at
it, rewrite it to make it easier to add new entries in the future.
testsuite: refactor tests on TAP support in view of future changes
* tests/defs (fetch_tap_driver): New subroutine; it fetches the
automake-provided TAP driver from the `lib/' directory into the
current directory, and edits its shebang line so that it will be
run with the perl interpreter determined at configure time.
* tests/tap-setup.sh: Use it.
* tests/tap-common-setup.test: There's no need to AC_SUBST `PERL'
anymore, nor to use it in the Makefile to run the TAP driver.
Also, use the `fetch_tap_driver' function instead of copying the
`tap-driver' auxiliary script directly.
* tests/tap-bad-prog.tap: Likewise.
* tests/tap-diagnostic-custom.test: Likewise.
* tests/tap-doc.test: Likewise.
* tests/tap-merge-stdout-stderr.test: Likewise.
* tests/tap-more.test: Likewise.
* tests/tap-more2.test: Likewise.
* tests/tap-recheck.test: Likewise.
* tests/tap-summary-aux.sh: Likewise.
* tests/tap-basic.test: Likewise, and fix a grammaro in comments
since we are at it.
* test-protocols:
coverage: missing tap plan and non-zero exit status
testsuite: fix an unportable use of sed
testsuite: avoid spurious failure if rst2html program is missing
fix: two recent ChangeLog entries
tap: improve diagnosing and reporting of plan mismatches
tap: log all TAP stream, even after a "Bail out!"
coverage: TAP diagnostics after "Bail out!" aren't reported
coverage: TAP errors after a "Bail out!" directive aren't reported
testsuite: reorganize tests on TAP plans
coverage: more TAP driver consistency w.r.t. 'prove'
tap: improve some comments in the TAP driver
tap: non-zero exit status after "Bail out!" should not be reported
tap: a "plan with SKIP" after test results is an error
test harness: avoid possible fork bomb
coverage: possible infinite recursion in the test harness
coverage: possible infinite recursion in the test harness
test defs: yet more environment cleanup
* tests/self-check-exit.tap: Disable the `errexit' shell flag;
this allow to remove some boilerplate idioms and to make some
code slightly less cumbersome.
* tests/self-check-me.tap: Likewise.
maintcheck: look for TAP-based tests without TAP plan
* Makefile.am (sc_tests_tap_plan): New maintainer check.
(syntax_check_rules): Add it.
* tests/suffix10.tap: Declare a TAP plan.
* tests/add-missing.tap: Likewise.
testsuite: avoid confusing SKIP messages for errored tests
* tests/defs [$use_tap] (exit trap): Don't print the test plan
in case the client script exited with a non-zero status. This
avoids the display of redundant "SKIP" messages for tests with
lazy plans that error out before any test is run.
testsuite: keep more debugging info around in 'add-missing.tap'
* tests/add-missing.tap: Use a different directory for each test,
so that in case of failure more information is kept around.
Since we are at it, remove redundant TAP diagnostic, and define
`$am_create_testdir' to "empty" to avoid bringing in unused
auxiliary files.
testsuite: more granular count of test results in our TAP library
* tests/tap-functions.sh ($tap_bad_count_): Removed, superseded
by ...
($tap_fail_count_, $tap_xpass_count_): ... these new variables,
which keep more granular counts.
($tap_pass_count_: New variable.
* tests/defs (exit trap): Adjust and simplify accordingly.
testsuite: fixlets and improvements in two long TAP-based tests
* tests/depmod.tap: Clean up the subdirectories used by tests that
passed, to avoid ending up with a too big test directory. This is
especially important since, on each and every system, at least some
tests in this script are expected to be skipped (which ones exactly
depends on the system), thus causing the test directory not to be
removed when the script terminates.
* tests/instspc.tap: Likewise (with the difference that tests here
are not expected to be skipped, but to xfail). Give more explicit
error message in case of setup failure. Skip the proper number of
tests when required, to avoid spurious errors due to mismatched
plan. Also, really avoid to run the unsupported test cases when
the need to skip them is detected.
tap: improve diagnosing and reporting of plan mismatches
Problem exposed by a failure in the test 'tap-plan-errors.test'.
* lib/tap-driver (NO_PLAN, EARLY_PLAN, LATE_PLAN): New constants.
(Throughout the file): Use more complex semantics for `$plan_seen',
which now also remember whether the plan that has been seen is an
"early" or "late" plan; in turn, this makes ...
($tap_stopped): ... this variable redundant; remove it.
(handle_tap_plan): Adjust to avoid producing spurious or confusing
error messages about extra or mismatched "late" TAP plans.
* tests/tap-plan-corned.test: Adjust and extend.
* tests/tap-color.test: Likewise.
* lib/tap-driver ($bailed_out): New global boolean variable,
telling whether a "Bail out!" directive has been seen or not.
(handle_tap_bailout): This function does not anymore stop the
reading from TAP stream; instead, it sets `$bailed_out' to a
true value, so that only the subsequent parsing of the input
TAP stream is stopped.
(finish): Remove, no more needed, its contents inlined into ...
(main): ... this function, with related adjustments in the code
flow.
(get_test_exit_message): Do not "flush" the input TAP stream
to fetch the exit status of test script, it is not anymore
required. Add a sanity check.
* tests/tap-bailout-and-logging.test: New test.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
* tests/tap-skip-whole.test: Rename ...
* tests/tap-planskip.test: ... to this
* tests/tap-skip-whole-badexit.test: Rename ...
* tests/tap-planskip-badexit.test: ... to this.
* tests/tap-skip-whole-bailout.test: Rename ...
* tests/tap-planskip-bailout.test: ... to this.
* tests/tap-skip-whole-bailout.test: Rename ...
* tests/tap-planskip-later-errors.test: ... to this.
* tests/tap-skip-whole-unplanned.test: Rename ...
* tests/tap-planskip-unplanned.test: ... to this.
* tests/tap-skip-whole-whitespace.test: Rename ...
* tests/tap-planskip-whitespace.test: ... to this.
* tests/tap-skip-whole-continue.test: Rename ...
* tests/tap-planskip-and-logging.test: ... to this, extend to try
more cases, and fix heading comments.
* tests/tap-skip-whole-lastline.test: Rename ...
* tests/tap-planskip-late.test: ... to this, fix heading comments,
and remove some checks that are just a duplication of those already
performed in the previous script.
* tests/tap-results-with-planskip.test: Rename to ...
* tests/tap-planskip-unplanned-corner.test: ... this, and improve
hading comments.
* tests/tap-planskip-later-errors.test: New test.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
coverage: more TAP driver consistency w.r.t. 'prove'
* tests/skip-results-with-planskip.test: New test, verifies that
test results seen in a TAP stream that has a "plan with SKIP" are
flagged as errors.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
tap: non-zero exit status after "Bail out!" should not be reported
This new behaviour is more consistent with that of the `prove'
utility. Also, experience with the use of TAP in the Automake's
own test suite has shown that reporting non-zero exit status from
a script after it has issued a "Bail out!" directive is mostly
redundant, more introducing noise rather than conveying useful
information.
* lib/tap-driver (finish): Inline the part of it that was getting
the script exit status ...
(main): ... in here.
* tests/tap-bailout-badexit.test: New test.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
tap: a "plan with SKIP" after test results is an error
This new behaviour is both more natural and more consistent with
that of the `prove' utility.
* lib/tap-driver (handle_tap_plan): Do not stop TAP parsing when
a "plan with SKIP" line (e.g., "1..0 # SKIP") is encountered, and
do not print a "SKIP" line if some TAP result has already been
seen.
* tests/tap-skip-whole.test: Adapt.
* tests/tap-skip-whole-lastline.test: Likewise.
* tests/tap-global-result.test: Adapt and extend.
* tests/tap-skip-plan-errors.test: Likewise, and fix an obsolete
small part of a comment.
* tests/tap-skip-whole-bailout.test: New test.
* tests/tap-skip-whole-unplanned.test: Likewise.
* tests/tap-skip-whole-continue.test: Likewise.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
This fixes a regression w.r.t. the master branch, exposed by
test 'parallel-tests-fork-bomb.test'.
* lib/am/check.am (am--redo-logs): Detect possible infinite
recursion due to a test log in $(TEST_LOGS) being the same
as $(TEST_SUITE_LOG).
* tests/parallel-tests-fork-bomb.test: Enhance and extend a
little now that we have more explicit semantics.
coverage: possible infinite recursion in the test harness
Motivated by a regression in the 'test-protocols' branch.
* tests/parallel-tests-fork-bomb.test: New test, checking that
if $(TEST_SUITE_LOG) is in $(TEST_LOGS), we obtain a diagnosed
error rather than a make hang or a fork bomb.
* tests/Makefile.am (TESTS): Update.
coverage: possible infinite recursion in the test harness
Motivated by a regression in the 'test-protocols' branch.
* tests/parallel-tests-fork-bomb.test: New test, checking that
if $(TEST_SUITE_LOG) is in $(TEST_LOGS), we obtain a diagnosed
error rather than a make hang or a fork bomb.
* tests/Makefile.am (TESTS): Update.
tests: fix various blunders in 'suffix-chain.test'
* tests/suffix-chain.test: Fix various blunders that were causing
this test to fail spuriously: append to `configure.in', not to
`configure.ac'; add a `;' character after the `return' statement
in an input file; use tabs, not spaces, to indent Makefile recipes;
and use more uncommon suffixes that don't trigger built-in recipes
with FreeBSD make.
testsuite: improvements to our shell TAP library and APIs
These improvements impact only the Automake's own testsuite.
* tests/tap-functions.sh (plan_): Allow new special arguments
"unknown" a.k.a. "later" and "lazy" a.k.a. "now". Perform
sanity check the given arguments. Adjust incorrect comments.
(lazy_plan_): Removed, superseded by the new version of `plan_'.
(not): New function, run a command and inverts its exit status.
(ok_, not_ok_, skip_): These are just shorthands for common
`result_' usages.
(xfail_, xpass_, reset_test_count_): Removed, they are not used,
nor does a legitimate use for them seem in sight.
(not): New function, to run a command inverting its exit status.
(command_not_ok_): Removed, superseded by `command_ok_' + `not'.
(command_ok_): Extended to accept TAP directives.
* tests/defs (trap): Don't use `late_plan_' anymore, use
`plan_ "now"' instead.
* tests/ac-output-old.tap: Update to take advantage of the new
APIs.
* tests/self-check-env-sanitize.tap: Likewise.
* tests/suffix-chain.tap: Likewise.
* tests/suffix10.tap: Likewise.
* tests/suffix11.tap: Likewise.
* tests/suffix3.tap: Likewise.
* tests/suffix8.tap: Likewise.
* tests/tap-bad-prog.tap: Likewise.
* tests/self-check-exit.tap: Likewise, plus a small improvement
in the name of a variable.
This will allow us to explicitly declare some expected failures
involving Solaris make as such, instead of partly papering over
them with skips and partly letting them fail too loudly.
* tests/suffix-chain.test: Rename ...
* tests/suffix-chain.tap: ... to this, and convert to the use of
TAP. Also, fix some bugs since we are at it, and ...
($required): ... require `cc'.
* tests/suffix3.test: Rename ...
* tests/suffix3.tap: ... to this, and convert to the use of TAP.
Also, since we are at it, throw in some formatting fixes.
* tests/suffix8.test: Rename ...
* tests/suffix8.tap: ... to this, and convert to the use of TAP.
Also, since we are at it, throw in some formatting fixes.
* tests/suffix10.test: Rename ...
* tests/suffix10.tap: ... to this, and convert to the use of TAP.
* tests/suffix11.test: Rename ...
* tests/suffix11.tap: ... to this, and convert to the use of TAP.
* tests/Makefile.am (TESTS): Update.
tests: don't let a known Solaris make bug poison too many tests
See automake bug#7670 and bug#7824.
* tests/defs (make_can_chain_suffix_rules): New function, tell
whether the make implementation in use can chain suffix rules
automatically or not.
* tests/suffix3.test: Use it to avoid issuing calls to make that
are unportable to make implementations that are not smart enough
to chain suffix rules automatically.
* tests/suffix8.test: Use it to avoid issuing calls to make that
* tests/suffix10.test: Use it to avoid issuing calls to make that
* tests/suffix11.test: Use it to avoid issuing calls to make that
* tests/suffix-chain.test: New test, exposes the limitation that
we have papered over in the tests above.
* .gitignore: Anchor files that are intended to be ignored only
if found in the same directory of the `.gitignore' file, not also
in its subdirectories.
* doc/.gitignore, doc/amhello/.gitignore, lib/Automake/.gitignore,
lib/Automake/tests/.gitignore, tests/.gitignore: Likewise. Also,
where needed, add new entries that were once implied by the
non-anchored entries in the upper-level `.gitignore' files.
testsuite: avoid spurious failure of self checks on Solaris
This is just a quick workaround to prevent having too much noise
in the testsuite; more proper fixes have been committed into the
'testsuite-work' branch.
* tests/self-check-dir.test: Relax a test to avoid unimportant
failure on Solaris 10.
* tests/self-check-exit.test: Likewise.
self tests: avoid spurious output on console, and related fixlets
* tests/self-check-cleanup.test: Set `$stderr_fileno_' to `2'
in the self tests, to avoid spurious diagnostic on console.
* tests/self-check-exit.tap: Only unset `am_explicit_skips' and
`$stderr_fileno_' in the self tests, not at the global level, it
is more correct.
($init): New variable, to reduce code duplication.
Some refactoring and adjustments to the name of the tests.
* tests/defs: Catch some more (very unlikely) set-up or internal
errors, and diagnose them explicitly. This is intended mostly
to be a theoretic-only improvement.
Motivated by a spurious failure of test `instsh3-w.shtst' on
Solaris 10.
* tests/defs: Make really "set -e" clean, to avoid spurious
failures in case this file is sourced by a test script that has
already set the `errexit' flag. To be sure we don't regress,
move the `set -e' call much earlier, and make the comment telling
about the expected "set -e" cleanliness more loud and outstanding.
self tests: register an expected failures with Solaris /bin/sh
Solaris 10 /bin/sh erroneously exit with success right away when
the following three conditions are met at the same time:
1. the `errexit' flag is active,
2. an exit trap is installed, and
3. a non-existing command is issued.
* tests/self-exit.tap: When that bug is detected, issue an XFAIL
rather than a FAIL. Since we are at it, improve by avoiding
creation of stray temporary files when testing for non-executable
commands.
gitignore: update and improve for the testsuite files
* tests/.gitignore: Don't ignore temporary partial log files,
autogenerated tests and makefile fragments that are not used
anymore. Anchor all the remaining patterns.
* tests/self-check-me.test: Renamed ...
* tests/self-check-me.tap: .. to this, and converted to use TAP.
* tests/self-check-exit.test: Renamed ...
* tests/self-check-exit.tap: .. to this, and converted to use TAP.
Set `$stderr_fileno_' to `2', to avoid spurious messages on the
console. Add a new test, and relax another one.
* tests/self-check-env-sanitize.test: Renamed ...
* tests/self-check-env-sanitize.tap: .. to this, and converted to
the use TAP.
* tests/Makefile.am (TESTS): Update.
test defs: test directory creation more configurable
After this change, the individual tests will be able to tell
`./defs' whether or not to create a temporary test directory
for them, and if yes, whether or not to pre-populate it.
* tests/defs: Only create the temporary test directory if the
variable `$am_create_testdir' is not set to "no". In this
case, only pre-populate it if that variable is not set to
`$empty'.
* tests/defs-static.in: Check that `$am_create_testdir' is not
exported in the environment.
* tests/self-check-env-sanitize.test: Update.
* tests/self-check-dir.test: Extend, and make deeper and more
robust.
* tests/README (Section "Writing test cases" subsection "Do"):
Update.
* tests/autodist.test: Set `$am_create_testdir' instead of
removing by hand files created by ./defs.
* tests/tap-doc2.test: Likewise.
* tests/parallel-tests-exit-statuses.test: Use the `seq_' function
from `tests/defs', instead of duplicating its semantics by hand.
* tests/testsuite-summary-count.test: Likewise.
testsuite: drop redundant environment sanitization
* tests/Makefile.am (TESTS_ENVIRONMENT): Don't remove `planned'
from the environment.
* tests/defs-static.in: Don't complain if `planned' is set in
the environment.
* tests/self-check-env-sanitize.test: Update.
* tests/defs: Where possible, prefer `framework_failure_' or
`fatal_' to `Exit 99'. Try to catch more possible hard errors
and setup failures. Do not hide expected error messages. Avoid
a couple of forks when copying files from `$(top_srcdir)/lib'.
($signal): Remove, no more needed.
testsuite: improve interface for giving early TAP plan
Since commit `v1.11-976-g4740f56', an early TAP plan can also be
specified after input lines have already been seen, as long as
no one of them is a TAP result line. We can now take advantage
of this in our own testsuite, allowing the TAP-based scripts to
specify their earlier plan after having included `./defs'. In
particular, this offers the benefit of allowing those scripts to
issue a "skip all" plan (e.g., "1..0 # SKIP reason ...") after
the inclusion of `./defs', which wasn't possible with the older
interface (introduced in commit `v1.11-1114-g0286308').
* tests/defs: Don't try to call `plan_' if `$planned' is defined,
the calling scripts can do that directly now.
* tests/ac-output-old.tap: Use `plan_' directly instead of
setting `$planned' before including `./defs'.
* tests/automake-cmdline.tap: Likewise.
* tests/depmod.tap: Likewise.
* tests/instspc.tap: Likewise.
* tests/tap-bad-prog.tap: Likewise.
* tests/add-missing.tap: Don't set `$planned'.
Merge branch 'test-protocols' into tap-testsuite-work
* test-protocols:
test defs: more environment cleanup
test harness: more minor VPATH-rewrite issues
test harness: "make -n check" works correctly with Solaris make
* tests/am/check.am (am__set_TESTS_bases, $(TEST_SUITE_LOGS),
am--redo-logs, recheck, recheck-html): Cosmetic fixlets to
minimize the risk of unwanted VPATH rewrites.
(check-TESTS): Likewise, and normalize trailing whitespace
since we are at it.
Bugs exposed by test cases `check6-p.test' and `check7-p.test'.
test harness: "make -n check" works correctly with Solaris make
To avoid some recipes being executed with "make -n", we used the
trick of issuing recursive make calls not directly with $(MAKE),
but with a simple indirection instead:
...
am__stealth_MAKE = $(MAKE)
...
target:
... $(am__stealth_MAKE) ...
...
This hack worked with GNU make, but, unfortunately, not with
Solaris make. The present change removes the hack, and uses more
proper idioms to avoid unwanted recipe executions with "make -n".
Bug exposed by test case `parallel-tests-dry-run.test'.
* lib/am/check.am (am__stealth_MAKE): Remove, no more used.
(.log.trs): Break recipe in multiple shell invocations.
($(TEST_SUITE_LOG)): Likewise, and move some corner-case parts of
the recipe to ...
(am--redo-logs): ... this new internal helper .PHONY target.
* tests/distcheck-configure-flags-am.test: Avoid using `+=' too
liberally with AM_DISTCHECK_CONFIGURE_FLAGS, since the line breaks
so introduced, in conjunction with single quotes, might confuse
Solaris make.
* lib/am/check.am (am__set_TESTS_bases): Add forgotten end-of-line
anchor to sed expression. In comments, add reference to ...
* tests/test-trs-basic.test: ... this test, which has been adapted
and tightened (and tweaked).
Merge branch 'test-protocols' into tap-testsuite-work
* test-protocols:
fix: regenerate Makefiles
test harness: work around a VPATH rewrite issue
parallel-tests: work around Solaris XPG4 make segfault
testsuite: fix weird spurious failure with Solaris /bin/sh
testsuite: fix another spurious failure on Solaris make
testsuite: fix two spurious failures on Solaris make
tap driver: refactor and remove dead code
testsuite: fix spurious failure in a test on TAP support
tap: plan location is more liberal w.r.t. non-TAP lines
testsuite: remove now-passing test from XFAIL_TESTS
* lib/am/check.am (am__TEST_BASES): Removed, it's role taken
over by ...
(am__set_TESTS_bases): ... these new variable.
($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues.
* automake.in (handle_tests): Update the code for the cleanup
of the `.trs' file to use `$(TEST_LOGS)' instead of
`$(am__TEST_BASES)'.
* tests/test-harness-vpath-rewrite.test: New test, better
exposing the problem fixed by this change
* tests/Makefile.am (TESTS): Add it.
* tests/test-trs-basic.test: Update and extend.
parallel-tests: work around Solaris XPG4 make segfault
Issue exposed by test `posixsubst-tests-p.test', and similar to
the problem solved by commit `v1.11-159-ge7aa360'.
* lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing
whitespace from $list, to avoid triggering a nasty bug (potential
segfault) on Solaris XPG4 make and Heirloom make.
testsuite: fix weird spurious failure with Solaris /bin/sh
Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can
apparently end up exiting with exit status 208, instead of leaving
the correct wide exit status to the parent. See:
<http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
Work around this incompatibility.
* tap-signal.test: Look for the above Solaris bug.
(signal_caught): Adapt to handle it.
testsuite: fix another spurious failure on Solaris make
* tests/parallel-tests-log-override-recheck.test: Filter make
output before grepping it, for make implementations that, like
Solaris' one, print the whole of the failed recipe on failure.
testsuite: fix two spurious failures on Solaris make
* tests/tests-environment-and-log-compiler.test: Do not fail if
"make -n check" fails, as that is issued only for debugging, its
semantics being tested in other test scripts.
* tests/test-driver-strip-vpath.test: Likewise.
* lib/tap-driver (get_tap_line): Removed, ...
(all callers): ... updated to use $parser->next directly instead.
(peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not
used anymore.
(TAP_PEEKING): Block deleted, all its subroutines and variables
having been removed.
testsuite: fix spurious failure in a test on TAP support
* tests/tap-passthrough-exit.test: When the `--ignore-exit' flag
of the TAP driver is used, don't look for a message reporting the
non-zero exit statuses of tests in the log files; such message is
not expected to be there anymore. Related simplifications.
tap: plan location is more liberal w.r.t. non-TAP lines
With this change, only lines that are TAP results will matter
w.r.t. the position of the TAP plan in the input; for example,
this input:
this is a non-TAP line
# and this a TAP diagnostic line
1..1
ok 1
was considered to be an error, diagnosed with a message "test
plan in middle of output"; as effect of the current change, such
input is now valid. This is more consistent with the behaviour
of the `prove' utility.
* lib/tap-driver ($lineno): Removed, no more needed.
($tap_stopped): New global variable.
(stringify_test_result): Return "ERROR" if a TAP result is found
when `$tap_stopped' is set to true.
(handle_tap_test): Diagnose TAP results that comes after a late
plan. Add a couple of blank lines, for clarity.
(handle_tap_plan): Set `$tap_stopped' to true after a late plan
is encountered. Do not complain anymore for extra non-TAP lines
preceding or following the plan. Adjust comments.
(main): Don't increment $lineno anymore.
* tests/tap-plan.test: Extend a bit, and remove stale comment.
* tests/tap-color.test: Adjust.
* tests/tap-passthrough.test: Likewise.
* tests/tap-plan-corner.test: Adjust and extend.
* tests/tap-plan-errors.test: Likewise.
* tests/tap-plan-middle.test: New test.
* tests/tap-plan-corner2.test: Delete, it's obsolete now.
* tests/Makefile.am (XFAIL_TESTS): Remove it.
(tap_with_common_setup_tests): Likewise, and add
`tap-plan-corner.test'.
This change fixes the names of the temporary directories used
by the TAP-based tests (e.g., test `foo.tap' now uses `foo.dir'
rather than `foo.tap.dir'); more importantly, this make the
variable `$destdir' defined in `tests/defs' usable also in the
TAP tests.
($me): Also strip the any `.tap' suffix from the name of the
test.
testsuite: TAP tests properly decide when to remove tempdirs
Before this change, the TAP tests in the Automake testsuite were
removing the temporary test directory even when they failed or
were skipped, thus making debugging more difficult.
* tests/tap-functions.sh (incr_tap_count): Removed, superseded
by ...
(incr_): ... this function, which can increment the value of any
variable passed to it.
(result_): Updated to use `incr_' instead of the now-removed
`incr_tap_count_'. Keep count of failures, xfailures, xpasses,
and skips, using ...
($tap_skip_count_, $tap_bad_count, _$tap_xfail_count_): ...
these new variables.
* tests/defs (trap): Try to use their values to decide whether
the temporary directory being used by the test script should be
removed or not. Other code reorganizations. And move the code
for the removal of the temporary directory out to ...
(rm_rf_): ... this new subroutine.
(Main code): Use that instead of duplicating the code.
testsuite: improve and refactor our custom TAP shell library
This change is mostly done in preparation for the next one.
* tests/tap-functions.sh (result_with_directive_): Merge into ...
(result_): ... this function, which has been greatly extended and
improved in various ways.
(ok_, not_ok_, skip_, xfail_, xpass_): Adjust.
Adjust comments. Remove an extra empty line.
* tests/instspc.tap: Use the new improved interface of `result_'.
* tests/tap-bad-prog.tap: Likewise.
projects / thirdparty / automake.git / log
