From: Stefano Lattarini Date: Thu, 18 Aug 2011 19:53:34 +0000 (+0200) Subject: Merge branch 'master' into test-protocols X-Git-Tag: ng-0.5a~89^2~99^2~1 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=94aa58693d1a6fd09db715524e1785d062f1a252;p=thirdparty%2Fautomake.git Merge branch 'master' into test-protocols * master: * THANKS (Daniel Richard G.): Update e-mail address. java: avoid compilation errors when CLASSPATH is empty parallel-tests: no more spurious successes for FreeBSD make hacking: we now require autoconf 2.68 --- 94aa58693d1a6fd09db715524e1785d062f1a252 diff --cc ChangeLog index 4d83efbca,ea42a25c5..bbcedb670 --- a/ChangeLog +++ b/ChangeLog @@@ -1,251 -1,44 +1,292 @@@ + 2011-08-17 Stefano Lattarini + + * THANKS (Daniel Richard G.): Update e-mail address. + + 2011-08-16 Daniel Richard G. (tiny change) + Stefano Lattarini + + java: avoid compilation errors when CLASSPATH is empty + * lib/am/java.am (CLASSPATH_ENV): When redefining `$CLASSPATH', + do not append an empty component in case the previous value of + CLASSPATH is empty or unset. + * tests/java-empty-classpath.test: New test. + * tests/Makefile.am (TESTS): Update. + Fixes automake bug#9306. + + 2011-08-16 Stefano Lattarini + + parallel-tests: no more spurious successes for FreeBSD make + Work around a bug of FreeBSD make bug that was causing the + automake-generated "check" target to complete with success + even if some tests failed; this happened only when FreeBSD + make was run in concurrent mode (as in, e.g., "make -j2 + check"). The bug is not present in NetBSD make. + This change fixes automake bug#9245: + + See also FreeBSD PR bin/159730: + + * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a + more "safe" (and apparently redundant) idiom to exit with error, + so that the non-zero exit status is picked up also by FreeBSD + make when it's running in concurrent mode. + * NEWS: Update. + * tests/check-concurrency-bug9245.test: New test. + * tests/Makefile.am (TESTS): Update. + + 2011-08-11 Stefano Lattarini + + hacking: we now require autoconf 2.68 + * HACKING ("Working in git"): Use autoconf and autom4te 2.68, + not 2.67, in the examples. + +2011-08-17 Stefano Lattarini + + tap: improve granularity for tests on problematic TAP messages + * tests/tap-message-0.test: Break up into ... + * tests/tap-msg0-result.test, tests/tap-msg0-directive.test, + tests/tap-msg0-planskip.test, tests/tap-msg0-misc.test, + tests/tap-msg0-bailout.test: ... these new tests, and extend + a little. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-17 Stefano Lattarini + + tap: correctly handle string "0" in TAP messages + * lib/tap-driver.pl (is_null_string): New function, can be used + to determine whether a given string variable is empty or undefined. + Useful to avoid pitfalls like: + if ($message) { print "$message\n"; } + which wouldn't print anything if $message is the literal "0". + (handle_tap_test, handle_tap_plan, handle_tap_bailout): Use it, + to avoid missing messages composed only by a literal "0" in TAP + result descriptions and in skip, todo and bailout explanations. + * tests/tap-message-0.test: Enhance. + * tests/Makefile.am (XFAIL_TESTS): Remove it, it passes now. + +2011-08-17 Stefano Lattarini + + tap: a minor simplification in the perl TAP driver + * lib/tap-driver.pl: The `--disable-hard-errors' option is a + no-op, so just ignore it and its argument. + +2011-08-17 Stefano Lattarini + + parallel-tests: fix help screen for test driver scripts + * lib/tap-driver.pl ($USAGE): The `--trs-file' option is mandatory + too. + * lib/tap-driver.sh (print_usage): Likewise. + * lib/test-driver (print_usage): Likewise. + ($scriptversion): Update. + +2011-08-17 Stefano Lattarini + + tap: add a dummy TAP driver script implemented in shell + awk + The user can also now decide which implementation of the TAP driver + to use in the testsuite by defining the `$am_tap_implementation' + variable to either "perl" or "shell". Future enhancements will + allow the testsuite to automatically run the test scripts on TAP + support with both the TAP driver implementations, to improve + coverage. + * tests/defs (fetch_tap_driver): Honor the `$am_tap_implementation' + variable to decide which implementation of the TAP driver to fetch. + ($am_tap_implementation): Default to "perl". + * tests/tap-common-setup.test: Do not fetch the TAP driver, the + code in tap-setup.sh does that already (and respecting runtime + overriding of `$am_tap_implementation'). + * lib/tap-driver: Renamed ... + * lib/tap-driver.pl: ... to this, and ... + ($ME): ... adjusted this. + * doc/automake.texi: Adjust to the renaming. + * tests/Makefile.am (TAP_LOG_DRIVER): Likewise. + * tests/tap-doc2.test: Likewise. + * lib/tap-driver.sh: New script, still mostly dummy. + * lib/Makefile.am (dist_script_DATA): Update, and since we are at + it, rewrite it to make it easier to add new entries in the future. + +2011-08-17 Stefano Lattarini + + testsuite: refactor tests on TAP support in view of future changes + * tests/defs (fetch_tap_driver): New subroutine; it fetches the + automake-provided TAP driver from the `lib/' directory into the + current directory, and edits its shebang line so that it will be + run with the perl interpreter determined at configure time. + * tests/tap-setup.sh: Use it. + * tests/tap-common-setup.test: There's no need to AC_SUBST `PERL' + anymore, nor to use it in the Makefile to run the TAP driver. + Also, use the `fetch_tap_driver' function instead of copying the + `tap-driver' auxiliary script directly. + * tests/tap-bad-prog.tap: Likewise. + * tests/tap-diagnostic-custom.test: Likewise. + * tests/tap-doc.test: Likewise. + * tests/tap-merge-stdout-stderr.test: Likewise. + * tests/tap-more.test: Likewise. + * tests/tap-more2.test: Likewise. + * tests/tap-recheck.test: Likewise. + * tests/tap-summary-aux.sh: Likewise. + * tests/tap-basic.test: Likewise, and fix a grammaro in comments + since we are at it. + +2011-08-14 Stefano Lattarini + + coverage: missing tap plan and non-zero exit status + * tests/tap-missing-plan-and-bad-exit.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + testsuite: fix an unportable use of sed + * tests/tap-todo-skip-whitespace.test: Alternation with "\|" in + sed regular expressions is not portable to (at least) FreeBSD + sed; do not use it. + +2011-08-12 Stefano Lattarini + + testsuite: avoid spurious failure if rst2html program is missing + * tests/test-driver-custom-no-html.test: Account for the + possibility of a lacking `rst2html' program in our final + grepping of make output. + +2011-08-12 Stefano Lattarini + + tap: improve diagnosing and reporting of plan mismatches + Problem exposed by a failure in the test 'tap-plan-errors.test'. + * lib/tap-driver (NO_PLAN, EARLY_PLAN, LATE_PLAN): New constants. + (Throughout the file): Use more complex semantics for `$plan_seen', + which now also remember whether the plan that has been seen is an + "early" or "late" plan; in turn, this makes ... + ($tap_stopped): ... this variable redundant; remove it. + (handle_tap_plan): Adjust to avoid producing spurious or confusing + error messages about extra or mismatched "late" TAP plans. + * tests/tap-plan-corned.test: Adjust and extend. + * tests/tap-color.test: Likewise. + +2011-08-12 Stefano Lattarini + + tap: log all TAP stream, even after a "Bail out!" + * lib/tap-driver ($bailed_out): New global boolean variable, + telling whether a "Bail out!" directive has been seen or not. + (handle_tap_bailout): This function does not anymore stop the + reading from TAP stream; instead, it sets `$bailed_out' to a + true value, so that only the subsequent parsing of the input + TAP stream is stopped. + (finish): Remove, no more needed, its contents inlined into ... + (main): ... this function, with related adjustments in the code + flow. + (get_test_exit_message): Do not "flush" the input TAP stream + to fetch the exit status of test script, it is not anymore + required. Add a sanity check. + * tests/tap-bailout-and-logging.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + coverage: TAP diagnostics after "Bail out!" aren't reported + This is compatible with the behaviour of the `prove' utility. + * tests/tap-bailout-suppress-later-diagnostic.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + coverage: TAP errors after a "Bail out!" directive aren't reported + This is compatible with the behaviour of the `prove' utility. + * tests/tap-bailout-suppress-later-errors.test: New test. + * tests/tap-bailout-badexit.test: Renamed ... + * tests/tap-bailout-suppress-badexit.test: ... to this, for + consistency. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + testsuite: reorganize tests on TAP plans + * tests/tap-skip-whole.test: Rename ... + * tests/tap-planskip.test: ... to this + * tests/tap-skip-whole-badexit.test: Rename ... + * tests/tap-planskip-badexit.test: ... to this. + * tests/tap-skip-whole-bailout.test: Rename ... + * tests/tap-planskip-bailout.test: ... to this. + * tests/tap-skip-whole-bailout.test: Rename ... + * tests/tap-planskip-later-errors.test: ... to this. + * tests/tap-skip-whole-unplanned.test: Rename ... + * tests/tap-planskip-unplanned.test: ... to this. + * tests/tap-skip-whole-whitespace.test: Rename ... + * tests/tap-planskip-whitespace.test: ... to this. + * tests/tap-skip-whole-continue.test: Rename ... + * tests/tap-planskip-and-logging.test: ... to this, extend to try + more cases, and fix heading comments. + * tests/tap-skip-whole-lastline.test: Rename ... + * tests/tap-planskip-late.test: ... to this, fix heading comments, + and remove some checks that are just a duplication of those already + performed in the previous script. + * tests/tap-results-with-planskip.test: Rename to ... + * tests/tap-planskip-unplanned-corner.test: ... this, and improve + hading comments. + * tests/tap-planskip-later-errors.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + coverage: more TAP driver consistency w.r.t. 'prove' + * tests/skip-results-with-planskip.test: New test, verifies that + test results seen in a TAP stream that has a "plan with SKIP" are + flagged as errors. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-12 Stefano Lattarini + + tap: improve some comments in the TAP driver + * lib/tap-driver (handle_tap_plan): Improve comments describing + possible errors in a "plan with SKIP" directive. + +2011-08-12 Stefano Lattarini + + tap: non-zero exit status after "Bail out!" should not be reported + This new behaviour is more consistent with that of the `prove' + utility. Also, experience with the use of TAP in the Automake's + own test suite has shown that reporting non-zero exit status from + a script after it has issued a "Bail out!" directive is mostly + redundant, more introducing noise rather than conveying useful + information. + * lib/tap-driver (finish): Inline the part of it that was getting + the script exit status ... + (main): ... in here. + * tests/tap-bailout-badexit.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-11 Stefano Lattarini + + tap: a "plan with SKIP" after test results is an error + This new behaviour is both more natural and more consistent with + that of the `prove' utility. + * lib/tap-driver (handle_tap_plan): Do not stop TAP parsing when + a "plan with SKIP" line (e.g., "1..0 # SKIP") is encountered, and + do not print a "SKIP" line if some TAP result has already been + seen. + * tests/tap-skip-whole.test: Adapt. + * tests/tap-skip-whole-lastline.test: Likewise. + * tests/tap-global-result.test: Adapt and extend. + * tests/tap-skip-plan-errors.test: Likewise, and fix an obsolete + small part of a comment. + * tests/tap-skip-whole-bailout.test: New test. + * tests/tap-skip-whole-unplanned.test: Likewise. + * tests/tap-skip-whole-continue.test: Likewise. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + +2011-08-09 Stefano Lattarini + + test harness: avoid possible fork bomb + This fixes a regression w.r.t. the master branch, exposed by + test 'parallel-tests-fork-bomb.test'. + * lib/am/check.am (am--redo-logs): Detect possible infinite + recursion due to a test log in $(TEST_LOGS) being the same + as $(TEST_SUITE_LOG). + +2011-08-09 Stefano Lattarini + + coverage: enhance test against infinite recursion in test harness + * tests/parallel-tests-fork-bomb.test: Enhance and extend a + little now, for soon-to-be-implemented more explicit and safer + semantics. + 2011-08-09 Stefano Lattarini coverage: possible infinite recursion in the test harness diff --cc lib/Automake/tests/Makefile.in index aca691c99,967a95286..6c1599ee3 --- a/lib/Automake/tests/Makefile.in +++ b/lib/Automake/tests/Makefile.in @@@ -455,21 -429,9 +455,21 @@@ $(TEST_SUITE_LOG): $(TEST_LOGS col="$$grn"; \ else \ col="$$red"; \ + test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \ + fi; \ + echo "$${col}$$br$${std}"; \ + echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \ + echo "$${col}$$br$${std}"; \ + create_testsuite_report --maybe-color; \ + echo "$$col$$br$$std"; \ + if $$success; then :; else \ + echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \ + if test -n "$(PACKAGE_BUGREPORT)"; then \ + echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \ + fi; \ + echo "$$col$$br$$std"; \ fi; \ - $$success - echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ - $$exit || exit 1 ++ $$success || exit 1 # Run all the tests. check-TESTS: diff --cc lib/am/check.am index 27388df02,a65d1fe79..0628c8661 --- a/lib/am/check.am +++ b/lib/am/check.am @@@ -278,25 -225,14 +278,30 @@@ $(TEST_SUITE_LOG): $(TEST_LOGS col="$$grn"; \ else \ col="$$red"; \ + test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \ + fi; \ +## Multi line coloring is problematic with "less -R", so we really need +## to color each line individually. + echo "$${col}$$br$${std}"; \ + echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \ + echo "$${col}$$br$${std}"; \ +## This is expected to go to the console, so it might have to be colorized. + create_testsuite_report --maybe-color; \ + echo "$$col$$br$$std"; \ + if $$success; then :; else \ + echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \ + if test -n "$(PACKAGE_BUGREPORT)"; then \ + echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \ + fi; \ + echo "$$col$$br$$std"; \ fi; \ - ## Be sure to exit with the proper exit status. - $$success - echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ -## The use of "exit 1" below is required to work around a FreeBSD make bug -## (present only when running in concurrent mode). See automake bug#9245: ++## Be sure to exit with the proper exit status. The use of "exit 1" below ++## is required to work around a FreeBSD make bug (present only when running ++## in concurrent mode). See automake bug#9245: + ## + ## and FreeBSD PR bin/159730: + ## . - $$exit || exit 1 ++ $$success || exit 1 RECHECK_LOGS = $(TEST_LOGS) diff --cc tests/Makefile.am index 853831d7f,850240974..2d630a9c3 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@@ -237,9 -237,10 +237,10 @@@ check10.test check11.test \ check12.test \ check-exported-srcdir.test \ +check-fd-redirect.test \ check-tests-in-builddir.test \ -check-tests_environment.test \ +check-no-test-driver.test \ + check-concurrency-bug9245.test \ -tests-environment-backcompat.test \ checkall.test \ clean.test \ clean2.test \ diff --cc tests/Makefile.in index a1c152fb0,5b5714123..dc7cb80b8 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@@ -290,14 -300,14 +290,14 @@@ XFAIL_TESTS = all.test auxdir2.test con override-conditional-2.test pr8365-remake-timing.test \ yacc-dist-nobuild-subdir.test txinfo5.test \ $(instspc_xfail_tests) - parallel_tests = backcompat5-p.test check-exported-srcdir-p.test \ - check-fd-redirect-p.test check-tests-in-builddir-p.test \ - check-p.test check11-p.test check12-p.test check2-p.test \ - check3-p.test check4-p.test check5-p.test check6-p.test \ - check7-p.test check8-p.test check9-p.test color-p.test \ - color2-p.test comment9-p.test dejagnu-p.test exeext4-p.test \ - maken3-p.test maken4-p.test posixsubst-tests-p.test \ - tests-environment-p.test + parallel_tests = backcompat5-p.test check-concurrency-bug9245-p.test \ - check-exported-srcdir-p.test check-tests-in-builddir-p.test \ - check-tests_environment-p.test check-p.test check10-p.test \ - check11-p.test check12-p.test check2-p.test check3-p.test \ - check4-p.test check5-p.test check6-p.test check7-p.test \ - check8-p.test check9-p.test color-p.test color2-p.test \ - comment9-p.test dejagnu-p.test exeext4-p.test maken3-p.test \ - maken4-p.test posixsubst-tests-p.test ++ check-exported-srcdir-p.test check-fd-redirect-p.test \ ++ check-tests-in-builddir-p.test check-p.test check11-p.test \ ++ check12-p.test check2-p.test check3-p.test check4-p.test \ ++ check5-p.test check6-p.test check7-p.test check8-p.test \ ++ check9-p.test color-p.test color2-p.test comment9-p.test \ ++ dejagnu-p.test exeext4-p.test maken3-p.test maken4-p.test \ ++ posixsubst-tests-p.test tests-environment-p.test instspc_tests = instspc-squote-build.test instspc-squote-install.test \ instspc-dquote-build.test instspc-dquote-install.test \ instspc-bquote-build.test instspc-bquote-install.test \ @@@ -496,9 -510,10 +496,10 @@@ check10.test check11.test \ check12.test \ check-exported-srcdir.test \ +check-fd-redirect.test \ check-tests-in-builddir.test \ -check-tests_environment.test \ +check-no-test-driver.test \ + check-concurrency-bug9245.test \ -tests-environment-backcompat.test \ checkall.test \ clean.test \ clean2.test \ @@@ -1625,21 -1473,9 +1626,21 @@@ $(TEST_SUITE_LOG): $(TEST_LOGS col="$$grn"; \ else \ col="$$red"; \ + test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \ fi; \ - echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ - $$exit || exit 1 + echo "$${col}$$br$${std}"; \ + echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \ + echo "$${col}$$br$${std}"; \ + create_testsuite_report --maybe-color; \ + echo "$$col$$br$$std"; \ + if $$success; then :; else \ + echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \ + if test -n "$(PACKAGE_BUGREPORT)"; then \ + echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \ + fi; \ + echo "$$col$$br$$std"; \ + fi; \ - $$success ++ $$success || exit 1 # Run all the tests. check-TESTS: @@@ -1838,23 -1664,24 +1839,24 @@@ uninstall-am .MAKE: check-am check-html install-am install-strip recheck \ recheck-html -.PHONY: all all-am check check-TESTS check-am check-html clean \ - clean-generic clean-local distclean distclean-generic distdir \ - dvi dvi-am html html-am info info-am install install-am \ - install-data install-data-am install-dvi install-dvi-am \ - install-exec install-exec-am install-html install-html-am \ - install-info install-info-am install-man install-pdf \ - install-pdf-am install-ps install-ps-am install-strip \ - installcheck installcheck-am installdirs maintainer-clean \ - maintainer-clean-generic mostlyclean mostlyclean-generic pdf \ - pdf-am ps ps-am recheck recheck-html uninstall uninstall-am +.PHONY: all all-am am--redo-logs check check-TESTS check-am check-html \ + clean clean-generic clean-local distclean distclean-generic \ + distdir dvi dvi-am html html-am info info-am install \ + install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-ps install-ps-am \ + install-strip installcheck installcheck-am installdirs \ + maintainer-clean maintainer-clean-generic mostlyclean \ + mostlyclean-generic pdf pdf-am ps ps-am recheck recheck-html \ + uninstall uninstall-am backcompat5-p.log: backcompat5.test + check-concurrency-bug9245-p.log: check-concurrency-bug9245.test check-exported-srcdir-p.log: check-exported-srcdir.test +check-fd-redirect-p.log: check-fd-redirect.test check-tests-in-builddir-p.log: check-tests-in-builddir.test -check-tests_environment-p.log: check-tests_environment.test check-p.log: check.test -check10-p.log: check10.test check11-p.log: check11.test check12-p.log: check12.test check2-p.log: check2.test