From: Stefano Lattarini Date: Mon, 8 Aug 2011 08:40:28 +0000 (+0200) Subject: Merge branch 'master' into test-protocols X-Git-Tag: ng-0.5a~89^2~131^2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=b9dfd77aa5be5a91bfc82ce398d27e8266733554;p=thirdparty%2Fautomake.git Merge branch 'master' into test-protocols * master: tests: fix spurious failure with Solaris make --- b9dfd77aa5be5a91bfc82ce398d27e8266733554 diff --cc ChangeLog index 579e84286,81e2569f6..96cf40016 --- a/ChangeLog +++ b/ChangeLog @@@ -1,338 -1,11 +1,346 @@@ + 2011-08-08 Stefano Lattarini + + tests: fix spurious failure with Solaris make + * tests/distcheck-configure-flags-am.test: Avoid using `+=' too + liberally with AM_DISTCHECK_CONFIGURE_FLAGS, since the line breaks + so introduced, in conjunction with single quotes, might confuse + Solaris make. + +2011-08-08 Stefano Lattarini + + test harness: fixlet in workaround for VPATH rewrite issue + Fixes a minor in previous commit `v1.11-983-gfda3de5'. + * lib/am/check.am (am__set_TESTS_bases): Add forgotten end-of-line + anchor to sed expression. In comments, add reference to ... + * tests/test-trs-basic.test: ... this test, which has been adapted + and tightened (and tweaked). + +2011-08-08 Stefano Lattarini + + test harness: work around a VPATH rewrite issue + * lib/am/check.am (am__TEST_BASES): Removed, it's role taken + over by ... + (am__set_TESTS_bases): ... these new variable. + ($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues. + * automake.in (handle_tests): Update the code for the cleanup + of the `.trs' file to use `$(TEST_LOGS)' instead of + `$(am__TEST_BASES)'. + * tests/test-harness-vpath-rewrite.test: New test, better + exposing the problem fixed by this change + * tests/Makefile.am (TESTS): Add it. + * tests/test-trs-basic.test: Update and extend. + +2011-08-07 Stefano Lattarini + + parallel-tests: work around Solaris XPG4 make segfault + Issue exposed by test `posixsubst-tests-p.test', and similar to + the problem solved by commit `v1.11-159-ge7aa360'. + * lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing + whitespace from $list, to avoid triggering a nasty bug (potential + segfault) on Solaris XPG4 make and Heirloom make. + +2011-08-07 Stefano Lattarini + + testsuite: fix weird spurious failure with Solaris /bin/sh + Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can + apparently end up exiting with exit status 208, instead of leaving + the correct wide exit status to the parent. See: + + Work around this incompatibility. + * tap-signal.test: Look for the above Solaris bug. + (signal_caught): Adapt to handle it. + +2011-08-07 Stefano Lattarini + + testsuite: fix another spurious failure on Solaris make + * tests/parallel-tests-log-override-recheck.test: Filter make + output before grepping it, for make implementations that, like + Solaris' one, print the whole of the failed recipe on failure. + +2011-08-07 Stefano Lattarini + + testsuite: fix two spurious failures on Solaris make + * tests/tests-environment-and-log-compiler.test: Do not fail if + "make -n check" fails, as that is issued only for debugging, its + semantics being tested in other test scripts. + * tests/test-driver-strip-vpath.test: Likewise. + +2011-08-07 Stefano Lattarini + + tap driver: refactor and remove dead code + * lib/tap-driver (get_tap_line): Removed, ... + (all callers): ... updated to use $parser->next directly instead. + (peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not + used anymore. + (TAP_PEEKING): Block deleted, all its subroutines and variables + having been removed. + +2011-08-07 Stefano Lattarini + + testsuite: fix spurious failure in a test on TAP support + * tests/tap-passthrough-exit.test: When the `--ignore-exit' flag + of the TAP driver is used, don't look for a message reporting the + non-zero exit statuses of tests in the log files; such message is + not expected to be there anymore. Related simplifications. + +2011-08-07 Stefano Lattarini + + tap: plan location is more liberal w.r.t. non-TAP lines + With this change, only lines that are TAP results will matter + w.r.t. the position of the TAP plan in the input; for example, + this input: + this is a non-TAP line + # and this a TAP diagnostic line + 1..1 + ok 1 + was considered to be an error, diagnosed with a message "test + plan in middle of output"; as effect of the current change, such + input is now valid. This is more consistent with the behaviour + of the `prove' utility. + * lib/tap-driver ($lineno): Removed, no more needed. + ($tap_stopped): New global variable. + (stringify_test_result): Return "ERROR" if a TAP result is found + when `$tap_stopped' is set to true. + (handle_tap_test): Diagnose TAP results that comes after a late + plan. Add a couple of blank lines, for clarity. + (handle_tap_plan): Set `$tap_stopped' to true after a late plan + is encountered. Do not complain anymore for extra non-TAP lines + preceding or following the plan. Adjust comments. + (main): Don't increment $lineno anymore. + * tests/tap-plan.test: Extend a bit, and remove stale comment. + * tests/tap-color.test: Adjust. + * tests/tap-passthrough.test: Likewise. + * tests/tap-plan-corner.test: Adjust and extend. + * tests/tap-plan-errors.test: Likewise. + * tests/tap-plan-middle.test: New test. + * tests/tap-plan-corner2.test: Delete, it's obsolete now. + * tests/Makefile.am (XFAIL_TESTS): Remove it. + (tap_with_common_setup_tests): Likewise, and add + `tap-plan-corner.test'. + +2011-08-07 Stefano Lattarini + + testsuite: remove now-passing test from XFAIL_TESTS + * tests/Makefile.am (XFAIL_TESTS): Remove `tap-signal.test', + which is passing since previous commit `v1.11-974-gc7fa872'. + +2011-08-06 Stefano Lattarini + + tap driver: handle signals received by the tests being run + * lib/tap-driver (get_test_exit_message): Also deal with signals, + by using the `wait' method of the TAP::Parser object instead of + the `exit' method. This required the use of the standard perl + module `POSIX'. + * doc/automake.texi (Use TAP with the Automake test harness): + Document that `--ignore-exit' has effect also on terminating + signals. Add a "synchronizing" comment that references the tests + 'tap-exit.test' and 'tap-signal.test'. + * tests/tap-signal.test: Extend and adjust. + +2011-08-06 Stefano Lattarini + + test driver: a preparatory refactoring (2) + * lib/tap-driver (finish): Move code to fetch the message about + the exit status of the test being run to ... + (get_test_exit_message): ... this new subroutine. + +2011-08-06 Stefano Lattarini + + tap driver: a preparatory refactoring (1) + * lib/tap-driver (finish): Do not flush the TAP stream nor + fetch the exit status of the TAP producer if running with + the `--ignore-exit' option. Obviously, don't try to write + such exit status in the test log anymore. + +2011-08-06 Stefano Lattarini + + cosmetics: typofix in the name of a function in the TAP driver + * lib/tap-driver (testuite_error): Renamed ... + (testsuite_error): ... to this, and minor related reformatting. + +2011-08-06 Stefano Lattarini + + tap: scripts with a SKIP plan but with exit status != 0 must error + This change has been motivated by Automake's own testsuite. Some + TAP tests there were erroring out (due to unexpected, unhandled + failures) before having encountered TAP result, so that the + simple-minded shell library implementing TAP generation ended up + printing a "1..0" trailing test plan; this caused the script to be + reported as a SKIP rather than an ERROR -- a nasty false negative. + * lib/tap-driver: Add prototypes for each subroutine, to free up + the order in which they can be defined and called. + (main): Move the code checking for a bad exit status of the TAP + producer ... + (finish): ... here, and flush the TAP stream to ensure that the + parser always obtains the producer's exit status. + * tests/tap-skip-whole-badexit.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Add it. + +2011-08-06 Stefano Lattarini + + tap: fix whitespace munging of diagnostic messages + * lib/tap-driver (extract_tap_comment): Pass the `g' flag to the + substitution operator, to strip also trailing whitespaces. Fixes + a failure in test `tap-whitespace-normalization.test'. + +2011-08-05 Stefano Lattarini + + tap: fix typo in TAP driver version message + * lib/tap-driver (anonymous subroutine printing the script version + message): Add missing trailing newline to the version message. + +2011-08-05 Stefano Lattarini + + tap docs: we don't support pragma or version directives (yet) + * doc/automake.texi (Incompatibilities with other TAP parsers + and drivers): Document that limitation. Normalize an unrelated + "TODO" item. + +2011-08-05 Stefano Lattarini + + tap: new option to change the string designating TAP diagnostic + Here we introduce a new option `--diagnostic-string' in our TAP + test driver, that allows the user to specify which string should + denote the beginning of a TAP diagnostic line. This change is + not gratuitous, nor result if over-engineering: it is motivated + by real issues that have emerged during the use of TAP in the + Automake's own testsuite (see the commit `v1.11-1082-g9b967c2' + "testsuite: yet more use of TAP, and related extensions"). + * doc/automake.texi (Use TAP with Automake test harness): Document + the new option. + (Incompatibilities with other TAP parsers and drivers): Report it + as a potential source of incompatibility. + * lib/tap-driver ($diag_string): New global variable, defaulting + to "#", and whose value can be changed ... + (Getopt::Long::GetOptions): ... by the newly recognized option + `--diagnostic-string'. + (handle_tap_comment): Subroutine removed, some of its simple logic + inlined ... + (main): ... in here, where now ... + (extract_tap_comment): ... this new subroutine is used. + ($USAGE): Adjust. + * tests/tap-diagnostic.test: Make one check slightly stricter. + * tests/tap-diag-custom.test: New test. + * tests/Makefile.am (tap_other_tests): Add it. + +2011-08-05 Stefano Lattarini + + check: add small "synchronization" comment to `test-driver' + * lib/test-driver: Add comment to the code initializing ANSI color + escapes, telling to keep it in sync with the similar initialization + in `lib/am/check.am'. + +2011-08-05 Stefano Lattarini + + maintcheck: fix spurious failure w.r.t. use of bare "exit" + * tests/check12.test: Cosmetic changes to avoid triggering a + spurious failure of the `sc_tests_Exit_not_exit' maintainer + check. + +2011-08-05 Stefano Lattarini + + check: update comments to reflect recent heavy changes + * lib/am/check.am [%?PARALLEL_TESTS%]: Update the comments to + reflect the recent changes. Remove quite many comments that + were merely duplicating excerpts from the Automake manual. + +2011-08-05 Stefano Lattarini + + check: small cleanups and refactorings in test harness and drivers + * lib/tap-driver (yn): New subroutine, converts a boolean value to + a "yes" or "no" string. + (write_test_results): Use it, and related reformatting. + * lib/test-driver (trap): Also remove the `.trs' file on signals, + for extra safety. + (fatal): Remove this function, it's never used. + ($logfile, $trsfile): Renamed ... + ($log_file, $trs_file): ... to these, for clarity, and in order to + be more consistent with the `tap-driver' script. + Improve a couple of comments. + * automake.in (handle_tests): Don't define anymore the now-obsolete + make macro `$(TEST_LOGS_TMP)', nor add it to the list of files to + be removed upon "make mostlyclean". + * lib/am/check.am ($(TEST_SUITE_LOG)): New shell function `f_ok', + tells whether a path refers to an existing, regular, readable file. + Use it throughout. + (recheck): Be safer w.r.t. make implementation that run recipes + with `errexit' shell flag active. + +2011-08-05 Stefano Lattarini + + testsuite: use AM_TESTS_FD_REDIRECT where appropriate + * parallel-tests-interrupt.test: Use AM_TESTS_FD_REDIRECT, not + AM_TESTS_ENVIRONMENT, to define file descriptors' redirection + for "make check". Fix botched comment. + +2011-08-05 Stefano Lattarini + + docs: some fixlets in documentation on testsuites support + * doc/automake.texi (Script-based Testsuites): Rename node ... + (Scripts-based Testsuites): ... to this. Break overly long lines + in an example (were causing problems with PDF output). + (Parallel Test Harness): Don't intend paragraphs that come after + examples or bulleted list. + (Overview of Custom Test Drivers Support): Fix typo s/the the/the/. + (Declaring Custom Test Drivers): When giving an aside, prefer comma + to parentheses, as it disrupts the reading flow less. + (API for Custom Test Drivers): Remove a sentence that was basically + duplicated from the previous subsection. + (Command-line arguments for test drivers): Don't intend paragraph + coming after an itemized list. Clarify wording and fix grammaros. + Add a couple of cross-references to earlier explanations of `.log' + and `.trs' files. + (Links and external resources): Node renamed ... + (Links and external resources on TAP): ... to this, for clarity. + Slightly extend and clarify introductory sentence. + +2011-08-04 Stefano Lattarini + + test harness: be aware of more metadata, simplify test drivers + * lib/am/check.am ($(TEST_SUITE_LOG)): When producing the + global test log, take into account the new metadata field + `:global-test-result:, to write nicely formatted RST section + titles, instead of leaving this chore to the individual test + drivers. + (am_rst_section): Re-introduce this variable, as removed in + commit `v1.11-870-ga27c9c4'. + * lib/test-driver, lib/tap-driver: Write the global test result + as metadata in the `.trs' file, not as (part of) a formatted RST + subsection title in the `.log' file. Related simplifications; + in particular, get rid of temporary files usage. + * doc/automake.texi (Log files generation and test results + recording): Document the new metadata. + * tests/check12.test: Update. + * tests/parallel-tests-harderror.test: Likewise. + * tests/parallel-tests-interrupt.test: Likewise. + * tests/parallel-tests-log-override-2.test: Likewise. + * tests/parallel-tests-log-override-recheck.test: Likewise. + * tests/parallel-tests-unreadable.test: Likewise. + * tests/tap-global-result.test: Likewise. + * tests/test-metadata-results.test: Likewise. + * tests/test-log.test: Likewise, and another minor unrelated fix. + * tests/test-metadata-global-result.test: New test. + * tests/Makefile.am (TESTS): Update. + +2011-08-03 Stefano Lattarini + + tap: check that also a trailing TAP plan can hold a skip directive + * tests/tap-skip-whole-lastline.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Add it. + +2011-08-03 Stefano Lattarini + + maintcheck: fix more maintainer-check failures + * tests/test-driver-custom-no-html.test (no-rst): Use `EOF', + not `EoL', as the here-document delimiter. + * tests/test-trs-basic.test: Use `cat + here-doc' rather + than `echo' when creating the dummy test scripts, to please + maintainer-check. + * tests/test-trs-recover.test: Use creative quoting where + needed, to please maintainer-check. + * tests/parallel-tests-no-color-in-log.test: Likewise. + * tests/parallel-tests-dry-run.test: Likewise. + 2011-08-03 Stefano Lattarini maintcheck: fix maintainer-check failures, both real and spurious