From: Stefano Lattarini Date: Wed, 29 Jun 2011 08:06:20 +0000 (+0200) Subject: Merge branch 'master' into test-protocols X-Git-Tag: ng-0.5a~89^2~155^2~26 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=3605250b571309a41f52ba110c1f14d2a2ca25f7;p=thirdparty%2Fautomake.git Merge branch 'master' into test-protocols * master: docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated docs: fix unportable example of AM_TESTS_ENVIRONMENT usage docs: avoid a footnote, some related rewordings and improvements Revert "docs: parallel-tests is not experimental anymore" docs: minor cosmetic fixes help: improve text about automatically-distributed files refactor: split 'usage' subroutine in automake tests: fix bug in 'autodist.test' parallel-tests: stricter checks on DISABLE_HARD_ERRORS support docs: parallel-tests is not experimental anymore check: document and test $(TEST_SUITE_LOG) overriding --- 3605250b571309a41f52ba110c1f14d2a2ca25f7 diff --cc ChangeLog index d6dbd1341,14364825b..c99944af6 --- a/ChangeLog +++ b/ChangeLog @@@ -1,138 -1,69 +1,204 @@@ + 2011-06-29 Stefano Lattarini + + docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated + * doc/automake.texi (Simple Tests using parallel-tests): Ditto, and + related adjustments. + Suggestion by Ralf Wildenhues. + + 2011-06-29 Stefano Lattarini + + docs: fix unportable example of AM_TESTS_ENVIRONMENT usage + * doc/automake.texi (Simple Tests using parallel-tests): The + old example on AM_TESTS_ENVIRONMENT relied on unportable shell + features, and in particular didn't work with various Korn + Shells (see also commit `v1.11-925-g29ca903'). Give another + example, simpler this time, but still inspired to real-world + usage (the GNU coreutils testsuite). + + 2011-06-23 Stefano Lattarini + + docs: avoid a footnote, some related rewordings and improvements + * doc/automake.texi (Dist): Reword the part about automatically + distributed files to avoid a footnote. Since we are at it, extend + a bit, and add an example and a reference to a relevant test case. + + 2011-06-23 Stefano Lattarini + + docs: minor cosmetic fixes + * doc/automake.texi: Break few overly long lines, throughout the + file. + ("Simple Tests"): Move @vindex for XFAIL_TESTS to the correct + position, i.e., before and not after the paragraph where it is + introduced. + ("Options" @item ansi2knr): Use @pxref instead of @xref. This + fixes a texinfo warning. + ("Other things Automake recognizes" @item AM_C_PROTOTYPES): Use + @pxref instead of @ref. + + 2011-06-23 Stefano Lattarini + + help: improve text about automatically-distributed files + This change fixes automake bug#7819. + * automake.in (usage): Distinguish between files that are always + automatically distributed when found, and those which are only + "under certain conditions". + * doc/automake.texi (Basics of Distribution): Update accordingly. + * tests/autodist-subdir.test: Update. + * tests/autodist-no-duplicate.test: Likewise. + * tests/autodist.test: Likewise. + (configure.in): Remove useless call to AM_MAINTAINER_MODE. + + 2011-06-23 Stefano Lattarini + + refactor: split 'usage' subroutine in automake + This change is related to automake bug#7819. + * automake.in (print_autodist_files): New subroutine, + extracted from ... + (usage): ... this, which now uses it. + * tests/autodist-no-duplicate.test: New test. + * tests/Makefile.am (TESTS): Update. + + 2011-06-23 Stefano Lattarini + + tests: fix bug in 'autodist.test' + * tests/autodist.test: Avoid spurious failure due to no + `defs-static' file being found in the parent directory. + +2011-06-21 Stefano Lattarini + + parallel-tests: allow each test to have multiple results + With this change, we improve the code creating the `test-suite.log' + global log and the console testsuite summary to make it able to + grasp multiple results per test script. This is required in order + to introduce the planned support for test protocols, like TAP and + SubUnit, which can indeed run multiple tests per test script, each + with its individual result. + The implementation makes use of a custom reStructuredText field + `:test-result:'. + Note that no new documentation is added by this change; that is + be left for follow-up changes. + * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files, + recognize a report of a test's result only if it is declared with + the custom `:test-result:' reStructuredText field placed at the + beginning of a line. Extend and add explanatory comments. + (recheck, recheck-html): Add explanatory comments. + * lib/test-driver: Write an appropriate reStructuredText field + `:test-result:' in the generated log file. Use a reStructuredText + transition to better separate the test outcome report from the + test script's registered output. Improve comments. + * tests/test-driver-custom-xfail-tests.test: Adjust. + * tests/parallel-tests7.test: Adjust. + * tests/parallel-tests-empty-testlogs.test: New test. + * tests/parallel-tests-recheck-override.test: Likewise. + * tests/parallel-tests2.test: Extend and keep more in-sync with ... + * tests/test-driver-custom-html.test: ... this new related test. + * tests/test-driver-custom-no-html.test: New test. + * tests/test-driver-custom-multitest.test: Likewise. + * tests/test-driver-custom-multitest-recheck.test: Likewise. + * tests/test-driver-custom-multitest-recheck2.test: Likewise. + * tests/trivial-test-driver: New file, used by the last four tests + above. + * tests/Makefile.am (TESTS): Update. + (EXTRA_DIST): Distribute `trivial-test-driver'. + (test-driver-custom-multitest.log): Depend on `trivial-test-driver'. + (test-driver-custom-multitest-recheck.log): Likewise. + (test-driver-custom-multitest-recheck2.log): Likewise. + (test-driver-custom-html.log): Likewise. + +2011-06-21 Stefano Lattarini + + parallel-tests: allow custom driver scripts + Allow suffix-based definition of custom "driver script" for the + test scripts. These driver scripts will be responsible of + launching the tests (or their corresponding $(LOG_COMPILER), if + they have an associated one), interpreting and displaying the + test results, and writing the `.log' files. + This new API should allow easy and flexible use of different + test protocols in the future; in particular, we plan to use it + to implement TAP and SubUnit harnesses. + Note that no new documentation is added by this change; that is + be left for follow-up changes. + * automake.in (handle_tests): Define default for $(LOG_DRIVER), + and, for any registered test extension `', define defaults + for $(_LOG_DRIVER). Substitute %DRIVER% using these new + variables, instead of the old internal $(am__test_driver). When + processing check2.am, also substitute %DRIVER_FLAGS%. + Require auxiliary script `test-driver' only if no driver has been + explicitly defined for the test script kinds. + * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log, + ?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call. + * tests/parallel-tests-no-extra-driver.test: New test. + * tests/test-driver-custom.test: Likewise. + * tests/test-driver-custom-xfail-tests.test: Likewise. + * tests/test-driver-fail.test: Likewise. + * tests/Makefile.am: Update. + * NEWS: Update. + +2011-06-21 Stefano Lattarini + + parallel-tests: add auxiliary script 'test-driver', refactor + This refactoring should cause no API of functionality change, + and is meant only to simplify the future implementation of TAP + and SubUnit testsuite drivers. More precisely, our roadmap is + to move most of the "testsuite driving" features out of the + Automake-generated Makefiles, and into external scripts with + well-defined interfaces. This will allow the user to define + its own personalized testsuite drivers, and will also offer us + a framework upon which to implement our new TAP and SubUnit + drivers, all in a very unobtrusive way and retaining an high + degree of code reuse and backward-compatibility. + * lib/test-driver: New auxiliary script. + * lib/Makefile.am (dist_SCRIPT_DATA): Add it. + * automake.in (handle_tests): Require the new auxiliary script + `test-driver', and define a new internal makefile variable + `$(am__test_driver)', used to call it. Perform new substitution + on `DRIVER' when processing the `check2.am' file. + * lib/check.am (am__tty_colors): Define new shell variable + `$am__color_tests'. + (am__rst_section): Removed, its role taken over by the new + `test-driver' script. + (am__test_driver_flags): New variable, contains the command + line options passed to `test-driver'. + (am__check_pre): Do not deal with temporary files and exit + traps anymore, as the `test-driver' script takes care of that + now. Define shell variable `$am__enable_hard_errors', used by + `$(am__test_driver_flags)'. Reorder so that we don't need to + save and restore the value of the `TERM' environment variable + anymore. + Other related adjustments. + (am__check_post): Remove, as its role has been completely taken + over by the `test-driver' script. + * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log, + ?!GENERIC?%OBJ%): Call the test script through the Automake + substituted `%DRIVER%', and honor the command-line options + in `$(am__test_driver_flags)'. Do not call the obsoleted + `$(am__check_post)' anymore. + * doc/automake.texi (Auxiliary Programs): Mention the new + `test-driver' script. + (Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR. + Since we are at it, break the list of auxiliary scripts by + placing one per line, to simplify potential future additions + of new scripts. + * tests/check.test: Adjust. + * tests/check2.test : Likewise. + * tests/check3.test : Likewise. + * tests/check4.test : Likewise. + * tests/check10.test: Likewise. + * tests/color.test: Likewise. + * tests/color2.test: Likewise. + * tests/comment9.test: Likewise. + * tests/dejagnu.test: Likewise. + * tests/exeext4.test: Likewise. + * tests/maken3.test: Likewise. + * tests/maken4.test: Likewise. + * tests/parallel-tests-interrupt.test: Likewise. + * tests/posixsubst-tests.test: Likewise. + * tests/repeated-options.test: Likewise. + * tests/check-no-test-driver.test: New test. + * tests/parallel-test-driver-install.test: Likewise. + * tests/Makefile.am (TESTS): Update. + * NEWS: Update. + 2011-06-21 Stefano Lattarini maintcheck: extend 'sc_tests_plain_*' checks diff --cc tests/Makefile.am index f1ccb7b21,3883dc009..b49e754e6 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@@ -723,17 -723,10 +724,21 @@@ parallel-tests-unreadable-log.test parallel-tests-subdir.test \ parallel-tests-interrupt.test \ parallel-tests-reset-term.test \ + parallel-tests-harderror.test \ + parallel-tests-log-override-1.test \ + parallel-tests-log-override-2.test \ + parallel-tests-log-override-recheck.test \ +parallel-tests-empty-testlogs.test \ +parallel-test-driver-install.test \ +test-driver-custom-no-extra-driver.test \ +test-driver-custom.test \ +test-driver-custom-xfail-tests.test \ +test-driver-custom-multitest.test \ +test-driver-custom-multitest-recheck.test \ +test-driver-custom-multitest-recheck2.test \ +test-driver-custom-html.test \ +test-driver-custom-no-html.test \ +test-driver-fail.test \ parse.test \ percent.test \ percent2.test \ diff --cc tests/Makefile.in index f06562b45,74538637b..0219eea21 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@@ -983,17 -996,10 +984,21 @@@ parallel-tests-unreadable-log.test parallel-tests-subdir.test \ parallel-tests-interrupt.test \ parallel-tests-reset-term.test \ + parallel-tests-harderror.test \ + parallel-tests-log-override-1.test \ + parallel-tests-log-override-2.test \ + parallel-tests-log-override-recheck.test \ +parallel-tests-empty-testlogs.test \ +parallel-test-driver-install.test \ +test-driver-custom-no-extra-driver.test \ +test-driver-custom.test \ +test-driver-custom-xfail-tests.test \ +test-driver-custom-multitest.test \ +test-driver-custom-multitest-recheck.test \ +test-driver-custom-multitest-recheck2.test \ +test-driver-custom-html.test \ +test-driver-custom-no-html.test \ +test-driver-fail.test \ parse.test \ percent.test \ percent2.test \