From: Stefano Lattarini Date: Fri, 5 Aug 2011 19:14:53 +0000 (+0200) Subject: Merge branch 'test-protocols' into tap-testsuite-work X-Git-Tag: ng-0.5a~89^2~148 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=3027d901f4c4698241e074be5063df5b18fa7879;p=thirdparty%2Fautomake.git Merge branch 'test-protocols' into tap-testsuite-work * test-protocols: tap: fix typo in TAP driver version message tap docs: we don't support pragma or version directives (yet) tap: new option to change the string designating TAP diagnostic check: add small "synchronization" comment to `test-driver' maintcheck: fix spurious failure w.r.t. use of bare "exit" check: update comments to reflect recent heavy changes check: small cleanups and refactorings in test harness and drivers testsuite: use AM_TESTS_FD_REDIRECT where appropriate docs: some fixlets in documentation on testsuites support test harness: be aware of more metadata, simplify test drivers tap: check that also a trailing TAP plan can hold a skip directive --- 3027d901f4c4698241e074be5063df5b18fa7879 diff --cc ChangeLog index 0b46b4c1c,b44766c6b..b2b45e33a --- a/ChangeLog +++ b/ChangeLog @@@ -1,67 -1,148 +1,212 @@@ + 2011-08-05 Stefano Lattarini + + tap: fix typo in TAP driver version message + * lib/tap-driver (anonymous subroutine printing the script version + message): Add missing trailing newline to the version message. + + 2011-08-05 Stefano Lattarini + + tap docs: we don't support pragma or version directives (yet) + * doc/automake.texi (Incompatibilities with other TAP parsers + and drivers): Document that limitation. Normalize an unrelated + "TODO" item. + + 2011-08-05 Stefano Lattarini + + tap: new option to change the string designating TAP diagnostic + Here we introduce a new option `--diagnostic-string' in our TAP + test driver, that allows the user to specify which string should + denote the beginning of a TAP diagnostic line. This change is + not gratuitous, nor result if over-engineering: it is motivated + by real issues that have emerged during the use of TAP in the + Automake's own testsuite (see the commit `v1.11-1082-g9b967c2' + "testsuite: yet more use of TAP, and related extensions"). + * doc/automake.texi (Use TAP with Automake test harness): Document + the new option. + (Incompatibilities with other TAP parsers and drivers): Report it + as a potential source of incompatibility. + * lib/tap-driver ($diag_string): New global variable, defaulting + to "#", and whose value can be changed ... + (Getopt::Long::GetOptions): ... by the newly recognized option + `--diagnostic-string'. + (handle_tap_comment): Subroutine removed, some of its simple logic + inlined ... + (main): ... in here, where now ... + (extract_tap_comment): ... this new subroutine is used. + ($USAGE): Adjust. + * tests/tap-diagnostic.test: Make one check slightly stricter. + * tests/tap-diag-custom.test: New test. + * tests/Makefile.am (tap_other_tests): Add it. + + 2011-08-05 Stefano Lattarini + + check: add small "synchronization" comment to `test-driver' + * lib/test-driver: Add comment to the code initializing ANSI color + escapes, telling to keep it in sync with the similar initialization + in `lib/am/check.am'. + + 2011-08-05 Stefano Lattarini + + maintcheck: fix spurious failure w.r.t. use of bare "exit" + * tests/check12.test: Cosmetic changes to avoid triggering a + spurious failure of the `sc_tests_Exit_not_exit' maintainer + check. + + 2011-08-05 Stefano Lattarini + + check: update comments to reflect recent heavy changes + * lib/am/check.am [%?PARALLEL_TESTS%]: Update the comments to + reflect the recent changes. Remove quite many comments that + were merely duplicating excerpts from the Automake manual. + + 2011-08-05 Stefano Lattarini + + check: small cleanups and refactorings in test harness and drivers + * lib/tap-driver (yn): New subroutine, converts a boolean value to + a "yes" or "no" string. + (write_test_results): Use it, and related reformatting. + * lib/test-driver (trap): Also remove the `.trs' file on signals, + for extra safety. + (fatal): Remove this function, it's never used. + ($logfile, $trsfile): Renamed ... + ($log_file, $trs_file): ... to these, for clarity, and in order to + be more consistent with the `tap-driver' script. + Improve a couple of comments. + * automake.in (handle_tests): Don't define anymore the now-obsolete + make macro `$(TEST_LOGS_TMP)', nor add it to the list of files to + be removed upon "make mostlyclean". + * lib/am/check.am ($(TEST_SUITE_LOG)): New shell function `f_ok', + tells whether a path refers to an existing, regular, readable file. + Use it throughout. + (recheck): Be safer w.r.t. make implementation that run recipes + with `errexit' shell flag active. + + 2011-08-05 Stefano Lattarini + + testsuite: use AM_TESTS_FD_REDIRECT where appropriate + * parallel-tests-interrupt.test: Use AM_TESTS_FD_REDIRECT, not + AM_TESTS_ENVIRONMENT, to define file descriptors' redirection + for "make check". Fix botched comment. + + 2011-08-05 Stefano Lattarini + + docs: some fixlets in documentation on testsuites support + * doc/automake.texi (Script-based Testsuites): Rename node ... + (Scripts-based Testsuites): ... to this. Break overly long lines + in an example (were causing problems with PDF output). + (Parallel Test Harness): Don't intend paragraphs that come after + examples or bulleted list. + (Overview of Custom Test Drivers Support): Fix typo s/the the/the/. + (Declaring Custom Test Drivers): When giving an aside, prefer comma + to parentheses, as it disrupts the reading flow less. + (API for Custom Test Drivers): Remove a sentence that was basically + duplicated from the previous subsection. + (Command-line arguments for test drivers): Don't intend paragraph + coming after an itemized list. Clarify wording and fix grammaros. + Add a couple of cross-references to earlier explanations of `.log' + and `.trs' files. + (Links and external resources): Node renamed ... + (Links and external resources on TAP): ... to this, for clarity. + Slightly extend and clarify introductory sentence. + + 2011-08-04 Stefano Lattarini + + test harness: be aware of more metadata, simplify test drivers + * lib/am/check.am ($(TEST_SUITE_LOG)): When producing the + global test log, take into account the new metadata field + `:global-test-result:, to write nicely formatted RST section + titles, instead of leaving this chore to the individual test + drivers. + (am_rst_section): Re-introduce this variable, as removed in + commit `v1.11-870-ga27c9c4'. + * lib/test-driver, lib/tap-driver: Write the global test result + as metadata in the `.trs' file, not as (part of) a formatted RST + subsection title in the `.log' file. Related simplifications; + in particular, get rid of temporary files usage. + * doc/automake.texi (Log files generation and test results + recording): Document the new metadata. + * tests/check12.test: Update. + * tests/parallel-tests-harderror.test: Likewise. + * tests/parallel-tests-interrupt.test: Likewise. + * tests/parallel-tests-log-override-2.test: Likewise. + * tests/parallel-tests-log-override-recheck.test: Likewise. + * tests/parallel-tests-unreadable.test: Likewise. + * tests/tap-global-result.test: Likewise. + * tests/test-metadata-results.test: Likewise. + * tests/test-log.test: Likewise, and another minor unrelated fix. + * tests/test-metadata-global-result.test: New test. + * tests/Makefile.am (TESTS): Update. + + 2011-08-03 Stefano Lattarini + + tap: check that also a trailing TAP plan can hold a skip directive + * tests/tap-skip-whole-lastline.test: New test. + * tests/Makefile.am (tap_with_common_setup_tests): Add it. + +2011-08-04 Stefano Lattarini + + testsuite: more correct names for a couple of tests + * tests/parallel-tests-ext-driver.test: Rename ... + * tests/parallel-tests-log-compiler-1.test: ... to this, and + adjust heading comments. + * tests/parallel-tests-ext-driver-prog.test: Rename ... + * tests/parallel-tests-log-compiler-2.test: ... to this, and + adjust heading comments. + * tests/Makefile.am (TESTS): Update. + +2011-08-04 Stefano Lattarini + + testsuite: fix another spurious failure + * tests/parallel-tests-ext-driver.test: Name out custom/dummy + log compilers as `foo-compiler' rather `foo-driver'; not only + this is less confusing, it also avoids errors due to the dummy + log compiler `test-driver' overriding the `test-driver' helper + script installed by Automake. + * tests/parallel-tests-ext-driver-prog.test: Adjust heading + comments. + +2011-08-04 Stefano Lattarini + + testsuite: fix a spurious failure + * tests/parallel-tests-ext-driver.test: Exporting of environment + variables directly from $(LOG_COMPILER) variables is not allowed + anymore. Adjust to this. + +2011-08-04 Stefano Lattarini + + testsuite: yet more use of TAP, and related extensions + * tests/defs (show_): New function, display the contents of one or + more files on stdout, with bells & whistles (both for cosmetic and + practical reasons, the latter aimed at avoiding spurious TAP + diagnostic). + (AUTOMAKE_run): Extend and adapt to make it compatible with TAP + based tests. Since we are at it, make its implementation more + namespace-safe, and improve its command-line interface. + (AUTOMAKE_fails): Adapt to the new command-line interface of + `AUTOMAKE_run'. + * tests/add-missing.test: Renamed ... + * tests/add-missing.tap: ... to this, and converted to the use + of the TAP protocol, and of the new `show_' function. + * tests/automake.test: Renamed ... + * tests/automake-cmdline.tap: ... to this, and converted to the + use of the TAP protocol. + * tests/getopt.test: Adapt to the new command-line interface + of `AUTOMAKE_run'. + * tests/cond44.test: Likewise. + * tests/cond45.test: Likewise. + * tests/configure.test: Likewise. + * tests/license2.test: Likewise. + * tests/parallel-am.test: Likewise. + * tests/parallel-am3.test: Likewise. + * tests/tags2.test: Likewise. + * tests/werror3.test: Likewise. + * tests/werror4.test: Likewise. + * tests/amopts-variable-expansion.test: Likewise. + * tests/warnings-win-over-strictness.test: Likewise. + * tests/Makefile.am (TESTS): Update. + (TAP_LOG_DRIVER_FLAGS): Add `--merge', so that TAP diagnostic is + reported in the testsuite progress output. + 2011-08-03 Stefano Lattarini maintcheck: fix more maintainer-check failures diff --cc tests/Makefile.in index 4ac322088,f03ec163a..5286f6dcc --- a/tests/Makefile.in +++ b/tests/Makefile.in @@@ -179,26 -177,9 +180,25 @@@ am__test_logs2 = $(am__test_logs1:.test TEST_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver TEST_LOG_COMPILE = $(TEST_LOG_COMPILER) $(AM_TEST_LOG_FLAGS) \ $(TEST_LOG_FLAGS) -TEST_LOGS = $(am__test_logs2:.tap.log=.log) +am__test_logs3 = $(am__test_logs2:.tap.log=.log) TAP_LOG_COMPILE = $(TAP_LOG_COMPILER) $(AM_TAP_LOG_FLAGS) \ $(TAP_LOG_FLAGS) +am__test_logs4 = $(am__test_logs3:.ptest.log=.log) +PTEST_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver +PTEST_LOG_COMPILE = $(PTEST_LOG_COMPILER) $(AM_PTEST_LOG_FLAGS) \ + $(PTEST_LOG_FLAGS) +am__test_logs5 = $(am__test_logs4:.instspc.log=.log) +INSTSPC_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver +INSTSPC_LOG_COMPILE = $(INSTSPC_LOG_COMPILER) $(AM_INSTSPC_LOG_FLAGS) \ + $(INSTSPC_LOG_FLAGS) +am__test_logs6 = $(am__test_logs5:.depmod.log=.log) +DEPMOD_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver +DEPMOD_LOG_COMPILE = $(DEPMOD_LOG_COMPILER) $(AM_DEPMOD_LOG_FLAGS) \ + $(DEPMOD_LOG_FLAGS) +TEST_LOGS = $(am__test_logs6:.shtst.log=.log) +SHTST_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver +SHTST_LOG_COMPILE = $(SHTST_LOG_COMPILER) $(AM_SHTST_LOG_FLAGS) \ + $(SHTST_LOG_FLAGS) - TEST_LOGS_TMP = $(TEST_LOGS:.log=.log-t) DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@