From: Stefano Lattarini Date: Fri, 22 Jun 2012 22:42:06 +0000 (+0200) Subject: Merge branch 'master' into ng/master X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=bbd39b37ac1478d34fe9724ad35ffd6de68c295e;p=thirdparty%2Fautomake.git Merge branch 'master' into ng/master * master: tests: avoid one last `...` command substitution in 'test-init.sh' maintcheck: guard against `...` for command substitution in test cases tests: more uses of $(...) over `...` for command substitution cosmetics: quote `like this', not 'like this', in a couple of tests readme: clarify/extend few entries in 't/README' readme: better separation of entries in 't/README' readme: subsections "Do" and "Do not" in 't/README' merged tests: new requirement 'grep-nonprint' cosmetics: fix description of an expected error message in a test tests: assume automake quotes 'like this', not `like this' Signed-off-by: Stefano Lattarini --- bbd39b37ac1478d34fe9724ad35ffd6de68c295e diff --cc syntax-checks.mk index 8f3c4b772,77d2f42a4..4c6698f94 --- a/syntax-checks.mk +++ b/syntax-checks.mk @@@ -55,21 -54,11 +55,22 @@@ sc_no_am_cd sc_perl_at_uscore_in_scalar_context \ sc_perl_local \ sc_AMDEP_TRUE_in_automake_in \ -sc_tests_make_without_am_makeflags \ +sc_tests_no_gmake_requirement \ +sc_tests_no_gmake_checking \ +sc_tests_make_can_chain_suffix_rules \ +sc_tests_make_dont_do_useless_vpath_rebuilds \ +sc_no_dotmake_target \ +sc_no_am_makeflags \ $(sc_obsolete_requirements_rules) \ +$(sc_renamed_variables_rules) \ +sc_no_RECHECK_LOGS \ +sc_tests_no_make_e \ +sc_docs_no_make_e \ +sc_make_simple_include \ +sc_tests_make_simple_include \ sc_tests_obsolete_variables \ sc_tests_here_document_format \ + sc_tests_command_subst \ sc_tests_Exit_not_exit \ sc_tests_automake_fails \ sc_tests_required_after_defs \ diff --cc t/README index 1d630c0a4,3f56400f2..31608e293 --- a/t/README +++ b/t/README @@@ -183,16 -184,16 +183,16 @@@ Writing test case set the shell variable 'am_serial_tests' to "yes" before including ./defs. - Some tests in the Automake-NG testsuite are auto-generated; those tests -* Some tests in the Automake testsuite are auto-generated; those tests -- might have custom extensions, but their basename (that is, with such -- extension stripped) is expected to end with "-w" string, optionally -- followed by decimal digits. For example, the name of a valid -- auto-generated test can be 'color-w.sh' or 'tap-signal-w09.tap'. ++* Some tests in the Automake-NG testsuite are auto-generated; those ++ tests might have custom extensions, but their basename (that is, ++ with such extension stripped) is expected to end with "-w" string, ++ optionally followed by decimal digits. For example, the name of a ++ valid auto-generated test can be 'color-w.sh' or 'tap-signal-w09.tap'. Please don't name hand-written tests in a way that could cause them -- to be confused with auto-generated tests; for example, 'u-v-w.sh' -- or 'option-w0.tap' are *not* valid name for hand-written tests. ++ to be confused with auto-generated tests; for example, 'u-v-w.sh' or ++ 'option-w0.tap' are *not* valid name for hand-written tests. - ./defs brings in some commonly required files, and sets a skeleton + * ./defs brings in some commonly required files, and sets a skeleton configure.ac. If possible, append to this file. In some cases you'll have to overwrite it, but this should be the exception. Note that configure.ac registers Makefile.in but do not output anything by @@@ -228,14 -229,14 +228,14 @@@ likely to fail, display its output even in the failure case, before exiting. - Use 'Exit' rather than 'exit' to abort for leave early from a test + * Use 'Exit' rather than 'exit' to abort for leave early from a test case. - Use '$PATH_SEPARATOR', not hard-coded ':', as the separator of + * Use '$PATH_SEPARATOR', not hard-coded ':', as the separator of PATH's entries. - It's more important to make sure that a feature works, than make sure -* It's more important to make sure that a feature works, than make - sure that Automake's output looks correct. It might look correct ++* It's more important to make sure that a feature works, than make sure + that the output of Automake-NG looks correct. It might look correct and still fail to work. In other words, prefer running 'make' over grepping Makefile.in (or do both). @@@ -249,9 -252,9 +251,9 @@@ code preserves filenames with spaces. This will catch errors like `echo $filename | ...`. - Make sure your test script can be used to faithfully check an - installed version of Automake-ng (as with "make installcheck"). For - example, if you need to copy or grep an Automake-ng provided script, + * Make sure your test script can be used to faithfully check an - installed version of automake (as with "make installcheck"). For - example, if you need to copy or grep an automake-provided script, ++ installed version of Automake-NG (as with "make installcheck"). For ++ example, if you need to copy or grep an Automake-NG provided script, do not assume that they can be found in the '$top_srcdir/lib' directory, but use '$am_scriptdir' instead. The complete list of such "$am_...dir" variables can be found in the 'defs-static.in' @@@ -269,15 -276,11 +275,11 @@@ by a C++ compiler, for similar reasons (i.e., the isatty(3) function from that same unistd.h header would be required otherwise). - Before commit: make sure the test is executable, add the tests to + * Before commit: make sure the test is executable, add the tests to TESTS in Makefile.am, add it to XFAIL_TESTS in addition if needed, - write a ChangeLog entry, send the diff to . + write a ChangeLog entry, send the diff to . - - Do not - ------ - - In test scripts, prefer using POSIX constructs over their old + * In test scripts, prefer using POSIX constructs over their old Bourne-only equivalents: - use $(...), not `...`, for command substitution; @@@ -297,13 -300,18 +299,12 @@@ years we'll discover that this test failed for some other bogus reason. This happened many times. Better use something like - Do not test an Automake-NG error with "$AUTOMAKE && Exit 1", or in ++* Do not test an Automake-NG error with "$AUTOMAKE && Exit 1", or in + three years we'll discover that this test failed for some other bogus + reason. This happened many times. Better use something like + AUTOMAKE_fails grep 'expected diagnostic' stderr - (Note this doesn't prevent the test from failing for another - reason, but at least it makes sure the original error is still - here.) + Note this doesn't prevent the test from failing for another reason, + but at least it makes sure the original error is still here. - -* Do not override Makefile variables using make arguments, as in e.g.: - - $MAKE prefix=/opt install - - This is not portable for recursive targets (targets that call a - sub-make may not pass "prefix=/opt" along). Use the following - instead: - - prefix=/opt $MAKE -e install diff --cc t/color.sh index 66f349368,5726ac1bd..9e4ac04fb --- a/t/color.sh +++ b/t/color.sh @@@ -96,8 -89,36 +89,10 @@@ test_color ( test_no_color () { - # With make implementations that, like Solaris make, in case of errors - # print the whole failing recipe on standard output, we should content - # ourselves with a laxer check, to avoid false positives. - # Keep this in sync with lib/am/check.am:$(am__color_tests). - if $FGREP '= Xalways; then' stdout; then - # Extra verbose make, resort to laxer checks. - # Note that we also want to check that the testsuite summary is - # not unduly colorized. - ( - set +e # In case some grepped regex below isn't matched. - # Not a useless use of cat; see above comments "grep-nonprinting" - # requirement in 'test-init.sh'. - cat stdout | grep "TOTAL.*:" - cat stdout | grep "PASS.*:" - cat stdout | grep "FAIL.*:" - cat stdout | grep "SKIP.*:" - cat stdout | grep "XFAIL.*:" - cat stdout | grep "XPASS.*:" - cat stdout | grep "ERROR.*:" - cat stdout | grep 'test.*expected' - cat stdout | grep 'test.*not run' - cat stdout | grep '====' - cat stdout | grep '[Ss]ee .*test-suite\.log' - cat stdout | grep '[Tt]estsuite summary' - ) | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - else - cat stdout | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - fi ++ # Not a useless use of cat; see above comments "grep-nonprinting" ++ # requirement in 'test-init.sh'. + cat stdout | grep "$esc" && Exit 1 + : } for vpath in false :; do diff --cc t/color2.sh index 406b7e505,72087a03b..841c585d4 --- a/t/color2.sh +++ b/t/color2.sh @@@ -103,8 -119,36 +96,10 @@@ test_color ( test_no_color () { - # With make implementations that, like Solaris make, in case of errors - # print the whole failing recipe on standard output, we should content - # ourselves with a laxer check, to avoid false positives. - # Keep this in sync with lib/am/check.am:$(am__color_tests). - if $FGREP '= Xalways; then' stdout; then - # Extra verbose make, resort to laxer checks. - # Note that we also want to check that the testsuite summary is - # not unduly colorized. - ( - set +e # In case some grepped regex below isn't matched. - # Not a useless use of cat; see above comments "grep-nonprinting" - # requirement in 'test-init.sh'. - cat stdout | grep "TOTAL.*:" - cat stdout | grep "PASS.*:" - cat stdout | grep "FAIL.*:" - cat stdout | grep "SKIP.*:" - cat stdout | grep "XFAIL.*:" - cat stdout | grep "XPASS.*:" - cat stdout | grep "ERROR.*:" - cat stdout | grep 'test.*expected' - cat stdout | grep 'test.*not run' - cat stdout | grep '====' - cat stdout | grep '[Ss]ee .*test-suite\.log' - cat stdout | grep '[Tt]estsuite summary' - ) | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - else - cat stdout | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - fi ++ # Not a useless use of cat; see above comments "grep-nonprinting" ++ # requirement in 'test-init.sh'. + cat stdout | grep "$esc" && Exit 1 + : } cat >expect-make <<'END' diff --cc t/parallel-tests-no-color-in-log.sh index 2f8d3d7c0,01e1bdd32..2f19fbc64 --- a/t/parallel-tests-no-color-in-log.sh +++ b/t/parallel-tests-no-color-in-log.sh @@@ -56,7 -46,9 +46,9 @@@ $AUTOMAKE --add-missin ./configure mv config.log config-log # Avoid possible false positives below. -AM_COLOR_TESTS=always $MAKE -e check && Exit 1 +$MAKE check AM_COLOR_TESTS=always && Exit 1 - $FGREP "$esc" *.log && Exit 1 + # Not a useless use of cat; see above comments "grep-nonprinting" + # requirement in 'test-init.sh'. + cat *.log | grep "$esc" && Exit 1 : diff --cc t/tap-color.sh index 0999504f0,5a8e1bc37..6773b2800 --- a/t/tap-color.sh +++ b/t/tap-color.sh @@@ -124,12 -117,39 +117,14 @@@ test_color ( test_no_color () { - # With make implementations that, like Solaris make, in case of errors - # print the whole failing recipe on standard output, we should content - # ourselves with a laxer check, to avoid false positives. - # Keep this in sync with lib/am/check.am:$(am__color_tests). - if $FGREP '= Xalways; then' stdout; then - # Extra verbose make, resort to laxer checks. - # But we also want to check that the testsuite summary is not unduly - # colorized. - ( - set +e # In case some grepped regex below isn't matched. - # Not a useless use of cat; see above comments "grep-nonprinting" - # requirement in 'test-init.sh'. - cat stdout | grep "TOTAL.*:" - cat stdout | grep "PASS.*:" - cat stdout | grep "FAIL.*:" - cat stdout | grep "SKIP.*:" - cat stdout | grep "XFAIL.*:" - cat stdout | grep "XPASS.*:" - cat stdout | grep "ERROR.*:" - cat stdout | grep "^#" - cat stdout | grep 'test.*expected' - cat stdout | grep 'test.*not run' - cat stdout | grep '====' - cat stdout | grep '[Ss]ee .*test-suite\.log' - cat stdout | grep '[Tt]estsuite summary' - ) | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - else - cat stdout | grep "$esc" && Exit 1 - : For shells with broken 'set -e' - fi ++ # Not a useless use of cat; see above comments "grep-nonprinting" ++ # requirement in 'test-init.sh'. + cat stdout | grep "$esc" && Exit 1 + : } +# It should be possible to override AM_COLOR_TESTS also from the +# environment. # Forced colorization should take place also with non-ANSI terminals; # hence the "TERM=dumb" definition. TERM=dumb AM_COLOR_TESTS=always $MAKE check >stdout \