* lib/autotest/general.m4 (HELP_TUNING_BEGIN): New diversion.
(HELP_TUNING, HELP_OTHER, HELP_END): Bump diversion numbers.
(AT_INIT): Accept
--color and --color=never|auto|always. If desired, colorize
test results and testsuite summary on standard output.
[HELP_TUNING]: Divert content instead to ...
[HELP_TUNING_BEGIN]: ... this diversion, m4_wrapped until the
end, when we know whether AT_COLOR_TESTS has been specified.
(AT_COLOR_TESTS): New macro, set the default for color to auto.
* doc/autoconf.texi (Writing Testsuites): Document it.
(testsuite Invocation): Document --color* options.
* tests/local.at: Call AT_COLOR_TESTS for Autoconf's testsuite.
* tests/autotest.at (color test results): New test, mirroring
color.test from Automake.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
+2010-06-17 Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
+
+ Autotest: enable colored test results.
+ * lib/autotest/general.m4 (HELP_TUNING_BEGIN): New diversion.
+ (HELP_TUNING, HELP_OTHER, HELP_END): Bump diversion numbers.
+ (AT_INIT): Accept
+ --color and --color=never|auto|always. If desired, colorize
+ test results and testsuite summary on standard output.
+ [HELP_TUNING]: Divert content instead to ...
+ [HELP_TUNING_BEGIN]: ... this diversion, m4_wrapped until the
+ end, when we know whether AT_COLOR_TESTS has been specified.
+ (AT_COLOR_TESTS): New macro, set the default for color to auto.
+ * doc/autoconf.texi (Writing Testsuites): Document it.
+ (testsuite Invocation): Document --color* options.
+ * tests/local.at: Call AT_COLOR_TESTS for Autoconf's testsuite.
+ * tests/autotest.at (color test results): New test, mirroring
+ color.test from Automake.
+ * NEWS: Update.
+
2010-06-15 Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
Avoid texinfo bug with backslashes in macro arguments.
** In configure scripts, loading CONFIG_SITE no longer searches PATH,
and problems in loading the configuration site files are diagnosed.
+** Autotest testsuites may optionally provide colored test results.
+
* Major changes in Autoconf 2.65 (2009-11-21) [stable]
Released by Eric Blake, based on git versions 2.64.*.
@command{testsuite} and in @samp{testsuite --version}.
@end defmac
+@defmac AT_COLOR_TESTS
+@atindex{COLOR_TESTS}
+Enable colored test results by default when the output is connected to
+a terminal.
+@end defmac
+
@defmac AT_TESTED (@var{executables})
@atindex{TESTED}
Log the file name and answer to @option{--version} of each program in
Force more verbosity in the detailed output of what is being done. This
is the default for debugging scripts.
+@item --color
+@itemx --color@r{[}=never@r{|}auto@r{|}always@r{]}
+Enable colored test results. Without an argument, or with @samp{always},
+test results will be colored. With @samp{never}, color mode is turned
+off. Otherwise, if either the macro @code{AT_COLOR_TESTS} is used by
+the testsuite author, or the argument @samp{auto} is given, then test
+results are colored if standard output is connected to a terminal.
+
@item --debug
@itemx -d
Do not remove the files after a test group was performed ---but they are
# - HELP_MODES
# Modes help text. Additional modes can be appended as self-contained
# cat'd here-docs as generated by AS_HELP_STRING.
+# - HELP_TUNING_BEGIN
+# Tuning help text. This is for Autotest-provided text.
# - HELP_TUNING
-# Tuning help text. Additional tuning options can be appended as
+# Additional tuning options' help text can be appended here as
# self-contained cat'd here-docs as generated by AS_HELP_STRING.
# - HELP_OTHER
# User help can be appended to this as self-contained cat'd here-docs.
m4_define([_m4_divert(PARSE_ARGS_END)], 202)
m4_define([_m4_divert(HELP)], 300)
m4_define([_m4_divert(HELP_MODES)], 301)
-m4_define([_m4_divert(HELP_TUNING)], 302)
-m4_define([_m4_divert(HELP_OTHER)], 303)
-m4_define([_m4_divert(HELP_END)], 304)
+m4_define([_m4_divert(HELP_TUNING_BEGIN)], 302)
+m4_define([_m4_divert(HELP_TUNING)], 303)
+m4_define([_m4_divert(HELP_OTHER)], 304)
+m4_define([_m4_divert(HELP_END)], 305)
m4_define([_m4_divert(VERSION)], 350)
m4_define([_m4_divert(VERSION_NOTICES)], 351)
m4_define([_m4_divert(VERSION_END)], 352)
[m4_ifdef([AT_ingroup], [m4_fatal([missing AT_CLEANUP detected])])dnl
m4_divert_text([DEFAULTS],
[
+# Whether to enable colored test results.
+at_color=m4_ifdef([AT_color], [AT_color], [no])
# List of the tested programs.
at_tested='m4_ifdef([AT_tested],
[m4_translit(m4_dquote(m4_defn([AT_tested])), [ ], m4_newline)])'
at_clean=:
;;
+ --color )
+ at_color=always
+ ;;
+ --color=* )
+ case $at_optarg in
+ no | never | none) at_color=never ;;
+ auto | tty | if-tty) at_color=auto ;;
+ always | yes | force) at_color=always ;;
+ *) at_optname=`echo " $at_option" | sed 's/^ //; s/=.*//'`
+ AS_ERROR([unrecognized argument to $at_optname: $at_optarg]) ;;
+ esac
+ ;;
+
--debug | -d )
at_debug_p=:
;;
# Sort the tests, removing duplicates.
at_groups=`AS_ECHO(["$at_groups"]) | tr ' ' "$as_nl" | sort -nu`
fi
+
+if test x"$at_color" = xalways \
+ || { test x"$at_color" = xauto && test -t 1; }; then
+ at_red=`printf '\033@<:@0;31m'`
+ at_grn=`printf '\033@<:@0;32m'`
+ at_lgn=`printf '\033@<:@1;32m'`
+ at_blu=`printf '\033@<:@1;34m'`
+ at_std=`printf '\033@<:@m'`
+else
+ at_red= at_grn= at_lgn= at_blu= at_std=
+fi
m4_divert_pop([PARSE_ARGS_END])dnl
m4_divert_push([HELP])dnl
-l, --list describes all the tests, or the selected TESTS
_ATEOF
m4_divert_pop([HELP_MODES])dnl
-m4_divert_push([HELP_TUNING])dnl
+m4_wrap([m4_divert_push([HELP_TUNING_BEGIN])dnl
cat <<_ATEOF || at_write_fail=1
dnl extra quoting prevents emacs whitespace mode from putting tabs in output
Execution tuning:
-C, --directory=DIR
[ change to directory DIR before starting]
+ --color[[=never|auto|always]]
+[ ]m4_ifdef([AT_color],
+ [disable colored test results, or enable even without terminal],
+ [enable colored test results on terminal, or always])
-j, --jobs[[=N]]
[ Allow N jobs at once; infinite jobs with no arg (default 1)]
-k, --keywords=KEYWORDS
[ default for debugging scripts]
-x, --trace enable tests shell tracing
_ATEOF
-m4_divert_pop([HELP_TUNING])dnl
+m4_divert_pop([HELP_TUNING_BEGIN])])dnl
m4_divert_push([HELP_END])dnl
cat <<_ATEOF || at_write_fail=1
at_msg="UNEXPECTED PASS"
at_res=xpass
at_errexit=$at_errexit_p
+ at_color=$at_red
;;
no:0)
at_msg="ok"
at_res=pass
at_errexit=false
+ at_color=$at_grn
;;
*:77)
at_msg='skipped ('`cat "$at_check_line_file"`')'
at_res=skip
at_errexit=false
+ at_color=$at_blu
;;
no:* | *:99)
at_msg='FAILED ('`cat "$at_check_line_file"`')'
at_res=fail
at_errexit=$at_errexit_p
+ at_color=$at_red
;;
yes:*)
at_msg='expected failure ('`cat "$at_check_line_file"`')'
at_res=xfail
at_errexit=false
+ at_color=$at_lgn
;;
esac
echo "$at_res" > "$at_job_dir/$at_res"
# In parallel mode, output the summary line only afterwards.
if test $at_jobs -ne 1 && test -n "$at_verbose"; then
- AS_ECHO(["$at_desc_line $at_msg"])
+ AS_ECHO(["$at_desc_line $at_color$at_msg$at_std"])
else
# Make sure there is a separator even with long titles.
- AS_ECHO([" $at_msg"])
+ AS_ECHO([" $at_color$at_msg$at_std"])
fi
at_log_msg="$at_group. $at_desc ($at_setup_line): $at_msg"
case $at_status in
at_result="$at_result $at_were run, one failed"
fi
at_result="$at_result unexpectedly and inhibited subsequent tests."
+ at_color=$at_red
else
# Don't you just love exponential explosion of the number of cases?
+ at_color=$at_red
case $at_xpass_count:$at_fail_count:$at_xfail_count in
# So far, so good.
- 0:0:0) at_result="$at_result $at_were successful." ;;
- 0:0:*) at_result="$at_result behaved as expected." ;;
+ 0:0:0) at_result="$at_result $at_were successful." at_color=$at_grn ;;
+ 0:0:*) at_result="$at_result behaved as expected." at_color=$at_lgn ;;
# Some unexpected failures
0:*:0) at_result="$at_result $at_were run,
esac
if test $at_unexpected_count = 0; then
- echo "$at_result"
+ echo "$at_color$at_result$at_std"
echo "$at_result" >&AS_MESSAGE_LOG_FD
else
- echo "ERROR: $at_result" >&2
+ echo "${at_color}ERROR: $at_result$at_std" >&2
echo "ERROR: $at_result" >&AS_MESSAGE_LOG_FD
{
echo
[m4_default([$2], [m4_newline])([$1])])])# AT_COPYRIGHT
+# AT_COLOR_TESTS
+# --------------
+# Enable colored test results if standard error is connected to a terminal.
+m4_define([AT_COLOR_TESTS],
+[m4_define([AT_color], [auto])])
+
# AT_SETUP(DESCRIPTION)
# ---------------------
# Start a group of related tests, all to be executed in the same subshell.
AT_CLEANUP
+
+# --color
+AT_CHECK_AT_TEST([colored test results],
+ [AT_CHECK([:])
+ AT_CLEANUP
+ AT_SETUP([fail])
+ AT_CHECK([exit 1])
+ AT_CLEANUP
+ AT_SETUP([xpass])
+ AT_XFAIL_IF([:])
+ AT_CHECK([:])
+ AT_CLEANUP
+ AT_SETUP([xfail])
+ AT_XFAIL_IF([:])
+ AT_CHECK([exit 1])
+ AT_CLEANUP
+ AT_SETUP([skip])
+ AT_CHECK([exit 77])
+ AT_CLEANUP
+ AT_SETUP([hardfail])
+ AT_XFAIL_IF([:])
+ AT_CHECK([exit 99])
+], [], [], [], [], [], [
+
+TERM=ansi
+export TERM
+
+red=`printf '\033@<:@0;31m'`
+grn=`printf '\033@<:@0;32m'`
+lgn=`printf '\033@<:@1;32m'`
+blu=`printf '\033@<:@1;34m'`
+std=`printf '\033@<:@m'`
+
+# Check that grep can parse nonprinting characters.
+# BSD 'grep' works from a pipe, but not a seekable file.
+# GNU or BSD 'grep -a' works on files, but is not portable.
+AT_CHECK([case `echo "$std" | grep .` in #'' restore font-lock
+ $std) :;;
+ *) Exit 77;;
+ esac], [], [ignore], [],
+ [echo "grep can't parse nonprinting characters" >&2])
+
+if echo 'ab*c' | grep -F 'ab*c' >/dev/null 2>&1; then
+ FGREP="grep -F"
+else
+ FGREP=fgrep
+fi
+
+# No color.
+AT_CHECK([$CONFIG_SHELL ./micro-suite], [1], [stdout], [stderr])
+for color in "$red" "$grn" "$lgn" "$blu"; do
+ AT_CHECK([cat stdout stderr | $FGREP "$color"], [1])
+done
+
+# Color of test group results.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always], [1], [stdout], [stderr])
+AT_CHECK([cat stdout | grep " only " | $FGREP "$grn"], [], [ignore])
+AT_CHECK([cat stdout | grep " fail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " xfail " | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([cat stdout | grep " xpass " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " skip " | $FGREP "$blu"], [], [ignore])
+AT_CHECK([cat stdout | grep " hardfail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+
+# The summary is green if all tests were successful, light green if all
+# behaved as expected, and red otherwise.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k skip],
+ [0], [stdout])
+AT_CHECK([cat stdout | grep 'test.*successful' | $FGREP "$grn"],
+ [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k xfail -k skip],
+ [0], [stdout])
+AT_CHECK([cat stdout | grep 'as expected' | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k fail],
+ [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k xpass],
+ [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k hardfail],
+ [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+# Reset color on verbose output.
+printf %s\\n "$std"
+], [1])
+
+
## ------------------- ##
## srcdir propagation. ##
## ------------------- ##
# Programs this package provides
AT_TESTED([autom4te autoconf autoheader autoupdate autoreconf ifnames])
+# Enable colored test output.
+AT_COLOR_TESTS
## ---------------- ##
## Utility macros. ##
projects / thirdparty / autoconf.git / commitdiff
