Simple Tests
-* Script-based Testsuites:: Automake-specific concepts and terminology
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
* Serial Test Harness:: Older (and obsolescent) serial test harness
* Parallel Test Harness:: Generic concurrent test harness
* Introduction to TAP::
* Use TAP with the Automake test harness::
* Incompatibilities with other TAP parsers and drivers::
-* Links and external resources::
+* Links and external resources on TAP::
Custom Test Drivers
@section Simple Tests
@menu
-* Script-based Testsuites:: Automake-specific concepts and terminology
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
* Serial Test Harness:: Older (and obsolescent) serial test harness
* Parallel Test Harness:: Generic concurrent test harness
@end menu
-@node Script-based Testsuites
-@subsection Script-based Testsuites
+@node Scripts-based Testsuites
+@subsection Scripts-based Testsuites
If the special variable @code{TESTS} is defined, its value is taken to be
a list of programs or scripts to run in order to do the testing. Under
@c Keep in sync with tests-environment-backcompat.test.
@example
AM_TESTS_ENVIRONMENT = \
-## Some environment initializations are kept in a separate shell file
-## `tests-env.sh', which can make it easier to also run tests from
-## the command line.
+## Some environment initializations are kept in a separate shell
+## file `tests-env.sh', which can make it easier to also run tests
+## from the command line.
. $(srcdir)/tests-env.sh; \
## On Solaris, prefer more POSIX-compliant versions of the standard
## tools by default.
PATH=/usr/xpg4/bin:$$PATH; export PATH; \
fi;
@c $$ restore font-lock
-## With this, the test scripts will be able to print diagnostic messages
-## to the original standard error stream, even if the test driver
-## redirects the stderr of the test scripts to a log file before executing
-## them.
+## With this, the test scripts will be able to print diagnostic
+## messages to the original standard error stream, even if the test
+## driver redirects the stderr of the test scripts to a log file
+## before executing them.
AM_TESTS_FD_REDIRECT = 9>&2
@end example
@trindex check-html
@vindex RST2HTML
@vindex TEST_SUITE_HTML
+@noindent
With @code{make check-html}, the log files may be converted from RST
(reStructuredText, see @uref{http://docutils.sourceforge.net/@/rst.html})
to HTML using @samp{RST2HTML}, which defaults to @command{rst2html} or
@file{.log} file in HTML format, like the @code{check-html} target.
@end itemize
+@noindent
In order to guarantee an ordering between tests even with @code{make
-j@var{N}}, dependencies between the corresponding @file{.log} files
may be specified through usual @command{make} dependencies. For example,
It's very important to note that, even when using custom test drivers,
most of the infrastructure described in the previous section about the
-the parallel harness remains in place; this includes:
+parallel harness remains in place; this includes:
@itemize
@item
The developer-reserved variable @code{AM_LOG_DRIVER_FLAGS} and the
user-reserved variable @code{LOG_DRIVER_FLAGS} can be used to define
-flags that will be passed to each invocation of @code{LOG_DRIVER}
-(with the user-defined flags obviously taking precedence over the
-developer-reserved ones). Similarly, for each extension @var{ext}
+flags that will be passed to each invocation of @code{LOG_DRIVER},
+with the user-defined flags obviously taking precedence over the
+developer-reserved ones. Similarly, for each extension @var{ext}
declared in @code{TEST_EXTENSIONS}, flags listed in
@code{AM_@var{ext}_LOG_DRIVER_FLAGS} and
@code{@var{ext}_LOG_DRIVER_FLAGS} will be passed to
The main characteristic of these APIs is that they are designed to share
as much infrastructure, semantics, and implementation details as possible
-with the parallel test harness and its default driver. So everything
-said in the previous section should apply here too, unless explicitly
-stated otherwise.
+with the parallel test harness and its default driver.
@menu
* Command-line arguments for test drivers::
mostly meant to be used in console reports about testsuite advancements and
results (@pxref{Testsuite progress output}).
@item --log-file=@file{@var{PATH}.log}
-The @file{.log} file the test driver must create. If it has a directory
-component (as in e.g., @file{sub/foo.log}), the test harness will ensure
-that such directory exist @emph{before} the test driver is called.
+The @file{.log} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
+@file{sub/foo.log}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
@item --trs-file=@file{@var{PATH}.trs}
-The @file{.trs} file the test driver must create. If it has a directory
-component (as in e.g., @file{sub/foo.trs}), the test harness will ensure
-that such directory exist @emph{before} the test driver is called.
+The @file{.trs} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
+@file{sub/foo.trs}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
@item --color-tests=@{yes|no@}
Whether the console output should be colorized or not (@pxref{Simple
tests and color-tests}, to learn when this option gets activated and
Explicitly terminate the list of options.
@end table
-The first of the remaining arguments passed to the test driver is the
-program to be run, and the other arguments are command-line options
-and arguments for this program.
+@noindent
+The first non-option argument passed to the test driver is the program to
+be run, and all the following ones are command-line options and arguments
+for this program.
Note that the exact semantics attached to the @option{--color-tests},
@option{--expect-failure} and @option{--enable-hard-errors} options are
* Introduction to TAP::
* Use TAP with the Automake test harness::
* Incompatibilities with other TAP parsers and drivers::
-* Links and external resources::
+* Links and external resources on TAP::
@end menu
@node Introduction to TAP
@emph{TODO}: there's surely something else ...
@end itemize
-@node Links and external resources
-@subsection Links and external resources
+@node Links and external resources on TAP
+@subsection Links and external resources on TAP
@noindent
Here are some links to more extensive official or third-party
-documentation and resources:
+documentation and resources about the TAP protocol and related
+tools and libraries.
@itemize @bullet
@item
@uref{http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod,