[ng] refactor: read 'serial-tests.am' more verbatim
* lib/am/serial-tests.am: Adjust to use the "ifeq" built-in and
the internal variable '$(am.conf.handle-exeext)' instead of the
transform '%?HANDLE-EXEEXT%'.
* automake.in (handle_tests): Use '&almost_verbatim' rather than
'&file_contents' to read and process 'serial-tests.am'.
We can do so now that the 'parallel-tests.am' file is now
processed "almost verbatim" by Automake.
* lib/am/parallel-tests.am (check-TESTS): Here, use "ifeq" to check
whether $(AM_LAZY_CHECK) is "yes". This removes the need for ...
(am__remove_if_not_lazy_check): ... this variable, which has thus
been removed.
[ng] refactor: read 'parallel-tests.am' more verbatim
* automake.in (handle_tests): Process 'parallel-tests.am' with
'&almost_verbatim', not with '&file_contents'. As a related
change, we now *must* include 'color-tests.am' explicitly.
* lib/am/parallel-tests.am: Related adjustments. In particular,
remove use of explicit '!' verbatim-escaping of some sections,
and do not try to include 'color-tests.am'.
* lib/am/serial-tests.am: Do not include 'color-tests.am'.
[ng] check: reduce code duplication in some auxiliary variables
* lib/am/parallel-tests.am (am.test-suite.awk-trs-loop/BEGIN,
am.test-suite.awk-trs-loop/END, am.test-suite.awk-functions): New, hold
fragments of awk scripts extracted from (and common to) ...
(am__create_global_log, am__count_test_result): ... these variables.
Some related improvements to comments, and minor reformatting.
[ng] header vars: prefer '#' comments over '##' ones; improve few comments
* lib/am/header-vars.am: Here, throughout the file. This will make it
easier for someone reading the generated Makefile.in to understand what
it's going on. And improve/fix few existing comments while at it.
[ng] header vars: fix comment on why DESTDIR is not explicitly initialized
* lib/am/header-vars.am (DESTDIR): It's because we want to allow it to
be defined from the environment, for compatibility with mainline Automake.
For better clarity, define an (empty) default with the line "DESTDIR ?=",
instead of with the commented-out definition "# DESTDIR=".
The comments to that variable claimed that "set +e" in the shell was
an XSI extension, but the Automake testsuite since at least 1.11 (which
was run several times with some weak, non-POSIX vendor /bin/sh) used it
in the test script, with not problem whatsoever. So we don't actually
need the safety net provided by $(am__sh_e_setup): we can simply use
'set +e' in the relevant recipes.
* master: (39 commits)
typofix: in a comment in Makefile.am
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
...
+ Extra non-trivial edits:
* Makefile.am (perf_TESTS): Define here, as 't/list-of-tests.mk' is gone
in the 'ng/master' branch.
(all_TESTS): Don't list tests in 't/perf'; this reflects the removal of
$(perf_TESTS) from $(handwritten_TESTS) done in 't/list-of-tests.mk' on
the 'master' branch.
* t/depcomp-shuffle.sh: Source 'test-lib.sh', not './defs-static'.
Also, simply use:
. depcomp-shuffle.sh
instead of:
. "$am_testauxdir"/depcomp-shuffle.sh
to source the helper shell script 'depcomp-shuffle.sh'. See recent
commit 'v1.12.2-49-g42fb45b' for a rationale.
* t/depcomp-shuffle-sub-vpath.sh: Likewise.
* t/depcomp-shuffle-sub.sh: Likewise.
* t/depcomp-shuffle-vpath.sh: Likewise.
* maint: (38 commits)
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
test init: refactor: new function 'process_requirements'
...
* testsuite-refactor: (33 commits)
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
test init: refactor: new function 'process_requirements'
...
tests: never source test-defs.sh directly, source test-lib.sh instead
After the recent re-organization, sourcing 'test-defs.sh' directly might
not work well and cause spurious failures or other unexpected behaviours.
We should source 'test-lib.sh' instead, which contains not more direct
code execution (only definition of shell variables/functions, or sourcing
of other '*.sh' with the same property), is protected against multiple
inclusions, and sources 'test-defs.sh' automatically in in a proper way.
* t/testsuite-summary-count.sh, t/tap-summary.sh, t/tap-summary-color.sh,
t/testsuite-summary-color.sh: Source 'test-lib.sh', not 'test-defs.sh'.
* gen-testsuite-part: Likewise, in the generated wrapper scripts.
* runtest.in: Here, similarly to what is done by AM_TESTS_ENVIRONMENT
in Makefile.am, unset variables that should be under the complete control
of the test framework, and that could create havoc if inherited from the
environment. This remove the need to check against possible environment
"pollution" ...
* t/ax/test-defs.in: ... in here.
* Makefile.am (AM_TESTS_ENVIRONMENT): Add a comment about the need of
synchronization with 'runtest.in'.
* t/self-check-env-sanitize.tap: Remove as obsolete.
* t/list-of-tests.mk: Adjust.
* t/self-check-tap.sh: This. The recent reorganization and code
moving between 'test-defs.sh' and 'test-lib.sh' has made it
brittle and prone to failures. Since the usefulness of this self
check is extremely limited, it's not worth trying to fix it. Just
remove it.
* t/list-of-tests.mk: Adjust.
And similarly, "am_using_tap=no" -> "am_test_protocol=none".
The new '$am_test_protocol' name is clearer, and will allow the easy
addition of further test protocols in the future. This is not truly
relevant for automake, but we are trying to make some parts of our
testsuite framework as general as possible, in view of a future move
to a more generic project like Gnulib.
* Makefile.am, t/ax/am-test-lib.sh, t/ax/test-defs.in, t/ax/test-lib.sh,
t/self-check-env-sanitize.tap, t/self-check-tap.sh: Adjust.
* syntax-checks.mk (sc_tests_obsolete_variables): Add 'am_using_tap' to
the list of obsolete variables to check against.
tests: protect test libs against multiple inclusion
* t/ax/test-lib.sh, t/ax/am-test-lib.sh: Return early if already sourced.
Use the witness variables '$test_lib_sourced' and '$am_test_lib_sourced',
respectively, for this purpose.
* runtest.in, Makefile.am (AM_TESTS_ENVIRONMENT): Unset 'test_lib_sourced'
and 'am_test_lib_sourced', to avoid interferences from the environment.
test defs: no need to re-add $srcdir/t/ax to $PATH
* t/ax/test-defs.sh: Here. That is already done by both 'runtest' and
AM_TESTS_ENVIRONMENT, and the presence of '$srcdir/t/ax' in $PATH is
anyway required, after the recent changes, for our testsuite framework
to work at all.
tests: split test libs into "generic" and "automake-specific"
This is the first step in the quest to merge the generically useful
parts of our test suite framework in a more generic project, like
Gnulib. Time will tell if we'll succeed, and whether the success will
be worth the extra hassle.
* t/ax/test-init.sh: Split out ...
* t/ax/am-test-lib.sh, t/ax/test-lib.sh: ... into these two tests.
* defs, Makefile.am: Adjust.
test setup: merge definitions of function for simple tests
* t/ax/plain-functions.sh: Delete, moving the definitions of ...
($stderr_fileno_): ... this variable ...
(warn_, fail_, skip_, skip_all_, fatal_, framework_failure_): ... and
these functions ...
* t/ax/test-init.sh: ... in here. This allow us to use those functions
earlier in this file (instead of having to duplicate their behaviour
with inlined code). The TAP-based tests are still able to override these
functions later to their TAP-enhanced equivalents when 'tap-functions.sh'
is sourced.
* Makefile.am (dist_noinst_DATA): Remove 't/ax/plain-functions.sh'.
* t/ax/tap-functions.sh: Adjust a comment.
* t/ax/test-init.sh (am_test_setup): Here.
Call it from the main code. Remove other calls of functions and
settings that are now duly called by 'am_test_setup'.
configure: testsuite shell set exit traps in shell functions
* configure.ac: Check that the shell selected to run the test
scripts can set an exit trap in a shell function, without having
that trap executed at the termination of the function rather
than of the scripts. According to the Autoconf manual, at least
AUX 5.3 /bin/sh suffers of such a bug.
test init: remove a couple of paranoid sanity checks
* t/ax/test-init.sh: Here, about 'am_top_srcdir' and 'am_top_builddir';
the values of those variables are AC_SUBST'd, so it's nigh impossible
for the to be wrong or get "messed up".
Now that the early $PATH setup in both 'runtest' and AM_TESTS_ENVIRONMENT
allow the '.' built-in to find to-be-sourced shell scripts in the 't/ax/'
directory automatically, we can simplify several usages like:
. "$am_testauxdir"/foo.sh
to just:
. foo.sh
Also, because our test scripts run with the 'errexit' flag active, and
because POSIX mandates that, when the '.' built-in is used,
... if no readable file is found, a non-interactive shell shall abort ...
We can do so with minimal churn, now that the early setup of $PATH in
both 'runtest' and AM_TESTS_ENVIRONMENT allow the '.' built-in to find
to-be-sourced shell scripts in 't/ax' automatically, both in in-tree
and VPATH builds.
With this change, we take another step forward a more rational and
"segregated" organization of our testsuite framework.
tests: update PATH early from AM_TESTS_ENVIRONMENT and runtest
Because the POSIX standards mandate that sourcing a file with ". FILE"
will cause FILE (assuming it is a relative containing no slashes) to be
looked for in PATH, such a move will allow us to simplify our sourcing
of shell testing libraries like (currently) 't/ax/test-init.sh' and
'defs', and to modularize and re-organize them better in the feature,
with minimal churn.
* runtest.in, Makefile.am (AM_TEST_ENVIRONMENT): Prepend $(abs_srcdir)/t/ax
and $(abs_builddir)/t/ax (in that order) to PATH; do so avoiding repetition
in the common case of non-VPATH builds.
runtest: export $srcdir for use by the test scripts
This is mostly a preparatory change in view of future ones. But it
also make the behaviour of runtest more similar to that of the test
harness used by "make check".
* runtest.in: Export srcdir (was already defined to '@srcdir@').
* gen-testsuite-part ($testauxdir): New, defined to "$testdir/ax".
Use it throughout, for better encapsulation and less duplication.
($auxdir): Remove as unneeded.
gen-tests: less hard-coding of 't' as the test directory
This will make life easier for us in case we decide to rename the
testsuite directory in the future (perhaps to something like 'tests',
which is somewhat less cryptic than the current 't').
* gen-testsuite-part ($testdir): New variable, used throughout instead
of hard-coding the name of the test directory to 't'.
All the script adjusted to make proper use of '$testdir'.
($auxdir): Move its definition earlier, for consistency with the one
of '$testdir'.
build: auxiliary testsuite files/scripts built by "make all"
This will allow the developers to run a tests case by hand out of
a newly extracted tarball simply doing:
$ ./configure && make
$ ./runtest t/the-test-case.sh
while before this change one has to resort to:
$ ./configure && make && make check TESTS=
$ ./runtest t/the-test-case.sh
or, with some non-GNU makes, even:
$ ./configure && make && make check TESTS= AM_MAKEFLAGS="TESTS="
$ ./runtest t/the-test-case.sh
This come very handy sometimes, especially when doing one-shot
debugging. Admittedly not a big deal, but one less friction in
the build system is always nice.
* Makefile.am (check_SCRIPTS, dist_check_DATA, nodist_check_DATA):
Move their content to ...
(noinst_SCRIPTS, dist_noinst_DATA, nodist_noinst_DATA): ... these
variables.
Adjust comments.
* t/list-of-tests.mk (handwritten_TESTS): Don't list ($perf_TESTS) in
here anymore.
* Makefile.am (EXTRA_DIST): Add them explicitly.
(perf): New target, run the performance tests and save the logs of
the ones with unexpected results in the file 't/perf/test-suite.log',
which is thus ...
(PERF_TEST_SUITE_LOG): ... defined in this variable ...
(CLEANFILES): ... and added to this.
(test_subdirs): Remove 't/perf': the tests in there are no more
listed in $(TESTS).
* t/ax/test-init.sh: Don't explicitly skip "perf" test here.
runtest: pass *all* the given shell options to the test invocation
* runtest.in: Here. This didn't work previously because, in our
option parsing loop, we were mistakenly redefining '$shell_opts'
each time, instead of appending to it.
* defs-static.in: Here: do not state anymore that this file should execute
correctly with any system's /bin/sh shell, as this is not true anymore: the
testsuite assumes a POSIX shell throughout these days.
runtest: support option --shell, tell which shell should run the test
* runtest.in: Here. The same effect could have been obtained by
exporting AM_TEST_RUNNER_SHELL to the desired value, but the new
API is simpler and requires less typing and less thinking.
tests: avoid spurious failure when running as root
Fixes automake bug#12041.
* t/primary-prefix-couples-force-valid.sh: If run as root, don't expect
a "test -x" on a non-executable files to fail: for root, all files are
executable (as well as readable and writable), and at least on Solaris
10 that causes "test -x" to succeed also on non-executable files.
[ng] tests: fix spurious failure in 'color-tests2.sh'
* t/color-tests2.sh: Change the idioms used to specify the value of
'AM_COLOR_TESTS' to make. This is required since, in contrast with
what happens in the mainline Automake counterpart, this script doesn't
invoke make with the '-e' option.
* master:
tests: avoid non-textual output in a '.log' file
tags: refactor to reduce code duplication (2)
tags: refactor to reduce code duplication (1)
tests: better name for a couple of tests
check: support colorized testsuite output by default
tags: unify recursion for tags targets with that of "usual" targets
+ Extra non-trivial edits:
* automake.in (handle_tests): Drop the '%COLOR%' transform in the
processing of both 'serial-tests.am' and 'parallel-tests.am', which
have superseded the 'check.am' from mainline Automake.
* lib/am/color-tests.am: Remove use of the '%?COLOR%' transform;
just act as if it were unconditionally TRUE.
(am__tty_colors_dummy): Merge into ...
(am__tty_colors): ... this.
* t/dist-formats.tap: Here. Otherwise, some inferior awk implementations,
like OpenIndiana /usr/xpg4/bin/awk, could be confused and cause spurious
errors in the testsuite harness.
* lib/am/tags.am (am__define_uniq_tagged_files): New variable,
factor out some code common to ...
(ctags-am, tags-am, ID): ... the recipes of these targets.
* lib/am/header-vars.am (am.chars.empty): This, move earlier. This both
fixes the associated comments (which referred to a variable defined above
it as "defined below"), and ensures that this variable is not used before
being defined (not a big deal in practice, since it expands to empty; but
better be consistent).
[ng] vars: names of iterating vars in $(foreach) loops shorter & simpler
In GNU make, after a "$(foreach x,...)" loop, the make variable $(x):
- is restored to its original variable it had before the 'foreach'
loop, if it was already set;
- re-unset otherwise.
This behaviour is checked by the recent spy test 't/spy-foreach.sh'.
This means that, in most situations, there's no actual need to use
a "namespace safe" variable name for the iterating variable in a
"$(foreach ...)" loop, because that variable won't linger in the
environment after the loop anyway.
* lib/am/compile.am, lib/am/header-vars.am, lib/am/parallel-tests.am,
lib/am/serial-tests.am: Adjust to use simpler variables names in their
foreach loops; for example, simply 'v' instead of 'am__v', etc.
[ng] spy: iteration variables in "$(foreach ...)" loops are temporary
* t/spy-foreach.sh: New test. Check that, after a "$(foreach x,...)"
loop, the make variable $(x) is:
- restored to the original variable it had before the 'foreach'
loop, if it was already set; and
- is re-unset otherwise.
The test 't/memoize.sh' was producing a '.log' file with few overly-long
lines (more than 12k characters long) and, when Solaris XPG4 awk was in
use, that was causing the 'test-driver.sh' script to experience a spurious
failure:
/usr/xpg4/bin/awk: line 382 (B): Record too long (LIMIT: 19999 bytes)
tap-driver.sh: fatal: I/O or internal error
* t/memoize.sh: Temporary disable shell tracing and make output in a few
places, to prevent the resulting '.log' file to contain overly long lines.
* master:
tests: avoid a spurious failure with Solaris /bin/sh
tests: fix buglet in t/missing3.sh
self tests: avoid spurious failures on older bash
tests: remove t/parallel-tests-many.sh (fixes spurious failure)
typofix: in a comment in the automake script
news: minor fixlets and reordering
+ Extra non-trivial edits:
* t/parallel-tests-many.sh: Do not deleted; keep the correct and
improved version from the 'ng/master' branch.
* t/subpkg.sh: Here: do not expect the C compiler to be simply
named 'cc'. This was causing spurious failures when $CC was
overridden at configure time or though config.site.
* ng/many-tests:
[ng] check: in recipes, don't use pipe when redirection suffices
[ng] recheck: don't exceed command line limits, even with many failed tests
[ng] coverage: recheck with many failed tests
[ng] check: refactor for less duplication and better performances
[ng] check: use awk rather than grep+xargs to count test results
[ng] parallel-tests: do not exceed command line length limits
[ng] coverage: testing with lots of test scripts
tests: avoid a spurious failure with Solaris /bin/sh
The /bin/sh shell on Solaris is dumb enough not to set the exit
status to 127 after the execution of a non-existing command is
attempted:
$ /bin/sh -c 'nonesuch'; echo stat = $?
/bin/sh: nonesuch: not found
stat = 1
This means that the missing script, when run through that shell,
cannot discriminate between a real failure of a maintainer tool
and a failure due to its absence. This is not a big deal in
practice (especially because all the 'missing' invocations in
our Makefiles are done with $(SHELL), and that is almost surely
set by configure to a proper POSIX shell), but was causing an
annoying failure in our testsuite. Fix it.
* t/missing3.sh: If 'missing' is run with a /bin/sh shell suffering
from the just-described bug, skip the check that would spuriously
fail due to that bug.
* t/missing3.sh: Be sure to use 'run_cmd' also in the first invocation
of 'missing', to ensure the stderr will actually be saved in a file we
can later grep.
projects / thirdparty / automake.git / log
