[ng] refactor: consider $(EXEEXT) used if non-empty at make runtime
To check whether we should handle $(EXEEXT), simply check for
"ifdef EXEEXT" in our Makefile fragments, rather than also checking
whether EXEEXT has been set as a configure-time AC_SUBST. This
simplifies the logic, and move it completely at make runtime, rather
than having it split between automake runtime (to decide whether
EXEEXT is AC_SUBST) and make runtime (to decide whether $(EXEEXT) is
empty).
* lib/am/header-vars.am (EXEEXT): Default to empty. This is required
to avoid environment interferences.
(am.conf.handle-exeext): Define this no more.
* lib/am/parallel-tests.am, lib/am/serial-tests.am: Simply look
at $(EXEEXT), rather than using $(am.conf.handle-exeext), or the
'%?HANDLE-EXEEXT%' transform.
* automake.in (process_file): Don't define that transform anymore.
[ng] tests: do not run very expensive tests by default
Some tests in the Automake testsuite (especially those testing for bugs
w.r.t. command-line length limits) require a *long* time to run. For
example, the test 'parallel-tests-many.sh' (the most extreme example of
this issue) takes ~ 11 minutes to execute on a modern, blazingly fast
multi-core ppc64 system (64 cores at 3.5 GHz each).
This slow-down is unacceptable during routine runs of the testsuite,
that I do every bunch of commits (or every commit, for more tricky
changes). And is even less acceptable for the casual user that just
run the testsuite before installing Automake, maybe on an aging and
slower system (or maybe on Cygwin *shudder*).
So, let's follow the route of GNU coreutils here: declare some tests as
"expensive", and let them run only if called if the RUN_EXPENSIVE_TESTS
environment variable is set to "yes".
* test-lib.sh (expensive_): New function; used in a test, cause it to be
skipped unless the RUN_EXPENSIVE_TESTS variable is defined to "yes".
* t/parallel-tests-many.sh, t/testsuite-summary-count-many.sh,
t/dist-many.sh: Call 'expensive_'.
* runtest.in: Export RUN_EXPENSIVE_TESTS to "yes": if a test is called
directly through './runtest', it should be run even it it's expensive.
* Makefile.am (check-expensive, installcheck-expensive): New convenience
targets that run the testsuite with RUN_EXPENSIVE_TESTS exported to
"yes", so that the expensive tests are not skipped.
* t/README: Update.
[ng] serial-tests: remove one level of indentation in a recipe
* lib/am/serial-tests.am (check-TESTS): Here. And add a few comments.
These changes make the already quite complicated recipe slightly easier
to follow.
* lib/am/parallel-tests.am (am.test-suite.handle-suffix.helper):
New, extracted from ...
(am.test-suite.handle-suffix): ... this, which now uses it.
(am__tpfx): Remove, inlined into 'am.test-suite.handle-suffix'.
* t/parallel-tests-internals.sh: Remove as obsolete.
[ng] refactor: use new function 'am.vars.is-undef' ...
* lib/am/header-vars.am (SUBDIRS): ... in the definition of this (instead
of using hand-rolled almost-equivalent) ...
($(1)LOG_DRIVER, TEST_EXTENSIONS): ... and of these (instead of resorting
to weaker and unsafer '?=' assignment).
Sometimes, in our makefiles, we want to assign a default value to
a variable that might not have been assigned yet. One might think
that using the GNU make assignment '?=' is enough:
VAR ?= DEFAULT-VALUE
But alas, such a usage allows interferences from the environment; i.e.,
if an environment variable named 'VAR' is defined to, say, BAD-VALUE,
the construct above will result in the $(VAR) make variable being
defined to BAD-VALUE, not to DEFAULT-VALUE.
Use the 'am.vars.is-undef' function to avoid this situation. It tells
whether the given make variable is not "safely" defined, i.e., either
undefined, or defined to a value that is inherited from the environment.
With such a tool, we can safely declare default values for variables
that avoids environmental interferences, as follow:
ifeq ($(call am.vars.is-undef,VAR),yes)
VAR = DEFAULT-VALUE
endif
* lib/am/header-vars.am (am.vars.is-undef): New.
* t/internals.tap: Test it, and add few sanity checks.
[ng] tests: simplify munging of header-vars.am ...
* t/am-xargs-map.sh, t/ensure-dir-exists.sh, t/internals.tap,
t/memoize.tap: ... here, since that files no more contains
'@foo@' substitutions nor '?COND?' transforms.
[ng] refactor: read 'color-tests.am' more verbatim
* lib/am/color-tests.am (am.test-suite.tty-colors): Define with the
'define' built-in rather than with '='.
* automake.in (handle_tests): Use '&almost_verbatim' rather than
'&file_contents' to read and process 'color-tests.am'.
[ng] refactor: read 'serial-tests.am' more verbatim
* lib/am/serial-tests.am: Adjust to use the "ifeq" built-in and
the internal variable '$(am.conf.handle-exeext)' instead of the
transform '%?HANDLE-EXEEXT%'.
* automake.in (handle_tests): Use '&almost_verbatim' rather than
'&file_contents' to read and process 'serial-tests.am'.
We can do so now that the 'parallel-tests.am' file is now
processed "almost verbatim" by Automake.
* lib/am/parallel-tests.am (check-TESTS): Here, use "ifeq" to check
whether $(AM_LAZY_CHECK) is "yes". This removes the need for ...
(am__remove_if_not_lazy_check): ... this variable, which has thus
been removed.
[ng] refactor: read 'parallel-tests.am' more verbatim
* automake.in (handle_tests): Process 'parallel-tests.am' with
'&almost_verbatim', not with '&file_contents'. As a related
change, we now *must* include 'color-tests.am' explicitly.
* lib/am/parallel-tests.am: Related adjustments. In particular,
remove use of explicit '!' verbatim-escaping of some sections,
and do not try to include 'color-tests.am'.
* lib/am/serial-tests.am: Do not include 'color-tests.am'.
[ng] check: reduce code duplication in some auxiliary variables
* lib/am/parallel-tests.am (am.test-suite.awk-trs-loop/BEGIN,
am.test-suite.awk-trs-loop/END, am.test-suite.awk-functions): New, hold
fragments of awk scripts extracted from (and common to) ...
(am__create_global_log, am__count_test_result): ... these variables.
Some related improvements to comments, and minor reformatting.
[ng] header vars: prefer '#' comments over '##' ones; improve few comments
* lib/am/header-vars.am: Here, throughout the file. This will make it
easier for someone reading the generated Makefile.in to understand what
it's going on. And improve/fix few existing comments while at it.
[ng] header vars: fix comment on why DESTDIR is not explicitly initialized
* lib/am/header-vars.am (DESTDIR): It's because we want to allow it to
be defined from the environment, for compatibility with mainline Automake.
For better clarity, define an (empty) default with the line "DESTDIR ?=",
instead of with the commented-out definition "# DESTDIR=".
The comments to that variable claimed that "set +e" in the shell was
an XSI extension, but the Automake testsuite since at least 1.11 (which
was run several times with some weak, non-POSIX vendor /bin/sh) used it
in the test script, with not problem whatsoever. So we don't actually
need the safety net provided by $(am__sh_e_setup): we can simply use
'set +e' in the relevant recipes.
* master: (39 commits)
typofix: in a comment in Makefile.am
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
...
+ Extra non-trivial edits:
* Makefile.am (perf_TESTS): Define here, as 't/list-of-tests.mk' is gone
in the 'ng/master' branch.
(all_TESTS): Don't list tests in 't/perf'; this reflects the removal of
$(perf_TESTS) from $(handwritten_TESTS) done in 't/list-of-tests.mk' on
the 'master' branch.
* t/depcomp-shuffle.sh: Source 'test-lib.sh', not './defs-static'.
Also, simply use:
. depcomp-shuffle.sh
instead of:
. "$am_testauxdir"/depcomp-shuffle.sh
to source the helper shell script 'depcomp-shuffle.sh'. See recent
commit 'v1.12.2-49-g42fb45b' for a rationale.
* t/depcomp-shuffle-sub-vpath.sh: Likewise.
* t/depcomp-shuffle-sub.sh: Likewise.
* t/depcomp-shuffle-vpath.sh: Likewise.
* maint: (38 commits)
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
test init: refactor: new function 'process_requirements'
...
* testsuite-refactor: (33 commits)
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
test init: refactor: new function 'process_requirements'
...
tests: never source test-defs.sh directly, source test-lib.sh instead
After the recent re-organization, sourcing 'test-defs.sh' directly might
not work well and cause spurious failures or other unexpected behaviours.
We should source 'test-lib.sh' instead, which contains not more direct
code execution (only definition of shell variables/functions, or sourcing
of other '*.sh' with the same property), is protected against multiple
inclusions, and sources 'test-defs.sh' automatically in in a proper way.
* t/testsuite-summary-count.sh, t/tap-summary.sh, t/tap-summary-color.sh,
t/testsuite-summary-color.sh: Source 'test-lib.sh', not 'test-defs.sh'.
* gen-testsuite-part: Likewise, in the generated wrapper scripts.
* runtest.in: Here, similarly to what is done by AM_TESTS_ENVIRONMENT
in Makefile.am, unset variables that should be under the complete control
of the test framework, and that could create havoc if inherited from the
environment. This remove the need to check against possible environment
"pollution" ...
* t/ax/test-defs.in: ... in here.
* Makefile.am (AM_TESTS_ENVIRONMENT): Add a comment about the need of
synchronization with 'runtest.in'.
* t/self-check-env-sanitize.tap: Remove as obsolete.
* t/list-of-tests.mk: Adjust.
* t/self-check-tap.sh: This. The recent reorganization and code
moving between 'test-defs.sh' and 'test-lib.sh' has made it
brittle and prone to failures. Since the usefulness of this self
check is extremely limited, it's not worth trying to fix it. Just
remove it.
* t/list-of-tests.mk: Adjust.
And similarly, "am_using_tap=no" -> "am_test_protocol=none".
The new '$am_test_protocol' name is clearer, and will allow the easy
addition of further test protocols in the future. This is not truly
relevant for automake, but we are trying to make some parts of our
testsuite framework as general as possible, in view of a future move
to a more generic project like Gnulib.
* Makefile.am, t/ax/am-test-lib.sh, t/ax/test-defs.in, t/ax/test-lib.sh,
t/self-check-env-sanitize.tap, t/self-check-tap.sh: Adjust.
* syntax-checks.mk (sc_tests_obsolete_variables): Add 'am_using_tap' to
the list of obsolete variables to check against.
tests: protect test libs against multiple inclusion
* t/ax/test-lib.sh, t/ax/am-test-lib.sh: Return early if already sourced.
Use the witness variables '$test_lib_sourced' and '$am_test_lib_sourced',
respectively, for this purpose.
* runtest.in, Makefile.am (AM_TESTS_ENVIRONMENT): Unset 'test_lib_sourced'
and 'am_test_lib_sourced', to avoid interferences from the environment.
test defs: no need to re-add $srcdir/t/ax to $PATH
* t/ax/test-defs.sh: Here. That is already done by both 'runtest' and
AM_TESTS_ENVIRONMENT, and the presence of '$srcdir/t/ax' in $PATH is
anyway required, after the recent changes, for our testsuite framework
to work at all.
tests: split test libs into "generic" and "automake-specific"
This is the first step in the quest to merge the generically useful
parts of our test suite framework in a more generic project, like
Gnulib. Time will tell if we'll succeed, and whether the success will
be worth the extra hassle.
* t/ax/test-init.sh: Split out ...
* t/ax/am-test-lib.sh, t/ax/test-lib.sh: ... into these two tests.
* defs, Makefile.am: Adjust.
test setup: merge definitions of function for simple tests
* t/ax/plain-functions.sh: Delete, moving the definitions of ...
($stderr_fileno_): ... this variable ...
(warn_, fail_, skip_, skip_all_, fatal_, framework_failure_): ... and
these functions ...
* t/ax/test-init.sh: ... in here. This allow us to use those functions
earlier in this file (instead of having to duplicate their behaviour
with inlined code). The TAP-based tests are still able to override these
functions later to their TAP-enhanced equivalents when 'tap-functions.sh'
is sourced.
* Makefile.am (dist_noinst_DATA): Remove 't/ax/plain-functions.sh'.
* t/ax/tap-functions.sh: Adjust a comment.
* t/ax/test-init.sh (am_test_setup): Here.
Call it from the main code. Remove other calls of functions and
settings that are now duly called by 'am_test_setup'.
configure: testsuite shell set exit traps in shell functions
* configure.ac: Check that the shell selected to run the test
scripts can set an exit trap in a shell function, without having
that trap executed at the termination of the function rather
than of the scripts. According to the Autoconf manual, at least
AUX 5.3 /bin/sh suffers of such a bug.
test init: remove a couple of paranoid sanity checks
* t/ax/test-init.sh: Here, about 'am_top_srcdir' and 'am_top_builddir';
the values of those variables are AC_SUBST'd, so it's nigh impossible
for the to be wrong or get "messed up".
Now that the early $PATH setup in both 'runtest' and AM_TESTS_ENVIRONMENT
allow the '.' built-in to find to-be-sourced shell scripts in the 't/ax/'
directory automatically, we can simplify several usages like:
. "$am_testauxdir"/foo.sh
to just:
. foo.sh
Also, because our test scripts run with the 'errexit' flag active, and
because POSIX mandates that, when the '.' built-in is used,
... if no readable file is found, a non-interactive shell shall abort ...
We can do so with minimal churn, now that the early setup of $PATH in
both 'runtest' and AM_TESTS_ENVIRONMENT allow the '.' built-in to find
to-be-sourced shell scripts in 't/ax' automatically, both in in-tree
and VPATH builds.
With this change, we take another step forward a more rational and
"segregated" organization of our testsuite framework.
tests: update PATH early from AM_TESTS_ENVIRONMENT and runtest
Because the POSIX standards mandate that sourcing a file with ". FILE"
will cause FILE (assuming it is a relative containing no slashes) to be
looked for in PATH, such a move will allow us to simplify our sourcing
of shell testing libraries like (currently) 't/ax/test-init.sh' and
'defs', and to modularize and re-organize them better in the feature,
with minimal churn.
* runtest.in, Makefile.am (AM_TEST_ENVIRONMENT): Prepend $(abs_srcdir)/t/ax
and $(abs_builddir)/t/ax (in that order) to PATH; do so avoiding repetition
in the common case of non-VPATH builds.
runtest: export $srcdir for use by the test scripts
This is mostly a preparatory change in view of future ones. But it
also make the behaviour of runtest more similar to that of the test
harness used by "make check".
* runtest.in: Export srcdir (was already defined to '@srcdir@').
* gen-testsuite-part ($testauxdir): New, defined to "$testdir/ax".
Use it throughout, for better encapsulation and less duplication.
($auxdir): Remove as unneeded.
gen-tests: less hard-coding of 't' as the test directory
This will make life easier for us in case we decide to rename the
testsuite directory in the future (perhaps to something like 'tests',
which is somewhat less cryptic than the current 't').
* gen-testsuite-part ($testdir): New variable, used throughout instead
of hard-coding the name of the test directory to 't'.
All the script adjusted to make proper use of '$testdir'.
($auxdir): Move its definition earlier, for consistency with the one
of '$testdir'.
build: auxiliary testsuite files/scripts built by "make all"
This will allow the developers to run a tests case by hand out of
a newly extracted tarball simply doing:
$ ./configure && make
$ ./runtest t/the-test-case.sh
while before this change one has to resort to:
$ ./configure && make && make check TESTS=
$ ./runtest t/the-test-case.sh
or, with some non-GNU makes, even:
$ ./configure && make && make check TESTS= AM_MAKEFLAGS="TESTS="
$ ./runtest t/the-test-case.sh
This come very handy sometimes, especially when doing one-shot
debugging. Admittedly not a big deal, but one less friction in
the build system is always nice.
* Makefile.am (check_SCRIPTS, dist_check_DATA, nodist_check_DATA):
Move their content to ...
(noinst_SCRIPTS, dist_noinst_DATA, nodist_noinst_DATA): ... these
variables.
Adjust comments.
* t/list-of-tests.mk (handwritten_TESTS): Don't list ($perf_TESTS) in
here anymore.
* Makefile.am (EXTRA_DIST): Add them explicitly.
(perf): New target, run the performance tests and save the logs of
the ones with unexpected results in the file 't/perf/test-suite.log',
which is thus ...
(PERF_TEST_SUITE_LOG): ... defined in this variable ...
(CLEANFILES): ... and added to this.
(test_subdirs): Remove 't/perf': the tests in there are no more
listed in $(TESTS).
* t/ax/test-init.sh: Don't explicitly skip "perf" test here.
runtest: pass *all* the given shell options to the test invocation
* runtest.in: Here. This didn't work previously because, in our
option parsing loop, we were mistakenly redefining '$shell_opts'
each time, instead of appending to it.
* defs-static.in: Here: do not state anymore that this file should execute
correctly with any system's /bin/sh shell, as this is not true anymore: the
testsuite assumes a POSIX shell throughout these days.
runtest: support option --shell, tell which shell should run the test
* runtest.in: Here. The same effect could have been obtained by
exporting AM_TEST_RUNNER_SHELL to the desired value, but the new
API is simpler and requires less typing and less thinking.
tests: avoid spurious failure when running as root
Fixes automake bug#12041.
* t/primary-prefix-couples-force-valid.sh: If run as root, don't expect
a "test -x" on a non-executable files to fail: for root, all files are
executable (as well as readable and writable), and at least on Solaris
10 that causes "test -x" to succeed also on non-executable files.
[ng] tests: fix spurious failure in 'color-tests2.sh'
* t/color-tests2.sh: Change the idioms used to specify the value of
'AM_COLOR_TESTS' to make. This is required since, in contrast with
what happens in the mainline Automake counterpart, this script doesn't
invoke make with the '-e' option.
* master:
tests: avoid non-textual output in a '.log' file
tags: refactor to reduce code duplication (2)
tags: refactor to reduce code duplication (1)
tests: better name for a couple of tests
check: support colorized testsuite output by default
tags: unify recursion for tags targets with that of "usual" targets
+ Extra non-trivial edits:
* automake.in (handle_tests): Drop the '%COLOR%' transform in the
processing of both 'serial-tests.am' and 'parallel-tests.am', which
have superseded the 'check.am' from mainline Automake.
* lib/am/color-tests.am: Remove use of the '%?COLOR%' transform;
just act as if it were unconditionally TRUE.
(am__tty_colors_dummy): Merge into ...
(am__tty_colors): ... this.
* t/dist-formats.tap: Here. Otherwise, some inferior awk implementations,
like OpenIndiana /usr/xpg4/bin/awk, could be confused and cause spurious
errors in the testsuite harness.
* lib/am/tags.am (am__define_uniq_tagged_files): New variable,
factor out some code common to ...
(ctags-am, tags-am, ID): ... the recipes of these targets.
* lib/am/header-vars.am (am.chars.empty): This, move earlier. This both
fixes the associated comments (which referred to a variable defined above
it as "defined below"), and ensures that this variable is not used before
being defined (not a big deal in practice, since it expands to empty; but
better be consistent).
projects / thirdparty / automake.git / log
