t: detect errors outside of test cases

We have recently merged a patch series that had a simple misspelling of
`test_expect_success`. Instead of making our tests fail though, this
typo went completely undetected and all of our tests passed, which is of
course unfortunate. This is a more general issue with our test suite:
all commands that run outside of a specific test case can fail, and if
we don't explicitly check for such failure then this failure will be
silently ignored.

Improve the status quo by enabling the errexit option so that any such
unchecked failures will cause us to abort immediately.

Note that for now, we only enable this option for Bash 5 and newer. This
is because other shells have wildly different behaviour, and older
versions of Bash (especially on macOS) are buggy. The list of enabled
shells may be extended going forward.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 28cfe730ee5aedfce1104bb8aae53c13e591cf05..de08a08d59fc8f598ebe0f666a0454b7085e88fe 100755 (executable)
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -7,6 +7,12 @@
 
 export TEST_CONTRIB_TOO=yes
 
+case "$jobname" in
+almalinux-*|debian-*|fedora-*|linux-*)
+       export GIT_TEST_USE_SET_E=yes
+       ;;
+esac
+
 case "$jobname" in
 fedora-breaking-changes-musl|linux-breaking-changes)
        export WITH_BREAKING_CHANGES=YesPlease

diff --git a/t/test-lib.sh b/t/test-lib.sh
index de7d9e7b925049cee92526d4ee3d28122b65cefb..cded7bd693e6f615d32ad0ce9ad0bb88eb585357 100644 (file)
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -15,6 +15,31 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see https://www.gnu.org/licenses/ .
 
+# Enable the use of errexit so that any unexpected failures will cause us to
+# abort tests, even when outside of a specific test case.
+#
+# Note that we only enable this on Bash 5 and newer, or when explicitly
+# requested by the user via `GIT_TEST_USE_SET_E=true`. This ib secause `set -e`
+# has wildly different behaviour across shells. The list of default-enabled
+# shells may be extended going forward.
+if test -z "$GIT_TEST_USE_SET_E" && test "${BASH_VERSINFO:=0}" -ge 5
+then
+       GIT_TEST_USE_SET_E=true
+fi
+
+# We cannot use `test-tool env-helper` here, as it's not yet available.
+case "${GIT_TEST_USE_SET_E:-false}" in
+1|on|true|yes)
+       set -e
+       ;;
+0|off|false|no)
+       ;;
+*)
+       echo "GIT_TEST_USE_SET_E requires a boolean" >&2
+       exit 1
+       ;;
+esac
+
 # Test the binaries we have just built.  The tests are kept in
 # t/ subdirectory and are run in 'trash directory' subdirectory.
 if test -z "$TEST_DIRECTORY"