test-lib: check malloc debug LD_PRELOAD before using

This fixes test failures across the suite on glibc platforms that don't
have libc_malloc_debug.so.0.

We added support for glibc's malloc checking routines long ago in
a731fa916e (Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test
suite for detecting heap corruption, 2012-09-14). Back then we didn't
need to do any checks to see if the platform supported it. We were just
setting some environment variables which would either enable it or not.

That changed in 131b94a10a (test-lib.sh: Use GLIBC_TUNABLES instead of
MALLOC_CHECK_ on glibc >= 2.34, 2022-03-04). Now that glibc split this
out into libc_malloc_debug.so, we have to add it to LD_PRELOAD. We only
do that when we detect glibc, but it's possible to have glibc but not
the malloc debug library. In that case LD_PRELOAD will complain to
stderr, and tests which check for an empty stderr will fail.

You can work around this by setting TEST_NO_MALLOC_CHECK, which disables
the feature entirely. But it's not obvious to know you need to do that.
Instead, since this malloc checking is best-effort anyway, let's just
automatically disable it when the LD_PRELOAD appears not to work. We can
check it by running something simple that should work (and produce
nothing on stderr) like "git version".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/t/test-lib.sh b/t/test-lib.sh
index b1a8ee5c002b3572ba7eafee22ac08eeb3daf688..e715d07f397d3d350147ee3bc892e12986fbc7bb 100644 (file)
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -593,9 +593,12 @@ then
        }
 else
        _USE_GLIBC_TUNABLES=
+       _USE_GLIBC_PRELOAD=libc_malloc_debug.so.0
        if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
           _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
-          expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
+          expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null &&
+          stderr=$(LD_PRELOAD=$_USE_GLIBC_PRELOAD git version 2>&1 >/dev/null) &&
+          test -z "$stderr"
        then
                _USE_GLIBC_TUNABLES=YesPlease
        fi
@@ -607,7 +610,7 @@ else
                if test -n "$_USE_GLIBC_TUNABLES"
                then
                        g=
-                       LD_PRELOAD="libc_malloc_debug.so.0"
+                       LD_PRELOAD=$_USE_GLIBC_PRELOAD
                        for t in \
                                glibc.malloc.check=1 \
                                glibc.malloc.perturb=165