selftests: Deescalate error reporting

Commit 7e47389142b8 ("selftests: Preserve subtarget failures in
all/install") updated the propagation of errors from indivdual kselftest
targets to be similar to that seen with FORCE_TARGETS.  While it would
be really nice to be in a position to do this currently it is premature
to do this as the default behaviour.

At present we default to trying to build all selftests but a combination
of code quality issues and build dependencies mean that it is almost
certain that at least one of them will fail to build (for example,
several depend on clang so don't work in a GCC container) and a top
level failure in the kselftest build reported.  Further, the resulting
failures mean that the install target does not run at all so any build
problem is escallated to a complete failure to produce a kselftest
tarball so CI systems that run into issues loose all selftests coverage.
This has been causing disruption to a range of CI systems including
KernelCI, mine and Arm's internal one.

Revert the commit, users who need this behaviour should be able to use
FORCE_TARGETS for the time being.  At present users that do this (such
as linux-next) are most likely building a subset of targets known to
succeed in their environments.

Link: https://lore.kernel.org/r/20260416-selftests-deescalate-error-reporting-v1-1-38e7c0536227@kernel.org
Fixes: 7e47389142b8 ("selftests: Preserve subtarget failures in all/install")
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 0949f370ad7866e8eb2af68099619c2ca3c75df8..450f13ba4cca98836bc8a2fe18a2eb43ce14b2d5 100644 (file)
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -209,14 +209,14 @@ export KHDR_INCLUDES
 .DEFAULT_GOAL := all
 
 all:
-       @ret=0;                                                 \
+       @ret=1;                                                 \
        for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do     \
                BUILD_TARGET=$$BUILD/$$TARGET;                  \
                mkdir $$BUILD_TARGET  -p;                       \
                $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET       \
                                O=$(abs_objtree)                \
                                $(if $(FORCE_TARGETS),|| exit); \
-               [ $$? -eq 0 ] || ret=1;                 \
+               ret=$$((ret * $$?));                            \
        done; exit $$ret;
 
 run_tests: all
@@ -274,7 +274,7 @@ ifdef INSTALL_PATH
        install -m 744 kselftest/ksft.py $(INSTALL_PATH)/kselftest/
        install -m 744 run_kselftest.sh $(INSTALL_PATH)/
        rm -f $(TEST_LIST)
-       @ret=0; \
+       @ret=1; \
        for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do \
                BUILD_TARGET=$$BUILD/$$TARGET;  \
                $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET install \
@@ -283,7 +283,7 @@ ifdef INSTALL_PATH
                                OBJ_PATH=$(INSTALL_PATH) \
                                O=$(abs_objtree)                \
                                $(if $(FORCE_TARGETS),|| exit); \
-               [ $$? -eq 0 ] || ret=1;         \
+               ret=$$((ret * $$?));            \
        done; exit $$ret;