From: Dan Fandrich <dan@coneharvesters.com>
Date: Fri, 15 Sep 2023 20:42:04 +0000 (-0700)
Subject: CI: ignore the "flaky" and "timing-dependent" test results
X-Git-Tag: curl-8_4_0~192
X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=589dca761;p=thirdparty%2Fcurl.git

CI: ignore the "flaky" and "timing-dependent" test results

CI builds will now run these tests, but will ignore the results if they
fail. The relevant tests are ones that are sensitive to timing or
have edge conditions that make them more likely to fail on CI servers,
which are often heavily overloaded and slow.

This change only adds two additional tests to be ignored, since the
others already had the flaky keyword.

Closes #11865
---

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 55d551f1a6..17e9ad0493 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -71,8 +71,9 @@ TEST_F = -a -p -r
 TEST_T = -a -t
 TEST_E = -a -e
 
-# !flaky means that it'll skip all tests using the flaky keyword
-TEST_NF = -a -p !flaky
+# ~<keyword> means that it will run all tests matching the keyword, but will
+# ignore their results (since these ones are likely to fail for no good reason)
+TEST_NF = -a -p ~flaky ~timing-dependent
 
 # special CI target derived from nonflaky with CI-specific flags
 TEST_CI = $(TEST_NF) -rm