CI: ignore the "flaky" and "timing-dependent" test results

CI builds will now run these tests, but will ignore the results if they
fail. The relevant tests are ones that are sensitive to timing or
have edge conditions that make them more likely to fail on CI servers,
which are often heavily overloaded and slow.

This change only adds two additional tests to be ignored, since the
others already had the flaky keyword.

Closes #11865

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 55d551f1a636754ef9563f5fc4cbe7990b82fcaa..17e9ad04938ebb7eda1c8625fb5720895e0a3222 100644 (file)
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -71,8 +71,9 @@ TEST_F = -a -p -r
 TEST_T = -a -t
 TEST_E = -a -e
 
-# !flaky means that it'll skip all tests using the flaky keyword
-TEST_NF = -a -p !flaky
+# ~<keyword> means that it will run all tests matching the keyword, but will
+# ignore their results (since these ones are likely to fail for no good reason)
+TEST_NF = -a -p ~flaky ~timing-dependent
 
 # special CI target derived from nonflaky with CI-specific flags
 TEST_CI = $(TEST_NF) -rm