From: Tom Krizek <tkrizek@isc.org>
Date: Thu, 17 Aug 2023 08:30:46 +0000 (+0200)
Subject: Add custom flaky decorator to handle unstable tests
X-Git-Tag: v9.19.17~49^2~1
X-Git-Url: http://git.ipfire.org/gitweb/?a=commitdiff_plain;h=5b703de733b1f53f0452e8e654459d7adeb9aa00;p=thirdparty%2Fbind9.git

Add custom flaky decorator to handle unstable tests

If the flaky plugin for pytest is available, use its decorator to
support re-running unstable tests. In case the package is missing,
execute the test as usual without attempts to re-run it in case of
failure.

This is mostly intended to increase the test stability in CI. Using a
custom decorator enables us to keep the flaky package as an optional
dependency.
---

diff --git a/bin/tests/system/pytest_custom_markers.py b/bin/tests/system/pytest_custom_markers.py
index 7411afeafd2..ba3a9d4894a 100644
--- a/bin/tests/system/pytest_custom_markers.py
+++ b/bin/tests/system/pytest_custom_markers.py
@@ -40,3 +40,21 @@ have_libxml2 = pytest.mark.skipif(
 have_json_c = pytest.mark.skipif(
     not feature_test("--have-json-c"), reason="json-c support disabled in the build"
 )
+
+
+try:
+    import flaky as flaky_pkg
+except ModuleNotFoundError:
+    # In case the flaky package is not installed, run the tests as usual
+    # without any attempts to re-run them.
+    # pylint: disable=unused-argument
+    def flaky(*args, **kwargs):
+        """Mock decorator that doesn't do anything special, just returns the function."""
+
+        def wrapper(wrapped_obj):
+            return wrapped_obj
+
+        return wrapper
+
+else:
+    flaky = flaky_pkg.flaky