Adjust the respdiff failure threshold for a new dataset

This is just a slight tweak for the respdiff CI test. The new dataset
has a different set of queries and it results in a slightly more
SERVFAILs rather than timeouts in the respdiff-long-third-party test.

In our comparison script, timeouts are not counted towards the
threshold. While the total number of differences remains roughly the
same, the different distributions of them (among SERVFAIL vs timeout)
warrants a slight bump in the threshold in order to avoid test failures.

Related isc-private/bind-qa!65

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 73205c395070893837d123a0852728e9ddaa8f59..8fded88571b677bce07d0e29bac177c4a38830f3 100644 (file)
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1476,7 +1476,7 @@ respdiff-long-third-party:
   variables:
     CC: gcc
     CFLAGS: "${CFLAGS_COMMON} -Og"
-    MAX_DISAGREEMENTS_PERCENTAGE: "0.1"
+    MAX_DISAGREEMENTS_PERCENTAGE: "0.2"
   script:
     - bash respdiff.sh -s third_party -q "${PWD}/100k_mixed.txt" -c 1 -w "${PWD}/rspworkdir" "${CI_PROJECT_DIR}"