Set stricter respdiff:recent-named threshold

This job is testing the current BIND implementation against the latest
released version. Unless there has been a behaviour change, there should
be no difference.

In practice, there is a small number of differences caused by upstream
discrepencies. Some of those cause "upstream unstable" answers which are
excluded from the results, but statistically, some of those will end up
being detected as differences on the resolver under test.

Currently, there seems to be about 300 upstream unstable answers with
typically around 50-60 differences. Setting the threshold to 0.1 should
be stable enough to pass if there are no changes, yet sensitive enough
to detect even fairly small changes to behaviour.

(cherry picked from commit 92ae68764954b3606a899f5f3ecd16846cda7324)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ef95cd16c279fea08741829392b663a7608e8c5c..99afebdb74cd23c7a5310884ac1465f970563136 100644 (file)
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2151,7 +2151,7 @@ respdiff:recent-named:
   variables:
     CC: gcc
     CFLAGS: "${CFLAGS_COMMON} -Og -DISC_TRACK_PTHREADS_OBJECTS"
-    MAX_DISAGREEMENTS_PERCENTAGE: "0.3"
+    MAX_DISAGREEMENTS_PERCENTAGE: "0.1"
 
 # Performance tests