The third-party comparison is right at the 0.4 % threshold, making it
fail quite often. The current range of observed values ranges from
0.3-0.5 %. Raise the threshold to 0.5 %.
The recent-named comparison produces values ranging from 0.05 to 0.12 %,
but appears to be more sensitive to time of the day when the test runs.
Raise the threshold to 0.15 %.
Both results are stable within the specified ranges across the last
three releases in both main and bind-9.20 series.
CC: gcc
CFLAGS: "${CFLAGS_COMMON}"
EXTRA_CONFIGURE: "-Doptimization=g"
- MAX_DISAGREEMENTS_PERCENTAGE: "0.4"
+ MAX_DISAGREEMENTS_PERCENTAGE: "0.5"
script:
- bash respdiff.sh -s third_party -q "${PWD}/100k_mixed.txt" -c 1 -w "${PWD}/rspworkdir" -r "${PWD}/respdiff" "${CI_PROJECT_DIR}"
- cd ../.. && ninja -C build clean >/dev/null 2>&1
CC: gcc
CFLAGS: "${CFLAGS_COMMON} -DISC_TRACK_PTHREADS_OBJECTS"
EXTRA_CONFIGURE: "-Doptimization=g"
- MAX_DISAGREEMENTS_PERCENTAGE: "0.1"
+ MAX_DISAGREEMENTS_PERCENTAGE: "0.15"
# Performance tests