From: Dan Fandrich <dan@coneharvesters.com>
Date: Thu, 22 Jun 2023 17:03:16 +0000 (-0700)
Subject: CI: enable parallel testing in CI builds
X-Git-Tag: curl-8_10_0~420
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=0324d557e4b4f754ea89636ea9164065f6446560;p=thirdparty%2Fcurl.git

CI: enable parallel testing in CI builds

The test-ci target now uses 2 processes by default, but the amount of
parallelism is tuned for each CI service and build environment based on
results of a number of test runs.  Some CI services use super-
oversubscribed build machines that can barely run the curl tests
already with no parallelism without frequently failing with
timing-induced failures. These continue to be run without parallelism.
Other services provide two fast, unloaded cores and these run with 14
processes, which is a good default for this kind of environment.

Here's a summary of the number of test processes by CI service:

  Appveyor - 2 (Windows MSVC), 1 (others)
  Azure - 2
  Circle CI - 14
  Cirrus - 28 (macOS), 14 (Linux), 7 (FreeBSD), 5 (macOS torture), 2 (Windows)
  GitHub Actions - 3 (macOS), 2 (Linux)

Some of these are a bit conservative to keep timing-induced flakiness down.

The net result is that the first test results should arrive only
3 minutes after a commit submission.

Changes merged via separate commits:
- 2a7c8b27fdd266894fe24d6d0f5d853ea843dff4 #14171
- 72341068a2d8f1b94a26add16830c725cc4054e6
- efce544418971b064d9fc8183a6caa582c54559b #14244
- c6cf411bacf97f230db160e543d0cc3bbe5b9aba

Ref: #10818
Closes #11510
---

diff --git a/appveyor.sh b/appveyor.sh
index a2925e647a..5f39dddd27 100644
--- a/appveyor.sh
+++ b/appveyor.sh
@@ -137,6 +137,7 @@ if [[ "${TFLAGS}" != 'skipall' ]] && \
   elif [ -x "$(cygpath 'C:/msys64/usr/bin/curl.exe')" ]; then
     TFLAGS+=" -ac $(cygpath 'C:/msys64/usr/bin/curl.exe')"
   fi
+  TFLAGS+=' -j0'
   if [ "${BUILD_SYSTEM}" = 'CMake' ]; then
     cmake --build _bld --config "${PRJ_CFG}" --target test-ci
   else
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 499d01150f..8c40135ecb 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -88,6 +88,6 @@ add_runtests(test-am        "-a -am")
 add_runtests(test-full      "-a -p -r")
 # ~flaky means that it'll ignore results of tests using the flaky keyword
 add_runtests(test-nonflaky  "-a -p ~flaky ~timing-dependent")
-add_runtests(test-ci        "-a -p ~flaky ~timing-dependent -r -rm")
+add_runtests(test-ci        "-a -p ~flaky ~timing-dependent -r -rm -j2")
 add_runtests(test-torture   "-a -t")
 add_runtests(test-event     "-a -e")
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4b58fad24b..54b9af66a8 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -133,7 +133,7 @@ TEST_E = -a -e
 TEST_NF = -a -p ~flaky ~timing-dependent
 
 # special CI target derived from nonflaky with CI-specific flags
-TEST_CI = $(TEST_NF) -r -rm
+TEST_CI = $(TEST_NF) -r -rm -j2
 endif
 
 CD2NROFF = $(top_srcdir)/scripts/cd2nroff $< >$@