Travis: Add a macOS chutney job, but don't wait for it to finish

Since Travis macOS has IPv6 support (and Travis Linux does not), chutney
will now run its IPv6 networks as part of Travis CI.

But since chutney is slow, don't wait for the macOS chutney to finish.
(Travis have fixed the duplicate notification bug in fast_finish. So we
can use fast_finish and allow_failure to finish early. Unfortunately,
allow_failure also means we ignore failures in macOS chutney.)

Also make sure that we have:
* a compile on each platform, with each compiler,
* a check on each platform, and
* a check on each compiler.

Finally, sort builds: allow fail last, macOS first, slowest first.

Closes ticket 30860.
Closes ticket 31859 for 0.2.9.

diff --git a/.travis.yml b/.travis.yml
index f47fe8bb2c6fdb4ffa534efde0c692a9fb4900a6..564c97dbfd9904785a718dd1530c0ea61449cf45 100644 (file)
--- a/.travis.yml
+++ b/.travis.yml
@@ -27,34 +27,33 @@ env:
 matrix:
   ## include creates builds with gcc, linux, unless we override those defaults
   include:
-    ## clang is the default macOS compiler, so we use it for the macOS job
+    ## We run basic tests on macOS
     - compiler: clang
       os: osx
-    ## We include a single coverage build with the best options for coverage
-    - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS=""
-    ## We only want to check these build option combinations once
-    ## (they shouldn't vary by compiler or OS)
-    ## We run coverage with hardening off, which seems like enough
-    # - env: HARDENING_OPTIONS=""
+    ## We run chutney on Linux, because it's faster than chutney on macOS
+    - env: CHUTNEY="yes" CHUTNEY_ALLOW_FAILURES="2" SKIP_MAKE_CHECK="yes"
     ## We check asciidoc with distcheck, to make sure we remove doc products
+    ## We use Linux clang, because there are no other Linux clang jobs
     - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" SKIP_MAKE_CHECK="yes"
-    # We also try running a hardened clang build with chutney on Linux.
-    - env: CHUTNEY="yes" SKIP_MAKE_CHECK="yes" CHUTNEY_ALLOW_FAILURES="2"
       compiler: clang
+    ## We include a single coverage build with the best options for coverage
+    - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS=""
+    ## We run chutney on macOS, because macOS Travis has IPv6
+    - env: CHUTNEY="yes" CHUTNEY_ALLOW_FAILURES="2" SKIP_MAKE_CHECK="yes"
+      os: osx
 
-  ## Uncomment to allow the build to report success (with non-required
-  ## sub-builds continuing to run) if all required sub-builds have
-  ## succeeded.  This is somewhat buggy currently: it can cause
-  ## duplicate notifications and prematurely report success if a
-  ## single sub-build has succeeded.  See
-  ## https://github.com/travis-ci/travis-ci/issues/1696
-  # fast_finish: true
+  ## Allow the build to report success (with non-required sub-builds
+  ## continuing to run) if all required sub-builds have succeeded.
+  fast_finish: true
 
   ## Careful! We use global envs, which makes it hard to allow failures by env:
   ## https://docs.travis-ci.com/user/customizing-the-build#matching-jobs-with-allow_failures
-  # allow_failures:
-  #   - compiler: gcc
-  #     os: linux
+  allow_failures:
+    ## macOS chutney is very slow, so we let the build finish before it's done
+    ## We'd like to fast finish, but still eventually show failures.
+    ## But Travis doesn't have that option.
+    - env: CHUTNEY="yes" CHUTNEY_ALLOW_FAILURES="2" SKIP_MAKE_CHECK="yes"
+      os: osx
 
 ## (Linux only) Use the latest Linux image (Ubuntu Trusty)
 dist: trusty

diff --git a/changes/ticket30860 b/changes/ticket30860
new file mode 100644 (file)
index 0000000..b946f73
--- /dev/null
+++ b/changes/ticket30860
@@ -0,0 +1,3 @@
+  o Testing:
+    - Run the chutney IPv6 networks as part of Travis CI.
+      Closes ticket 30860.

diff --git a/changes/ticket31859 b/changes/ticket31859
index 0eb8a42e9a4e753a9906c114daa196e5b020bfeb..dbc591e00b268a7dcba5150d4d0df381c5bd9866 100644 (file)
--- a/changes/ticket31859
+++ b/changes/ticket31859
@@ -1,3 +1,3 @@
   o Testing:
-    - Remove some redundant Travis CI jobs, to speed up CI.
+    - Simplify the Travis CI build matrix, and optimise for build time.
       Closes ticket 31859.