GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs

Re-implement old mingw-w64 jobs in GHA. This allows to use the latest
Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows
Server 2016 (gcc 7, 9) with Windows Server 2022.

GHA runners are also significantly faster, and allow running tests in
parallel (`-j14`). It also offloads 3 more long-running jobs from
AppVeyor CI.

These jobs download (then cache) the mingw-w64 packages from their
original location, which allows flexibility in choosing which versions
and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI.
The new jobs use these distros:
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/ (for gcc 7, same as on AppVeyor)
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/ (for gcc 6, same as on AppVeyor)
- https://winlibs.com/ (for gcc 9)

I matched existing AppVeyor job configs, with these differences:
- gcc 6.4.0 instead of 6.3.0.
  (same distro as on AppVeyor, but the latest bugfix release)
- gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro.
  (in AppVeyor this relies on an old MSYS2 pre-installed on the runner)
- using win32 builds instead of posix for gcc 6.4.0 and 7.3.0.
- websockets enabled.
- always build examples.
- always build tests (this wasn't done for 6.4.0 with AppVeyor CI).

I did not replicate existing test exclusions, and oddly enough the few
failures (so far) were different from MSYS2 jobs and also from their
AppVeyor CI counterparts.

Also:
- delete redundant (default) `-u` option from `cygpath` calls.
- allow matrix options to override default ones in CMake.
- detect and use Windows-supplied curl for `TFLAGS` `-ac` option.
  (it's available in modern runners.)
- delete the 3 AppVeyor CI jobs now replicated in GHA.
- appveyor: prefer `SYSTEMROOT` over `WINDIR`.
- tidy-up quotes.

Job performance:
```
                                                                 AppVeyor  GHA
                                                                           w/examples
                                                                           w/tests
                                                                 --------  ----------
CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity   1m25s     8m50s
CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode   31m45s     9m39s
CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static            28m25s    13m38s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49880799
https://github.com/curl/curl/actions/runs/9218292508

Notice that building examples and tests is time consuming.

We can tweak any build parameter as necessary to make them more useful
and/or without clogging the job queue or introducing flakiness.

Closes #13759

diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml
index d010c9a2cda4aec15fcaf6d53adc8240b2af6085..e83f5db5cb0f155f0cd9da9603039440a77355a6 100644 (file)
--- a/.github/workflows/windows.yml
+++ b/.github/workflows/windows.yml
@@ -41,7 +41,7 @@ permissions: {}
 
 jobs:
   cygwin:
-    name: "cygwin (${{ matrix.build }}, ${{ matrix.platform }}, ${{ matrix.config }})"
+    name: 'cygwin (${{ matrix.build }}, ${{ matrix.platform }}, ${{ matrix.config }})'
     runs-on: windows-latest
     timeout-minutes: 45
     env:
@@ -105,8 +105,8 @@ jobs:
         shell: C:\cygwin\bin\bash.exe '{0}'
         run: |
           export TFLAGS='-j8 ${{ matrix.tflags }}'
-          if [ -x "$(cygpath -u "${SYSTEMROOT}/System32/curl.exe")" ]; then
-            TFLAGS+=" -ac $(cygpath -u "${SYSTEMROOT}/System32/curl.exe")"
+          if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then
+            TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")"
           fi
           make -C bld -j3 V=1 test-ci
 
@@ -116,13 +116,14 @@ jobs:
         shell: C:\cygwin\bin\bash.exe '{0}'
         run: |
           export PATH="/usr/bin:$(cygpath ${SYSTEMROOT})/system32"
-          cmake -B bld ${options} ${{ matrix.config }} \
+          cmake -B bld ${options} \
             "-DCMAKE_C_FLAGS=${cflags}" \
             -DCMAKE_UNITY_BUILD=ON \
             -DCURL_WERROR=ON \
             -DBUILD_EXAMPLES=ON \
             -DENABLE_WEBSOCKETS=ON \
-            -DCURL_BROTLI=ON -DUSE_NGHTTP2=ON
+            -DCURL_BROTLI=ON -DUSE_NGHTTP2=ON \
+            ${{ matrix.config }}
 
       - name: 'cmake build'
         if: ${{ matrix.build == 'cmake' }}
@@ -135,7 +136,7 @@ jobs:
           bld/src/curl.exe --disable --version
 
   msys2:
-    name: "msys2 (${{ matrix.build }}, ${{ matrix.sys }}, ${{ matrix.env }}, ${{ matrix.config }})"
+    name: 'msys2 (${{ matrix.build }}, ${{ matrix.sys }}, ${{ matrix.env }}, ${{ matrix.config }})'
     runs-on: windows-latest
     timeout-minutes: 45
     strategy:
@@ -209,8 +210,8 @@ jobs:
         shell: msys2 {0}
         run: |
           export TFLAGS='-j14 ${{ matrix.tflags }}'
-          if [ -x "$(cygpath -u 'C:/msys64/usr/bin/curl.exe')" ]; then
-            TFLAGS+=" -ac $(cygpath -u 'C:/msys64/usr/bin/curl.exe')"
+          if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then
+            TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")"
           fi
           make -C bld -j3 V=1 test-ci
 
@@ -224,7 +225,7 @@ jobs:
           else
             options='-DCMAKE_C_COMPILER=gcc'
           fi
-          cflags='-Wno-deprecated-declarations'
+          cflags='-Wno-deprecated-declarations'  # for examples
           if [ '${{ matrix.test }}' = 'uwp' ]; then
             options+=' -DCMAKE_SYSTEM_NAME=WindowsStore -DCMAKE_SYSTEM_VERSION=10.0'
             pacman --noconfirm --ask 20 --noprogressbar --sync --needed 'mingw-w64-${{ matrix.env }}-winstorecompat-git'
@@ -240,15 +241,16 @@ jobs:
           [ '${{ matrix.type }}' = 'Debug' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG='
           [ '${{ matrix.type }}' = 'Release' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE='
           [ '${{ matrix.type }}' = 'Release' ] && [[ '${{ matrix.config }}' = *'ENABLE_DEBUG=ON'* ]] && cflags+=' -DDEBUGBUILD'
-          cmake -B bld ${options} ${{ matrix.config }} \
+          cmake -B bld ${options} \
             "-DCMAKE_C_FLAGS=${cflags}" \
             "-DCMAKE_RC_COMPILE_OBJECT=${rcopts}" \
-            "-DCMAKE_BUILD_TYPE=${{ matrix.type }}" \
+            '-DCMAKE_BUILD_TYPE=${{ matrix.type }}' \
             -DCMAKE_UNITY_BUILD=ON \
             -DCURL_WERROR=ON \
             -DBUILD_EXAMPLES=ON \
             -DENABLE_WEBSOCKETS=ON \
-            -DCURL_BROTLI=ON -DUSE_NGHTTP2=ON
+            -DCURL_BROTLI=ON -DUSE_NGHTTP2=ON \
+            ${{ matrix.config }}
 
       - name: 'cmake build'
         if: ${{ matrix.build == 'cmake' }}
@@ -273,8 +275,110 @@ jobs:
         shell: msys2 {0}
         run: |
           export TFLAGS='-j14 ${{ matrix.tflags }}'
-          if [ -x "$(cygpath -u 'C:/msys64/usr/bin/curl.exe')" ]; then
-            TFLAGS+=" -ac $(cygpath -u 'C:/msys64/usr/bin/curl.exe')"
+          if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then
+            TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")"
+          fi
+          ls bld/lib/*.dll >/dev/null 2>&1 && cp -f -p bld/lib/*.dll bld/tests/libtest/
+          cmake --build bld --config '${{ matrix.type }}' --target test-ci
+
+  old-mingw-w64:
+    name: 'old-mingw-w64 (${{ matrix.build }}, ${{ matrix.env }}, ${{ matrix.config }})'
+    runs-on: windows-latest
+    timeout-minutes: 45
+    strategy:
+      matrix:
+        include:
+          - build: 'cmake'
+            env: '9.5.0-x86_64'
+            url: 'https://github.com/brechtsanders/winlibs_mingw/releases/download/9.5.0-10.0.0-msvcrt-r1/winlibs-x86_64-posix-seh-gcc-9.5.0-mingw-w64msvcrt-10.0.0-r1.7z'
+            config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=OFF'
+            type: 'Debug'
+            tflags: '~1139 ~1177 ~1477 ~2301 ~2302 ~3027'
+          - build: 'cmake'
+            env: '7.3.0-x86_64'
+            url: 'https://downloads.sourceforge.net/mingw-w64/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-win32/seh/x86_64-7.3.0-release-win32-seh-rt_v5-rev0.7z'
+            config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON'
+            type: 'Debug'
+            tflags: '~1139 ~1177 ~1477 ~2301 ~2302 ~3027'
+          - build: 'cmake'
+            env: '6.4.0-i686'
+            url: 'https://downloads.sourceforge.net/mingw-w64/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/6.4.0/threads-win32/dwarf/i686-6.4.0-release-win32-dwarf-rt_v5-rev0.7z'
+            config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=OFF -DCMAKE_UNITY_BUILD=OFF'
+            type: 'Debug'
+            tflags: 'skiprun'
+      fail-fast: false
+    steps:
+      - name: 'cache compiler (gcc ${{ matrix.env }})'
+        uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4
+        id: cache-compiler
+        with:
+          path: C:\my-cache\mingw-w64-${{ matrix.env }}
+          key: ${{ runner.os }}-mingw-w64-${{ matrix.env }}
+
+      - name: 'install compiler (gcc ${{ matrix.env }})'
+        if: ${{ steps.cache-compiler.outputs.cache-hit != 'true' }}
+        timeout-minutes: 5
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          mkdir -p /c/my-cache/
+          cd /c/my-cache/ || exit 1
+          curl --fail --silent --show-error --retry 3 --retry-connrefused --output pack.bin --location --proto-redir =https '${{ matrix.url }}'
+          mkdir -p mingw-w64-${{ matrix.env }}
+          cd mingw-w64-${{ matrix.env }} || exit 1
+          pwd
+          7z x -y ../pack.bin >/dev/null
+
+      - run: git config --global core.autocrlf input
+      - uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
+
+      - name: 'update PATH (gcc ${{ matrix.env }})'
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          echo "PATH=/c/my-cache/mingw-w64-${{ matrix.env }}/mingw64/bin:/c/msys64/usr/bin:$PATH" >> $GITHUB_ENV
+
+      - name: 'cmake configure'
+        if: ${{ matrix.build == 'cmake' }}
+        timeout-minutes: 5
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          cflags='-Wno-deprecated-declarations'  # for examples
+          [ '${{ matrix.type }}' = 'Debug' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG='
+          [ '${{ matrix.type }}' = 'Release' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE='
+          cmake -B bld ${options} \
+            '-GMSYS Makefiles' \
+            -DCMAKE_C_COMPILER=gcc \
+            "-DCMAKE_C_FLAGS=${cflags}" \
+            '-DCMAKE_BUILD_TYPE=${{ matrix.type }}' \
+            -DCMAKE_UNITY_BUILD=ON \
+            -DCURL_WERROR=ON \
+            -DBUILD_EXAMPLES=ON \
+            -DENABLE_WEBSOCKETS=ON \
+            ${{ matrix.config }}
+
+      - name: 'cmake build'
+        if: ${{ matrix.build == 'cmake' }}
+        timeout-minutes: 10
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          cmake --build bld --config '${{ matrix.type }}' --parallel 3
+          [[ '${{ matrix.config }}' != *'BUILD_SHARED_LIBS=OFF'* ]] && cp -f -p bld/lib/*.dll bld/src/
+          bld/src/curl.exe --disable --version
+
+      - name: 'cmake build tests'
+        if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skipall' }}
+        timeout-minutes: 10
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          cmake --build bld --config '${{ matrix.type }}' --parallel 3 --target testdeps
+
+      - name: 'cmake run tests'
+        if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skipall' && matrix.tflags != 'skiprun' }}
+        timeout-minutes: 40
+        shell: C:\msys64\usr\bin\bash.exe {0}
+        run: |
+          export TFLAGS='-j14 ${{ matrix.tflags }}'
+          if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then
+            TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")"
           fi
           ls bld/lib/*.dll >/dev/null 2>&1 && cp -f -p bld/lib/*.dll bld/tests/libtest/
           cmake --build bld --config '${{ matrix.type }}' --target test-ci

diff --git a/appveyor.sh b/appveyor.sh
index 9a71c9b7cf56523128f85de7705b1730d7513262..6716319f4eb44c999a22ec7862b003feb286627f 100644 (file)
--- a/appveyor.sh
+++ b/appveyor.sh
@@ -33,7 +33,7 @@ if [ "${APPVEYOR_BUILD_WORKER_IMAGE}" = 'Visual Studio 2022' ]; then
 else
   openssl_root_win='C:/OpenSSL-v111-Win64'
 fi
-openssl_root="$(cygpath -u "${openssl_root_win}")"
+openssl_root="$(cygpath "${openssl_root_win}")"
 
 if [ "${BUILD_SYSTEM}" = 'CMake' ]; then
   options=''
@@ -121,10 +121,10 @@ fi
 
 if [ "${TESTING}" = 'ON' ]; then
   export TFLAGS=''
-  if [ -x "$(cygpath -u "${WINDIR}/System32/curl.exe")" ]; then
-    TFLAGS+=" -ac $(cygpath -u "${WINDIR}/System32/curl.exe")"
-  elif [ -x "$(cygpath -u 'C:/msys64/usr/bin/curl.exe')" ]; then
-    TFLAGS+=" -ac $(cygpath -u 'C:/msys64/usr/bin/curl.exe')"
+  if [ -x "$(cygpath "${SYSTEMROOT}/System32/curl.exe")" ]; then
+    TFLAGS+=" -ac $(cygpath "${SYSTEMROOT}/System32/curl.exe")"
+  elif [ -x "$(cygpath 'C:/msys64/usr/bin/curl.exe')" ]; then
+    TFLAGS+=" -ac $(cygpath 'C:/msys64/usr/bin/curl.exe')"
   fi
   TFLAGS+=" ${DISABLED_TESTS:-}"
   if [ "${BUILD_SYSTEM}" = 'CMake' ]; then

diff --git a/appveyor.yml b/appveyor.yml
index 33f5b35caf8d6a0626cba945aa7b99b1c41d32fa..9d854803449d6bbe14ad2f290385cd2c911197cc 100644 (file)
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -129,51 +129,6 @@ environment:
       TESTING: 'ON'
       DISABLED_TESTS: '!1139 !1501 !1177 !1477'
 
-    # generated CMake-based MSYS Makefiles builds (mingw cross-compiling)
-
-    - job_name: 'CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode'
-      APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
-      BUILD_SYSTEM: CMake
-      PRJ_GEN: 'MSYS Makefiles'
-      PRJ_CFG: Debug
-      SCHANNEL: 'ON'
-      ENABLE_UNICODE: 'ON'
-      HTTP_ONLY: 'OFF'
-      TESTING: 'ON'
-      # test 286 disabled due to https://github.com/curl/curl/issues/12040
-      DISABLED_TESTS: '~286 !1086 !1139 !1451 !1501 !1177 !1477'
-      ADD_PATH: 'C:/mingw-w64/x86_64-7.2.0-posix-seh-rt_v5-rev1/mingw64/bin'
-      MSYS2_ARG_CONV_EXCL: '/*'
-      BUILD_OPT: -k
-    - job_name: 'CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static'
-      APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
-      BUILD_SYSTEM: CMake
-      PRJ_GEN: 'MSYS Makefiles'
-      PRJ_CFG: Debug
-      SCHANNEL: 'ON'
-      ENABLE_UNICODE: 'OFF'
-      HTTP_ONLY: 'OFF'
-      TESTING: 'ON'
-      # test 286 disabled due to https://github.com/curl/curl/issues/12040
-      DISABLED_TESTS: '~286 !1086 !1139 !1451 !1501 !1177 !1477'
-      ADD_PATH: 'C:/msys64/mingw64/bin'
-      MSYS2_ARG_CONV_EXCL: '/*'
-      BUILD_OPT: -k
-    - job_name: 'CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity'
-      APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2015'
-      BUILD_SYSTEM: CMake
-      PRJ_GEN: 'MSYS Makefiles'
-      PRJ_CFG: Debug
-      SCHANNEL: 'ON'
-      ENABLE_UNICODE: 'OFF'
-      HTTP_ONLY: 'OFF'
-      TESTING: 'OFF'
-      DISABLED_TESTS: ''
-      ADD_PATH: 'C:/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin'
-      MSYS2_ARG_CONV_EXCL: '/*'
-      BUILD_OPT: -k
-      UNITY: 'OFF'
-
     # winbuild-based builds
 
     - job_name: 'winbuild, VS2015, Debug, x64, OpenSSL 1.1.1, Build-only'
@@ -243,16 +198,7 @@ environment:
       VC_VERSION: VC12
 
 install:
-  - ps: |
-      if($env:ADD_SHELL -ne $null) {
-        $env:PATH = "$env:ADD_SHELL;$env:PATH"
-      }
-      else {
-        $env:PATH = "C:/msys64/usr/bin;$env:PATH"
-      }
-      if($env:ADD_PATH -ne $null) {
-        $env:PATH = "$env:ADD_PATH;$env:PATH"
-      }
+  - ps: $env:PATH = "C:/msys64/usr/bin;$env:PATH"
 
 build_script:
   - cmd: sh -c ./appveyor.sh