Add Windows to GitLab CI

Ensure BIND can be tested on Windows in GitLab to more quickly catch
build and test errors on that operating system.

Some notes:

  - While build jobs are triggered for all pipelines, system test jobs
    are not - due to the time it takes to run the complete system test
    suite on Windows (about 20 minutes), the latter are only run for
    pipelines created through GitLab's web interface and for pipelines
    created for Git tags.

  - Only the "Release" build configuration is currently used.  Adding
    "Debug" builds is a matter of extending .gitlab-ci.yml, but it was
    not done for the time being due to questionable usefulness of
    performing such builds in GitLab CI.

  - Only a 64-bit build is performed.  Adding support for 32-bit builds
    is not planned to be implemented.

  - Unit tests are still not run on Windows, but adding support for that
    is on the roadmap.

  - All Windows GitLab CI jobs are run inside Windows Server containers,
    using the Custom executor feature of GitLab Runner as Windows Server
    2016 is not supported by GitLab Runner's native Docker on Windows
    executor and Windows Server 2019 is not yet widely available from
    hosting providers.

  - The Windows Docker image used by GitLab CI is not stored in the
    GitLab Container Registry as it is over 27 GB in size and thus
    passing it between GitLab and its runners is impractical.

  - There is no vcvarsall.bat variant written in PowerShell and batch
    scripts are no longer supported by GitLab Runner Custom executor, so
    the environment variables set by vcvarsall.bat are injected back
    into the PowerShell environment by processing the output of "set".

  - Visual Studio parallel builds are a bit different than "make -jX"
    builds as parallelization happens in two tiers: project parallelism
    (controlled by the "/maxCpuCount" msbuild.exe switch) and compiler
    parallelism (controlled by the "/MP" cl.exe switch).  To limit the
    total number of compiler processes spawned concurrently to a value
    similar to the one used for Unix builds, msbuild.exe is allowed to
    build at most 2 projects at once, each of which can spawn up to half
    of BUILD_PARALLEL_JOBS worth of compiler processes.  Using such
    parameters is a fairly arbitrary decision taken to solve the
    trade-off between compilation speed and runner load.

  - Configuring network addresses in Windows Server containers is
    tricky.  Adding 10.53.0.1/24 and similar addresses to the vEthernet
    interface created by Docker never causes ifconfig.bat to fail, but
    in fact only one container can have any given IP address configured
    at any given time (the request to add the same address in another
    container is silently ignored).  Thus, in order to allow multiple
    system test jobs to be run in parallel, the addresses used in system
    tests are configured on the loopback interfaces.  Interestingly
    enough, the addresses set on the loopback interfaces... persist
    between containers.  Fortunately, this is acceptable for the time
    being and only requires ifconfig.bat failures to be ignored (as
    ifconfig.bat will fail if it attempts to configure an already
    existing address on an interface).  We also need to wait for a brief
    moment after calling ifconfig.bat as the addresses the latter
    attempts to configure may not be immediately available after it
    returns (and that causes runall.sh to error out).  Finally, for some
    reason we also need to signal that the DNS servers on each loopback
    interface are to be configured using DHCP or else ifconfig.bat will
    fail to add the requested addresses.

  - Since named.pid files created by named instances used in system
    tests contain Windows PIDs instead of Cygwin PIDs and various
    versions of Cygwin "kill" react differently when passed Windows PIDs
    without the -W switch, all "kill" invocations in GitLab CI need to
    use that switch (otherwise they would print error messages which
    would cause stop.pl to assume the process being killed died
    prematurely).  However, to preserve compatibility with older Cygwin
    versions used in our other Windows test environments, we alter the
    relevant scripts "on the fly" rather than in the Git repository.

  - In the containers used for running system tests, Windows Error
    Reporting is configured to automatically create crash dumps in
    C:\CrashDumps.  This directory is examined after the test suite is
    run to ensure no crashes went under stop.pl's radar.

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d56f50bdb4e35904b9fd474baef4b900f4963176..29b31eb4ecdbe91b6c39dc01b85fdc6979e6fe3a 100644 (file)
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -737,3 +737,62 @@ unit:nolibtool:sid:amd64:
   dependencies:
     - nolibtool:sid:amd64
   needs: ["nolibtool:sid:amd64"]
+
+# Jobs for Visual Studio 2017 builds on Windows (amd64)
+
+msvc:windows:amd64:
+  <<: *default_triggering_rules
+  stage: build
+  tags:
+    - windows
+    - amd64
+  variables:
+    VSCONF: Release
+  script:
+    - 'Push-Location "C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Auxiliary/Build"'
+    - '& cmd.exe /C "vcvarsall.bat x64 & set" | Foreach-Object { if ($_ -match "(.*?)=(.*)") { Set-Item -force -path "Env:\$($matches[1])" -value "$($matches[2])" } }'
+    - 'Pop-Location'
+    - 'Set-Location win32utils'
+    - '& "C:/Strawberry/perl/bin/perl.exe" Configure
+         "with-tools-version=15.0"
+         "with-platform-toolset=v141"
+         "with-platform-version=10.0.17763.0"
+         "with-vcredist=C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Redist/MSVC/14.16.27012/vcredist_x64.exe"
+         "with-openssl=C:/OpenSSL"
+         "with-libxml2=C:/libxml2"
+         "without-python"
+         "with-system-tests"
+         x64'
+    - 'Set-Item -path "Env:CL" -value "/MP$([Math]::Truncate($BUILD_PARALLEL_JOBS/2))"'
+    - '& msbuild.exe /maxCpuCount:2 /t:Build /p:Configuration=$VSCONF bind9.sln'
+  artifacts:
+    untracked: true
+    expire_in: "1 hour"
+
+system:msvc:windows:amd64:
+  stage: system
+  tags:
+    - windows
+    - amd64
+  variables:
+    VSCONF: Release
+  script:
+    - 'Push-Location bin/tests/system'
+    - '$ifIndex = Get-NetIPInterface -AddressFamily IPv4 -InterfaceMetric 75 | Select-Object -ExpandProperty ifIndex'
+    - '& C:/tools/cygwin/bin/sed.exe -i "s/^exit.*/netsh interface ipv4 set dnsservers $ifIndex dhcp/; s/\(name\|interface\)=Loopback/$ifIndex/;" ifconfig.bat'
+    - '& C:/tools/cygwin/bin/sed.exe -i "s/kill -f/kill -W/;" conf.sh stop.pl'
+    - '& cmd.exe /C ifconfig.bat up; ""'
+    - 'Start-Sleep 2'
+    - '$Env:Path = "C:/tools/cygwin/bin;$Env:Path"'
+    - '& sh.exe runall.sh $TEST_PARALLEL_JOBS'
+    - 'If (Test-Path C:/CrashDumps/*) { dir C:/CrashDumps; Throw }'
+  dependencies:
+    - msvc:windows:amd64
+  needs: ["msvc:windows:amd64"]
+  artifacts:
+    untracked: true
+    expire_in: "1 week"
+    when: on_failure
+  only:
+    - tags
+    - web