ci: add support for GitLab CI

We already support Azure Pipelines and GitHub Workflows in the Git
project, but until now we do not have support for GitLab CI. While it is
arguably not in the interest of the Git project to maintain a ton of
different CI platforms, GitLab has recently ramped up its efforts and
tries to contribute to the Git project more regularly.

Part of a problem we hit at GitLab rather frequently is that our own,
custom CI setup we have is so different to the setup that the Git
project has. More esoteric jobs like "linux-TEST-vars" that also set a
couple of environment variables do not exist in GitLab's custom CI
setup, and maintaining them to keep up with what Git does feels like
wasted time. The result is that we regularly send patch series upstream
that fail to compile or pass tests in GitHub Workflows. We would thus
like to integrate the GitLab CI configuration into the Git project to
help us send better patch series upstream and thus reduce overhead for
the maintainer. Results of these pipeline runs will be made available
(at least) in GitLab's mirror of the Git project at [1].

This commit introduces the integration into our regular CI scripts so
that most of the setup continues to be shared across all of the CI
solutions. Note that as the builds on GitLab CI run as unprivileged
user, we need to pull in both sudo and shadow packages to our Alpine
based job to set this up.

[1]: https://gitlab.com/gitlab-org/git

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644 (file)
index 0000000..cd98bcb
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,53 @@
+default:
+  timeout: 2h
+
+workflow:
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+    - if: $CI_COMMIT_TAG
+    - if: $CI_COMMIT_REF_PROTECTED == "true"
+
+test:
+  image: $image
+  before_script:
+    - ./ci/install-docker-dependencies.sh
+  script:
+    - useradd builder --create-home
+    - chown -R builder "${CI_PROJECT_DIR}"
+    - sudo --preserve-env --set-home --user=builder ./ci/run-build-and-tests.sh
+  after_script:
+    - |
+      if test "$CI_JOB_STATUS" != 'success'
+      then
+        sudo --preserve-env --set-home --user=builder ./ci/print-test-failures.sh
+      fi
+  parallel:
+    matrix:
+      - jobname: linux-sha256
+        image: ubuntu:latest
+        CC: clang
+      - jobname: linux-gcc
+        image: ubuntu:20.04
+        CC: gcc
+        CC_PACKAGE: gcc-8
+      - jobname: linux-TEST-vars
+        image: ubuntu:20.04
+        CC: gcc
+        CC_PACKAGE: gcc-8
+      - jobname: linux-gcc-default
+        image: ubuntu:latest
+        CC: gcc
+      - jobname: linux-leaks
+        image: ubuntu:latest
+        CC: gcc
+      - jobname: linux-asan-ubsan
+        image: ubuntu:latest
+        CC: clang
+      - jobname: pedantic
+        image: fedora:latest
+      - jobname: linux-musl
+        image: alpine:latest
+  artifacts:
+    paths:
+      - t/failed-test-artifacts
+    when: on_failure

diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
index 6e8452836802c1d57ac162de6f1cbb0c19ef95c1..48c43f0f90774e65fcc8b388c6a52444fde50506 100755 (executable)
--- a/ci/install-docker-dependencies.sh
+++ b/ci/install-docker-dependencies.sh
@@ -16,11 +16,22 @@ linux32)
        '
        ;;
 linux-musl)
-       apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+       apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
                pcre2-dev python3 musl-libintl perl-utils ncurses \
                apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
                bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
        ;;
+linux-*)
+       # Required so that apt doesn't wait for user input on certain packages.
+       export DEBIAN_FRONTEND=noninteractive
+
+       apt update -q &&
+       apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \
+               libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \
+               perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \
+               libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \
+               apache2 cvs cvsps gnupg libcgi-pm-perl subversion
+       ;;
 pedantic)
        dnf -yq update >/dev/null &&
        dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null

diff --git a/ci/lib.sh b/ci/lib.sh
index eab0e24080e6dd2de643229dcc20404f10709bc7..6dfc90d7f53b1b171d08abb12876da45d83e46c9 100755 (executable)
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -14,6 +14,22 @@ then
                need_to_end_group=
                echo '::endgroup::' >&2
        }
+elif test true = "$GITLAB_CI"
+then
+       begin_group () {
+               need_to_end_group=t
+               printf "\e[0Ksection_start:$(date +%s):$(echo "$1" | tr ' ' _)\r\e[0K$1\n"
+               trap "end_group '$1'" EXIT
+               set -x
+       }
+
+       end_group () {
+               test -n "$need_to_end_group" || return 0
+               set +x
+               need_to_end_group=
+               printf "\e[0Ksection_end:$(date +%s):$(echo "$1" | tr ' ' _)\r\e[0K\n"
+               trap - EXIT
+       }
 else
        begin_group () { :; }
        end_group () { :; }
@@ -229,6 +245,35 @@ then
 
        GIT_TEST_OPTS="--github-workflow-markup"
        JOBS=10
+elif test true = "$GITLAB_CI"
+then
+       CI_TYPE=gitlab-ci
+       CI_BRANCH="$CI_COMMIT_REF_NAME"
+       CI_COMMIT="$CI_COMMIT_SHA"
+       case "$CI_JOB_IMAGE" in
+       macos-*)
+               CI_OS_NAME=osx;;
+       alpine:*|fedora:*|ubuntu:*)
+               CI_OS_NAME=linux;;
+       *)
+               echo "Could not identify OS image" >&2
+               env >&2
+               exit 1
+               ;;
+       esac
+       CI_REPO_SLUG="$CI_PROJECT_PATH"
+       CI_JOB_ID="$CI_JOB_ID"
+       CC="${CC_PACKAGE:-${CC:-gcc}}"
+       DONT_SKIP_TAGS=t
+       handle_failed_tests () {
+               create_failed_test_artifacts
+               return 1
+       }
+
+       cache_dir="$HOME/none"
+
+       runs_on_pool=$(echo "$CI_JOB_IMAGE" | tr : -)
+       JOBS=$(nproc)
 else
        echo "Could not identify CI type" >&2
        env >&2

diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0c8b6117fd71e9adb3a526baa05ef1..c33ad4e3a2220a97f6e0708afe915010f4f44561 100755 (executable)
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -51,6 +51,12 @@ do
                        tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
                        continue
                        ;;
+               gitlab-ci)
+                       mkdir -p failed-test-artifacts
+                       cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
+                       tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
+                       continue
+                       ;;
                *)
                        echo "Unhandled CI type: $CI_TYPE" >&2
                        exit 1