Bake CI tooling and registry-mirror config into custom images

Two additions to docker-refresh.yml's matrix:

  - Extend the existing self-hosted-ubuntu24 image (scripts/ci/Dockerfile)
    to install docker.io, docker-buildx and m4, and pre-set git's
    safe.directory='*' system config. These are the per-job knobs
    setup-dind currently apt-installs and git-configures on every CI
    run; baking them in saves ~20-30s per dind-based job.

  - New scripts/ci/Dockerfile.dind, published as
    docker.internal.networkradius.com/fr-dind. It's docker:24-dind
    with /etc/docker/daemon.json baked in pointing at the internal
    NetworkRADIUS registry as a Docker Hub pull-through mirror, and
    the internal CA pre-trusted. This is what the services: dind:
    block in our dind-based workflows should reference once the image
    is built and pushed - it removes the per-job anonymous Docker Hub
    pulls that have been hitting the 100/6h rate limit.

The dind image needs the internal CA cert which lives on the runner
host (not in the repo); a new "Stage internal CA into build context"
step copies it into the build context just before docker build for
matrix entries that set needs_internal_ca: true.

Once docker-refresh is run on master, the dind-based workflows can be
simplified to reference the new images and drop most of setup-dind.

diff --git a/.github/workflows/docker-refresh.yml b/.github/workflows/docker-refresh.yml
index f600149bb7f1f6c255cd1718fe83aedfc3295e58..4b6c5ecf69403d911d34da76a4dc42f500a5f326 100644 (file)
--- a/.github/workflows/docker-refresh.yml
+++ b/.github/workflows/docker-refresh.yml
@@ -32,6 +32,18 @@ jobs:
               - docker.internal.networkradius.com/self-hosted
             dockerfile: scripts/ci/Dockerfile
 
+          #
+          #  Custom dind image used as the services: sidecar by the
+          #  dind-based workflows (docker.yml, crossbuild.yml, multi-
+          #  server-tests). Bakes in registry-mirror -> internal cache
+          #  and trusts the internal CA, so per-job apt-installing and
+          #  daemon.json wiring goes away.
+          #
+          - base_image: docker:24-dind
+            image_name: docker.internal.networkradius.com/fr-dind
+            dockerfile: scripts/ci/Dockerfile.dind
+            needs_internal_ca: true
+
           - base_image: mariadb
           - base_image: postgres
           - base_image: 4teamwork/389ds
@@ -76,6 +88,20 @@ jobs:
             ${{ matrix.os.base_image }} \
             sh -c './build-dep-pkg.sh'
 
+      #
+      #  Stage the internal CA cert into the build context for any
+      #  matrix entry that asks for it (currently the dind sidecar
+      #  image, which needs to trust docker.internal.networkradius.com).
+      #  The cert lives on the self-hosted runner host outside the
+      #  repo, so we copy it in just before the build.
+      #
+      - name: Stage internal CA into build context
+        if: ${{ matrix.os.needs_internal_ca }}
+        shell: bash
+        run: |
+          cp /usr/local/share/ca-certificates/networkradius.com.crt \
+             scripts/ci/networkradius.com.crt
+
       - name: Build main Docker image
         if: ${{ matrix.os.image_name && matrix.os.dockerfile }}
         shell: bash

diff --git a/scripts/ci/Dockerfile b/scripts/ci/Dockerfile
index 8b6ccf308110ee1fba018c769d558787f44f3c5f..cf65b9c364ae54e39172a71786b3a40bbb0fc997 100644 (file)
--- a/scripts/ci/Dockerfile
+++ b/scripts/ci/Dockerfile
@@ -21,7 +21,13 @@ RUN apt-get update && \
        apt-get dist-upgrade -y
 
 #
-#  Install packages needed by the build
+#  Install packages needed by the build.
+#
+#  docker.io, docker-buildx and m4 are needed when this image is used
+#  as the job container of a dind-based workflow (docker.yml,
+#  crossbuild.yml, multi-server-tests): the in-container docker CLI
+#  talks to the dind sidecar over DOCKER_HOST, and m4 regenerates the
+#  per-distro Dockerfiles from the m4 templates.
 #
 RUN apt-get install -y --no-install-recommends \
        apt-transport-https \
@@ -29,6 +35,8 @@ RUN apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
        devscripts \
+       docker-buildx \
+       docker.io \
        equivs \
        gawk \
        git \
@@ -36,6 +44,7 @@ RUN apt-get install -y --no-install-recommends \
        gnupg \
        libasan6 \
        lsb-release \
+       m4 \
        python3-pip \
        quilt \
        ruby-dev \
@@ -163,6 +172,14 @@ RUN apt-get install -y --no-install-recommends \
 #      samba \
 
 
+#
+#  Trust any workspace path. The job container runs as root but the
+#  bind-mounted runner workspace is owned by the runner user; without
+#  this, git refuses with "dubious ownership" the moment a Makefile
+#  runs `git rev-parse`. system-level so it applies to every shell.
+#
+RUN git config --system --add safe.directory '*'
+
 #
 #  Additional improvements
 #  - install eapol_test

diff --git a/scripts/ci/Dockerfile.dind b/scripts/ci/Dockerfile.dind
new file mode 100644 (file)
index 0000000..f7ec670
--- /dev/null
+++ b/scripts/ci/Dockerfile.dind
@@ -0,0 +1,32 @@
+FROM docker:24-dind
+
+#
+#  Pre-bake the registry-mirror config so the dockerd inside this
+#  container pulls public base images via the internal NetworkRADIUS
+#  registry (which acts as a Docker Hub pull-through cache). Without
+#  this, every CI job's fresh dockerd anonymously pulls from
+#  registry-1.docker.io and the 31 self-hosted runners share egress
+#  IPs that quickly blow through Docker Hub's 100/6h anonymous limit.
+#
+RUN mkdir -p /etc/docker && cat > /etc/docker/daemon.json <<'EOF'
+{
+  "registry-mirrors": ["https://docker.internal.networkradius.com"]
+}
+EOF
+
+#
+#  Trust the internal CA so dockerd can talk HTTPS to the mirror at
+#  docker.internal.networkradius.com. The base docker:dind image only
+#  has Alpine's default CA bundle, which doesn't include the internal
+#  NR root.
+#
+#  The cert file is staged into the build context by docker-refresh.yml
+#  from /usr/local/share/ca-certificates/networkradius.com.crt on the
+#  runner host before this image is built.
+#
+COPY scripts/ci/networkradius.com.crt /usr/local/share/ca-certificates/networkradius.com.crt
+RUN apk add --no-cache ca-certificates && \
+    update-ca-certificates && \
+    mkdir -p /etc/docker/certs.d/docker.internal.networkradius.com && \
+    cp /usr/local/share/ca-certificates/networkradius.com.crt \
+       /etc/docker/certs.d/docker.internal.networkradius.com/ca.crt