From: Viktor Szakats Date: Wed, 10 Sep 2025 08:25:17 +0000 (+0200) Subject: GHA: minimize installed packages in http3-linux and Windows cross-builds X-Git-Tag: rc-8_17_0-2~525 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=0455d8772a1af20ce63c46c5738582aa9b1b8441;p=thirdparty%2Fcurl.git GHA: minimize installed packages in http3-linux and Windows cross-builds In the last couple of months some jobs started taking a lot of time and often timing out due to slow `apt install` from the Azure Ubuntu mirror. The jobs affected were those that installed large packages: GHA/http3-linux and the 3 cross-build jobs in GHA/windows. This patch reduces the installed packaged to the minimum required to complete the jobs. Saving a minute+ for each http3-linux job (a total of 20+ minutes for the workflow.) Also saving bandwidth and reducing the chance for long downloads or timeouts with slow Azure repos. Details: - http3: delete redundant packages from the `build-cache` job. - http3: install gnutls dependencies for gnutls jobs only. - http3: do not install test dependencies in jobs not running tests. - http3: drop redundant packages from the curl jobs. - Windows-cross: replace `mingw-w64` with `gcc-mingw-w64-x86-64-win32` for the 3 Windows cross-build job. Dropping C++, 32-bit, and 64-bit POSIX-threaded parts. Saving time and significant bandwidth for each of the 3 jobs: Download size: 277 MB -> 65 MB (installed: 1300 MB -> 400 MB) - Windows-cross: restore previous job time limit of 15m (from 45m) Follow-up to ff5140a25f42fef80325c6e28c4802fdb7e06386 #18163 Before: https://github.com/curl/curl/actions/runs/17611514207 (http3) https://github.com/curl/curl/actions/runs/17611514185/job/50034354923 (Windows cross) After: https://github.com/curl/curl/actions/runs/17628406362?pr=18509 (http3) https://github.com/curl/curl/actions/runs/17627562551/job/50088055529?pr=18509 (Windows cross) http3 job | Bef. | Aft. | :------------------ | ------: | ------: | Build caches (hot) | 10s | 12s | AM awslc | 3m 0s | 1m 54s | CM awslc | 4m 32s | 3m 4s | AM boringssl | 3m 9s | 1m 48s | CM boringssl | 3m 43s | 3m 2s | AM gnutls | 3m 9s | 2m 18s | CM gnutls | 4m 19s | 2m 55s | AM libressl | 2m 14s | 1m 24s | CM libressl | 5m 30s | 2m 57s | AM openssl | 5m 16s | 4m 17s | CM openssl | 1m 50s | 1m 47s | AM openssl-quic | 2m 58s | 1m 7s | CM openssl-quic | 4m 16s | 2m 43s | AM quiche | 2m 54s | 1m 34s | CM quiche | 5m 0s | 3m 15s | AM quictls | 2m 34s | 1m 13s | CM quictls | 4m 20s | 3m 17s | AM wolfssl | 2m 48s | 1m 30s | CM wolfssl | 4m 49s | 3m 22s | Total: | 66m 21s | 43m 27s | Gain: | | 22m 54s | Out of curiousity, build times as seen in the http3 build-cache job: - TLS backends: - openssl: 2m25s - libressl: 27s - aws-lc: 41s - boringssl: 1m8s - quictls: 1m46s - gnutls: 6m30s - wolfssl: 51s - quiche + boringssl: 1m9s - ng* libs (not yet optimized for build speed): - nghttp3: 13s - ngtcp2: 52s (with 6 backends, 3 runs) - ngtcp2: 19s (boringssl) - nghttp2: 21s Ref: https://github.com/curl/curl/actions/runs/17626120054/job/50083344805 A similar effort in curl-for-win, affecting 2 GHA/curl-for-win Windows jobs (though they use the default Debian repo, with no issues): - with llvm/clang: Download size: 648 MB -> 430 MB (installed: 3344 MB -> 2333 MB) - with gcc: Download size: 550 MB -> 328 MB (installed: 2815 MB -> 1804 MB) Ref: https://github.com/curl/curl-for-win/commit/e19665d9486bdca60f996ed2e198a66128cfba38 Ref: https://github.com/curl/curl-for-win/commit/6b14c3946a8c89dc1d3847afc9501fc71f3ac628 Bug: https://github.com/curl/curl/pull/18502#issuecomment-3270259744 Closes #18509 --- diff --git a/.github/workflows/http3-linux.yml b/.github/workflows/http3-linux.yml index a317f99783..1a7ff07d5f 100644 --- a/.github/workflows/http3-linux.yml +++ b/.github/workflows/http3-linux.yml @@ -192,12 +192,8 @@ jobs: sudo apt-get -o Dpkg::Use-Pty=0 update sudo rm -f /var/lib/man-db/auto-update sudo apt-get -o Dpkg::Use-Pty=0 install \ - libtool autoconf automake pkgconf stunnel4 \ - libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libev-dev libc-ares-dev \ - nettle-dev libp11-kit-dev libtspi-dev libunistring-dev guile-2.2-dev libtasn1-bin \ - libtasn1-6-dev libidn2-0-dev gawk gperf libtss2-dev dns-root-data bison gtk-doc-tools \ - texinfo texlive texlive-extra-utils autopoint libev-dev \ - apache2 apache2-dev libnghttp2-dev dante-server + libtool autoconf automake pkgconf \ + nettle-dev libp11-kit-dev libev-dev autopoint bison gperf gtk-doc-tools libtasn1-bin # for gnutls echo 'CC=gcc-12' >> "$GITHUB_ENV" echo 'CXX=g++-12' >> "$GITHUB_ENV" @@ -350,6 +346,7 @@ jobs: timeout-minutes: 45 env: MATRIX_BUILD: ${{ matrix.build.generate && 'cmake' || 'autotools' }} + MATRIX_INSTALL_PACKAGES: '${{ matrix.build.install_packages }}' strategy: fail-fast: false matrix: @@ -430,6 +427,7 @@ jobs: -DUSE_NGTCP2=ON -DCURL_DISABLE_NTLM=ON - name: 'gnutls' + install_packages: nettle-dev libp11-kit-dev install_steps: skipall PKG_CONFIG_PATH: /home/runner/gnutls/build/lib/pkgconfig:/home/runner/nghttp3/build/lib/pkgconfig:/home/runner/ngtcp2/build/lib/pkgconfig:/home/runner/nghttp2/build/lib/pkgconfig configure: >- @@ -438,6 +436,7 @@ jobs: --with-gnutls=/home/runner/gnutls/build --enable-ssls-export - name: 'gnutls' + install_packages: nettle-dev libp11-kit-dev PKG_CONFIG_PATH: /home/runner/gnutls/build/lib/pkgconfig:/home/runner/nghttp3/build/lib/pkgconfig:/home/runner/ngtcp2/build/lib/pkgconfig:/home/runner/nghttp2/build/lib/pkgconfig generate: >- -DCURL_USE_GNUTLS=ON @@ -493,17 +492,20 @@ jobs: steps: - name: 'install prereqs' + env: + INSTALL_PACKAGES: >- + ${{ !contains(matrix.build.install_steps, 'skipall') && !contains(matrix.build.install_steps, 'skiprun') && 'stunnel4 ' || '' }} + ${{ !contains(matrix.build.install_steps, 'skipall') && !contains(matrix.build.install_steps, 'skiprun') && 'apache2 apache2-dev libnghttp2-dev vsftpd dante-server' || '' }} + run: | sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list sudo apt-get -o Dpkg::Use-Pty=0 update sudo rm -f /var/lib/man-db/auto-update sudo apt-get -o Dpkg::Use-Pty=0 install \ - libtool autoconf automake pkgconf stunnel4 \ - libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libev-dev libc-ares-dev \ - nettle-dev libp11-kit-dev libtspi-dev libunistring-dev guile-2.2-dev libtasn1-bin \ - libtasn1-6-dev libidn2-0-dev gawk gperf libtss2-dev dns-root-data bison gtk-doc-tools \ - texinfo texlive texlive-extra-utils autopoint libev-dev libuv1-dev \ - apache2 apache2-dev libnghttp2-dev vsftpd dante-server + libtool autoconf automake pkgconf \ + libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libidn2-0-dev libuv1-dev \ + ${INSTALL_PACKAGES} \ + ${MATRIX_INSTALL_PACKAGES} python3 -m venv ~/venv echo 'CC=gcc-12' >> "$GITHUB_ENV" echo 'CXX=g++-12' >> "$GITHUB_ENV" diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index 27b29ce429..0938142df9 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -706,7 +706,7 @@ jobs: linux-cross-mingw-w64: name: "linux-mingw, ${{ matrix.build == 'cmake' && 'CM' || 'AM' }} ${{ matrix.compiler }}" runs-on: ubuntu-latest - timeout-minutes: 45 + timeout-minutes: 15 env: MAKEFLAGS: -j 5 TRIPLET: 'x86_64-w64-mingw32' @@ -725,7 +725,7 @@ jobs: INSTALL_PACKAGES: ${{ matrix.compiler == 'clang-tidy' && 'clang' || '' }} run: | sudo rm -f /var/lib/man-db/auto-update - sudo apt-get -o Dpkg::Use-Pty=0 install mingw-w64 ${INSTALL_PACKAGES} + sudo apt-get -o Dpkg::Use-Pty=0 install gcc-mingw-w64-x86-64-win32 ${INSTALL_PACKAGES} - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5 with: