From: Joe Orton Date: Mon, 10 Aug 2026 10:58:18 +0000 (+0000) Subject: Consolidate Linux CI pytest-based test execution: X-Git-Url: http://git.ipfire.org/gitweb/index.cgi?a=commitdiff_plain;h=d53bd3dd00b3428583c4e5b1cd95bfcb19272f61;p=thirdparty%2Fapache%2Fhttpd.git Consolidate Linux CI pytest-based test execution: * test/travis_run_linux.sh: Replace TEST_CORE/TEST_PROXY/TEST_H2 with one TEST_PYTEST block, running `make check-all-pytest` over every test/modules/*/ suite except modules/md (needs an ACME/pebble server not available here) and mod_http2 if not built. * .github/workflows/linux.yml: Replace the "HTTP/2 test suite" job and the disabled "ACME test suite" job with one "Python pytest test suites" job setting TEST_PYTEST=1. Drop TEST_INSTALL: check-all-pytest tests the check/ build tree directly. Drop the python3-pytest/ python3-cryptography/etc. packages, since both suites now manage their own dependencies via uv; install uv via pipx instead. Set TEST_PYTEST=1 for the UBSan, ASAN jobs. * test/README.ci: Updated. Assisted-by: Claude Sonnet 5 GitHub: PR #701 git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1937026 13f79535-47bb-0310-9956-ffa450edef68 --- diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index 38d9af194e..0992098c20 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -219,56 +219,46 @@ jobs: env: | NOTEST_LIBS=-lubsan TEST_UBSAN=1 + TEST_PYTEST=1 # ------------------------------------------------------------------------- - name: ASan notest-cflags: -ggdb -fsanitize=address -fno-sanitize-recover=address -fno-omit-frame-pointer config: --enable-mods-shared=reallyall + pkgs: nghttp2-client env: | APR_VERSION=1.7.x APU_VERSION=1.7.x APU_CONFIG="--with-crypto --with-ldap" TEST_ASAN=1 + TEST_PYTEST=1 CLEAR_CACHE=1 # ------------------------------------------------------------------------- - name: ASan, pool-debug notest-cflags: -ggdb -fsanitize=address -fno-sanitize-recover=address -fno-omit-frame-pointer config: --enable-mods-shared=reallyall + pkgs: nghttp2-client env: | APR_VERSION=1.7.x APR_CONFIG="--enable-pool-debug" APU_VERSION=1.7.x APU_CONFIG="--with-crypto --with-ldap" TEST_ASAN=1 + TEST_PYTEST=1 CLEAR_CACHE=1 # ------------------------------------------------------------------------- - - name: HTTP/2 test suite + # Runs every pytest-based test suite (pytest_suite/ + all + # test/modules/*/ pyhttpd suites except modules/md, which needs a + # local ACME/pebble server that isn't available here) via `make + # check-all-pytest`. See TEST_PYTEST in test/travis_run_linux.sh. + - name: Python pytest test suites config: --enable-mods-shared=reallyall --with-mpm=event --enable-mpms-shared=all - pkgs: curl python3-pytest nghttp2-client python3-cryptography python3-requests python3-multipart python3-filelock python3-websockets + pkgs: nghttp2-client env: | APR_VERSION=1.7.6 APU_VERSION=1.6.3 APU_CONFIG="--with-crypto" NO_TEST_FRAMEWORK=1 - TEST_INSTALL=1 - TEST_H2=1 - TEST_CORE=1 - TEST_PROXY=1 - # ------------------------------------------------------------------------- - ### TODO: if: *condition_not_24x - ### TODO: pebble install is broken. - # - name: ACME test suite - # config: --enable-mods-shared=reallyall --with-mpm=event --enable-mpms-shared=event - # pkgs: >- - # python3-pytest nghttp2-client python3-cryptography python3-requests python3-filelock - # golang-1.17 curl - # env: | - # APR_VERSION=1.7.6 - # APU_VERSION=1.6.3 - # APU_CONFIG="--with-crypto" - # GOROOT=/usr/lib/go-1.17 - # NO_TEST_FRAMEWORK=1 - # TEST_INSTALL=1 - # TEST_MD=1 + TEST_PYTEST=1 # ------------------------------------------------------------------------- ### TODO: if: *condition_not_24x - name: Configured w/reduced exports @@ -365,9 +355,9 @@ jobs: - name: Install prerequisites run: sudo apt-get install -o Acquire::Retries=5 cpanminus libtool-bin libapr1-dev libaprutil1-dev - liblua5.3-dev libbrotli-dev libcurl4-openssl-dev + liblua5.3-dev libbrotli-dev libcurl4-openssl-dev libnghttp2-dev libjansson-dev libpcre2-dev gdb - perl-doc libsasl2-dev ${{ matrix.pkgs }} check + perl-doc libsasl2-dev curl pipx ${{ matrix.pkgs }} check - uses: actions/checkout@v6 - uses: actions/checkout@v6 with: @@ -394,6 +384,11 @@ jobs: name: config.log-${{ env.JOBID }} path: | /home/runner/build/**/config.log + - name: Install uv + if: env.TEST_PYTEST == '1' + run: | + pipx install uv + echo "$HOME/.local/bin" >> $GITHUB_PATH - name: Build and test run: ./test/travis_run_linux.sh - uses: actions/upload-artifact@v7 @@ -403,3 +398,5 @@ jobs: path: | **/config.log test/perl-framework/t/logs/error_log + test/pytest_suite/t/logs/error_log + test/gen/apache/logs/error_log diff --git a/test/README.ci b/test/README.ci index 7b28d28d8b..784652574b 100644 --- a/test/README.ci +++ b/test/README.ci @@ -29,22 +29,62 @@ The CI scripts use the following environment variables: * NO_TEST_FRAMEWORK - if set, the Perl-based test framework and libcheck-based unit tests are skipped +* TEST_LOGNO - if set, checks for APLOGNO() macro errors (duplicates + or empty arguments) and exits; no build or testing is done. + * TEST_UBSAN - set for job using UBSan ("Undefined Behaviour Sanitizer") -* TEST_MALLOC - set for job using enhanced malloc debugging. +* TEST_ASAN - set for job using ASan ("Address Sanitizer") + +* TEST_MALLOC - set for job using enhanced malloc debugging + (sets MALLOC_PERTURB_, MALLOC_CHECK_ and LIBC_FATAL_STDERR_). * TEST_INSTALL - set for job testing "make install" -* TEST_VPATH - set for job testing srcdir!=builddir +* TEST_VPATH - set for job testing srcdir!=builddir (VPATH build) * TEST_LDAP - set for job with slapd, running LDAP tests -* TEST_SSL - set for job with SSL/TLS testing variants +* TEST_SSL - set for job with SSL/TLS testing variants (encrypted + private keys, session cache backends via redis/memcache/shmcb, etc.) + +* TEST_OPENSSL3 - if set to a version string (e.g. "3.4.6"), that + version of OpenSSL is built from source and used for mod_ssl. + Requires non-system APR/APR-util (APR_VERSION must also be set). + +* TEST_OPENSSL3_BRANCH - if set together with TEST_OPENSSL3, the + OpenSSL source is cloned from the named git branch instead of + downloading a release tarball. + +* OPENSSL_CONFIG - extra arguments passed to OpenSSL's ./Configure + when building from source (e.g. "no-engine"). + +* TEST_PYTEST - if set, runs all available pytest-based test suites + via "make check-all-pytest" (pytest_suite/ and test/modules/*/ + pyhttpd suites). Replaces the old per-suite TEST_CORE / TEST_PROXY + / TEST_H2 / TEST_MD flags. + +* TEST_MOD_TLS - if set (together with RUSTLS_VERSION), builds + rustls-ffi from source and configures httpd with --with-tls + --with-rustls for mod_tls testing. + +* TEST_MD - (currently unused) if set, builds pebble and runs the + mod_md ACME test suite. Preserved for future use once a working + pebble build is available; see comments in travis_run_linux.sh. + +* TEST_APR_TARBALL - if set, APR/APR-util source is fetched as a + release tarball from archive.apache.org rather than cloned from git. * TESTS - a list of Perl framework tests to run * TEST_ARGS - arguments to pass to ./t/TEST in the Perl test framework +* LITMUS - if set, runs the litmus WebDAV test suite against mod_dav. + +* PHP_FPM - path to the php-fpm binary. If set, PHP tests are + included in the pytest test suites. Set globally in the workflow + (e.g. "/usr/sbin/php-fpm8.3"). + * CLEAR_CACHE - if set, the cached $HOME/root is removed before each build Caching diff --git a/test/travis_run_linux.sh b/test/travis_run_linux.sh index 67378192a3..d7771148e1 100755 --- a/test/travis_run_linux.sh +++ b/test/travis_run_linux.sh @@ -238,39 +238,50 @@ if test -v LITMUS -a $RV -eq 0; then popd fi -if test -v TEST_CORE -a $RV -eq 0; then - # Run core module tests. - MPM=event py.test-3 test/modules/core - RV=$? -fi - -if test -v TEST_PROXY -a $RV -eq 0; then - # Run proxy tests. - py.test-3 test/modules/proxy - RV=$? -fi - -if test -v TEST_H2 -a $RV -eq 0; then - # Build the test clients +if test -v TEST_PYTEST -a $RV -eq 0; then + # Run all available pytest-based test suites against this build, via + # the unified `make check-all-pytest` target: pytest_suite/ (the + # self-contained port of the classic Apache::Test suite, incl. PHP + # tests if PHP_FPM is set) and every test/modules/*/ pyhttpd suite + # (core, http1, http2, proxy, ssl, aaa, ...). This replaces the old + # per-suite TEST_CORE / TEST_PROXY / TEST_H2 / TEST_MD flags, which + # each invoked py.test-3 directly against a `make install`ed tree; + # check-all-pytest instead builds and tests entirely from the in-tree + # check/ build, needing no install. + # + # modules/md is excluded: its ACME tests need a local pebble CA server, + # which isn't available here (built from source, pebble's Go module + # currently fails to build against modern Go -- see the old commit + # history for the details of that dead end). + # + # modules/http2 is excluded when mod_http2 wasn't built (e.g. the + # UBSan job's --disable-http2): its pytest package hard-requires + # both http2 and proxy_http2 to load, and errors at fixture setup + # otherwise rather than skipping. (cd test/clients && make) - # Run HTTP/2 tests. - MPM=event py.test-3 test/modules/http2 + targets="" + for d in test/modules/*/; do + name=$(basename "$d") + case "$name" in + md|__pycache__) continue ;; + http2) test -f modules/http2/.libs/mod_http2.so || continue ;; + esac + targets="$targets modules/$name" + done + PYHTTPD_TARGETS="$targets" make check-all-pytest RV=$? - if test $RV -eq 0; then - MPM=worker py.test-3 test/modules/http2 - RV=$? - fi fi if test -v TEST_MD -a $RV -eq 0; then - # Run ACME tests. - # need the go based pebble as ACME test server - # which is a package on debian sid, but not on focal - # FAILS on TRAVIS with + # Preserved for reference only: nothing sets TEST_MD, so this never + # runs. modules/md is covered by TEST_PYTEST's check-all-pytest run above + # for everything except its ACME tests, which need a local pebble CA + # server -- building pebble from source last failed with: # package github.com/letsencrypt/pebble/cmd/pebble # imports crypto/ed25519: unrecognized import path "crypto/ed25519" (import path does not begin with hostname) # - # but works on a docker ubuntu-focal image. ??? + # Revive this (e.g. once a working pebble build/package is available) + # by setting TEST_MD=1 on a job and ensuring GOROOT/GOPATH are usable. export GOPATH=${PREFIX}/gocode mkdir -p "${GOPATH}" export PATH="${GOROOT}/bin:${GOPATH}/bin:${PATH}"