test/README.testsuite

   1 The extended testsuite only works with UID=0. It contains of several
   2 subdirectories named "test/TEST-??-*", which are run one by one.
   3
   4 To run the extended testsuite do the following:
   5
   6 $ ninja -C build  # Avoid building anything as root later
   7 $ sudo test/run-integration-tests.sh
   8 ninja: Entering directory `/home/zbyszek/src/systemd/build'
   9 ninja: no work to do.
  10 --x-- Running TEST-01-BASIC --x--
  11 + make -C TEST-01-BASIC BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run
  12 make: Entering directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC'
  13 TEST-01-BASIC CLEANUP: Basic systemd setup
  14 TEST-01-BASIC SETUP: Basic systemd setup
  15 ...
  16 TEST-01-BASIC RUN: Basic systemd setup [OK]
  17 make: Leaving directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC'
  18 --x-- Result of TEST-01-BASIC: 0 --x--
  19 --x-- Running TEST-02-CRYPTSETUP --x--
  20 + make -C TEST-02-CRYPTSETUP BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run
  21
  22 If one of the tests fails, then $subdir/test.log contains the log file of
  23 the test.
  24
  25 To run just one of the cases:
  26
  27 $ sudo make -C test/TEST-01-BASIC clean setup run
  28
  29 Specifying the build directory
  30 ==============================
  31
  32 If the build directory is not detected automatically, it can be specified
  33 with BUILD_DIR=:
  34
  35 $ sudo BUILD_DIR=some-other-build/ test/run-integration-tests
  36
  37 or
  38
  39 $ sudo make -C test/TEST-01-BASIC BUILD_DIR=../../some-other-build/ ...
  40
  41 Note that in the second case, the path is relative to the test case directory.
  42 An absolute path may also be used in both cases.
  43
  44 Configuration variables
  45 =======================
  46
  47 TEST_NO_QEMU=1
  48     Don't run tests under QEMU
  49
  50 TEST_NO_NSPAWN=1
  51     Don't run tests under systemd-nspawn
  52
  53 TEST_NO_KVM=1
  54     Disable QEMU KVM auto-detection (may be necessary when you're trying to run the
  55     *vanilla* QEMU and have both qemu and qemu-kvm installed)
  56
  57 TEST_NESTED_KVM=1
  58     Allow tests to run with nested KVM. By default, the testsuite disables
  59     nested KVM if the host machine already runs under KVM. Setting this
  60     variable disables such checks
  61
  62 QEMU_MEM=512M
  63     Configure amount of memory for QEMU VMs (defaults to 512M)
  64
  65 QEMU_SMP=1
  66     Configure number of CPUs for QEMU VMs (defaults to 1)
  67
  68 KERNEL_APPEND='...'
  69     Append additional parameters to the kernel command line
  70
  71 NSPAWN_ARGUMENTS='...'
  72     Specify additional arguments for systemd-nspawn
  73
  74 QEMU_TIMEOUT=infinity
  75     Set a timeout for tests under QEMU (defaults to infinity)
  76
  77 NSPAWN_TIMEOUT=infinity
  78     Set a timeout for tests under systemd-nspawn (defaults to infinity)
  79
  80 INTERACTIVE_DEBUG=1
  81     Configure the machine to be more *user-friendly* for interactive debuggung
  82     (e.g. by setting a usable default terminal, suppressing the shutdown after
  83     the test, etc.)
  84
  85 The kernel and initramfs can be specified with $KERNEL_BIN and $INITRD.
  86 (Fedora's or Debian's default kernel path and initramfs are used by default)
  87
  88 A script will try to find your QEMU binary. If you want to specify a different
  89 one with $QEMU_BIN.
  90
  91 Debugging the qemu image
  92 ========================
  93
  94 If you want to log in the testsuite virtual machine, you can specify additional
  95 kernel command line parameter with $KERNEL_APPEND and then log in as root.
  96
  97 $ sudo make -C test/TEST-01-BASIC KERNEL_APPEND="systemd.unit=multi-user.target" run
  98
  99 Root password is empty.
 100
 101 Ubuntu CI
 102 =========
 103
 104 New PR submitted to the project are run through regression tests, and one set
 105 of those is the 'autopkgtest' runs for several different architectures, called
 106 'Ubuntu CI'.  Part of that testing is to run all these tests.  Sometimes these
 107 tests are temporarily deny-listed from running in the 'autopkgtest' tests while
 108 debugging a flaky test; that is done by creating a file in the test directory
 109 named 'deny-list-ubuntu-ci', for example to prevent the TEST-01-BASIC test from
 110 running in the 'autopkgtest' runs, create the file
 111 'TEST-01-BASIC/deny-list-ubuntu-ci'.
 112
 113 The tests may be disabled only for specific archs, by creating a deny-list file
 114 with the arch name at the end, e.g.
 115 'TEST-01-BASIC/deny-list-ubuntu-ci-arm64' to disable the TEST-01-BASIC test
 116 only on test runs for the 'arm64' architecture.
 117
 118 Note the arch naming is not from 'uname -m', it is Debian arch names:
 119 https://wiki.debian.org/ArchitectureSpecificsMemo
 120
 121 For PRs that fix a currently deny-listed test, the PR should include removal
 122 of the deny-list file.