Documentation/howto-tests.txt

   1
   2  util-linux regression tests
   3  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
   4
   5  It's expected that for each invasive change or important bugfix you will
   6  include a test to your patch.
   7
   8  Compile tests & run basic tests:
   9
  10         $ make check
  11
  12  Note that the configure option --disable-static disables many of libmount and
  13  libblkid unit tests.
  14
  15  Run all tests including tests that require root permissions:
  16
  17         # cd tests
  18         # ./run.sh [--verbose] [--memcheck]
  19
  20         Alternatively using sudo and make:
  21
  22         $ make check TS_COMMAND="true"
  23         $ sudo -E make check TS_OPTS="--parallel=1"
  24
  25  note that as root you have to manually remove output and diff directories
  26
  27         # rm -rf output diff
  28
  29  or run 'make clean' as root.
  30
  31
  32  Run subset of tests:
  33
  34         $ ./run.sh <test_directory-name>
  35
  36         for example:
  37
  38               $ ./run.sh blkid
  39               $ ./run.sh libmount
  40
  41  The tests is possible to exclude by ./run.sh --exclude=<list>  where the
  42  <list> is blank separated test names in format "testdir/testname", for example:
  43
  44         $ ./run.sh --exclude="mount/move"
  45
  46  The --exclude is evaluated by the ./run.sh script only. See below
  47
  48         TS_OPT_testdir_[testscript_]fake=
  49
  50  environment variable which provides more powerful functionality to skip tests.
  51
  52
  53  *** WARNING for root users ***
  54
  55  The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
  56  if executed with root permissions.
  57
  58  Please, be careful and use these tests only for development and never on
  59  production system.
  60
  61
  62 environment variables
  63 ---------------------
  64
  65 TS_COMMAND
  66
  67     Evaluated by "make check" to override the default command (run.sh).
  68     Example:
  69       - build all test dependencies, but skip the actual test
  70         $ make check TS_COMMAND="true"
  71
  72 TS_OPTS
  73
  74     Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
  75     Examples:
  76       - run utmp tests only
  77         $ make check TS_OPTS="--parallel=1 utmp"
  78
  79 TS_OPT_testdir_[testscript_]fake="<yes|no>"
  80
  81     Evaluated by any test script to skip certain tests.
  82     Examples:
  83       - skip all the tests within "fdisk" test-direcrory:
  84         $ make check TS_OPT_fdisk_fake="yes"
  85
  86       - skip only "fdisk/bsd" test:
  87         $ make check TS_OPT_fdisk_bsd_fake="yes"
  88
  89       - skip all "fdisk" tests except fdisk/bsd:
  90         $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"
  91
  92 TS_OPT_testdir_[testscript_]known_fail="<yes|no>"
  93
  94     Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
  95     test will run but (negative) results will be ignored. The build log and test
  96     diffs will still remind you about the issue.
  97
  98 TS_OPT_testdir_[testscript_]verbose="<yes|no>"
  99
 100     Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.
 101
 102 TS_OPT_testdir_[testscript_]memcheck="<yes|no>"
 103
 104     Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.
 105
 106
 107 External services
 108 -----------------
 109
 110 Travis CI - automatically executed for all github commits.
 111
 112     URL: https://travis-ci.org/karelzak/util-linux/
 113
 114     See .travis.yml for more details.
 115
 116     We require "sudo" to install additional stuff and to run the tests with
 117     UID=0, it means that tests are executed on travis "legacy infrastructure".
 118
 119
 120 Drone.oi - automatically executed for all github commits.
 121
 122     URL: https://drone.io/github.com/karelzak/util-linux
 123
 124     The drone.io does not use any in-tree config file (like travis-ci), the
 125     currently used configuration (maintained by Drone.oi web UI):
 126
 127         MAKE_CHECK="root"
 128         MAKE_CHECK_OPTS="--skip-loopdevs --exclude=mount/move"
 129         source ./.travis-functions.sh
 130         travis_install_script || exit
 131         travis_before_script || exit
 132         ret=0
 133         travis_script || ret=$?
 134         travis_after_script
 135         exit $ret
 136
 137     yes, it shares the setup functions with travis-ci.