Documentation/howto-tests.txt

   1
   2  util-linux regression tests
   3  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
   4
   5  It's expected that for each invasive change or important bugfix you will
   6  include a test to your patch.
   7
   8  Compile binaries, libs, extra test programs and run the basic tests:
   9
  10         $ make check
  11
  12  Compile extra test programs only:
  13
  14         $ make check-programs
  15
  16  Note that the configure option --disable-static disables many of libmount and
  17  libblkid unit tests.
  18
  19  Run all tests including tests that require root permissions:
  20
  21         # cd tests
  22         # ./run.sh [options, see --help]
  23
  24         Alternatively using sudo and make:
  25
  26         $ make check-programs
  27         $ sudo -E make check TS_OPTS="--parallel=1"
  28
  29  note that as root you have to manually remove output and diff directories
  30
  31         # rm -rf output diff
  32
  33  or run 'make clean' as root.
  34
  35
  36  Run subset of tests:
  37
  38         $ cd tests
  39         $ ./run.sh <test_directory-name>
  40
  41         for example:
  42
  43               $ ./run.sh blkid
  44               $ ./run.sh libmount
  45
  46         or individual test script, for example:
  47
  48               $ ./ts/cal/year
  49
  50  The tests is possible to exclude by ./run.sh --exclude=<list>  where the
  51  <list> is blank separated test names in format "testdir/testname", for example:
  52
  53         $ ./run.sh --exclude="mount/move"
  54
  55  The --exclude is evaluated by the ./run.sh script only. See below
  56
  57         TS_OPT_testdir_[testscript_]fake=
  58
  59  environment variable which provides more powerful functionality to skip tests.
  60
  61
  62  *** WARNING for root users ***
  63
  64  The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
  65  if executed with root permissions.
  66
  67  Please, be careful and use these tests only for development and never on
  68  production system.
  69
  70
  71 environment variables
  72 ---------------------
  73
  74 TS_COMMAND
  75
  76     Evaluated by "make check" to override the default command (run.sh).
  77     Example:
  78       - build all test dependencies, but skip the actual test
  79         $ make check TS_COMMAND="true"
  80
  81 TS_OPTS
  82
  83     Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
  84     Examples:
  85       - run utmp tests only
  86         $ make check TS_OPTS="--parallel=1 utmp"
  87
  88 TS_OPT_testdir_[testscript_]fake="<yes|no>"
  89
  90     Evaluated by any test script to skip certain tests.
  91     Examples:
  92       - skip all the tests within "fdisk" test-directory:
  93         $ make check TS_OPT_fdisk_fake="yes"
  94
  95       - skip only "fdisk/bsd" test:
  96         $ make check TS_OPT_fdisk_bsd_fake="yes"
  97
  98       - skip all "fdisk" tests except fdisk/bsd:
  99         $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"
 100
 101 TS_OPT_testdir_[testscript_]known_fail="<yes|no>"
 102
 103     Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
 104     test will run but (negative) results will be ignored. The build log and test
 105     diffs will still remind you about the issue.
 106
 107 TS_OPT_testdir_[testscript_]verbose="<yes|no>"
 108
 109     Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.
 110
 111 TS_OPT_testdir_[testscript_]memcheck="<yes|no>"
 112
 113     Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.
 114
 115
 116 External services
 117 -----------------
 118
 119 Travis CI - automatically executed for all github commits.
 120
 121     URL: https://travis-ci.org/karelzak/util-linux/
 122
 123     See .travis.yml for more details.
 124
 125     We require "sudo" to install additional stuff and to run the tests with
 126     UID=0, it means that tests are executed on travis "legacy infrastructure".
 127
 128
 129 Drone.io - automatically executed for all github commits.
 130
 131     URL: https://drone.io/github.com/karelzak/util-linux
 132
 133     The drone.io does not use any in-tree config file (like travis-ci), the
 134     currently used configuration (maintained by Drone.io web UI):
 135
 136         MAKE_CHECK="root"
 137         MAKE_CHECK_OPTS="--skip-loopdevs --exclude=mount/move"
 138         source ./.travis-functions.sh
 139         travis_install_script || exit
 140         travis_before_script || exit
 141         ret=0
 142         travis_script || ret=$?
 143         travis_after_script
 144         exit $ret
 145
 146     yes, it shares the setup functions with travis-ci.