Documentation/howto-tests.txt

   1
   2  util-linux regression tests
   3  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
   4
   5  It's expected that for each invasive change or important bugfix you will
   6  include a test to your patch.
   7
   8  Compile binaries, libs, extra test programs and run the basic tests:
   9
  10         $ make check
  11
  12  Compile extra test programs only:
  13
  14         $ make check-programs
  15
  16  Note that the configure option --disable-static disables many of libmount and
  17  libblkid unit tests.
  18
  19  Run all tests including tests that require root permissions:
  20
  21         # cd tests
  22         # ./run.sh [options, see --help]
  23
  24         Alternatively using sudo and make:
  25
  26         $ make check-programs
  27         $ sudo -E make check TS_OPTS="--parallel=1"
  28
  29  note that as root you have to manually remove output and diff directories
  30
  31         # rm -rf output diff
  32
  33  or run 'make clean' as root.
  34
  35
  36  Run subset of tests:
  37
  38         $ cd tests
  39         $ ./run.sh <test_directory-name>
  40
  41         for example:
  42
  43               $ ./run.sh blkid
  44               $ ./run.sh libmount
  45
  46         or individual test script, for example:
  47
  48               $ ./ts/cal/year
  49
  50  The tests is possible to exclude by ./run.sh --exclude=<list>  where the
  51  <list> is blank separated test names in format "testdir/testname", for example:
  52
  53         $ ./run.sh --exclude="mount/move"
  54
  55  The --exclude is evaluated by the ./run.sh script only. See below
  56
  57         TS_OPT_testdir_[testscript_]fake=
  58
  59  environment variable which provides more powerful functionality to skip tests.
  60
  61
  62  *** WARNING for root users ***
  63
  64  The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
  65  if executed with root permissions.
  66
  67  Please, be careful and use these tests only for development and never on
  68  production system.
  69
  70 fuzz targets
  71 ------------
  72
  73 The fuzz targets can be built and run along with the other tests (after installing
  74 clang):
  75
  76         $ ./tools/config-gen fuzz
  77         $ make check
  78
  79 environment variables
  80 ---------------------
  81
  82 TS_COMMAND
  83
  84     Evaluated by "make check" to override the default command (run.sh).
  85     Example:
  86       - build all test dependencies, but skip the actual test
  87         $ make check TS_COMMAND="true"
  88
  89 TS_OPTS
  90
  91     Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
  92     Examples:
  93       - run utmp tests only
  94         $ make check TS_OPTS="--parallel=1 utmp"
  95
  96 TS_OPT_testdir_[testscript_]fake="<yes|no>"
  97
  98     Evaluated by any test script to skip certain tests.
  99     Examples:
 100       - skip all the tests within "fdisk" test-directory:
 101         $ make check TS_OPT_fdisk_fake="yes"
 102
 103       - skip only "fdisk/bsd" test:
 104         $ make check TS_OPT_fdisk_bsd_fake="yes"
 105
 106       - skip all "fdisk" tests except fdisk/bsd:
 107         $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"
 108
 109 TS_OPT_testdir_[testscript_]known_fail="<yes|no>"
 110
 111     Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
 112     test will run but (negative) results will be ignored. The build log and test
 113     diffs will still remind you about the issue.
 114
 115 TS_OPT_testdir_[testscript_]verbose="<yes|no>"
 116
 117     Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.
 118
 119 TS_OPT_testdir_[testscript_]memcheck="<yes|no>"
 120
 121     Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.
 122
 123
 124 External services
 125 -----------------
 126
 127 Travis CI - automatically executed for all github commits.
 128
 129     URL: https://travis-ci.org/karelzak/util-linux/
 130
 131     See .travis.yml for more details.
 132
 133     We require "sudo" to install additional stuff and to run the tests with
 134     UID=0, it means that tests are executed on travis "legacy infrastructure".
 135
 136 lgtm CI - automatically executed security code analysis
 137
 138     URL: https://lgtm.com/projects/g/karelzak/util-linux/
 139
 140 Coverity Scan
 141
 142     URL: https://scan.coverity.com/projects/karelzak-util-linux
 143
 144 Fossies codespell report
 145
 146     URL: https://fossies.org/linux/test/util-linux-master.tar.gz/codespell.html
 147
 148 OSS-Fuzz
 149
 150     URL: https://google.github.io/oss-fuzz/
 151     URL: https://oss-fuzz-build-logs.storage.googleapis.com/index.html#util-linux
 152     URL: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_util-linux/latest
 153
 154 CIFuzz
 155
 156     URL: https://google.github.io/oss-fuzz/getting-started/continuous-integration/
 157     URL: https://github.com/karelzak/util-linux/actions?query=workflow%3ACIFuzz