docs: Provide more information about the TCK test suite

Most importantly, how to get it, how install dependencies and how
to run it.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

diff --git a/docs/testtck.rst b/docs/testtck.rst
index 51de095657df108a868ee36d883b673f17f2d8be..6264b40de7f10addfb63dc595eb358a36c0d9e66 100644 (file)
--- a/docs/testtck.rst
+++ b/docs/testtck.rst
@@ -2,6 +2,8 @@
 libvirt TCK : Technology Compatibility Kit
 ==========================================
 
+.. contents::
+
 The libvirt TCK provides a framework for performing testing of the integration
 between libvirt drivers, the underlying virt hypervisor technology, related
 operating system services and system configuration. The idea (and name) is
@@ -23,15 +25,90 @@ determine the level of compatibility of their platform, and evaluate whether it
 will meet their needs, and get awareness of any regressions that may have
 occurred since a previous test run.
 
-For more details you can look at:
+Libvirt-TCK is maintained using `a GIT
+repository <https://gitlab.com/libvirt/libvirt-tck>`__. GitLab is also the place
+where the whole TCK development workflow (issues, merge requests, comments)
+happens.
 
--  The initial `mail from Daniel
-   Berrange <https://www.redhat.com/archives/libvir-list/2009-April/msg00176.html>`__
-   presenting the project.
--  The `page describing
-   VirtTCK <https://fedoraproject.org/wiki/Features/VirtTCK>`__ the inclusion of
-   libvirt-TCK as a Fedora Feature.
+Using TCK
+---------
 
-Libvirt-TCK is maintained using `a GIT
-repository <https://gitlab.com/libvirt/libvirt-tck>`__, and comment, patches and
-reviews are carried on the `libvir-list <contact.html>`__ development list.
+TCK can be used independently of the enviroment, i.e. both on your local host
+or in a VM. We strongly recommend using a VM for the tests as TCK might affect
+your current host setup, see `Running TCK`_.
+
+Installing dependencies
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Since TCK is based on libvirt Perl bindings, you'll need to have the proper
+version of the bindings installed for the version of libvirt you wish to test
+in order to be able execute the TCK test suite successfully. Additionally, a
+number of Perl dependencies will need to be installed as well, some will be
+available through the system package manager and some will likely need to be
+installed from CPAN (Perl's equivalent of Python's PyPI). Here's where
+`libvirt-ci's <https://gitlab.com/libvirt/libvirt-ci.git>`__ lcitool can help
+with preparing a test environment in a fresh VM, taking care of the
+dependencies along the way:
+
+::
+
+    $ lcitool install --target fedora-36 tck-fedora36 --wait
+
+would get you a new Fedora 36 VM named ``tck-fedora36``. Then
+
+::
+
+    $ lcitool update tck-fedora36 libvirt,libvirt-perl,libvirt-tck+runtime
+
+will install all the necessary dependencies to build libvirt, the corresponding
+Perl bindings and all TCK runtime dependencies to be able to execute the tests.
+We also recommend executing TCK using the Avocado framework as the test harness
+engine which means that you'll have to install Avocado in the test environment
+as well. You can get it either from
+`PyPI <https://pypi.org/project/avocado-framework/>`__ (recommended), or if
+you're on Fedora you can make use of the Avocado `module <https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html#installing-from-packages>`__.
+Using Avocado is not mandatory for the time being and you can skip it, but
+in the future we plan on making the TCK internal coupling with Avocado tighter.
+
+Running TCK
+~~~~~~~~~~~
+
+Once you have all the dependencies installed, you can then proceed with running
+as root the test suite as root (when running with Avocado):
+
+::
+
+    # avocado --config avocado.config run --tap - ./scripts/
+
+from the TCK's git root.
+
+
+If you don't want to install Avocado you can execute tests using the
+``libvirt-tck`` binary directly (again, from the git root). You'll need to pass
+a few options that Avocado takes care of:
+
+::
+
+    # PERL5LIB=./lib perl bin/libvirt-tck -c <path_to_config> --force ./scripts
+
+Running with the ``--force`` argument is not necessary and you can safely omit
+it, but it becomes useful if you need to interrupt a test run for some
+reason. In such case using ``--force`` ensures the first thing TCK does before
+running any tests is that it will clean up all resources from the previous test
+run which may have been left behind if you had interrupted the previous TCK's
+execution.
+
+Note that running with root privileges is necessary since some tests need
+access to system resources or configs. This, along with the fact that some
+tests might affect the host system are good reasons to consider using a test VM
+as described above.
+
+Contributing a test
+-------------------
+
+We'd appreciate if you provided a functional test case whenever you're adding a
+new feature or fixing a bug in libvirt with the only complication being that
+in case you're adding a new public API then a Perl binding will have to be
+introduced first. After that, the best way to start is looking at some existing
+tests, copy-pasting one that fits your scenario the best and tweak the
+remaining bits.