@section testsOverview Testing Overview
+In DHCP, a unit test exercises a particular piece of code in
+isolation. There is a separate unit test per module or API. Each unit
+test lives in a directory beneath the code it is designed to exercise.
+So, we (will eventually) have:
+
+@verbatim
+server/tests/
+client/tests/
+common/tests/
+dhcpctl/tests/
+...
+@endverbatim
+
+And so on.
+
+Ideally each function would be invoked with every possible type of input, and
+each branch of every function would be checked. In practice we try to be a bit
+more pragmatic, and target the most basic operations, as well tricky code, and
+areas we have seen bugs in the past.
+
+We are using <a href="http://code.google.com/p/kyua/wiki/ATF">ATF (Automated
+Test Framework)</a> as a framework to run our unittests.
+
@section testsAtf ATF unit-tests
ATF stands for Automated Test Framework, and is the framework used for unit
To build the unit-tests, use the following:
@verbatim
-./configure --with-atf
-make
-make check
+$ ./configure --with-atf
+$ make
+$ make check
@endverbatim
The following syntax is supported as well:
@verbatim
-./configure --with-atf=/path/to/your/atf/install
+$ ./configure --with-atf=/path/to/your/atf/install
@endverbatim
-but it seems to have troubles detecting ATF installation, at least
+but it seems to have troubles sometimes detecting ATF installation, at least
with ATF 0.14 and Mac OS X 10.6.8.
Each code directory (e.g. server/) that has unit-tests has a sub-directory
executable. The typical way to run tests is:
@verbatim
-atf-run | atf-report
+$ atf-run | atf-report
@endverbatim
-atf-run will read the Atffile in the current directory and execute
-all the tests specified in it. Using atf-run - rather than calling the
-test binary directly - has several major benefits. The main one is that
-atf-run is able to recover from test segfault and continue execution
-from the next case onwards.
+atf-run will read the Atffile in the current directory and execute all the tests
+specified in it. Using atf-run - rather than calling the test binary directly -
+has several major benefits. The main one is that atf-run is able to recover from
+test segfault and continue execution from the next case onwards. Another is that
+it is possible to specify a timeout for a test. atf-run will kill the test in
+case of any infinite loops and will continue running next tests.
-It is possible to run atf-run without passing its output to atf-report,
-but its output is somewhat convoluted. That is useful in some situations,
-e.g. when one wants to see test output.
+It is possible to run atf-run without passing its output to atf-report, but its
+output is somewhat convoluted. That is useful in some situations, e.g. when one
+wants to see test output.
-It is possible to run test binary directly. The only required parameter
-is the test case name. The binary will print out a warning that direct
-binary execution is not recommended as it won't be able to recover
-from crash. However, such an approach is convenient for running the
-test under the debugger.
+It is possible to run test binary directly. The only required parameter is the
+test case name. The binary will print out a warning that direct binary execution
+is not recommended as it won't be able to recover from crash. However, such an
+approach is convenient for running the test under the debugger.
@section testsAtfAdding Adding new unit-tests
-There are a small number of unit-tests that are not ATF based. They will
-be converted to ATF soon. Please do not use any other frameworks.
+There are a small number of unit-tests that are not ATF based. They will be
+converted to ATF soon. Please do not use any other frameworks.
Sadly, the DHCP code was not written with unit-testing in mind: often a
-non-standard approach is required for writing unit-tests. The existing
-code often has many dependencies that make testing a single piece of code
-awkward to unit test. For example, to test hash tables, one needs to
-also include the OMAPI code. Rather than significantly refactoring the
-code (a huge task that could take months), we decided to link whatever
-is needed in the tests. If developing new test suite, it is recommended
-that you take a look at existing tests and just copy them. In particular,
-the following things should be done for adding new tests:
-
-1. Create new file that will hold test code. It is recommended you
-name it (tested_feature_name)_unittest.c and put the file in specified
-tests directory. For example tests related to hash tables used on the
-server side should be named server/tests/hash_unittest.c. If in doubt,
-it is convenient to name the test code after the file that holds tested
-code, e.g. server/mdb6.c is tested in server/tests/mdb6_unittest.c.
-
-2. Implement the test. The file server/tests/simple_unittest.c holds a
-template explaining the basic layout of the ATF tests. There may be many
-test cases in a single *_unittest.c file. Make sure that you register
-all your test cases using ATF_TP_ADD_TC() macro, and try to minimize
-modifications to the tested code if possible. Keep in mind that we are
-using modernized \ref codingGuidelines for test development. You are
-advised to also look at atf-c-api(3).
-
-3. Extend Makefile.am to build your test. In particular, add your binary
+non-standard approach is required for writing unit-tests. The existing code
+often has many dependencies that make testing a single piece of code awkward to
+unit test. For example, to test hash tables, one needs to also include the
+OMAPI code. Rather than significantly refactoring the code (a huge task that
+could take months), we decided to link whatever is needed in the tests. If
+developing new test suite, it is recommended that you take a look at existing
+tests and just copy them as a starting point.
+
+
+In particular, the following
+things should be done for adding new tests:
+
+<b>1. Tests directory.</b> For each code component (server, client, common,
+etc.) there should be a tests subdirectory. If it isn't there yet, then it must
+be created. This can be done by:
+
+a). Creating the directory:
+
+@verbatim
+ $ mkdir $subdir/tests
+ $ cvs add tests
+@endverbatim
+
+b). Adding the subdirectory to the build system:
+
+ Add to $subdir/Makefile.am:
+
+@verbatim
+ SUBDIRS = tests
+@endverbatim
+
+ Add to the AC_OUTPUT macro in configure.ac:
+
+@verbatim
+ subdir/tests/Makefile
+@endverbatim
+
+c. Create a Makefile.am in the new directory, something similar to this:
+
+@verbatim
+ AM_CPPFLAGS = -I../..
+
+ check_PROGRAMS = test_foo
+
+ TESTS = test_foo
+
+ test_foo_SOURCES = test_foo.c
+ test_foo_LDADD = ../../tests/libt_api.a # plus others...
+@endverbatim
+
+See existing Makefile.am for examples, and the Automake documentation:
+
+ http://www.gnu.org/software/automake/manual/html_node/Tests.html
+
+<b>2. Implement the test.</b> That typically means that you create a new file that will
+hold test code. It is recommended you name it (tested_feature_name)_unittest.c
+and put the file in specified tests directory. For example tests related to
+hash tables used on the server side should be named
+server/tests/hash_unittest.c. If in doubt, it is convenient to name the test
+code after the file that holds tested code, e.g. server/mdb6.c is tested in
+server/tests/mdb6_unittest.c.
+
+The file server/tests/simple_unittest.c holds a template explaining the basic
+layout of the ATF tests. There may be many test cases in a single *_unittest.c
+file. Make sure that you register all your test cases using ATF_TP_ADD_TC()
+macro, and try to minimize modifications to the tested code if possible. Keep in
+mind that we are using modernized \ref codingGuidelines for test
+development. You are advised to also look at atf-c-api(3) man page.
+
+To add a new test, such as when a new module is added or when you want to start
+testing existing code, you can copy the server/tests/simple_unittest.c as a new
+new file, add the new file as a target in Makefile.am, and begin adding
+tests. Reviewing that file is a good idea, even if you decide to write your test
+from scratch, as it give you quick overview of the essential capabilities of the
+ATF framework (how to write test, how to make checks, pass or fail test
+etc.). Do not forget to add your new file to git via "git add
+yourtest_unittest.c".
+
+<b>3. Extend Makefile.am</b> to build your test. In particular, add your binary
name to ATF_TESTS. The tests directory will be built only in case where
-ATF is enabled, using --enable-atf during configure phase.
+ATF is enabled, using --with-atf during configure phase.
-4. Modify Atffile to include your new test binary, if needed. If you
-followed naming convention proposed in step 2, your test will be included
-and will be included automatically.
+<b>4. Modify Atffile to include your new test</b>, if needed. Tests in specified
+directory must be registered in Atffile. See server/tests/Atffile for
+example. Currently every executable with name that falls into *_unittest pattern
+will be executed automatically. If you followed naming convention proposed in a
+previous step, your test will be included and will be included automatically.
-5. Enjoy your improved confidence in the code, as you can run the tests after
+<b>5. Enjoy your improved confidence in the code</b>, as you can run the tests after
any change you may want to do:
@verbatim
-make check
-atf-run | atf-report
+$ make check
+@endverbatim
+
+or run them manually
+
+@verbatim
+$ cd server/tests
+$ atf-run | atf-report
@endverbatim
@section testsAtfCoding ATF Coding Guidelines
Introduction
------------
+That is only a brief overview of tests in ISC DHCP. For more thorough
+description, see ISC DHCP Developer's Guide. You can generate it, by
+having Doxygen installed and doing:
+
+ cd doc
+ make devel
+
+and then opening doc/html/index.html
+
+Tests Overview
+--------------
+
In DHCP, a unit test exercises a particular piece of code in
isolation. There is a separate unit test per module or API. Each unit
test lives in a directory beneath the code it is designed to exercise.
And so on.
-Ideally each function would be invoked with every possible type of
-input, and each branch of every function would be checked. In practice
-we try to be a bit more pragmatic, and target the most basic
-operations, as well tricky code, and areas we have seen bugs in the
-past.
-
We are using ATF (Automated Test Framework) as a framework to run our
-unittests.
+unittests. See ISC DHCP Developer's Guide for much more thorough
+description of unit-test and ATF framework in general.
Running Unit Tests
------------------
In order to run the unit tests for DHCP, enable ATF support during configure:
-$ ./configure --enable-atf
+$ ./configure --with-atf
And then use:
Adding a New Unit Test
----------------------
-To add an additional test to an existing test program, you must create
-a function for the new test in the C source file. You can extend existing
-test suite or create a new one. For introductory example, see
-server/tests/simple_unittest.c. It is a skeleton example of ATF test.
-
-Tests in specified directory must be registered in Atffile. See
-server/tests/Atffile for example. Currently every executable with name
-that falls into *_unittest pattern will be executed automatically.
+See ISC DHCP Developer's Guide.
Adding a New Unit Test Program
------------------------------
-To add a new program, such as when a new module is added, you can copy
-the "simple_unittest.c" file (in this directory) to a new name, add
-the new file as a target in Makefile.am, and begin adding tests. Do
-not forget to add it to CVS via "cvs add".
-
-If there is no "tests" directory for a given subdirectory, then one
-must be created. This can be done by:
-
-1. Creating the directory:
-
- $ mkdir $subdir/tests
- $ cvs add tests
-
-2. Adding the subdirectory to the build system:
-
- Add to $subdir/Makefile.am:
-
- SUBDIRS = tests
-
- Add to the AC_OUTPUT macro in configure.ac:
-
- $subdir/tests/Makefile
-
-3. Create a Makefile.am in the new directory, something like this:
-
- AM_CPPFLAGS = -I../..
-
- check_PROGRAMS = test_foo
-
- TESTS = test_foo
-
- test_foo_SOURCES = test_foo.c
- test_foo_LDADD = ../../tests/libt_api.a # plus others...
-
-
-See existing Makefile.am for examples, and the Automake documentation:
-
- http://www.gnu.org/software/automake/manual/html_node/Tests.html
-
-
-Support Functions
------------------
-
-See http://www.netbsd.org/~jmmv/atf/index.html for details about ATF. Note
-that although ATF was originally used for NetBSD validation, it is used
-on many different platforms and is not NetBSD specific.
-
-Additional Testing
-------------------
-
-Other static or runtime testing is always an option. For instance, you
-can use valgrind to check for memory leaks.
-
-$Id: HOWTO-unit-test,v 1.2.570.1 2012/01/31 15:50:15 tomasz Exp $
+See ISC DHCP Developer's Guide.
projects / thirdparty / dhcp.git / commitdiff
