From: Marcin Siodelski Date: Thu, 18 Aug 2016 08:30:45 +0000 (+0200) Subject: [4489] Updated dev-guide with database prerequisites for unit tests. X-Git-Tag: trac4631_base~9^2~4 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=750fa1da22069a1899e8b3263600a7ab3314e071;p=thirdparty%2Fkea.git [4489] Updated dev-guide with database prerequisites for unit tests. --- diff --git a/doc/Makefile.am b/doc/Makefile.am index 69401e6749..1976fabeec 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -4,7 +4,7 @@ EXTRA_DIST = version.ent.in Doxyfile Doxyfile-xml EXTRA_DIST += devel/config-backend.dox EXTRA_DIST += devel/contribute.dox EXTRA_DIST += devel/mainpage.dox -EXTRA_DIST += devel/qa.dox +EXTRA_DIST += devel/unit-tests.dox nobase_dist_doc_DATA = examples/ddns/sample1.json nobase_dist_doc_DATA += examples/ddns/template.json diff --git a/doc/devel/contribute.dox b/doc/devel/contribute.dox index 31028decaa..3f43307887 100644 --- a/doc/devel/contribute.dox +++ b/doc/devel/contribute.dox @@ -94,7 +94,7 @@ written and observing the test fail, you can detect the situation where a bug in the test code will cause it to pass regardless of the code being tested. -See @ref qaUnitTests for instructions on how to run unit-tests. +See @ref unitTests for instructions on how to run unit-tests. If you happen to add new files or have modified any \c Makefile.am files, it is also a good idea to check if you haven't broken the diff --git a/doc/devel/mainpage.dox b/doc/devel/mainpage.dox index ebe1623d94..7b203dd464 100644 --- a/doc/devel/mainpage.dox +++ b/doc/devel/mainpage.dox @@ -1,4 +1,4 @@ -// Copyright (C) 2012-2015 Internet Systems Consortium, Inc. ("ISC") +// Copyright (C) 2012-2016 Internet Systems Consortium, Inc. ("ISC") // // This Source Code Form is subject to the terms of the Mozilla Public // License, v. 2.0. If a copy of the MPL was not distributed with this @@ -34,6 +34,12 @@ * @section contrib Contributor's Guide * - @subpage contributorGuide * + * @section buildingKeaWithUnitTests Building Kea with Unit tests + * - @subpage unitTests + * - @subpage unitTestsIntroduction + * - @subpage unitTestsEnvironmentVariables + * - @subpage unitTestsDatabaseConfig + * * @section hooksFramework Hooks Framework * - @subpage hooksdgDevelopersGuide * - @subpage dhcpv4Hooks @@ -107,9 +113,6 @@ * - @subpage configBackendAdding * - @subpage perfdhcpInternals * - * @section qualityAssurance Quality Assurance - * - @subpage qaUnitTests - * * @section miscellaneousTopics Miscellaneous Topics * - @subpage logKeaLogging * - @subpage logBasicIdeas diff --git a/doc/devel/qa.dox b/doc/devel/qa.dox deleted file mode 100644 index 8ad8c88840..0000000000 --- a/doc/devel/qa.dox +++ /dev/null @@ -1,68 +0,0 @@ -// Copyright (C) 2015 Internet Systems Consortium, Inc. ("ISC") -// -// This Source Code Form is subject to the terms of the Mozilla Public -// License, v. 2.0. If a copy of the MPL was not distributed with this -// file, You can obtain one at http://mozilla.org/MPL/2.0/. - -/** - -@page qa Kea Quality Assurance processes - - @section qaUnitTests Unit-tests - -Kea uses the Google C++ Testing Framework (also called googletest or gtest) as a -base for our C++ unit-tests. See http://code.google.com/p/googletest/ for -details. We used to have Python unit-tests that were inherited from BIND10 -days. Those tests are removed now, so please do not develop any new Python -tests in Kea. If you want to write DHCP tests in Python, we encourage you to -take a look at ISC Forge: http://kea.isc.org/wiki/IscForge. You must have \c -gtest installed or at least extracted in a directory before compiling Kea -unit-tests. To enable unit-tests in Kea, use: - -@code -./configure --with-gtest=/path/to/your/gtest/dir -@endcode - -or - -@code -./configure --with-gtest-source=/path/to/your/gtest/dir -@endcode - -Depending on how you compiled or installed \c gtest (e.g. from sources -or using some package management system) one of those two switches will -find \c gtest. After that you make run unit-tests: - -@code -make check - -@endcode - -The following environment variable can affect unit-tests: - -- KEA_LOCKFILE_DIR - Specifies a directory where the logging system should - create its lock file. If not specified, it is prefix/var/run/kea, where prefix - defaults to /usr/local. This variable must not end with a slash. There is one - special value: "none", which instructs Kea to not create lock file at - all. This may cause issues if several processes log to the same file. - Also see Kea User's Guide, section 15.3. - -- KEA_LOGGER_DESTINATION - Specifies logging destination. If not set, logged - messages will not be recorded anywhere. There are 3 special values: - stdout, stderr and syslog. Any other value is interpreted as a filename. - Also see Kea User's Guide, section 15.3. - -- KEA_PIDFILE_DIR - Specifies the directory which should be used for PID files - as used by dhcp::Daemon or its derivatives. If not specified, the default is - prefix/var/run/kea, where prefix defaults to /usr/local. This variable must - not end with a slash. - -- KEA_SOCKET_TEST_DIR - if set, it specifies the directory where Unix - sockets are created. There's OS limitation on how long a Unix socket - path can be. It is typcially slightly over 100 characters. If you - happen to build and run unit-tests in deeply nested directories, this - may become a problem. KEA_SOCKET_TEST_DIR can be specified to instruct - unit-test to use a different directory. Must not end with slash (e.g. - /tmp). - - */ diff --git a/doc/devel/unit-tests.dox b/doc/devel/unit-tests.dox new file mode 100644 index 0000000000..7801e1448f --- /dev/null +++ b/doc/devel/unit-tests.dox @@ -0,0 +1,277 @@ +// Copyright (C) 2015-2016 Internet Systems Consortium, Inc. ("ISC") +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at http://mozilla.org/MPL/2.0/. + +/** + + @page unitTests Building Kea with Unit Tests + +@section unitTestsIntroduction Introduction + +Kea uses the Google C++ Testing Framework (also called googletest or gtest) as a +base for our C++ unit-tests. See http://code.google.com/p/googletest/ for +details. We used to have Python unit-tests that were inherited from BIND10 +days. Those tests are removed now, so please do not develop any new Python +tests in Kea. If you want to write DHCP tests in Python, we encourage you to +take a look at ISC Forge: http://kea.isc.org/wiki/IscForge. You must have \c +gtest installed or at least extracted in a directory before compiling Kea +unit-tests. To enable unit-tests in Kea, use: + +@code +./configure --with-gtest=/path/to/your/gtest/dir +@endcode + +or + +@code +./configure --with-gtest-source=/path/to/your/gtest/dir +@endcode + +Depending on how you compiled or installed \c gtest (e.g. from sources +or using some package management system) one of those two switches will +find \c gtest. After that you make run unit-tests: + +@code +make check + +@endcode + +@section unitTestsEnvironmentVariables Environment Variables + +The following environment variable can affect unit-tests: + +- KEA_LOCKFILE_DIR - Specifies a directory where the logging system should + create its lock file. If not specified, it is prefix/var/run/kea, where prefix + defaults to /usr/local. This variable must not end with a slash. There is one + special value: "none", which instructs Kea to not create lock file at + all. This may cause issues if several processes log to the same file. + Also see Kea User's Guide, section 15.3. + +- KEA_LOGGER_DESTINATION - Specifies logging destination. If not set, logged + messages will not be recorded anywhere. There are 3 special values: + stdout, stderr and syslog. Any other value is interpreted as a filename. + Also see Kea User's Guide, section 15.3. + +- KEA_PIDFILE_DIR - Specifies the directory which should be used for PID files + as used by dhcp::Daemon or its derivatives. If not specified, the default is + prefix/var/run/kea, where prefix defaults to /usr/local. This variable must + not end with a slash. + +- KEA_SOCKET_TEST_DIR - if set, it specifies the directory where Unix + sockets are created. There's OS limitation on how long a Unix socket + path can be. It is typcially slightly over 100 characters. If you + happen to build and run unit-tests in deeply nested directories, this + may become a problem. KEA_SOCKET_TEST_DIR can be specified to instruct + unit-test to use a different directory. Must not end with slash (e.g. + /tmp). + +@section unitTestsDatabaseConfig Databases Configuration for Unit Tests + + With the use of databases requiring separate authorisation, there are + certain database-specific pre-requisites for successfully running the unit + tests. These are listed in the following sections. + + @subsection unitTestsDatabaseUsers Database Users Required for Unit Tests + + Unit tests validating database backends require that keatest database + is created. This database should be empty (should not include any relations). + Unit tests will create required tables for each test case, and drop these tables + when the test case ends. The unit tests also require that keatest user + is created and that this user is configured to access keatest + database with a keatest password. + + Unit tests use these credentials to create database schema, run test cases + and drop the schema. Thus, the keatest user must have sufficiently + high privileges to create and drop tables, as well as insert and modify the + data within those tables. + + The database backends, which support read only access to the host reservations + databases (currently MySQL and PostgreSQL), include unit tests verifying that + a database user, with read-only privileges, can be used to retrieve host + reservations. Those tests require that a user keatest_readonly, with + SQL SELECT privilege to the keatest database (without INSERT, UPDATE etc.), + is also created. + + The following sections provide step-by-step guidelines how to setup the + databases for running unit tests. + + @subsection mysqlUnitTestsPrerequisites MySQL Database + + A database called keatest must be created. A database user, also called + keatest (and with a password keatest) must also be created and + be given full privileges in that database. The unit tests create the schema + in the database before each test and delete it afterwards. + + In detail, the steps to create the database and user are: + + -# Log into MySQL as root: + @verbatim + % mysql -u root -p + Enter password: + : + mysql>@endverbatim\n + -# Create the test database. This must be called "keatest": + @verbatim + mysql> CREATE DATABASE keatest; + mysql>@endverbatim\n + -# Create the users under which the test client will connect to the database + (the apostrophes around the words keatest and localhost are + required): + @verbatim + mysql> CREATE USER 'keatest'@'localhost' IDENTIFIED BY 'keatest'; + mysql> CREATE USER 'keatest_readonly'@'localhost' IDENTIFIED BY 'keatest'; + mysql>@endverbatim\n + -# Grant the created users permissions to access the keatest database + (again, the apostrophes around the user names and localhost + are required): + @verbatim + mysql> GRANT ALL ON keatest.* TO 'keatest'@'localhost'; + mysql> GRANT SELECT ON keatest.* TO 'keatest_readonly'@'localhost'; + mysql>@endverbatim\n + -# Exit MySQL: + @verbatim + mysql> quit + Bye + %@endverbatim + + The unit tests are run automatically when "make check" is executed (providing + that Kea has been build with the \--with-dhcp-mysql switch (see the installation + section in the Kea Administrator + Reference Manual). + + @subsection pgsqlUnitTestsPrerequisites PostgreSQL Database + + Conceptually, the steps required to run PostgreSQL unit-tests are the same as + in MySQL. First, a database called keatest must be created. A database + user, also called keatest (that will be allowed to log in using password + keatest) must be created and given full privileges in that database. A + database user, called keatest_readonly (using password keatest) + must be created with SELECT privilege on all tables. + The unit tests create the schema in the database before each test and delete it + afterwards. + + PostgreSQL set up differs from system to system. Please consult your OS-specific + PostgreSQL documentation. The remainder of that section uses Ubuntu 13.10 x64 + (with PostgreSQL 9.0+) as an example. On Ubuntu, after installing PostgreSQL + (with sudo apt-get install postgresql), it is installed as user + postgres. To create new databases or add new users, initial commands + must be issued as user postgres: + +@verbatim +$ sudo -u postgres psql postgres +[sudo] password for thomson: +psql (9.1.12) +Type "help" for help. +postgres=# CREATE USER keatest WITH PASSWORD 'keatest'; +CREATE ROLE +postgres=# CREATE DATABASE keatest; +CREATE DATABASE +postgres=# GRANT ALL PRIVILEGES ON DATABASE keatest TO keatest; +GRANT +postgres=# \q +@endverbatim + + PostgreSQL versions earlier than 9.0 don't provide an SQL statement for granting + privileges on all tables in a database. In newer PostgreSQL versions, it is + possible to grant specific privileges on all tables within a schema. + However, this only affects tables which exist when the privileges are granted. + To ensure that the user has specific privileges to tables dynamically created + by the unit tests, the default schema privileges must be altered. + + The following example demonstrates how to create user keatest_readonly, + which has SELECT privilege to the tables within the keatest database, + in Postgres 9.0+. For earlier versions of Postgres, it is recommended to + simply grant full privileges to keatest_readonly user, using the + same steps as for the keatest user. + +@verbatim +$ psql -U postgres +Password for user postgres: +psql (9.1.12) +Type "help" for help. + +postgres=# CREATE USER keatest_readonly WITH PASSWORD 'keatest'; +CREATE ROLE +postgres=# \q + +$ psql -U keatest +Password for user keatest: +psql (9.1.12) +Type "help" for help. + +keatest=> ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES to keatest_readonly; +ALTER DEFAULT PRIVILEGES +keatest=> \q +@endverbatim + + Note that keatest user (rather than postgres) is used to grant + privileges to the keatest_readonly user. This ensures that the SELECT + privilege is granted only on the tables that the keatest user can access + within the public schema. + + Now we should be able to log into the newly created database using both user + names: +@verbatim +$ psql -d keatest -U keatest +Password for user keatest: +psql (9.1.12) +Type "help" for help. + +keatest=> \q + +$ psql -d keatest -U keatest_readonly +Password for user keatest_readonly: +psql (9.1.12) +Type "help" for help. + +keatest=> +@endverbatim + + If instead of seeing keatest=> prompt, your login will be refused with error + code about failed peer or indent authentication, it means that PostgreSQL is + configured to check unix username and reject login attepts if PostgreSQL names + are different. To alter that, PostgreSQL configuration must be changed. + /etc/postgresql/9.1/main/pg_hba.conf config file + has to be tweaked. It may be in a different location in your system. The following + lines: + +@verbatim +local all all peer +host all all 127.0.0.1/32 md5 +host all all ::1/128 md5 +@endverbatim + + were replaced with: + +@verbatim +local all all password +host all all 127.0.0.1/32 password +host all all ::1/128 password +@endverbatim + + Another possible problem is to get no password prompt, in general because + you have no pg_hba.conf config file and everybody is by default + trusted. As it has a very bad effect on the security you should have + been warned it is a highly unsafe config. The solution is the same, + i.e., require password or md5 authentication method. If you lose + the postgres user access you can add first: +@verbatim +local all postgres trust +@endverbatim + to trust only the local postgres user. Note the postgres user can + be pgsql on some systems. + + Please consult your PostgreSQL user manual before applying those changes as + those changes may expose your other databases that you run on the same system. + In general case, it is a poor idea to run anything of value on a system + that runs tests. Use caution! + + The unit tests are run automatically when "make check" is executed (providing + that Kea has been build with the \--with-dhcp-pgsql switch (see the installation + section in the Kea Administrator + Reference Manual). + + + */ diff --git a/src/lib/dhcpsrv/database_backends.dox b/src/lib/dhcpsrv/database_backends.dox index 104318ece2..682bc5dea0 100644 --- a/src/lib/dhcpsrv/database_backends.dox +++ b/src/lib/dhcpsrv/database_backends.dox @@ -83,137 +83,5 @@ - user - database user ID under which the database is accessed. If not specified, no user ID is used - the database is assumed to be open. - @section dhcp-backend-unittest Running Unit Tests - - With the use of databases requiring separate authorisation, there are - certain database-specific pre-requisites for successfully running the unit - tests. These are listed in the following sections. - - @subsection dhcp-mysql-unittest MySQL Unit Tests - - A database called keatest must be created. A database user, also called - keatest (and with a password keatest) must also be created and - be given full privileges in that database. The unit tests create the schema - in the database before each test and delete it afterwards. - - In detail, the steps to create the database and user are: - - -# Log into MySQL as root: - @verbatim - % mysql -u root -p - Enter password: - : - mysql>@endverbatim\n - -# Create the test database. This must be called "keatest": - @verbatim - mysql> CREATE DATABASE keatest; - mysql>@endverbatim\n - -# Create the user under which the test client will connect to the database - (the apostrophes around the words keatest and localhost are - required): - @verbatim - mysql> CREATE USER 'keatest'@'localhost' IDENTIFIED BY 'keatest'; - mysql>@endverbatim\n - -# Grant the created user permissions to access the keatest database - (again, the apostrophes around the words keatest and localhost - are required): - @verbatim - mysql> GRANT ALL ON keatest.* TO 'keatest'@'localhost'; - mysql>@endverbatim\n - -# Exit MySQL: - @verbatim - mysql> quit - Bye - %@endverbatim - - The unit tests are run automatically when "make check" is executed (providing - that Kea has been build with the \--with-dhcp-mysql switch (see the installation - section in the Kea Administrator - Reference Manual). - - @subsection dhcp-pgsql-unittest PostgreSQL Unit Tests - - Conceptually, the steps required to run PostgreSQL unit-tests are the same as - in MySQL. First, a database called keatest must be created. A database - user, also called keatest (that will be allowed to log in using password - keatest) must be created and given full privileges in that database. The - unit tests create the schema in the database before each test and delete it - afterwards. - - PostgreSQL set up differs from system to system. Please consult your OS-specific - PostgreSQL documentation. The remainder of that section uses Ubuntu 13.10 x64 as - example. On Ubuntu, after installing PostgreSQL (with sudo apt-get install - postgresql), it is installed as user postgres. To create new databases - or add new users, initial commands must be issued as user postgres: - -@verbatim -$ sudo -u postgres psql postgres -[sudo] password for thomson: -psql (9.1.12) -Type "help" for help. -postgres=# CREATE USER keatest WITH PASSWORD 'keatest'; -CREATE ROLE -postgres=# CREATE DATABASE keatest; -CREATE DATABASE -postgres=# GRANT ALL PRIVILEGES ON DATABASE keatest TO keatest; -GRANT -postgres=# \q -@endverbatim - - Now we are back to our regular, unprivileged user. Try to log into the newly - created database using keatest credentials: -@verbatim -$ psql -d keatest -U keatest -Password for user keatest: -psql (9.1.12) -Type "help" for help. - -keatest=> -@endverbatim - - If instead of seeing keatest=> prompt, your login will be refused with error - code about failed peer or indent authentication, it means that PostgreSQL is - configured to check unix username and reject login attepts if PostgreSQL names - are different. To alter that, PostgreSQL configuration must be changed. - Alternatively, you may set up your environment, so the tests would be run from - unix account keatest. /etc/postgresql/9.1/main/pg_hba.conf config file - had to betweaked. It may be in a different location in your system. The following - lines: - -@verbatim -local all all peer -host all all 127.0.0.1/32 md5 -host all all ::1/128 md5 -@endverbatim - - were replaced with: - -@verbatim -local all all password -host all all 127.0.0.1/32 password -host all all ::1/128 password -@endverbatim - - Another possible problem is to get no password prompt, in general because - you have no pg_hba.conf config file and everybody is by default - trusted. As it has a very bad effect on the security you should have - been warned it is a highly unsafe config. The solution is the same, - i.e., require password or md5 authentication method. If you lose - the postgres user access you can add first: -@verbatim -local all postgres trust -@endverbatim - to trust only the local postgres user. Note the postgres user can - be pgsql on some systems. - - Please consult your PostgreSQL user manual before applying those changes as - those changes may expose your other databases that you run on the same system. - In general case, it is a poor idea to run anything of value on a system - that runs tests. Use caution! - - The unit tests are run automatically when "make check" is executed (providing - that Kea has been build with the \--with-dhcp-pgsql switch (see the installation - section in the Kea Administrator - Reference Manual). */