From: Tomek Mrugalski Date: Tue, 17 Jul 2012 18:25:55 +0000 (+0200) Subject: [2040] README migrated to dhcp-perf-guide X-Git-Tag: trac2351_base~117^2~19 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=9cb393ea43b1a8a7f781732270a61dae64a147ff;p=thirdparty%2Fkea.git [2040] README migrated to dhcp-perf-guide --- diff --git a/tests/tools/dhcp-ubench/Makefile b/tests/tools/dhcp-ubench/Makefile index 95c081197c..6ec0d83b5a 100644 --- a/tests/tools/dhcp-ubench/Makefile +++ b/tests/tools/dhcp-ubench/Makefile @@ -8,6 +8,8 @@ SQLITE_LDFLAGS=`pkg-config sqlite3 --libs` all: mysql_ubench sqlite_ubench memfile_ubench +doc: dhcp-perf-guide.html dhcp-perf-guide.pdf + mysql_ubench.o: mysql_ubench.cc mysql_ubench.h $(CXX) $< -c $(CFLAGS) $(MYSQL_CFLAGS) diff --git a/tests/tools/dhcp-ubench/README b/tests/tools/dhcp-ubench/README deleted file mode 100644 index 54d8cde635..0000000000 --- a/tests/tools/dhcp-ubench/README +++ /dev/null @@ -1,149 +0,0 @@ - - DHCP micro-benchmarks ------------------------ - - Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") - - by Tomasz Mrugalski - -This directory contains simplified prototypes for various DB back-ends -that are planned or considered as a backend engine for BIND10 DHCP. -Athough trivial now, they are expected to evolve into useful tools -that will allow users to measure performance in their specific environment. - -Currently the following benchmarks are planned: -- in memory+flat file -- SQLite -- MySQL - -As they require additional (sometimes heavy) dependencies, they are not -built by default. Actually, their build system completely separated. -It will be eventually merged with the main BIND10 makefile system, but -that is a low priority for now. - -All planned benchmarks will follow the same pattern. - -1. prepare operation (connect to a database, create a file etc.) -2. Measure timestamp 0 -3. commit new lease4 (step repeated X times) -4. Measure timestamp 1 -5. search for random lease4 (step repeated X times) -6. Measure timestamp 2 -7. update existing lease4 (step repeated X times) -8. Measure timestamp 3 -9. delete existing lease4 (step repeated X times) -10. Measure timestamp 4 -11. Print out statistics, based on X and measured timestamps. - -Although this approach does not attempt to simulate actual DHCP server -operation that has mix of all steps intervening, it answers the -questions about basic database strenghts and weak points. In particular -it can show what is the impact of specific DB optimizations, like -changing engine, optimizing for writes/reads etc. - -The framework attempts to do the same amount of operations for every -backend thus allowing fair complarison between them. - - MySQL backend ---------------- - -MySQL backend requires MySQL client development libraries. It uses -mysql_config tool (that works similar to pkg-config) to discover -required compilation and linking options. To install required packages -on Ubuntu, use the following command: - -sudo apt-get install mysql-client mysql-server libmysqlclient-dev - -Running MySQL server is required. Make sure that you have your setup -configured so there is a user that is able to create databases. - -Before running tests, you need to initialize your database. You can -use mysql-schema.sql script for that purpose. - -WARNING: It will drop existing Kea database. Do not run this on -your production server. Assuming your MySQL user is kea, you can -initialize your test database by: - -mysql -u kea -p < mysql.schema - -After that step, you are ready to run the test: - -./mysql_ubench - -or - -./mysql_ubench > results-mysql.txt - -Redirecting output to a file is important, because for each operation -there is a single character printed to show progress. If you have a slow -terminal, this may considerably affect test perfromance. On the other hand, -printing something after each operation is required, as poor DB setting -may slow down operations to around 20 per second. Observant user is expected -to note that initial dots are printed too slowly and abort the test. - -Currently all parameters are hardcoded. To modify them, one needs to -modify source code and recompile. Fortunately, that is quite easy. -To modify MySQL parameters, see main() method at the end of mysql_ubench.c -file. That is the plase where one can modify MySQL connection -parameters (MySQL server hostname, user and password and database name). - -One parameter that has huge impact on performance is a a backend engine. -You can get a list of engines of your MySQL implementation by using - -> show engines; - -in your mysql client. Two notable engines are MyISAM and InnoDB. You -can tweak it by using the following command in mysql: - -> alter table lease4 engine=MyISAM; - -or - -> alter table lease4 engine=InnoDB; - - SQLite backend ----------------- -SQLite backend requires both sqlite3 development and run-time package. Their -names may vary from system to system, but on Ubuntu 12.04 they are called -sqlite3 libsqlite3-dev. To install them, use the following command: - -sudo apt-get install sqlite3 libsqlite3-dev - -To run SQLite3 tests, first create the database: - -cat sqlite.schema | sqlite3 sqlite.db - -A new database called sqlite.db will be created. That is the default name used -by sqlite_ubench test. If you prefer other name, make sure you update -sqlite_ubench.cc accordingly. Once the database is created, you can run -tests: - -./sqlite_ubench > results-sqlite.txt - -Make sure to tweak number of iterations in sqlite_ubench.cc. See num variable -in main() function near the end of sqlite_ubench.cc file. - -To improve performance, asynchronous mode is used (PRAGMA synchronous = OFF). -If you do not like it, modify SQLite_uBenchmark::connect() by commenting it out -and suffer the pain of having around 10 inserts per seconds. - -Another tweakable feature is journal mode. It can be turned to several modes -of operation. Its value can be modified in SQLite_uBenchmark::connect(). -See http://www.sqlite.org/pragma.html#pragma_journal_mode for detailed -explanantion. - - Performance optimizations ---------------------------- - -The following section is a collection of loose notes, ideas and comments that -could be investigated to possibly improve (or thrash) performance. Please -consider them as developer's scratchpad only. However, if you are experienced -programmer or a DB admin, you are welcome to try some of them. ISC engineers -are more than welcome to discuss pros and cons of various approaches on -bind10-dev@lists.isc.org list. - -- SQLite: Excellent article that discusses performance optimizations: - - http://stackoverflow.com/questions/1711631/how-do-i-improve-the-performance-of-sqlite - - http://blog.quibb.org/2010/08/fast-bulk-inserts-into-sqlite/ - - use prepared statements - - use journal mode in memory (or disable it completely) diff --git a/tests/tools/dhcp-ubench/dhcp-perf-guide.xml b/tests/tools/dhcp-ubench/dhcp-perf-guide.xml index 91b2ecdb0c..76ff3c7929 100644 --- a/tests/tools/dhcp-ubench/dhcp-perf-guide.xml +++ b/tests/tools/dhcp-ubench/dhcp-perf-guide.xml @@ -32,6 +32,10 @@ 2012Internet Systems Consortium, Inc. + + Tomasz + Mrugalski + BIND 10 is a framework that features Domain Name System @@ -71,6 +75,43 @@ pros an cons of various optimization techniques and tools. + + + + + ISC DHCP 4.x + + TODO: Write something about ISC DHCP4 here. + + + + + Kea + + + + +
+ Backend performance evaluation + + Kea will support several different database backends, using + both popular databases (like MySQL or SQLite) and + custom-developed solutions (like in-memory database). BIND 10 + source code features set of performance microbenchmarks. + These are small tools written in C/C++ that simulate expected + DHCP server behaviour and evaluate the performance of + considered databases. + + + + Those benchmarks are stored in tests/tools/dhcp-ubench + directory. This directory contains simplified prototypes for + various DB back-ends that are planned or considered as a + backend engine for BIND10 DHCP. Athough trivial now, they are + expected to evolve into useful tools that will allow users to + measure performance in their specific environment. + + Currently the following benchmarks are implemented: @@ -82,7 +123,7 @@ As they require additional (sometimes heavy) dependencies, they are not - built by default. Actually, their build system completely separated. + built by default. Actually, their build system is completely separated. It will be eventually merged with the main BIND10 makefile system, but that is a low priority for now. @@ -92,13 +133,13 @@ prepare operation (connect to a database, create a file etc.) Measure timestamp 0 - Commit new lease4 (step repeated X times) + Commit new lease4 (repeated X times) Measure timestamp 1 - Search for random lease4 (step repeated X times) + Search for random lease4 (repeated X times) Measure timestamp 2 - Update existing lease4 (step repeated X times) + Update existing lease4 (repeated X times) Measure timestamp 3 - Delete existing lease4 (step repeated X times) + Delete existing lease4 (repeated X times) Measure timestamp 4 Print out statistics, based on X and measured timestamps. @@ -114,44 +155,135 @@ The framework attempts to do the same amount of operations for every backend thus allowing fair complarison between them. +
- +
+ MySQL backend + MySQL backend requires MySQL client development libraries. It uses + mysql_config tool (that works similar to pkg-config) to discover required + compilation and linking options. To install required packages on Ubuntu, + use the following command: - - ISC DHCP 4.x - - TODO: Write something about ISC DHCP4 here. - - + $ sudo apt-get install mysql-client mysql-server libmysqlclient-dev - - Kea - + Make sure that MySQL server is running. Make sure that you have your setup + configured so there is a user that is able to modify used database. - + Before running tests, you need to initialize your database. You can + use mysql.schema script for that purpose. WARNING: It will drop existing + Kea database. Do not run this on your production server. Assuming your + MySQL user is kea, you can initialize your test database by: -
- Backend performance evaluation - - Kea will support several different database backends, using - both popular databases (like MySQL or SQLite) and - custom-developed solutions (like in-memory database). BIND 10 - source code features set of performance microbenchmarks. - These are small tools written in C/C++ that simulate expected - DHCP server behaviour and evaluate the performance of - considered databases. + $ mysql -u kea -p < mysql.schema - - Those benchmarks are stored in tests/tools/dhcp-ubench - directory. This directory contains simplified prototypes for - various DB back-ends that are planned or considered as a - backend engine for BIND10 DHCP. Athough trivial now, they are - expected to evolve into useful tools that will allow users to - measure performance in their specific environment. + After database is initialized, you are ready to run the test: + $ ./mysql_ubench + + or + + $ ./mysql_ubench > results->mysql.txt + + Redirecting output to a file is important, because for each operation + there is a single character printed to show progress. If you have a slow + terminal, this may considerably affect test perfromance. On the other hand, + printing something after each operation is required, as poor DB setting + may slow down operations to around 20 per second. Observant user is expected + to note that initial dots are printed too slowly and abort the test. + + Currently all parameters are hardcoded. To modify them, one needs to + modify source code and recompile. Fortunately, that is quite easy. + To modify MySQL parameters, see main() method at the end of mysql_ubench.c + file. That is the plase where one can modify MySQL connection + parameters (MySQL server hostname, user and password and database name). + +
+ MySQL tweaks + To reconfigure mysql_ubench parameters, a modification to the source + code and recompilation is required. All parameters are listed in main() + function in mysql_ubench.cc file, near the end of the file. Currently supported + parameter are (default values specified in brackets): + + hostname - name of the host to connect to ("localhost") + user - MySQL username ("root") + passwd - MySQL password ("secret") + dbname - MySQL database name ("kea") + num - number of iterations (100) + sync - should the operations be performend in synchronous (true) + or asynchronous (false) manner (true) + verbose - should the test print out progress? (true) + + + + One parameter that has huge impact on performance is a a backend engine. + You can get a list of engines of your MySQL implementation by using + + > show engines; + + in your mysql client. Two notable engines are MyISAM and InnoDB. mysql_ubench will + use MyISAM for synchronous mode and InnoDB for asynchronous. +
+
+ + +
+ SQLite-ubench + SQLite backend requires both sqlite3 development and run-time package. Their + names may vary from system to system, but on Ubuntu 12.04 they are called + sqlite3 libsqlite3-dev. To install them, use the following command: + + > sudo apt-get install sqlite3 libsqlite3-dev + + Before running the test the database has to be created. Use the following command for that: + > cat sqlite.schema | sqlite3 sqlite.db + + A new database called sqlite.db will be created. That is the default name used + by sqlite_ubench test. If you prefer other name, make sure you update + sqlite_ubench.cc accordingly. + + Once the database is created, you can run tests: + > ./sqlite_ubench + or + > ./sqlite_ubench > results-sqlite.txt +
+ SQLite tweaks + To reconfigure sqlite_ubench parameters, a modification to the source + code and recompilation is required. All parameters are listed in main() + function in sqlite_ubench.cc file, near the end of the file. Currently supported + parameter are (default values specified in brackets): + + filename - name of the database file ("sqlite.db") + num - number of iterations (100) + sync - should the operations be performend in synchronous (true) + or asynchronous (false) manner (true) + verbose - should the test print out progress? (true) + + + + SQLite can run in asynchronous or synchronous mode. This + mode can be controlled by using sync parameter. It is set + using (PRAGMA synchronous = ON or OFF). + + Another tweakable feature is journal mode. It can be + turned to several modes of operation. Its value can be + modified in SQLite_uBenchmark::connect(). See + http://www.sqlite.org/pragma.html#pragma_journal_mode for + detailed explanantion. +
+ +
+ memfile-ubench + TODO + +
+ SQLite tweaks + ... +
+
+