From: Tomek Mrugalski <tomasz@isc.org>
Date: Wed, 4 Nov 2015 14:27:48 +0000 (+0900)
Subject: [4088] Developer's guide written
X-Git-Tag: trac4088fd_base~8
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=b4c11fc8e81adea6a5d932097cd76ce3d99763c5;p=thirdparty%2Fkea.git

[4088] Developer's guide written
---

diff --git a/src/lib/eval/eval.dox b/src/lib/eval/eval.dox
index 5359d66548..be4380761d 100644
--- a/src/lib/eval/eval.dox
+++ b/src/lib/eval/eval.dox
@@ -1,4 +1,3 @@
-
 // Copyright (C) 2015  Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
@@ -14,8 +13,106 @@
 // PERFORMANCE OF THIS SOFTWARE.
 
 /**
-  @page dhcpEval Expression evaluation (client classification)
+  @page dhcpEval libeval - Expression evaluation and client classification
+
+  The core of the libeval library is a parser that is able to parse an
+  expression (e.g. option[123] == 'APC'). This is currently used for client
+  classification, but in the future may be also used for other applications.
+  The external interface to the library is @ref isc::eval::EvalContext class.
+  Once instantiated, it offers two major methods:
+  @ref isc::eval::EvalContext::parseFile that parses content of the file
+  and @ref isc::eval::EvalContext::parseString, which parses specified string.
+  Once expression is parsed, it is converted to a collection of tokens
+  that are stored in Reverse Polish Notation in EvalContext::expression.
+
+  Internally, the parser code is generated by flex and bison. Those two
+  tools convert lexer.ll and parser.yy files into a number of .cc and .hh files.
+  To avoid adding flex and bison as dependencies, the result of the
+  generation is checked into the github repo and is distributted in the
+  tarballs.
+
+  @section dhcpEvalLexer Lexer generation using flex
+
+  Flex is used to generate lexer, a piece of code that converts input
+  data into a series of tokens. It contains a small number of directives,
+  but the majority of the code consists of a definition of tokens. These
+  are regular expressions that define various tokens, e.g. strings,
+  numbers, parentheses etc. Once the expression is matched, the associated
+  code is called. In majority of the cases a generator method from
+  @ref isc::eval::EvalParser is called. It returns newly created
+  bison tokens. The purpose of the lexer is to generate a stream
+  of tokens that are consumed by the parser.
+
+  lexer.cc and lexer.hh files must not be edited. In case there is a need
+  to introduce changes, lexer.ll must be updated and the .cc .hh files
+  regenerated.
+
+  @section dhcpEvalParser Parser generation using bison
+
+  Bison is used to generate parser, a piece of code that consumes a
+  stream of tokens and attempts to match it against defined grammar.
+  Bison parser is created from the parser.yy file. It contains
+  a number of directives, but the two most important sections are:
+  a list of tokens (for each token defined here, bison will generate
+  make_NAMEOFTOKEN method in @ref isc::eval::EvalParser class) and
+  the grammar. Grammar is a tree like structure with possible loops.
+
+  Here's an over simplified version of the grammar:
+
+@code
+01. %start expression;
+02.
+03. expression:
+04. token EQUAL token
+05. | token;
+06.
+07. token:
+08. STRING {
+09.    TokenPtr str(new TokenString($1));
+10.    ctx.expression.push_back(str);
+11.}
+12.| OPTION {
+13.    TokenPtr opt(new TokenOption($1));
+14.    ctx.expression.push_back(opt);
+15.};
+@endcode
+
+This code determines that the grammar starts from expression (line 1).
+The actual definition of expression (lines 3-5) may either be a
+single token or an expression token equals token. Token is further
+defined in lines 7-15: it may either be a string (lines 8-11) or option
+(lines 12-15). When the actual case is defined, respective C++ code
+is called. For example, TokenString class is instantiated with
+appropriate values and put onto the expression stack.
+
+@section dhcpEvalMakefile Generating parser files
+
+ In general case, we do want to avoid generating parser files, so an
+ average user interested in just compiling Kea would not need flex or
+ bison. Therefore the generated files are already included in the
+ git repository and will be included in the tarball releases.
+
+ However, there will be cases when one of the developers would want
+ to tweak the lexer.ll and parser.yy files and then regenerate
+ the code. For this purpose, two makefile targets were defined:
+ @code
+ make parser
+ @endcode
+ will generate the parsers and
+ @code
+ make parser-clean
+ @endcode
+ will remove the files. Generated files removal was also hooked
+ into maintainer-clean target.
+
+@section dhcpEvalConfigure Configure options
 
-  @todo: Document how the expression evaluation is implemented.
+ Since the flex/bison tools are not necessary for a regular compilation,
+ there are checks conducted during configure, but lack of flex or
+ bison tools does not stop the configure process. There is a flag
+ --enable-generate-parser that tells configure script that the
+ parser will be generated. With this flag, the checks for flex/bison
+ are mandatory. Any missing or too old flex/bison will cause an
+ error.
 
- */
+*/
\ No newline at end of file