<!doctype linuxdoc system>
-<!-- $Id: ulogd.sgml,v 1.6 2002/06/13 12:56:16 laforge Exp $ -->
+<!-- $Id: ulogd.sgml,v 1.7 2003/02/08 12:21:36 laforge Exp $ -->
<article>
<title>ULOGD - the Userspace Logging Daemon</title>
<author>Harald Welte <laforge@gnumonks.org></author>
-<date>Revision $Revision: 1.6 $, $Date: 2002/06/13 12:56:16 $</date>
+<date>Revision $Revision: 1.7 $, $Date: 2003/02/08 12:21:36 $</date>
<abstract>
This is the documentation for <tt>ulogd</tt>, the Userspace logging daemon.
-ulogd makes use of the Linux 2.4 firewalling subsystem (netfilter) and the
-ULOG target for netfilter.
+ulogd makes use of the Linux >= 2.4.x packet filter subsystem (iptables) and
+the ULOG target for iptables
</abstract>
<toc>
depends on the communication between the output of the logging plugins
and input of the output plugins.
<p>
-Rusty advised me to use some kind of type-key-value triples, but I think
-this is the total overkill and is too complicated for me to implement it
-in a reasonable short period of time. (3 hours later) Hmm... Rusty finally
-convinced me to use linked lists of type-key-value triples - and it wasn't
-that difficult.
+Rusty advised me to use some kind of type-key-value triples, which is in fact
+what I implemented.
<p>
-Another issue is, of course, performance. Up to ulogd 0.3, ulogd did several
+One issue is, of course, performance. Up to ulogd 0.3, ulogd did several
linked list iterations and about 30 malloc() calls _per packet_. This
-changed with the new 0.9 revision:
+changed with the new >= 0.9 revisions:
<itemize>
<item>Not a single dynamic allocation in the core during runtime.
Everything is pre-allocated at start of ulogd to provide the highest
In order to put the ipt_ULOG module into your kernel source,you need the latest
iptables package, or even better: the latest CVS snapshot. A description how to
obtain this is provided on the netfilter
-homepage <URL URL="http://netfilter.gnumonks.org">.
+homepage <URL URL="http://www.netfilter.org/">.
<p>
To run patch-o-matic, just type
<tscreen><verb>
<sect1>ulogd
<sect2>Recompiling the source
<p>
-Download the ulogd package from <URL URL="http://www.gnumonks.org/ftp/pub/netfilter/"> and
+Download the ulogd package from <URL URL="http://ftp.netfilter.org/pub/ulogd/"> and
untar it.
<p>
If you want to build ulogd with MySQL support, type './configure --with-mysql'. You may also have to specify the path of the mysql libraries using '--with-mysql=path'. To build ulogd without MySQL support, just use './configure'.
<sect2>Using a precompiled package
<p>
-I also provide a SRPM, which should compile on almost any rpm-based distribution. It is available at <URL URL="http://www.gnumonks.org/ftp/pub/netfilter/">
+I also provide a SRPM, which should compile on almost any rpm-based distribution. It is available at <URL URL="http://ftp.netfilter.org/pub/ulogd/">
<p>
Just download the package and do the usual 'rpm --rebuild <file>'.
<sect>Configuration
-<sect1>netfilter
+<sect1>iptables ULOG target
+<sect2>Quick Setup
<p>
Just add rules using the ULOG target to your firewalling chain. A very basic
example:
<tscreen><verb>
-iptables -A FORWARD -j ULOG --ulog-nlgroup 32 --prefix foo
+iptables -A FORWARD -j ULOG --ulog-nlgroup 32 --ulog-prefix foo
</verb></tscreen>
<p>
To increase logging performance, try to use the
processed by ulogd. This reduces the number of context switches between kernel
and userspace.
<p>
-Of course you can combine the ULOG target with the different netfilter match modules.
-For a more detailed description, have a look at the netfilter HOWTO's, available on
-the netfilter homepage.
+Of course you can combine the ULOG target with the different netfilter match
+modules. For a more detailed description, have a look at the netfilter
+HOWTO's, available on the netfilter homepage.
+<sect2>ULOG target reference
+<p>
+<descrip>
+<tag>--ulog-nlgroup N</tag>
+The number of the netlink multicast group to which ULOG'ed packets are sent.
+You will have to use the same group number in the ULOG target and ulogd in
+order to make logging work.
+<tag>--ulog-cprange N</tag>
+Copyrange. This works like the 'snaplen' paramter of tcpdump. You can specify
+a number of bytes up to which the packet is copied. If you say '40', you will
+receive the first fourty bytes of every packet. Leave it to '0'
+<tag>--ulog-qthreshold N</tag>
+Queue threshold. If a packet is matched by the iptables rule, and already N
+packets are in the queue, the queue is flushed to userspace. You can use this
+to implement a policy like: Use a big queue in order to gain high performance,
+but still have certain packets logged immediately to userspace.
+<tag>--ulog-prefix STRING</tag>
+A string that is associated with every packet logged by this rule. You can use
+this option to later tell from which rule the packet was logged.
+</descrip>
+
+<sect2>ipt_ULOG module parameters
+<p>
+The ipt_ULOG kernel module has a couple of module loadtime parameters which can
+(and should) be tuned to accomodate the needs of the application:
+<descrip>
+<tag>nlbufsiz N</tag>
+Netlink buffer size. A buffer of the specified size N is allocated for every
+netlink group that is used. Please note that due to restrictions of the kernel
+memory allocator, we cannot have a buffer size > 128kBytes. Larger buffer
+sizes increase the performance, since less kernel/userspace context switches
+are needed for the same amount of packets. The backside of this performance
+gain is a potentially larger delay. The default value is 4096 bytes, which is
+quite small.
+<tag>flushtimeout N</tag>
+The flushtimeout determines, after how many clock ticks (on alpha: 1ms, on
+x86 and most other platforms: 10ms time units) the buffer/queue is to be
+flushed, even if it is not full. This can be used to have the advantage of a
+large buffer, but still a finite maximum delay introduced. The default value
+is set to 10 seconds.
+</descrip>
+Example:
+<tscreen><verb>
+modprobe ipt_ULOG nlbufsiz=65535 flushtimeout=100
+</verb></tscreen>
+This would use a buffer size of 64k and a flushtimeout of 100 clockticks (1 second on x86).
<sect1>ulogd
<p>
-All configurable parameters of ulogd are in the configfile '/etc/ulogd.conf'
+ulogd is what this is all about, so let's describe it's configuration...
+<sect2>ulogd configfile syntax reference
+<p>
+All configurable parameters of ulogd are in the configfile, typically located
+at '/etc/ulogd.conf'.
<p>
The following configuration parameters are available:
<descrip>
<tag>nlgroup</tag>
-The netlink multicast group, which ulgogd should bind to. This is the same as given with the '--ulog-nlgroup' option to iptables.
+The netlink multicast group, which ulgogd should bind to. This is the same as
+given with the '--ulog-nlgroup' option to iptables.
<tag>logfile</tag>
The main logfile, where ulogd reports any errors, warnings and other unexpected
conditions.
<tag>loglevel</tag>
-This specifies, how verbose the logging to logfile is. Currently defined loglevels are: 1=debug information, 3=informational messages, 5=noticable exceptional conditions, 7=error conditions, 8=fatal errors, program abort.
+This specifies, how verbose the logging to logfile is. Currently defined
+loglevels are: 1=debug information, 3=informational messages, 5=noticable
+exceptional conditions, 7=error conditions, 8=fatal errors, program abort.
<tag>plugin</tag>
-This option is followed by a filename of a ulogd plugin, which ulogd shold load upon initialization. This option may appear more than once.
+This option is followed by a filename of a ulogd plugin, which ulogd shold load
+upon initialization. This option may appear more than once.
+<tag>bufsize</tag>
+Size of the receive buffer. You should set this to at least the size of the
+kernel buffer (nlbufsiz parameter of the ipt_ULOG module).
+</descrip>
+<sect2>ulogd commandline option reference
+<p>
+Apart from the configfile, there are a couple of commandline options to ulogd:
+<descrip>
+<tag>-h --help</tag>
+Print a help message about the commandline options.
+<tag>-V --version</tag>
+Print version information about ulogd.
+<tag>-d --daemon</tag>
+For off into daemon mode. Unless you are debugging, you will want to use this
+most of the time.
+<tag>-c --configfile</tag>
+Using this commandline option, an alternate config file can be used. This is
+important if multiple instances of ulogd are to be run on a single machine.
</descrip>
<sect>Available plugins
<p>
-ulogd comes with the following plugins:
-<descrip>
-<tag>ulogd_BASE.so</tag>
-Basic interpreter plugin for nfmark, timestamp, mac address, ip header, tcp header, udp header, icmp header, ah/esp header.
-<tag>ulogd_PWSNIFF.so</tag>
-Example interpreter plugin to log plaintext passwords as used with FTP and POP3. Don't blame me for writing this plugin! The protocols are inherently insecure, and there are a lot of other tools for sniffing passwords... it's just an example.
-<tag>ulogd_OPRINT.so</tag>
+It is important to understand that ulogd without plugins does nothing. It will receive packets, and do nothing with them.
+<p>
+There are two kinds of plugins, interpreter and output plugins. Interpreter
+plugins parse the packet, output plugin write the interpreted information to
+some logfile/database/...
+
+<sect1>Interpreter plugins
+<p>
+ulogd comes with the following interpreter plugins:
+<sect2>ulogd_BASE.so
+<p>
+Basic interpreter plugin for nfmark, timestamp, mac address, ip header, tcp
+header, udp header, icmp header, ah/esp header... Most people will want to load
+this very important plugin.
+<sect2>ulogd_PWSNIFF.so
+<p>
+Example interpreter plugin to log plaintext passwords as used with FTP and
+POP3. Don't blame me for writing this plugin! The protocols are inherently
+insecure, and there are a lot of other tools for sniffing passwords... it's
+just an example.
+<sect2>ulogd_LOCAL.so
+<p>
+This is a 'virtual interpreter'. It doesn't really return any information on
+the packet itself, rather the local system time and hostname. Please note that
+the time is the time at the time of logging, not the packets receive time.
+
+<sect1>Output plugins
+<p>
+ulogd comes with the following output plugins:
+
+<sect2>ulogd_OPRINT.so
+<p>
A very simple output module, dumping all packets in the format
<tscreen><verb>
===>PACKET BOUNDARY
===>PACKET BOUNDARY
...
</verb></tscreen>
-to a file.
+to a file. The only useful application is debugging.
<p>The module defines the following configuration directives:
<descrip>
<tag>dumpfile</tag>
-The filename where it should log to. The default is <tt>/var/log/ulogd.pktlog</tt>
+The filename where it should log to. The default is
+<tt>/var/log/ulogd.pktlog</tt>
</descrip>
-<tag>ulogd_LOGEMU.so</tag>
-An output module which tries to emulate the old syslog-based LOG targed as far as possible. Logging is done to a textfile instead of syslog, though.
+<sect2>ulogd_LOGEMU.so
+<p>
+An output module which tries to emulate the old syslog-based LOG targed as far
+as possible. Logging is done to a seperate textfile instead of syslog, though.
<p>
The module defines the following configuration directives:
<descrip>
-<tag>syslogfile</tag>The filename where it should log to. The default is <tt>/var/log/ulogd.syslogemu</tt>
-</descrip>
-<descrip>
-<tag>syslogsync</tag>Set this to 1 if you want to have your logfile written synchronously. This may reduce performance, but makes your log-lines appear immediately. The default is <tt>0</tt>
+<tag>syslogfile</tag>The filename where it should log to. The default is
+<tt>/var/log/ulogd.syslogemu</tt>
+<tag>syslogsync</tag>Set this to 1 if you want to have your logfile written
+synchronously. This may reduce performance, but makes your log-lines appear
+immediately. The default is <tt>0</tt>
</descrip>
-<tag>ulogd_MYSQL.so</tag>
+<sect2>ulogd_MYSQL.so
+<p>
An output plugin for logging into a mysql database. This is only compiled if
you have the mysql libraries installed, and the configure script was able to
detect them. (that is: --with-mysql was specified for ./configure) <p>
Password for mysql
</descrip>
+<sect2>ulogd_PGSQL.so
+<p>
+An output plugin for logging into a postgresql database. This is only compiled
+if you have the mysql libraries installed, and the configure script was able to
+detect them. (that is: --with-pgsql was specified for ./configure) <p>
+
+The plugin automagically inserts the data into the configured table; It
+connects to mysql during the startup phase of ulogd and obtains a list of the
+columns in the table. Then it tries to resolve the column names against keys of
+interpreter plugins. This way you can easly select which information you want
+to log - just by the layout of the table. <p>
+
+If, for example, your table contains a field called 'ip_saddr', ulogd will
+resolve this against the key 'ip.saddr' and put the ip address as 32bit
+unsigned integer into the table. <p>
+
+You may want to have a look at the file '<tt>doc/mysql.table</tt>' as an
+example table including fields to log all keys from ulogd_BASE.so. Just delete
+the fields you are not interested in, and create the table. <p>
+
+The module defines the following configuration directives:
+<descrip>
+<tag>pgsqltable</tag>
+Name of the table to which ulogd should log
+<tag>pgsqldb</tag>
+Name of the mysql database
+<tag>pgsqlhost</tag>
+Name of the mysql database host
+<tag>pgsqluser</tag>
+Name of the mysql user
+<tag>pgsqlpass</tag>
+Password for mysql
+</descrip>
+
+<sect2>ulogd_PCAP.so
+<p>
+An output plugin that can be used to generate libpcap-style packet logfiles.
+This can be useful for later analysing the packet log with tools like tcpdump
+or ethereal.
+
+The module defines the following configuration directives:
+<descrip>
+<tag>pcapfile</tag>
+The filename where it should log to. The default is:
+<tt>/var/log/ulogd.pcap</tt>
+<tag>pcapsync</tag>
+Set this to <tt>1</tt> if you want to have your pcap logfile written
+synchronously. This may reduce performance, but makes your packets appear
+immediately in the file on disk. The default is <tt>0</tt>
</descrip>
+
<sect> QUESTIONS / COMMENTS
<p>
All comments / questions / ... are appreciated.
Please note also that there is now a mailinglist, ulogd@lists.gnumonks.org.
You can subscribe at
<URL URL="http://lists.gnumonks.org/mailman/listinfo/ulogd/">
+<p>
+The preferred method for reporting bugs is the netfilter bugzilla system,
+available at <URL URL="http://bugzilla.netfilter.org/">.
</article>
projects / thirdparty / ulogd2.git / commitdiff
