From: manolamancha <> Date: Mon, 3 May 2010 19:12:00 +0000 (+0000) Subject: Update logresolvemerge.pl documentation and added plugin development to index X-Git-Tag: AWSTATS_7_0_BETA2~52 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=958fe4572d1b60e27491175ea2c030aefd527987;p=thirdparty%2FAWStats.git Update logresolvemerge.pl documentation and added plugin development to index --- diff --git a/docs/awstats_tools.html b/docs/awstats_tools.html index cc277b98..078a0413 100644 --- a/docs/awstats_tools.html +++ b/docs/awstats_tools.html @@ -1,37 +1,34 @@ - - + -AWStats Documentation - Other tools - - - - +AWStats Documentation - Other tools + - + +
- - - + + - -
+

-AWStats logfile analyzer 6.95 Documentation
+AWStats logfile analyzer 6.95 Documentation

 +  
+ -

Other utilities

+

Other utilities


This is a list of other tools provided with AWStats.
@@ -51,7 +48,7 @@ All those tools are available in tools directory of AWStats distribution. -->
-

awstats_updateall.pl

 +

awstats_updateall.pl
awstats_updateall launches update process for all AWStats config files (except
awstats.model.conf) found in a particular directory, so you can easily setup a
cron/scheduler job. The scanned directory is by default /etc/awstats. @@ -66,7 +63,7 @@ All those tools are available in tools directory of AWStats distribution.

-

awstats_buildstaticpages.pl

 +

awstats_buildstaticpages.pl
awstats_buildstaticpages allows you to launch AWStats with -staticlinks option
to build all possible pages allowed by AWStats -output option.
@@ -95,73 +92,22 @@ All those tools are available in tools directory of AWStats distribution.

-

logresolvemerge.pl

 -
logresolvemerge allows you to merge several log files into one output, -
sorted on date. It also makes a fast reverse DNS lookup to replace -
all IP addresses into host names in resulting log file. -
logresolvemerge comes with ABSOLUTELY NO WARRANTY. It's a free software -
distributed with a GNU General Public License (See COPYING.txt file). -
logresolvemerge is part of AWStats but can be used alone as a log merger -
or resolver before using any other log analyzer. +

logresolvemerge.pl

 +logresolvemerge allows you to get one unique output log file, sorted on date,
built from particular sources:
 - It can read several input log files,
 - It can read .gz/.bz2 log files,
 - It can also makes a fast reverse DNS lookup to replace
   all IP addresses into host names in resulting log file.
logresolvemerge comes with ABSOLUTELY NO WARRANTY. It's a free software
distributed with a GNU General Public License (See COPYING.txt file).
logresolvemerge is part of AWStats but can be used alone as a log merger
or resolver before using any other log analyzer.

Usage:
  logresolvemerge.pl [options] file
  logresolvemerge.pl [options] file1 ... filen
  logresolvemerge.pl [options] *.*
  perl logresolvemerge.pl [options] *.* > newfile
Options:
  -dnslookup      make a reverse DNS lookup on IP adresses
  -dnslookup=n    same with a n parallel threads instead of serial requests
  -dnscache=file  make DNS lookup from cache file first before network lookup
  -showsteps      print on stderr benchmark information every 8192 lines
  -addfilenum     if used with several files, file number can be added in first
  -addfilename    if used with several files, file name can be added in first
                  +field of output file. This can be used to add a cluster id
                  +when log files come from several load balanced computers.
  -stoponfirsteof Stop processing when any logfile reaches end-of-file.
  -printfields    For IIS or W3C logs, prints the latest field header for
                  +the currentlog file when switching between log file entries
                  +so that the parsercan automatically determine which fields
                  are avaiable.
  -ignoremissing  will not fail if a log file is missing

This runs logresolvemerge in command line to open one or several
server log files to merge them (sorted on date) and/or to make a reverse
DNS lookup (if asked). The result log file is sent on standard output.
Note: logresolvemerge is not a 'sort' tool to sort one file. It's a
software able to output sorted log records (with a reverse DNS lookup
included or not) even if log records are dispatched in several files.
Each of thoose files must be already independently sorted itself
(but that is the case in all web server log files). So you can use it
for load balanced log files or to group several old log files.

Don't forget that the main goal of logresolvemerge is to send log records to
a log analyzer in a sorted order without merging files on disk (NO NEED
OF DISK SPACE AT ALL) and without loading files into memory (NO NEED
OF MORE MEMORY). Choose of output records is done on the fly.

So logresolvemerge is particularly usefull when you want to output several
and/or large log files in a fast process, with no use of disk or
more memory, and in a chronological order through a pipe (to be used by a log
analyzer).

Note: If input records are not 'exactly' sorted but 'nearly' sorted (this
occurs with heavy servers), this is not a problem, the output will also
be 'nearly' sorted but a few log analyzers (like AWStats) knowns how to deal
with such logs.

WARNING: If log files are old MAC text files (lines ended with CR char), you
can't run this tool on Win or Unix platforms.

WARNING: Because of memory holes in ActiveState Perl version, use another
Perl interpreter if you need to process large log files.

Now supports/detects:
  Automatic detection of log format
  Files can be .gz/.bz2 files if zcat/bzcat tools are available in PATH.
  Multithreaded reverse DNS lookup (several parallel requests) with Perl 5.8+.
New versions and FAQ at http://awstats.sourceforge.net

-
Usage: -
logresolvemerge.pl [options] file -
logresolvemerge.pl [options] file1 ... filen -
logresolvemerge.pl [options] *.* -
Options: -
-dnslookup make a reverse DNS lookup on IP adresses -
-dnscache=file make DNS lookup from cache file first before network lookup -
-showsteps print on stderr benchmark information every 8192 lines -
-addfilenum if used with several files, file number can be added in first -
field of output file. -
-
This runs logresolvemerge in command line to open one or several web -
server log files to merge them (sorted on date) and/or to make a reverse -
DNS lookup (if wanted). The result log file is sent on standard output. -
Note: logresolvemerge is not a 'sort' tool to sort one file. It's a -
software able to output sorted log records (with a reverse DNS lookup -
included or not) even if log records are dispatched in several files. -
Each of thoose files must be already independently sorted itself -
(but that is the case in all web server log files). So you can use it -
for load balanced log files or to process several old log files. -
-
Don't forget that the main goal of logresolvemerge is to send log records to -
a log analyzer in a sorted order without merging files on disk (NO NEED -
OF DISK SPACE AT ALL) and without loading files into memory (NO NEED -
OF MORE MEMORY). Choose of output records is done on the fly. -
-
So logresolvemerge is particularly useful when you want to output several -
and/or large log files in a fast process, with no use of disk or -
more memory and in a chronological order through a pipe (to be used by a log -
analyzer. -
-
Note: If input records are not 'exactly' sorted but 'nearly' sorted (this -
occurs with heavy servers), this is not a problem, the output will also -
be 'nearly' sorted but a few log analyzers (like AWStats) knowns how to deal -
with such logs. -
-
WARNING: If log files are old MAC text files (lines ended with CR char), you -
can't run this tool on Win or Unix platforms. -
-
WARNING: Because of important memory holes in ActiveState Perl version, use -
another Perl interpreter if you need to process large log files. -
-
Now supports/detects: -
Automatic detection of log format -
Files can be .gz/.bz2 files if zcat/bzcat tools are available in PATH. -
New versions and FAQ at http://awstats.sourceforge.net
-
- -
-

maillogconvert.pl

 +

maillogconvert.pl
maillogconvert is mail log preprocessor that convert a mail log file (from
postfix, sendmail or qmail servers) into a human readable format.
The output format is also ready to be used by a log analyzer, like AWStats.

Usage: -
perl maillogconvert.pl [standard|vadmin] [year] < logfile > output +
perl maillogconvert.pl [standard|vadmin] [year] < logfile > output

The first parameter specifies what format the mail logfile is :
standard - logfile is standard postfix,sendmail,qmail or mdaemon log format @@ -176,34 +122,35 @@ All those tools are available in tools directory of AWStats distribution.
For example, the following sample from postfix mail log server:
-
# 1 Mail fromuser@aol.com -> touser@toserver.com, forward touser@toserver.com -> touser@mainserver.com +
# 1 Mail fromuser@aol.com -> touser@toserver.com, forward touser@toserver.com -> touser@mainserver.com
Jan 01 07:27:31 apollon postfix/smtpd[1684]: connect from remt30.cluster1.charter.net[209.225.8.40]
Jan 01 07:27:32 apollon postfix/smtpd[1684]: 2BC793B8A4: client=remt30.cluster1.charter.net[209.225.8.40] -
Jan 01 07:27:32 apollon postfix/cleanup[1687]: 2BC793B8A4: message-id=<36027278@vneka> -
Jan 01 07:27:32 apollon postfix/qmgr[13860]: 2BC793B8A4: from=, size=2130, nrcpt=1 (queue active) +
Jan 01 07:27:32 apollon postfix/cleanup[1687]: 2BC793B8A4: message-id=<36027278@vneka> +
Jan 01 07:27:32 apollon postfix/qmgr[13860]: 2BC793B8A4: from=, size=2130, nrcpt=1 (queue active)
Jan 01 07:27:32 apollon postfix/smtpd[1684]: disconnect from remt30.cluster1.charter.net[209.225.8.40] -
Jan 01 07:27:38 apollon postfix/local[1689]: 2BC793B8A4: to=, orig_to=, relay=local, delay=6, status=sent ("|/usr/bin/procmail") +
Jan 01 07:27:38 apollon postfix/local[1689]: 2BC793B8A4: to=, orig_to=, relay=local, delay=6, status=sent ("|/usr/bin/procmail")
# 2 Reject: 450
Jan 01 14:05:44 apollon postfix/smtpd[2114]: connect from baby.mainframe.nl[81.29.4.2]
Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: client=baby.mainframe.nl[81.29.4.2] -
Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: reject: RCPT from baby.mainframe.nl[81.29.4.2]: 450 : User unknown in local recipient table; from=<> to= proto=ESMTP helo= -
Jan 01 14:10:16 juni postfix/smtpd[2568]: C34ED1432B: reject: RCPT from relay2.tp2rc.edu.tw[163.28.32.177]: 450 : User unknown in local recipient table; from=<> proto=ESMTP helo= +
Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: reject: RCPT from baby.mainframe.nl[81.29.4.2]: 450 : User unknown in local recipient table; from=<> to= proto=ESMTP helo= +
Jan 01 14:10:16 juni postfix/smtpd[2568]: C34ED1432B: reject: RCPT from relay2.tp2rc.edu.tw[163.28.32.177]: 450 : User unknown in local recipient table; from=<> proto=ESMTP helo=
# 1 From unknown
Jan 01 15:17:05 apollon postfix/smtpd[29866]: connect from tomts12.bellnexxia.net[209.226.175.56]
Jan 01 15:17:05 apollon postfix/smtpd[29866]: 578093B8B5: client=tomts12.bellnexxia.net[209.226.175.56] -
Jan 01 15:17:05 apollon postfix/cleanup[28931]: 578093B8B5: message-id=<20030905131913.EOVH11393.tomts12-srv.bellnexxia.net@tomts12-srv> -
Jan 01 15:17:06 apollon postfix/qmgr[965]: 578093B8B5: from=<>, size=109367, nrcpt=1 (queue active) -
Jan 01 15:17:06 apollon postfix/local[32432]: 578093B8B5: to=, orig_to=, relay=local, delay=1, status=sent ("|/usr/bin/procmail") +
Jan 01 15:17:05 apollon postfix/cleanup[28931]: 578093B8B5: +message-id=<20030905131913.EOVH11393.tomts12-srv.bellnexxia.net@tomts12-srv> +
Jan 01 15:17:06 apollon postfix/qmgr[965]: 578093B8B5: from=<>, size=109367, nrcpt=1 (queue active) +
Jan 01 15:17:06 apollon postfix/local[32432]: 578093B8B5: to=, orig_to=, relay=local, delay=1, status=sent ("|/usr/bin/procmail")
Jan 01 15:17:06 apollon postfix/smtpd[29866]: disconnect from tomts12.bellnexxia.net[209.226.175.56] - +

will give a file that looks like this:

2004-01-01 07:27:38 fromuser@aol.com touser@toserver.com remt30.cluster1.charter.net localhost SMTP - 1 2130 -
2004-01-01 14:05:44 <> touser2@toserver.com baby.mainframe.nl - SMTP - 450 0 -
2004-01-01 14:10:16 <> unknownuser@unknownserver.com relay2.tp2rc.edu.tw - SMTP - 450 0 -
2004-01-01 15:17:06 <> touser@toserver.com tomts12.bellnexxia.net localhost SMTP - 1 109367 +
2004-01-01 14:05:44 <> touser2@toserver.com baby.mainframe.nl - SMTP - 450 0 +
2004-01-01 14:10:16 <> unknownuser@unknownserver.com relay2.tp2rc.edu.tw - SMTP - 450 0 +
2004-01-01 15:17:06 <> touser@toserver.com tomts12.bellnexxia.net localhost SMTP - 1 109367

See FAQ-COM100 to see how to use maillogconvert.pl with AWStats to analyze mail log files. @@ -213,7 +160,7 @@ All those tools are available in tools directory of AWStats distribution.

-

urlaliasbuilder.pl

 +

urlaliasbuilder.pl
Urlaliasbuilder generates an 'urlalias' file from an input file (an urlalias file
is a file with two columns: url and clear title of url).
The input file must contain a list of URLs (It can be an AWStats history file). @@ -246,10 +193,9 @@ All those tools are available in tools directory of AWStats distribution.
- - - + \ No newline at end of file diff --git a/docs/index.html b/docs/index.html index de5e5fce..d08e8d7b 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,161 +1,152 @@ - - + -AWStats logfile analyzer Documentation - - - - +AWStats logfile analyzer Documentation + +
- +
- - - + + - -
+

-AWStats logfile analyzer 6.95 Documentation
+AWStats logfile analyzer 6.95 Documentation

 +  
+ -

Table of contents

+

Table of contents

-
+
- -
+ +
- - - - - + + + + +
  - + + + + + -
- - - -
- - - - - - - - -
+
+ + + + + + + + +
-
+
- +
- -
+
- -
+ + -
Release Notes
-What is AWStats / Features +What is AWStats / Features
-New Features / Changelog +New Features / Changelog
-Upgrade +Upgrade
+
 - -
+ + -
Reference manual
-Install, Setup and Use AWStats +Install, Setup and Use AWStats
-Configuration Directives/Options +Configuration Directives/Options
-Configuration for Extra Sections feature +Configuration for Extra Sections feature
-Contribs, plugins and resources +Contribs, plugins and resources
-Other provided utilities +Other provided utilities
-AWStats Webmin admin module +AWStats Webmin admin module
-Glossary of terms +Glossary of terms
+
+
- +
- +AWStats License + + - -
+
Other Topics
-Comparison with other log analyzers +Comparison with other log analyzers -FAQ and Troubleshooting +FAQ and Troubleshooting
-Benchmarks +Benchmarks -AWStats License -
Plugin Developement
-Security - -The Online Demo - - 
+
- +
- - + \ No newline at end of file