FAQ-COM025 <a href="#NOLOG">How to use AWStats with no server log</a><br>
FAQ-COM050 <a href="#LIMITLOG">What is the log size limit AWStats can analyze ?</a><br>
FAQ-COM090 <a href="#FTP">Setup for FTP server log files.</a><br>
-FAQ-COM100 <a href="#MAIL">Setup for MAIL log files (Sendmail, Postfix, QMail, Exchange).</a><br>
+FAQ-COM100 <a href="#MAIL">Setup for MAIL log files (Sendmail, Postfix, QMail, MDaemon, Exchange).</a><br>
FAQ-COM110 <a href="#MEDIASERVER">Setup for MEDIA SERVER log files (Realmedia, Windows media, Darwin streaming server).</a><br>
FAQ-COM120 <a href="#ROTATE">How to rotate my logs without loosing data.</a><br>
FAQ-COM130 <a href="#CRONTAB">How to run AWStats frequently ?</a><br>
Examples of used platforms (bold means 'tested by author', others were reported by AWStats users to work correctly) :<br>
<table bgcolor=#F4F4F4 width="100%" class=CFAQ><tr class=CFAQ><td class=CFAQ>
<u>OS:</u><br>
-<b>Windows 2000</b>, <b>Windows NT 4.0</b>, Windows Me, <b>Linux</b>, Macintosh, <b>Solaris</b>, <b>Aix</b>, BeOS, ...<br>
-<u>Web/Wap/Proxy/FTP/Mails/Streaming servers</u><br>
-<b>Apache 1.3.x and 2</b>, <b>IIS 5.0</b>, WebStar, WebLogic, WebSite, <b>Windows Media Server</b>, Tomcat, <b>Squid</b>, <b>Sendmail</b>, <b>Postfix</b>, QMail,
-Roxen, Resin, <b>ProFTP</b>, Lotus Notes/Domino, Darwin, IPlanet, IceCast, ZeroBrand, Zeus, Zope, Abyss, www4mail, ...<br>
+<b>Windows 2000</b>, <b>Windows NT 4.0</b>, Windows Me, <b>Linux (RedHat, Mandrake, Debian, Suse...)</b>, Macintosh, <b>Solaris</b>, <b>Aix</b>, BeOS, ...<br>
+<u>Web/Wap/Proxy/Streaming servers</u><br>
+<b>Apache 1.3.x and 2</b>, <b>IIS 5.0</b>, WebStar, WebLogic, WebSite, <b>Windows Media Server</b>, Tomcat, <b>Squid</b>,
+Roxen, Resin, Lotus Notes/Domino, Darwin, IPlanet, IceCast, ZeroBrand, Zeus, Zope, Abyss...<br>
+<u>FTP servers</u><br>
+<b>ProFTP</b>, ...<br>
+<u>Mails servers</u><br>
+<b>Sendmail</b>, <b>Postfix</b>, QMail, <b>Mdaemon</b>, www4mail, ...<br>
<u>Perl interpreters:</u><br>
<b>ActivePerl 5.6</b>, <b>Perl 5.8</b>, <b>Perl 5.6</b>, <b>Perl 5.0</b>, <b>mod_perl for Apache</b>, ...<br>
</td></tr></table>
<br>
<a name="MAIL"></a><br>
-<b><u>FAQ-COM100 : SETUP FOR MAIL LOG FILES (SENDMAIL, POSTFIX, QMAIL, EXCHANGE...)</u></b><br>
+<b><u>FAQ-COM100 : SETUP FOR MAIL LOG FILES (SENDMAIL, POSTFIX, QMAIL, MDAEMON, EXCHANGE...)</u></b><br>
<font class=CProblem>PROBLEM:</font><br>
What do I have to do to use AWStats to analyze my mail log files.<br>
<font class=CSolution>SOLUTION:</font><br>
<br>
This tip works with AWStats 5.5 or higher.<br>
<br>
-<u>For Postfix, Sendmail or QMail log files</u><br>
+<u>For Postfix, Sendmail, QMail or MDaemon log files</u><br>
<br>
You must setup AWStats to use a mail log file preprocessor (<i>maillogconvert.pl</i> is provided
into AWStats <i>tools</i> directory, but you can use the one of your choice):<br>
<i>
LogFile="perl /path/to/maillogconvert.pl standard < /pathtomaillog/maillog |"<br>\r
</i>
-And for VAdmin QMail logfiles (multi-host/virtualhost mail servers running vadmin software)<br>\r
+If the logfiles are compressed, they can be processed this way<br>\r
<i>
-LogFile="perl /path/to/maillogconvert.pl vadmin < /pathtomaillog/maillog |"<br>\r
+LogFile="gzip -cd /var/log/maillog.0.gz | /path/to/maillogconvert.pl standard |"<br>\r
</i>
-If the logfiles are compressed, they can be processed this way<br>\r
+And for VAdmin QMail logfiles (multi-host/virtualhost mail servers running vadmin software)<br>\r
<i>
-LogFile="gzip -cd /var/log/maillog.0.gz | /path/to/maillogconvert.pl vadmin |"<br>\r
+LogFile="perl /path/to/maillogconvert.pl vadmin < /pathtomaillog/maillog |"<br>\r
</i>
Then change:
<i>
<u>For Exchange log files</u><br>
<br>
-If someone can understand the Exchange log format, he can explain it to me at eldy@users.sourceforge.net.<br>
-Until this server will log messy data, just forget using AWStats or use a more serious
-mail server (sendmail, postfix, ...). Sorry.<br>
+Despite the high number of possible log format provided with Exchange,
+none of them is built enough seriously to offer an interseting analyze (missing informations,
+messy data, no id to join multiple records for same mail, etc...).
+For this reason, an "interesting" log analysis is a joke with Exchange log files (No data is
+better than false data). So forget using AWStats or use a more serious mail server (sendmail,
+postfix, qmail, mdaemon, ...). Sorry.<br>
<!--
Your Exchange mail log file might looks like this:<br>
<i>2002-9-29 0:3:2 GMT - - - S_EXCHANG01 - /O=INVESTMENT SCORECARD INC/OU=MAIL/CN=RECIPIENTS/CN=KJOHNSON 1027 1640EF7202084243942C5EC620117D376B68AA@s_exchang01.investmentscorecard.com 0 0 1943 2 2002-9-29 0:3:2 GMT 0 - c=US;a= ;p=Investment Score;l=S_EXCHANG01-020929000302Z-2583 Dev Request Late Notification: KS_CM00004271 EX:/O=INVESTMENT SCORECARD INC/OU=MAIL/CN=RECIPIENTS/CN=CLEARQUEST(KS_CM) -</i><br>
# Note 2: QMail logging is not 100% accurate. Some messages might
# not be logged correctly or completely.
#
-# A mail received to 2 different receivers, report 2 records instead of one.\r
+# A mail received to 2 different receivers, report 2 records.\r
# A mail received to a forwarded account is reported as to the original receiver, not the "forwarded to".
# A mail locally sent to a local alias is reported as n mails to all addresses of alias.
#-------------------------------------------------------\r
$NBOFENTRYFOFLUSH
$MailType
/;
-
+$Debug=0;
$NBOFENTRYFOFLUSH=8192; # Nb or records for flush of %entry (Must be a power of 2)
$MailType=''; # Mail server family (postfix, sendmail, qmail)
# "root@servername", "[123.123.123.123]"
# Return: servername or 123.123.123.123 if servername is 'unknown'
sub CleanHost {
- $_=shift;
+ $_=shift||'';
if (/^\[(.*)\]$/) { $_=$1; } # If [ip] we keep ip
if (/^unknown\s*\[/) { $_ =~ /\[(.*)\]/; $_=$1; } # If unknown [ip], we keep ip
else { $_ =~ s/\s*\[.*$//; }
my $code=shift;
my $size=shift||0;
my $forwardto=shift;
+ my $extinfo=shift||'-';
# Clean day and month
$day=sprintf("%02d",$day);
#if we don't have info for relay_s, we keep it unknown, awstats might then guess it
# Write line
- print "$year-$month-$day $time $from $to $relay_s $relay_r SMTP - $code $size\n";
+ print "$year-$month-$day $time $from $to $relay_s $relay_r SMTP $extinfo $code $size\n";
# If there was a redirect
if ($forwardto) {
perl maillogconvert.pl [standard|vadmin] [year] < logfile > output
The first parameter specifies what format the mail logfile is :
- standard - logfile is standard postfix,sendmail or qmail log format
+ standard - logfile is standard postfix,sendmail,qmail or mdaemon log format
vadmin - logfile is qmail log format with vadmin multi-host support
The second parameter specifies what year to timestamp logfile with, if current
#
# Get sender host for postfix
#
- elsif (/: client=/ ne undef) {
+ elsif (/: client=/) {
$MailType||='postfix';
my ($id,$relay_s)=m/\w+\s+\d+\s+\d+:\d+:\d+\s+[\w\-]+\s+(?:sendmail|postfix\/(?:local|lmtp|smtpd|smtp|virtual))\[\d+\]:\s+(.*?):\s+client=(.*)/;
$mailid=$id;
#
# See if we received postfix email reject error
#
- elsif (/: reject/ ne undef) {
+ elsif (/: reject/) {
$MailType||='postfix';
# Example:
# postfix: Jan 01 04:19:04 apollon postfix/smtpd[26553]: 1954F3B8A4: reject: RCPT from unknown[80.245.33.2]: 450 <partenaires@chiensderace.com>: User unknown in local recipient table; from=<httpd@fozzy2.dpi-europe.fr> to=<partenaires@chiensderace.com> proto=ESMTP helo=<fozzy2.dpi-europe.fr>
# postfix: Jan 01 04:26:39 halley postfix/smtpd[9245]: reject: RCPT from unknown[203.156.32.33]: 554 <charitha99@yahoo.com>: Recipient address rejected: Relay access denied; from=<1126448365@aol.com> to=<charitha99@yahoo.com>\r
my ($mon,$day,$time,$id,$code,$from,$to)=m/(\w+)\s+(\d+)\s+(\d+:\d+:\d+)\s+[\w\-]+\s+(?:postfix\/(?:local|lmtp|smtpd|smtp|virtual))\[\d+\]:\s+(.*?):\s+(.*)\s+from=([^\s,]*)\s+to=([^\s,]*)/;
- $mailid=($id eq 'reject'?999:$id); # id not provided in log, we take 999
+ $mailid=($id eq 'reject'?'999':$id); # id not provided in log, we take '999'
# $code='reject: RCPT from c66.191.66.89.dul.mn.charter.com[66.191.66.89]: 450 <partenaires@chiensderace.com>: User unknown in local recipient table;'
# or 'reject: RCPT from unknown[203.156.32.33]: 554 <charitha99@yahoo.com>: Recipient address rejected: Relay access denied;'
if ($mailid) {
#
# See if we received sendmail reject error
#
- elsif (/, reject/ ne undef) {
+ elsif (/, reject/) {
$MailType||='sendmail';
# Example:
# sm-mta: Jul 27 04:06:05 androneda sm-mta[6641]: h6RB44tg006641: ruleset=check_mail, arg1=<7ms93d4ms@topprodsource.com>, relay=crelay1.easydns.com [216.220.57.222], reject=451 4.1.8 Domain of sender address 7ms93d4ms@topprodsource.com does not resolve
#
# See if we received postfix email bounced error
#
- elsif (/stat(us)?=bounced/ ne undef) {
+ elsif (/stat(us)?=bounced/) {
$MailType||='postfix';
# Example:
# postfix: Sep 9 18:24:23 halley postfix/local[22003]: 12C6413EC9: to=<etavidian@partenor.com>, relay=local, delay=0, status=bounced (unknown user: "etavidian")
my ($mon,$day,$time,$id,$to,$relay_r)=m/(\w+)\s+(\d+)\s+(\d+:\d+:\d+)\s+[\w\-]+\s+(?:postfix\/(?:local|lmtp|smtpd|smtp|virtual))\[\d+\]:\s+(.*?):\s+to=([^\s,]*)[\s,]+relay=([^\s,]*)/;
- $mailid=($id eq 'reject'?999:$id); # id not provided in log, we take 999
+ $mailid=($id eq 'reject'?'999':$id); # id not provided in log, we take '999'
if ($mailid) {
- $mail{$mailid}{'code'}="999"; # Unkown error (bounced)
+ $mail{$mailid}{'code'}=999; # Unkown error (bounced)
$mail{$mailid}{'to'}=&trim($to);
$mail{$mailid}{'relay_r'}=&trim($relay_r);
$mail{$mailid}{'mon'}=$mon;
#
# See if we send a sendmail (with ctladdr tag) email
#
- elsif(/, ctladdr=/ ne undef) {
+ elsif (/, ctladdr=/) {
$MailType||='sendmail';
#
# Matched outgoing sendmail/postfix message
$mail{$id}{'to'}=&trim($to);
$mail{$id}{'from'}=&trim($from);
if (! defined($mail{$id}{'size'})) { $mail{$id}{'size'}='?'; }
- debug("For id=$id, found a sendmail outgoing message: to=$mail{$id}{'to'} from=$mail{$id}{'from'} size=$mail{$id}{'size'} relay_s=$mail{$id}{'relay_s'}");
+ debug("For id=$id, found a sendmail outgoing message: to=$mail{$id}{'to'} from=$mail{$id}{'from'} size=$mail{$id}{'size'} relay_r=".($mail{$id}{'relay_r'}||''));
}
#
# Matched incoming qmail message
#
- elsif (/info msg .* from/ ne undef) {
+ elsif (/info msg .* from/) {
# Example: Sep 14 09:58:09 gandalf qmail: 1063526289.292776 info msg 270182: bytes 10712 from <john@john.do> qp 54945 uid 82
$MailType||='qmail';
#my ($id,$size,$from)=m/(\d+)(?:\.\d+)? info msg \d+: bytes (\d+) from <(.*)>/;
#
# Matched incoming sendmail or postfix message
#
- elsif (/: from=/ ne undef) {
+ elsif (/: from=/) {
# sm-mta: Jul 28 06:55:13 androneda sm-mta[28877]: h6SDtCtg028877: from=<4cmkh79eob@webtv.net>, size=2556, class=0, nrcpts=1, msgid=<w1$kqj-9-o2m45@0h2i38.4.m0.5u>, proto=ESMTP, daemon=MTA, relay=smtp.easydns.com [205.210.42.50]
# postfix: Jul 3 15:32:26 apollon postfix/qmgr[13860]: 08FB63B8A4: from=<nobody@ns3744.ovh.net>, size=3302, nrcpt=1 (queue active)
my ($id,$from,$size)=m/\w+\s+\d+\s+\d+:\d+:\d+\s+[\w\-]+\s+(?:sm-mta|sendmail(?:-in|)|postfix\/qmgr|postfix\/nqmgr)\[\d+\]:\s+(.*?):\s+from=(.*?),\s+size=(.*?),/;
$mailid=$id;
if (! $mail{$id}{'code'}) { $mail{$id}{'code'}=1; } # If not already defined, we define it
- if ($mail{$id}{'from'} ne '<>') { $mail{$id}{'from'}=$from; }
+ if (! $mail{$id}{'from'} || $mail{$id}{'from'} ne '<>') { $mail{$id}{'from'}=$from; }
$mail{$id}{'size'}=$size;
if (m/\s+relay=([^\,]+)[\s\,]/ || m/\s+relay=([^\s\,]+)$/) { $mail{$id}{'relay_s'}=$1; }
- debug("For id=$id, found a sendmail/postfix incoming message: from=$mail{$id}{'from'} size=$mail{$id}{'size'} relay_s=$mail{$id}{'relay_s'}");
+ debug("For id=$id, found a sendmail/postfix incoming message: from=$mail{$id}{'from'} size=$mail{$id}{'size'} relay_s=".($mail{$id}{'relay_s'}||''));
}
#
# Matched exchange message
#
- elsif (/^([^\t]+)\t([^\t]+)\t[^\t]+\t([^\t]+)\t([^\t]+)\t([^\t]+)\t[^\t]+\t([^\t]+)\t([^\t]+)\t([^\t]+)\t[^\t]+\t[^\t]+\t([^\t]+)\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t([^\t]+)/ ne undef) {
+ elsif (/^([^\t]+)\t([^\t]+)\t[^\t]+\t([^\t]+)\t([^\t]+)\t([^\t]+)\t[^\t]+\t([^\t]+)\t([^\t]+)\t([^\t]+)\t[^\t]+\t[^\t]+\t([^\t]+)\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t[^\t]+\t([^\t]+)/) {
# date hour GMT ip_s relay_s partner relay_r ip_r to code id size from
# Example: 2003-8-12 0:58:14 GMT 66.218.66.69 n14.grp.scd.yahoo.com - PACKRAT 192.168.1.2 christina@pirnie.org 1019 bh9e3f+5qvo@eGroups.com 0 0 4281 1 2003-8-12 0:58:14 GMT 0 Version: 6.0.3790.0 - [SRESafeHaven] Re: More Baby Stuff jtluvs2cq@wmconnect.com -\r
$MailType||='exchange';
#
# Matched sendmail/postfix "to" message
#
- elsif (/: to=.*stat(us)?=sent/i ne undef) {
+ elsif (/: to=.*stat(us)?=sent/i) {
my ($mon,$day,$time,$id,$to)=m/(\w+)\s+(\d+)\s+(\d+:\d+:\d+)\s+[\w\-]+\s+(?:sm-mta|sendmail(?:-out|)|postfix\/(?:local|lmtp|smtpd|smtp|virtual))\[.*?\]:\s+(.*?):\s+to=(.*?),/;
$mailid=$id;
$mail{$id}{'code'}='1';
#
# Matched qmail "to" record
#
- elsif (/starting delivery/ ne undef) {
+ elsif (/starting delivery/) {
# Example: Sep 14 09:58:09 gandalf qmail: 1063526289.574100 starting delivery 251: msg 270182 to local spamreport@john.do
# Example: 2003-09-27 11:22:07.039237500 starting delivery 3714: msg 163844 to local name_also_removed@maildomain.com\r
$MailType||='qmail';
#
# Matched qmail status code record
#
- elsif (/delivery (\d+): (\w+):/ ne undef) {
+ elsif (/delivery (\d+): (\w+):/) {
# Example: Sep 14 09:58:09 gandalf qmail: 1063526289.744259 delivery 251: success: did_0+0+1/
# Example: 2003-09-27 11:22:07.070367500 delivery 3714: success: did_1+0+0/\r
$MailType||='qmail';
foreach my $delivery (keys %{$mail{$mailid}{'to'}}) { $mail{$id}{'code'}{$delivery}||=1; }
debug("For id=$id, found a qmail 'end msg' record. This replace 'delivery' record for delivery=".join(',',keys %{$mail{$id}{'code'}}));
}
-
+ #
+ # Matched MDaemon log file record
+ #
+ elsif (/^\"(\d\d\d\d)-(\d\d)-(\d\d) (\d\d:\d\d:\d\d)\",\"[^\"]*\",(\w+),\d+,\"([^\"]*)\",\"([^\"]*)\",\"[^\"]*\",\"[^\"]*\",\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\",([\.\d]+),(\d+),(\d+)/) {
+ # Example: "2003-11-06 00:00:42","2003-11-06 00:00:45",SMTPI,9443,"dillon_fm@aaaaa.net","cpeltier@domain.com","","","10.0.0.16","","",0,4563,1\r
+ $MailType||='mdaemon';
+ my ($id)=($numrecord);
+ if ($5 eq 'SMTPI' || $5 eq 'SMTPO') {
+ $mail{$id}{'year'}=$1;
+ $mail{$id}{'mon'}=$2;
+ $mail{$id}{'day'}=$3;
+ $mail{$id}{'time'}=$4;
+ $mail{$id}{'direction'}=($5 eq 'SMTPI'?'in':'out');
+ $mail{$id}{'from'}=$6;
+ $mail{$id}{'to'}=$7;
+ if ($5 eq 'SMTPI') {
+ $mail{$id}{'relay_s'}=$8;
+ $mail{$id}{'relay_r'}='localhost';
+ }
+ if ($5 eq 'SMTPO') {
+ $mail{$id}{'relay_s'}='localhost';
+ $mail{$id}{'relay_r'}=$8;
+ }
+ $mail{$id}{'code'}=1;
+ $mail{$id}{'size'}=$12;
+ $mail{$id}{'extinfo'}="?virus=$9&rbl=$10&heuristicspam=$11&ssl=$13";
+ $mail{$id}{'extinfo'}=~s/\s/_/g;
+ $mailid=$id;
+ }
+ }
+
+
#
# Write record if all required data were found
#
my $delivery=0;
my $canoutput=0;
- debug("ID:$mailid RELAY_S:$mail{$mailid}{'relay_s'} RELAY_R:$mail{$mailid}{'relay_r'} FROM:$mail{$mailid}{'from'} TO:$mail{$mailid}{'to'} CODE:$mail{$mailid}{'code'}");
+ debug("ID:$mailid RELAY_S:".($mail{$mailid}{'relay_s'}||'')." RELAY_R:".($mail{$mailid}{'relay_r'}||'')." FROM:".($mail{$mailid}{'from'}||'')." TO:".($mail{$mailid}{'to'}||'')." CODE:".($mail{$mailid}{'code'}||''));
# Check if we can output a mail line
if ($MailType eq 'qmail') {
# If we can
if ($canoutput) {
- &OutputRecord($mail{$mailid}{'year'}?$mail{$mailid}{'year'}:$year,$mail{$mailid}{'mon'},$mail{$mailid}{'day'},$mail{$mailid}{'time'},$mail{$mailid}{'from'},$to,$mail{$mailid}{'relay_s'},$mail{$mailid}{'relay_r'},$code,$mail{$mailid}{'size'},$mail{$mailid}{'forwardto'});
+ &OutputRecord($mail{$mailid}{'year'}?$mail{$mailid}{'year'}:$year,$mail{$mailid}{'mon'},$mail{$mailid}{'day'},$mail{$mailid}{'time'},$mail{$mailid}{'from'},$to,$mail{$mailid}{'relay_s'},$mail{$mailid}{'relay_r'},$code,$mail{$mailid}{'size'},$mail{$mailid}{'forwardto'},$mail{$mailid}{'extinfo'});
# Delete mail with generic unknown id (This id can by used by another mail)
- if ($mailid == 999) {
+ if ($mailid eq '999') {
debug(" Delete mail for id=$mailid",3);
delete $mail{$mailid};
}
projects / thirdparty / AWStats.git / commitdiff
