What do I have to do to use AWStats to analyze some FTP server log files.
SOLUTION:
AWStats was built to analyze web,wap or proxy server's log files. However it can be used with some FTP server log files.
-Setup your ProFTP logformat:
-Modify the proftpd.conf file to add the following two lines : +
+1- Setup your FTP log file format:
+
+With ProFTP, modify the proftpd.conf file to add the following two lines : +
|
- LogFormat awstats "%t %h %u %m %f %s %b" # WARNING: You must use a tab char between % tags and not a space ! +LogFormat awstats "%t %h %u %m %f %s %b" # WARNING: You must use a tab char between % tags and not a space ! ExtendedLog /var/log/xferlog read,write awstats # WARNING: ExtendedLog directive might need to be placed inside a virtual host context if you use them. - Then turn off old format Transfer log: - - TransferLog none # WARNING: TransferLog directive might need to be placed inside a virtual host context if you use them. - + |
| +TransferLog none # WARNING: TransferLog directive might need to be placed inside a virtual host context if you use them. + |
To have the change effective, stop your server, remove old log file /var/log/xferlog and restart the server.
-Setup AWStats to analyze those FTP log files:
+Download a file by FTP and check that your new log file looks like this:
+[01/Jan/2001:21:49:57 +0200] ftp.server.com user RETR /home/fileiget.txt 226 1499
+
+2- Then setup AWStats to analyze the FTP log file:
+
Copy config awstats.model.conf file to "awstats.proftp.conf".
Modify this new config file: +
|
- LogFile="/var/log/xferlog" +LogFile="/var/log/xferlog" LogFormat="%time1 %host %logname %method %url %code %bytesd" LogSeparator="\t" DNSLookup=0 SkipFiles="" NotPageList="" ShowLinksOnUrl=0 - ShowHeader=1 ShowMenu=1 ShowMonthDayStats=UVHB ShowDaysOfWeekStats=HB @@ -394,8 +402,8 @@ Modify this new config file: ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowHTTPErrorsStats=0 - - + |
Now you can use AWStats as usual (run the update process and read statistics).
@@ -408,14 +416,15 @@ What do I have to do to use AWStats to analyze my mail log files.
For Sendmail or PostFix log files
1- First, install a mail log convertor like SMA to use it -as a post-processor for AWStats update process.
-Setup SMA to convert your mail log into a format readable by AWStats:
-Copy sma (or sma.exe) file and sma.conf into same directory than awstats.pl file.
+as a pre-processor for AWStats update process (to convert your mail log into a readable format):
+For this, you can download an copy sma (or sma.exe) file and sma.conf into same directory +than awstats.pl file.
Edit sma.conf to change the ClogFormat and ClogSentOnly parameter to : - -
ClogFormat "%y-%m-%d %h:%n:%s %f %t %F %T SMTP - %S %z" +
|
+ClogFormat "%y-%m-%d %h:%n:%s %f %t %F %T SMTP - %S %z"
ClogSentOnly no + |
@@ -424,18 +433,17 @@ To check that your sma config is correct, run the following command:
The resulting convertedmaillog file must have records that match the following format:
2001-01-01 05:38:08 sender@mailsender.com receiver@mailreceiver.com hostrelaysender hostrelayreceiver SMTP - 1 31357
Warning: some sma versions (1.3) have been reported to not convert month correctly, giving -for example a date like this 2001-#01-01 instead of 2001-01-01. This makes AWStats analyze -fails so check your sma output and change version if error.
+for example a date like this 2001-#01-01 or 2001-00-01 instead of 2001-01-01. This makes AWStats +analyze fails so check your sma output and change version if error.
2- Then setup AWStats to analyze SMA processed mail log:
Copy config awstats.model.conf file to "awstats.mail.conf".
Modify this new config file: +
|
- LogFile="/pathtosma/sma -O clog -f /pathtosmaconf/sma.conf /pathtomaillog/maillog |" +LogFile="/pathtosma/sma -O clog -f /pathtosmaconf/sma.conf /pathtomaillog/maillog |" LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" - DNSLookup=0 ValidSMTPCodes="1" - ShowHeader=1 ShowMenu=1 ShowMonthDayStats=HB ShowDaysOfWeekStats=HB @@ -457,7 +465,8 @@ Modify this new config file: ShowKeywordsStats=0 ShowHTTPErrorsStats=0 - + |
Now you can use AWStats as usual (run the update process and read statistics).
For Exchange log files
@@ -473,17 +482,57 @@ Modify this new config file:
LogFile="yourmaillogfilename"
LogFormat="%time2 %other %host %other %other %other %email %method %other %url %code %bytesd %other %other %other %other %other %other %email_r"
LogSeparator="\t" -
DNSLookup=0 -->
-FAQ-SET272 : SETUP FOR WINDOWS MEDIA SERVER
+FAQ-SET272 : SETUP FOR A MEDIA SERVER (REALMEDIA, WINDOWS MEDIA SERVER)
PROBLEM:
-What do I have to do to use AWStats to analyze my Windows Media Server log files.
+What do I have to do to use AWStats to analyze my Media Server log files.
SOLUTION:
-If your Windows Media Server version allows it, setup your log format to write the following fields: +For Realmedia
+
+Your log file will probably looks like this:
+ +216.125.146.50 - - [16/Sep/2002:14:57:21 -0500] "GET cme/rhythmcity/rcitycaddy.rm?cloakport=8080,554,7070 RTSP/1.0" 200 6672 [Win95_4.0_6.0.9.374_play32_NS80_en-US_586] [80d280e1-c9ae-11d6-fa53-d52aaed98681] [UNKNOWN] 281712 141 3 0 0 494
+ +
+Copy config awstats.model.conf file to "awstats.mediaserver.conf". +Modify this new config file: +
|
+LogFile="/pathtomediaserver/mediaserverlog"
+ LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %uabracket %other %other %other %other %other %other %other %other" + LogSeparator="\s+" + ShowMenu=1 + ShowMonthDayStats=UHB + ShowDaysOfWeekStats=HB + ShowHoursStats=HB + ShowDomainsStats=HB + ShowHostsStats=HBL + ShowAuthenticatedUsers=0 + ShowRobotsStats=0 + ShowEMailSenders=0 + ShowEMailReceivers=0 + ShowSessionsStats=0 + ShowPagesStats=PB + ShowFileTypesStats=HB + ShowFileSizesStats=0 + ShowBrowsersStats=1 + ShowOSStats=1 + ShowOriginStats=PH + ShowKeyphrasesStats=0 + ShowKeywordsStats=0 + ShowHTTPErrorsStats=1 + + |
+Now you can use AWStats as usual (run the update process and read statistics).
+
+
+For Windows Media Server
+
+1- If your Windows Media Server version allows it, setup your log format to write the following fields:
c-ip
date @@ -519,15 +568,32 @@ Listen to streaming files and check that your new log file looks like this:
80.223.91.37 2002-10-08 14:18:58 mmst://mydomain.com/mystream 0 106 1 200 {F4A826EE-FA46-480F-A49B-76786320FC6B} 8.0.0.4477 fi-FI - - wmplayer.exe 8.0.0.4477 Windows_2000 5.1.0.2600 Pentium 0 0 20702 mms TCP Windows_Media_Audio_9 - - 277721 -
-Then setup AWStats to analyze your Media Server log:
+
+
+If your Windows Media Server version does not allow to define your log format:
+Just follow instructions in step 2 directly but use the log format string found in first +lines of your log files as value for AWStats LogFormat parameter, for example:
+ +LogFormat="c-ip date time c-dns cs-uri-stem c-starttime x-duration c-rate +c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) +cs(Referer) c-hostexe c-hostexever c-os c-osversion c-cpu filelength +filesize avgbandwidth protocol transport audiocodec videocodec channelURL +sc-bytes c-bytes s-pkts-sent c-pkts-received c-pkts-lost-client +c-pkts-lost-net c-pkts-lost-cont-net c-resendreqs c-pkts-recovered-ECC +c-pkts-recovered-resent c-buffercount c-totalbuffertime c-quality s-ip s-dns +s-totalclients s-cpu-util" + +
This means you don't use the AWStats tags but AWStats can often also understand all the IIS and/or +Windows Media Server tags.
+ +
+2- Then setup AWStats to analyze your Media Server log:
Copy config awstats.model.conf file to "awstats.mediaserver.conf".
Modify this new config file: +
|
- LogFile="/pathtomediaserver/mediaserverlog" +LogFile="/pathtomediaserver/mediaserverlog" LogFormat="c-ip date time cs-uri-stem c-starttime x-duration c-rate c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) cs(Referer) c-hostexe c-hostexever c-os c-osversion c-cpu filelength filesize avgbandwidth protocol transport audiocodec videocodec channelURL sc-bytes" - DNSLookup=0 - ShowHeader=1 ShowMenu=1 ShowMonthDayStats=UHB ShowDaysOfWeekStats=HB @@ -549,24 +615,9 @@ Modify this new config file: ShowKeywordsStats=0 ShowHTTPErrorsStats=1 - -Now you can use AWStats as usual (run the update process and read statistics). + |
-If your Windows Media Server version does not allow to define your log format, just try to use -same setup for AWStats than previously described but use the log format string found in first -lines of your log files as value for AWStats LogFormat parameter, for example:
- -LogFormat="c-ip date time c-dns cs-uri-stem c-starttime x-duration c-rate -c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) -cs(Referer) c-hostexe c-hostexever c-os c-osversion c-cpu filelength -filesize avgbandwidth protocol transport audiocodec videocodec channelURL -sc-bytes c-bytes s-pkts-sent c-pkts-received c-pkts-lost-client -c-pkts-lost-net c-pkts-lost-cont-net c-resendreqs c-pkts-recovered-ECC -c-pkts-recovered-resent c-buffercount c-totalbuffertime c-quality s-ip s-dns -s-totalclients s-cpu-util" - -
This means you don't use the AWStats tags but AWStats can often also understand the IIS and/or -Windows Media Server tags.
+Now you can use AWStats as usual (run the update process and read statistics).
diff --git a/wwwroot/cgi-bin/awstats.pl b/wwwroot/cgi-bin/awstats.pl index c227f622..a943d113 100644 --- a/wwwroot/cgi-bin/awstats.pl +++ b/wwwroot/cgi-bin/awstats.pl @@ -4397,7 +4397,7 @@ sub DefinePerlParsingFormat() { $PerlParsingFormat .= "([\\d|-]+)"; } elsif ($f =~ /%refererquot$/) { - $pos_referer = $i; $i++; push @fieldlib, 'host'; + $pos_referer = $i; $i++; push @fieldlib, 'referer'; $PerlParsingFormat .= "\\\"(.*)\\\""; # referer might be "" } elsif ($f =~ /%referer$/) { @@ -4408,6 +4408,10 @@ sub DefinePerlParsingFormat() { $pos_agent = $i; $i++; push @fieldlib, 'ua'; $PerlParsingFormat .= "\\\"([^\\\"]*)\\\""; # ua might be "" } + elsif ($f =~ /%uabracket$/) { + $pos_agent = $i; $i++; push @fieldlib, 'ua'; + $PerlParsingFormat .= "\\\[(.*)\\\]"; # ua might be [] + } elsif ($f =~ /%ua$/) { $pos_agent = $i; $i++; push @fieldlib, 'ua'; $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)"; @@ -7220,9 +7224,9 @@ if (scalar keys %HTMLOutput) { print "
| \n"; $max_v=$max_p=$max_h=$max_k=1; # Define total and max for (my $ix=1; $ix<=12; $ix++) { @@ -7268,6 +7271,8 @@ if (scalar keys %HTMLOutput) { # Define average # TODO # Show bars for month + print " | ||||||
| \n"; for (my $ix=1; $ix<=12; $ix++) { my $monthix=sprintf("%02s",$ix); my $bredde_u=0; my $bredde_v=0;my $bredde_p=0;my $bredde_h=0;my $bredde_k=0; @@ -7285,13 +7290,31 @@ if (scalar keys %HTMLOutput) { if ($ShowMonthDayStats =~ /B/i) { print " | "; print " | |||||
| "; + print " | ||||||
| << | "; +# } +# else { + print ""; +# } for (my $ix=1; $ix<=12; $ix++) { my $monthix=sprintf("%02s",$ix); - print " | $MonthLib{$monthix} | "; - } +# if ($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks) { +# print "$MonthLib{$monthix} $YearRequired | ";
+# }
+# else {
+ print "$MonthLib{$monthix} $YearRequired | ";
+# }
+ }
+# if ($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks) {
+# print ">> | "; +# } +# else { + print ""; +# } print " |
\n"; @@ -7308,7 +7331,7 @@ if (scalar keys %HTMLOutput) { for (my $ix=1; $ix<=12; $ix++) { my $monthix=sprintf("%02s",$ix); print "