From: Bostjan Skufca <bostjan@a2o.si>
Date: Thu, 30 Jul 2015 17:44:06 +0000 (+0000)
Subject: LogFormat: add partial support for literal strings in logfiles. Limited to data prefi... 
X-Git-Tag: AWSTATS_7_5~37^2
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=9c2218aa91b8105e38eff3b6c7014c36da7a63ee;p=thirdparty%2FAWStats.git

LogFormat: add partial support for literal strings in logfiles. Limited to data prefixes like "field=%field"

Reason:
Logging many fields results in very long log lines which in turn causes in very
unreadable log files. Visual cues in form of prefixes which describe data may
be used in log files. This commit adds support for ignoring such prefixes.

Why suffixes are not supported?
Awstats has data fields naming defined as %fieldName, but does not have all fields
joined in one place. This makes it nigh impossible to match, for example, this
specification: "dataOut=1024b" (note the "b" suffix which means data out is
measured in bytes). LogFormat specification for this would be "bytesOut=%bytesdb",
but in this case %bytesdb part would fail to parse out bytes transferred, as
awstats would look for field name called "bytesdb" instead of field name "%bytesd"
with "b" suffix denoting units of measurement.
---

diff --git a/docs/awstats_config.html b/docs/awstats_config.html
index 30185c63..5f3a14e5 100644
--- a/docs/awstats_config.html
+++ b/docs/awstats_config.html
@@ -355,6 +355,9 @@ when reading it), follow the example:<br>
 <br>#   If your log format has some fields not included in this list, use
 <br>#   %other            Means another field
 <br>#   %otherquot        Means another not used double quoted field
+<br>#   If your log format has some literal strings, which precede data fields, use
+<br>#   status=%code      Means your log files have HTTP status logged as "status=200"
+<br>#   Literal strings that follow data field must be separated from said data fields by space.
 <br>#
 <br># Examples for Apache combined logs (following two examples are equivalent):
 <br># LogFormat = 1
diff --git a/wwwroot/cgi-bin/awstats.model.conf b/wwwroot/cgi-bin/awstats.model.conf
index b4f04b62..616980d7 100644
--- a/wwwroot/cgi-bin/awstats.model.conf
+++ b/wwwroot/cgi-bin/awstats.model.conf
@@ -111,6 +111,9 @@ LogType=W
 #   If your log format has some fields not included in this list, use:
 #   %other            Means another not used field
 #   %otherquot        Means another not used double quoted field
+#   If your log format has some literal strings, which precede data fields, use
+#   status=%code      Means your log files have HTTP status logged as "status=200"
+#   Literal strings that follow data field must be separated from said data fields by space.
 #
 # Examples for Apache combined logs (following two examples are equivalent):
 # LogFormat = 1
diff --git a/wwwroot/cgi-bin/awstats.pl b/wwwroot/cgi-bin/awstats.pl
index c9dcee9a..3c043a39 100755
--- a/wwwroot/cgi-bin/awstats.pl
+++ b/wwwroot/cgi-bin/awstats.pl
@@ -9103,6 +9103,11 @@ sub DefinePerlParsingFormat {
 			# Add separator for next field
 			if ($PerlParsingFormat) { $PerlParsingFormat .= "$LogSeparator"; }
 
+			# If field is prefixed with custom string, just push it to regex literally
+			if ( $f =~ /^([^%]+)%/ ) {
+				$PerlParsingFormat .= "$1"
+                        }
+
 			# Special for logname
 			if ( $f =~ /%lognamequot$/ ) {
 				$pos_logname = $i;