OPTIM: halog: improve cold-cache behaviour when loading a file

Using posix_fadvise() it is possible to tell the system that we're
going to read a whole file at once. The kernel then doubles the
read-ahead size for this file. On Linux with an SSD, this has improved
cold-cache performance by around 20%. Hot-cache is not affected at all.

diff --git a/contrib/halog/halog.c b/contrib/halog/halog.c
index 250623d8275989fefd555df1e81259f8c8852931..448e5c0a8300ca80436c302368bb3400130c9446 100644 (file)
--- a/contrib/halog/halog.c
+++ b/contrib/halog/halog.c
@@ -672,6 +672,15 @@ int main(int argc, char **argv)
        else if (filter & FILT_COUNT_ONLY)
                line_filter = NULL;
 
+#if defined(POSIX_FADV_SEQUENTIAL)
+       /* around 20% performance improvement is observed on Linux with this
+        * on cold-cache. Surprizingly, WILLNEED is less performant. Don't
+        * use NOREUSE as it flushes the cache and prevents easy data
+        * manipulation on logs!
+        */
+       posix_fadvise(0, 0, 0, POSIX_FADV_SEQUENTIAL);
+#endif
+
        if (!line_filter &&
            !(filter & (FILT_HTTP_ONLY|FILT_TIME_RESP|FILT_ERRORS_ONLY|FILT_HTTP_STATUS|FILT_QUEUE_ONLY|FILT_QUEUE_SRV_ONLY|FILT_TERM_CODE_NAME))) {
                /* read the whole file at once first */