From: Thilo Bangert <bangert@users.noreply.github.com>
Date: Tue, 4 Oct 2016 23:33:50 +0000 (+0200)
Subject: Read the last timestamp from the RRD file, instead of setting it to zero. (#744)
X-Git-Tag: v1.7.0~30
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=36fcb9c37a744834438c02611a4d772c7c5625d4;p=thirdparty%2Frrdtool-1.x.git

Read the last timestamp from the RRD file, instead of setting it to zero. (#744)

This fixes a dataloss problem when trying to insert old data. rrdcached
will accept the data at first, but then fail the whole batch when trying
to flush the data to disk.

The last timestamp is initially set to 0, and later updated to the
timestamp of the first update. However, if the RRD file already contains
newer data,  fails later on. And since RRD cache writes all updates
in one go, the whole batch of samples fails and is discarded (even though,
just the first few updates are too old).

By reading in the last timestamp from the RRD file, we already fail
correctly on the individual update commands (ie. on the protocol layer).

RRD  ----------------------|
RRDcached         |------------------| <- Flush
                           |---------| <- Dataloss

With this patch, the initial updates, that overlap with data in the RRD
file will fail indivdually (at submission time).

RRD  ----------------------|
RRDcached         |xxxxxxxx----------| <- Flush
---

diff --git a/src/rrd_daemon.c b/src/rrd_daemon.c
index 0cb86dca..c325d8c7 100644
--- a/src/rrd_daemon.c
+++ b/src/rrd_daemon.c
@@ -1594,6 +1594,37 @@ static int handle_request_update (HANDLER_PROTO) /* {{{ */
       goto done;
     }
 
+    time_t last_update_from_file;
+    rrd_file_t * rrd_file;
+    rrd_t rrd;
+
+    rrd_clear_error();
+    rrd_init(&rrd);
+    rrd_file = rrd_open(file, &rrd, RRD_READONLY);
+    if (!rrd_file)
+    {
+      rrd_free(&rrd);
+      free (ci);
+      RRDD_LOG (LOG_ERR, "handle_request_update: Could not read RRD file.");
+
+      rc = send_response(sock, RESP_ERR, "RRD Error: %s\n", rrd_get_error());
+      goto done;
+    }
+    last_update_from_file = rrd.live_head->last_up;
+    rrd_close(rrd_file);
+    rrd_free(&rrd);
+
+    ci->last_update_stamp = last_update_from_file;
+
+    if(ci->last_update_stamp<1)
+    {
+      free (ci);
+      RRDD_LOG (LOG_ERR, "handle_request_update: Invalid timestamp from RRD file.");
+
+      rc = send_response(sock, RESP_ERR, "Error: rrdcached: Invalid timestamp returned\n");
+      goto done;
+    }
+
     wipe_ci_values(ci, now);
     ci->flags = CI_FLAGS_IN_TREE;
     pthread_cond_init(&ci->flushed, NULL);