Tobias Oetiker [Tue, 14 Aug 2007 21:59:31 +0000 (21:59 +0000)]
Fix for HoltWinters phase-shift bug described below.
When one or more primary data point times were missed, the SEASONAL and
DEVSEASONAL archives were marked as being up-to-date, so that they would not
be written to. It was correct not to write to these archives, but the code
failed to advance the pointers within the SEASONAL and DEVSEASONAL archives
so that future updates would go to the correct location in the archives.
Rather than mark these archives as up-to-date (by setting
rra_step_cnt[rra_idx] = 0), my patch allocates a new "skip_update" array
that is set to 1 for SEASONAL and DEVSEASONAL archives that have missed one
or more primary data points. When an RRA is written to, the cur_row pointer
advancement happens for all archives, but the skip_update array is checked
just before actually writing out the changes.
Please give it a whirl!
-- Evan Miller emiller imvu.com
* remove some experimental code from rrd_open
* rrd_dontneed will release all data except for header blocks and RRA hot blocks
which are going to be updated withing 10 minutes.
for now, do not release fetched data ... in the future we may call rrd_dontneed and let the users decied with an option
if they want to keep the data in cache ...
* rrd_open: rrd_close does not purge file from cache
* rrd_open: new function rrd_dontneed for purging un-needed pages from core
* rrd_open: in linux at least only fadivse DONTNEED has the power to purge
pages from cache, so letst call madvise as well as fadvise
* rrd_create: uses open/write/close now, flushes file to disk and keeps only hot pages in core
* rrd_update: keeps only hot pages in core
* configure enables FADVISE even when mmap is in use
Tobias Oetiker [Mon, 18 Jun 2007 16:05:07 +0000 (16:05 +0000)]
Bernhard Fischer:
- move several static struct option out of loops and makes them
non-static
- moves some functions from old-style definitions into new-style
definitions
Tobias Oetiker [Fri, 15 Jun 2007 07:59:01 +0000 (07:59 +0000)]
There are two popular variants of the Holt-Winters forecasting method; RRDtool
supports the "additive" method, which means that seasonal variation is simply
added to the baseline. For our application, it would be more appropriate to use
the "multiplicative" Holt-Winters method, where seasonal variation is a
coefficient multiplied by the baseline. Quick example to illustrate the
difference: if the average doubles season-over-season, the additive method
would predict the delta between min and max to be constant, whereas the
multiplicative method would predict the delta to double as well.
Attached is a patch against trunk to support the multiplicative method. I've
done this with a new consolidation function, MHWPREDICT, which is essentially
interchangeable with HWPREDICT. There is a noticeable improvement in prediction
deviations for certain types of functions; the attachments show HWPREDICT and
MHWPREDICT predictions for a function with an x*sin(x) component.
Because HWPREDICT and MHWPREDICT differ only in their equations, I've factored
out their math into rrd_hw_math.c. The appropriate smoothing functions are
passed to the update functions in a container of function pointers, which are
called where appropriate. Thus the additive and multiplicative methods use the
same update functions, and the right equations are evaluated without having
flag checks everywhere. This approach, I think, makes the algorithms quite
clear, with minimal duplicate code.
I have moved update_hwpredict, update_seasonal, update_devpredict,
update_devseasonal, and update_failures into a separate file, rrd_hw_update.c,
with some slight refactoring related to rrd_hw_math.c. I ran some
regression tests against trunk to make sure I didn't break anything with
the existing HWPREDICT code.
MHWPREDICT uses the same deviation smoothing and failure detection algorithms
as HWPREDICT.
Some helpful references on the multiplicative Holt-Winters method:
http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdf
(a student's quick overview of additive vs. multiplicative HW)
http://ideas.repec.org/p/msh/ebswps/1999-1.html (paper on variations to the
multiplicative Holt-Winters, including variance calculations; FYI, my
implementation uses "Model 1")
My employer and the owner of this patch (IMVU, Inc.) is happy to license it
under the same terms as RRDtool, i.e. give it to the project.
-- Evan Miller emiller imvu.com
Tobias Oetiker [Wed, 13 Jun 2007 16:51:06 +0000 (16:51 +0000)]
be more helpful when raising rb_eTypeError in string_arr string_arr_new(VALUE rb_strings). This patch indicates which index is in error as well as the type you are erroneously passing. -- anonymous
Tobias Oetiker [Fri, 1 Jun 2007 09:10:33 +0000 (09:10 +0000)]
Bernhard Fischer
- use 'z' length modifier when printing size_t
- add a cast to __rrd_read and document why it is there
- add RANDOM hint for the header
Tobias Oetiker [Thu, 31 May 2007 18:47:13 +0000 (18:47 +0000)]
Bernhard:
- plug tiny mem-leak in error path of rrd_close where freeing the
rrd_file was forgotten.
- improve two error messages
- use CHECK_MADVISE_OVERLAPS per default
Tobias Oetiker [Wed, 30 May 2007 12:39:30 +0000 (12:39 +0000)]
From Bernhard Fischer
- rrd_close(): call close on the file and use rrd_close consistently
- clean up some error paths. The fadvise error path is leaking memory (see XXX in these spots).
Tobias Oetiker [Tue, 29 May 2007 21:29:17 +0000 (21:29 +0000)]
More updates from Bernhard Fischer
- flag rrd_resize's old file with RRD_COPY
- cleanup error-handling pathes in rrd_update and fix a few typos in
comments
- rrd_close(): implement printing mincore results for the rrd if
DEBUG=2 was defined
- rrd_open(): madvise start addresses need to be page-aligned; implement
alternative path to the fine-grained (i.e. exact) madvise by flagging
just the first two pages as needed (see TWO_PAGES). Implement
alternative path that records the last madvise()ed area to avoid
redundant calls to madvise() on identical areas (due to
page-alignment constraints) -- see CHECK_MADVISE_OVERLAPS. Implement
path for USE_DIRECT_IO.
- configure: add check for O_DIRECT flag to open(2). Add option
--enable-direct-io. Add _GNU_SOURCE to CFLAGS to silence warnings
about chroot which is marked LEGACY since SUSv2 and is a non POSIX
extension. Make checks for posix_fadvise() dependant on
--disable-mmap, since we do not need fadvise for the mmap case.
Tobias Oetiker [Fri, 25 May 2007 13:14:35 +0000 (13:14 +0000)]
updates from Bernhard Fischer rep dot nop gmail com
- add some more rrd_sterror calls to failure paths.
- make rrdtool updatev work like 1.2.23 (i.e. multiple values are still
broken, but at least updating a single value works; see
http://oss.oetiker.ch/rrdtool-trac/ticket/8)
- indent touched files.
projects / thirdparty / rrdtool-1.x.git / log
