From: Dmitry Marakasov <amdmi3@amdmi3.ru>
Date: Wed, 26 Jun 2019 18:14:32 +0000 (+0300)
Subject: Fix UB when calculating median of all-NaN values
X-Git-Tag: v1.8.0~85
X-Git-Url: http://git.ipfire.org/gitweb/gitweb.cgi?a=commitdiff_plain;h=1d700bf3ba13a75b91047e6e979231abd755892b;p=thirdparty%2Frrdtool-1.x.git

Fix UB when calculating median of all-NaN values

The current code contains undefined behavior where all-NaN values
are passed to median. In that case we end up with final_elements==0
in the following branch:

    else {
        rpnstack->s[++stptr] =
            0.5 * (element_ptr[final_elements / 2] +
                   element_ptr[final_elements / 2 - 1]);
    }

and so we use 0 and -1 as element_ptr array indexes. The
latter is ill-formed and leads to a crash in my case. Move the
check which accounts for the last NaN earlier, so we could
push NaN and finish right away.
---

diff --git a/src/rrd_rpncalc.c b/src/rrd_rpncalc.c
index d3b6b7f1..787d05fe 100644
--- a/src/rrd_rpncalc.c
+++ b/src/rrd_rpncalc.c
@@ -1253,16 +1253,17 @@ short rpn_calc(
                     }
                 }
 
+                /* when goodvals and badvals meet, they might have met on a
+                 * NAN, which wouldn't decrease final_elements. so, check
+                 * that now. */
+                if (isnan(*goodvals))
+                    --final_elements;
+
                 stptr -= elements;
                 if (!final_elements) {
                     /* no non-NAN elements; push NAN */
                     rpnstack->s[++stptr] = DNAN;
                 } else {
-                    /* when goodvals and badvals meet, they might have met on a
-                     * NAN, which wouldn't decrease final_elements. so, check
-                     * that now. */
-                    if (isnan(*goodvals))
-                        --final_elements;
                     /* and finally, take the median of the remaining non-NAN
                      * elements. */
                     qsort(element_ptr, final_elements, sizeof(double),