Fix signed integer overflow in safe_fprintf()

The following "overflow" check in safe_fprintf() invokes undefined
behavior (signed integer overflow), which is incorrect.  Clang will
optimize it away.

	int old_length = fmtbuff_length;
	fmtbuff_length += fmtbuff_length / 4;
	if (old_length > fmtbuff_length) { ... }

The check is in the form:

	x > x + x / 4.

As per the C standard, the check is equivalent to:

	0 > x / 4,

since x + x / 4 is assumed to not overflow (otherwise undefined).

An earlier check ensures x >= 8192, so this check is always false,
and Clang doesn't emit code for it.

This patch uses unsigned integers to avoid undefined behavior.

diff --git a/tar/util.c b/tar/util.c
index 92b2804ae59de999d2e5e5037e141f9c4baef5cc..4ed2901d5f5a53ba0f6fa58765471c2333d84fc6 100644 (file)
--- a/tar/util.c
+++ b/tar/util.c
@@ -122,7 +122,9 @@ safe_fprintf(FILE *f, const char *fmt, ...)
                        fmtbuff_length *= 2;
                else {
                        int old_length = fmtbuff_length;
-                       fmtbuff_length += fmtbuff_length / 4;
+                       /* Convert to unsigned to avoid signed overflow,
+                        * otherwise the check may be optimized away. */
+                       fmtbuff_length += (unsigned)fmtbuff_length / 4;
                        if (old_length > fmtbuff_length) {
                                length = old_length;
                                fmtbuff_heap[length-1] = '\0';