From: Steffan Karger Date: Mon, 28 Nov 2016 14:26:40 +0000 (+0100) Subject: Clean up format_hex_ex() X-Git-Tag: v2.4_rc1~12 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=294040102fbceb4be320a45ba3c340ace1804a49;p=thirdparty%2Fopenvpn.git Clean up format_hex_ex() Fix a potential null-pointer dereference, and make the code a bit more readable while doing so. The NULL dereference could not be triggered, because the current code never called format_hex_ex() with maxouput == 0 and separator == NULL. But it's nicer to not depend on that. Our use of int vs size_t for lengths needs some attention too, but I'm not pulling that into this patch. Instead I decided to just make the (previously existing) assumption that INT_MAX <= SIZE_MAX explicit by adding a static_assert(). Signed-off-by: Steffan Karger Acked-by: Gert Doering Message-Id: <1480343200-25908-1-git-send-email-steffan.karger@fox-it.com> URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg13259.html Signed-off-by: Gert Doering --- diff --git a/src/openvpn/buffer.c b/src/openvpn/buffer.c index 52c6ab92f..5341d35e2 100644 --- a/src/openvpn/buffer.c +++ b/src/openvpn/buffer.c @@ -438,13 +438,16 @@ format_hex_ex (const uint8_t *data, int size, int maxoutput, unsigned int space_break_flags, const char* separator, struct gc_arena *gc) { - struct buffer out = alloc_buf_gc (maxoutput ? maxoutput : - ((size * 2) + (size / (space_break_flags & FHE_SPACE_BREAK_MASK)) * (int) strlen (separator) + 2), - gc); - int i; - for (i = 0; i < size; ++i) + const size_t bytes_per_hexblock = space_break_flags & FHE_SPACE_BREAK_MASK; + const size_t separator_len = separator ? strlen (separator) : 0; + static_assert (INT_MAX <= SIZE_MAX, "Code assumes INT_MAX <= SIZE_MAX"); + const size_t out_len = maxoutput > 0 ? maxoutput : + ((size * 2) + ((size / bytes_per_hexblock) * separator_len) + 2); + + struct buffer out = alloc_buf_gc (out_len, gc); + for (int i = 0; i < size; ++i) { - if (separator && i && !(i % (space_break_flags & FHE_SPACE_BREAK_MASK))) + if (separator && i && !(i % bytes_per_hexblock)) buf_printf (&out, "%s", separator); if (space_break_flags & FHE_CAPS) buf_printf (&out, "%02X", data[i]);