libutil/iconv: avoid overflow in surrogate pairs

Consider the non-conforment utf-8 sequence "\xf5\x80\x80\x80", which
would encode 0x140000. We would set the high byte of the first
surrogate to 0xd8 | (0x130000 >> 18), or 0xdc, which is an invalid
start for a high surrogate, making the sequence as a whole invalid (as
you would expect -- the Unicode range was set precisely to that
covered by utf-16 surrogates).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

diff --git a/lib/util/charset/iconv.c b/lib/util/charset/iconv.c
index 952b9e7911b7c20d16cc97858f5488eac6264c94..131df640986649f71a8a8dae3c6d41265978dc37 100644 (file)
--- a/lib/util/charset/iconv.c
+++ b/lib/util/charset/iconv.c
@@ -923,6 +923,16 @@ static size_t utf8_pull(void *cd, const char **inbuf, size_t *inbytesleft,
                                errno = EILSEQ;
                                goto error;
                        }
+                       if (codepoint > 0x10ffff) {
+                               /*
+                                * Unicode stops at 0x10ffff, and if
+                                * we ignore that, we'll end up
+                                * encoding the wrong characters in
+                                * the surrogate pair.
+                                */
+                               errno = EILSEQ;
+                               goto error;
+                       }
 
                        codepoint -= 0x10000;