This caused problems for code that uses this function to read from a
size-limited buffer that can contain NUL bytes; i.e. lib-smtp. In that case,
hitting a NUL byte yields the same result as hitting the end of the buffered
data, which is unacceptable. No other code relies on the function returning 0,
so this can be changed safely to returning -1 instead.
/* the following bytes must all be 10xxxxxx */
for (i = 1; i < len; i++) {
- if ((input[i] & 0xc0) != 0x80)
- return input[i] == '\0' ? 0 : -1;
+ if ((input[i] & 0xc0) != 0x80) {
+ return (max_len == SIZE_MAX && input[i] == '\0' ?
+ 0 : -1);
+ }
chr <<= 6;
chr |= input[i] & 0x3f;