Fix ::Parser::Tokenizer::prefix() limited token results

When he prefix() method is passed a set limit for characters to scan and
the matched characters do reach that limit the entire Tokenizer buffer
content is consumed and returned.

Correct operation is to only consume and return the matched characters.

diff --git a/src/parser/Tokenizer.cc b/src/parser/Tokenizer.cc
index 3d43c08b80e3429b55ba38fea63fe772a04ebc32..b9b9e26872913cab66d9dba01a8adc68c24489f9 100644 (file)
--- a/src/parser/Tokenizer.cc
+++ b/src/parser/Tokenizer.cc
@@ -76,6 +76,8 @@ Parser::Tokenizer::prefix(SBuf &returnedToken, const CharacterSet &tokenChars, c
         return false;
     if (prefixLen == SBuf::npos && (atEnd() || limit == 0))
         return false;
+    if (prefixLen == SBuf::npos && limit > 0)
+        prefixLen = limit;
     returnedToken = consume(prefixLen); // cannot be empty after the npos check
     return true;
 }