util/base64: don't reset decoded bytes in RFC4648

Old behavior:
With RFC4648, the decoded bytes were reset to 0 in case an unusual
character was encountered in the encoded string. This worked out fine
for small test cases where there weren't many bytes to be decoded.

Problem:
If a big encoded string had a character outside of the base alphabet,
the processing would stop and the number of decoded bytes were set to 0.
However, even though the processing should stop at the invalid
character, the number of decoded bytes should correctly store the bytes
decoded up until the point an invalid characted was encountered.

New behavor:
For any base64 encoded string given to the base64 decoder in RFC4648
mode, we make sure that the number of decoded bytes correctly reflect
the number of bytes processed up until the string was valid. This makes
sure any further calculations/use of the decoded data is done correctly.

Redmine ticket: 5885

(cherry picked from commit 418ddba38e008f0a57c07e7a872d0771d36a9bbd)

diff --git a/src/util-base64.c b/src/util-base64.c
index 8cc81130399b4ecfb6a1e703081d638d1694ee3a..e84224e5b01de2ad1daa901202fdc1b36d91b07c 100644 (file)
--- a/src/util-base64.c
+++ b/src/util-base64.c
@@ -117,10 +117,10 @@ Base64Ecode DecodeBase64(uint8_t *dest, uint32_t dest_size, const uint8_t *src,
             /* Invalid character found, so decoding fails */
             if (src[i] != '=') {
                 valid = false;
-                if (mode != BASE64_MODE_RELAX) {
+                ecode = BASE64_ECODE_ERR;
+                if (mode == BASE64_MODE_STRICT) {
                     *decoded_bytes = 0;
                 }
-                ecode = BASE64_ECODE_ERR;
                 break;
             }
             padding++;