CLEANUP: slz: fix the documented worst case size of flush() and finish()

The documented output buffer requirements of the flush() and finish()
functions date back to the 32-bit queue, where at most 7 bits could be
pending. Since the 64-bit queue was introduced (used on x86_64 and armv8),
up to 31 bits may be pending, and the accounting also forgot the EOB that
may have to be emitted before the empty block. As a result a caller
strictly sizing its output buffer from the documentation could be short
by one to two bytes and see the encoder write past the end of its buffer.

Let's update the document worst cases for these functions depending on
what they still have to emit: 31 pending bits + 7 for EOB + 3 for
BFINAL/BTYPE + 7 for EOB or 32 for LEN+NLEN, rounded up to the next
byte (easily forgotten):

  function           claimed   real
  rfc1951_flush()        9      10
  rfc1951_finish()       4       6
  rfc1950_flush()       11      12
  rfc1950_finish()       8      10
  rfc1952_flush()       19      20
  rfc1952_finish()      12      14

Note that all values are at least as large as the previously claimed ones
and that 32-bit systems never consume more than what was claimed, so the
new documented values are valid both for 32 and 64 bits.

Even though this patch only touches comments, it's marked as a bug so
that it is backported where it matters and users have a chance to spot
the new values.

This should be backported to 1.2.

This is libslz upstream commit 1d774851bc0fe2788ef75d9d74057ecd0c54f868.

Note: this only updates comments, no code was changed. It may be
      backported if it helps with context for other backports.

diff --git a/include/import/slz.h b/include/import/slz.h
index 5ff756c35fc74c8a199766606486f2f9e23ecdd7..10fd34c4216a9aaab73614c2db43864f841a28d7 100644 (file)
--- a/include/import/slz.h
+++ b/include/import/slz.h
@@ -154,12 +154,12 @@ static inline long slz_encode(struct slz_stream *strm, void *out,
 /* Flushes pending bits and sends the trailer for stream <strm> into buffer
  * <buf> if needed. When it's done, the stream state is updated to SLZ_ST_END.
  * It returns the number of bytes emitted. The trailer consists in flushing the
- * possibly pending bits from the queue (up to 24 bits), rounding to the next
- * byte, then 4 bytes for the CRC when doing zlib/gzip, then another 4 bytes
- * for the input length for gzip. That may about to 4+4+4 = 12 bytes, that the
- * caller must ensure are available before calling the function. Note that if
- * the initial header was never sent, it will be sent first as well (up to 10
- * extra bytes).
+ * possibly pending bits from the queue, the possible EOB and the final empty
+ * block, rounding to the next byte, then 4 bytes for the CRC when doing
+ * zlib/gzip, then another 4 bytes for the input length for gzip. That may
+ * amount to 6+4+4 = 14 bytes, that the caller must ensure are available before
+ * calling the function. Note that if the initial header was never sent, it
+ * will be sent first as well (up to 10 extra bytes).
  */
 static inline int slz_finish(struct slz_stream *strm, void *buf)
 {
@@ -179,8 +179,8 @@ static inline int slz_finish(struct slz_stream *strm, void *buf)
  * empty literal block to byte-align the output, allowing to completely flush
  * the queue. Note that if the initial header was never sent, it will be sent
  * first as well (0, 2 or 10 extra bytes). This requires that the output buffer
- * still has this plus the size of the queue available (up to 4 bytes), plus
- * one byte for (BFINAL,BTYPE), plus 4 bytes for LEN+NLEN, or a total of 19
+ * still has this plus the 6 bytes needed to flush the queue, the possible EOB
+ * and the (BFINAL,BTYPE) bits, plus 4 bytes for LEN+NLEN, or a total of 20
  * bytes in the worst case. The number of bytes emitted is returned. It is
  * guaranteed that the queue is empty on return. This may cause some overhead
  * by adding needless 5-byte blocks if called to often.

diff --git a/src/slz.c b/src/slz.c
index 08eb80de92fff7c8926b7701d7249d343d508269..e21a609b45fa9613e758efee5476e2c2ddba537e 100644 (file)
--- a/src/slz.c
+++ b/src/slz.c
@@ -783,11 +783,12 @@ int slz_rfc1951_init(struct slz_stream *strm, int level)
 /* Flushes any pending data for stream <strm> into buffer <buf>, then emits an
  * empty literal block to byte-align the output, allowing to completely flush
  * the queue. This requires that the output buffer still has the size of the
- * queue available (up to 4 bytes), plus one byte for (BFINAL,BTYPE), plus 4
- * bytes for LEN+NLEN, or a total of 9 bytes in the worst case. The number of
- * bytes emitted is returned. It is guaranteed that the queue is empty on
- * return. This may cause some overhead by adding needless 5-byte blocks if
- * called to often.
+ * queue (up to 31 bits when using the 64-bit queue), plus a possible EOB (7
+ * bits), plus (BFINAL,BTYPE) (3 bits), which once rounded up to the next byte
+ * make 6 bytes, plus 4 bytes for LEN+NLEN, or a total of 10 bytes in the worst
+ * case. The number of bytes emitted is returned. It is guaranteed that the
+ * queue is empty on return. This may cause some overhead by adding needless
+ * 5-byte blocks if called to often.
  */
 int slz_rfc1951_flush(struct slz_stream *strm, unsigned char *buf)
 {
@@ -819,9 +820,10 @@ int slz_rfc1951_flush(struct slz_stream *strm, unsigned char *buf)
 /* Flushes any pending for stream <strm> into buffer <buf>, then sends BTYPE=1
  * and BFINAL=1 if needed. The stream ends in SLZ_ST_DONE. It returns the number
  * of bytes emitted. The trailer consists in flushing the possibly pending bits
- * from the queue (up to 7 bits), then possibly EOB (7 bits), then 3 bits, EOB,
- * a rounding to the next byte, which amounts to a total of 4 bytes max, that
- * the caller must ensure are available before calling the function.
+ * from the queue (up to 31 bits when using the 64-bit queue), then possibly
+ * EOB (7 bits), then 3 bits, EOB, a rounding to the next byte, which amounts
+ * to a total of 6 bytes max, that the caller must ensure are available before
+ * calling the function.
  */
 int slz_rfc1951_finish(struct slz_stream *strm, unsigned char *buf)
 {
@@ -1155,10 +1157,10 @@ int slz_rfc1952_init(struct slz_stream *strm, int level)
  * empty literal block to byte-align the output, allowing to completely flush
  * the queue. Note that if the initial header was never sent, it will be sent
  * first as well (10 extra bytes). This requires that the output buffer still
- * has this plus the size of the queue available (up to 4 bytes), plus one byte
- * for (BFINAL,BTYPE), plus 4 bytes for LEN+NLEN, or a total of 19 bytes in the
- * worst case. The number of bytes emitted is returned. It is guaranteed that
- * the queue is empty on return. This may cause some overhead by adding
+ * has this plus the 6 bytes needed to flush the queue, the possible EOB and
+ * the (BFINAL,BTYPE) bits, plus 4 bytes for LEN+NLEN, or a total of 20 bytes
+ * in the worst case. The number of bytes emitted is returned. It is guaranteed
+ * that the queue is empty on return. This may cause some overhead by adding
  * needless 5-byte blocks if called to often.
  */
 int slz_rfc1952_flush(struct slz_stream *strm, unsigned char *buf)
@@ -1175,11 +1177,12 @@ int slz_rfc1952_flush(struct slz_stream *strm, unsigned char *buf)
 /* Flushes pending bits and sends the gzip trailer for stream <strm> into
  * buffer <buf>. When it's done, the stream state is updated to SLZ_ST_END. It
  * returns the number of bytes emitted. The trailer consists in flushing the
- * possibly pending bits from the queue (up to 24 bits), rounding to the next
- * byte, then 4 bytes for the CRC and another 4 bytes for the input length.
- * That may about to 4+4+4 = 12 bytes, that the caller must ensure are
- * available before calling the function. Note that if the initial header was
- * never sent, it will be sent first as well (10 extra bytes).
+ * possibly pending bits from the queue, the possible EOB and the final empty
+ * block, rounding to the next byte, then 4 bytes for the CRC and another 4
+ * bytes for the input length. That may amount to 6+4+4 = 14 bytes, that the
+ * caller must ensure are available before calling the function. Note that if
+ * the initial header was never sent, it will be sent first as well (10 extra
+ * bytes).
  */
 int slz_rfc1952_finish(struct slz_stream *strm, unsigned char *buf)
 {
@@ -1427,15 +1430,14 @@ int slz_rfc1950_init(struct slz_stream *strm, int level)
        return 0;
 }
 
-/* Flushes any pending data for stream <strm> into buffer <buf>, then emits an
- * empty literal block to byte-align the output, allowing to completely flush
- * the queue. Note that if the initial header was never sent, it will be sent
- * first as well (2 extra bytes). This requires that the output buffer still
- * has this plus the size of the queue available (up to 4 bytes), plus one byte
- * for (BFINAL,BTYPE), plus 4 bytes for LEN+NLEN, or a total of 11 bytes in the
- * worst case. The number of bytes emitted is returned. It is guaranteed that
- * the queue is empty on return. This may cause some overhead by adding
- * needless 5-byte blocks if called to often.
+/* Flushes pending bits and sends the gzip trailer for stream <strm> into
+ * buffer <buf>. When it's done, the stream state is updated to SLZ_ST_END. It
+ * returns the number of bytes emitted. The trailer consists in flushing the
+ * possibly pending bits from the queue, the possible EOB and the final empty
+ * block, rounding to the next byte, then 4 bytes for the CRC. That may amount
+ * to 6+4 = 10 bytes, that the caller must ensure are available before calling
+ * the function. Note that if the initial header was never sent, it will be
+ * sent first as well (2 extra bytes).
  */
 int slz_rfc1950_flush(struct slz_stream *strm, unsigned char *buf)
 {
@@ -1451,11 +1453,11 @@ int slz_rfc1950_flush(struct slz_stream *strm, unsigned char *buf)
 /* Flushes pending bits and sends the gzip trailer for stream <strm> into
  * buffer <buf>. When it's done, the stream state is updated to SLZ_ST_END. It
  * returns the number of bytes emitted. The trailer consists in flushing the
- * possibly pending bits from the queue (up to 24 bits), rounding to the next
- * byte, then 4 bytes for the CRC. That may about to 4+4 = 8 bytes, that the
- * caller must ensure are available before calling the function. Note that if
- * the initial header was never sent, it will be sent first as well (2 extra
- * bytes).
+ * possibly pending bits from the queue, the possible EOB and the final empty
+ * block, rounding to the next byte, then 4 bytes for the CRC. That may amount
+ * to 6+4 = 10 bytes, that the caller must ensure are available before calling
+ * the function. Note that if the initial header was never sent, it will be
+ * sent first as well (2 extra bytes).
  */
 int slz_rfc1950_finish(struct slz_stream *strm, unsigned char *buf)
 {