Fix yet another bug in archive streamer with LZ4 decompression.

The code path in astreamer_lz4_decompressor_content() that updated
the output pointers when the output buffer isn't full was wrong.
It advanced next_out by bytes_written, which could include previous
decompression output not just that of the current cycle.  The
correct amount to advance is out_size.  While at it, make the
output pointer updates look more like the input pointer updates.

This bug is pretty hard to reach, as it requires consecutive
compression frames that are too small to fill the output buffer.
pg_dump could have produced such data before 66ec01dc4, but
I'm unsure whether any files we use astreamer with would be
likely to contain problematic data.

Author: Chao Li <lic@highgo.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/0594CC79-1544-45DD-8AA4-26270DE777A7@gmail.com
Backpatch-through: 15

diff --git a/src/bin/pg_basebackup/bbstreamer_lz4.c b/src/bin/pg_basebackup/bbstreamer_lz4.c
index fb287f5b771148c01ac0a7255eda0eb0c18f1c7b..6f583cb41686f48546c799dfe144334ecf87bdd4 100644 (file)
--- a/src/bin/pg_basebackup/bbstreamer_lz4.c
+++ b/src/bin/pg_basebackup/bbstreamer_lz4.c
@@ -358,11 +358,14 @@ bbstreamer_lz4_decompressor_content(bbstreamer *streamer,
                avail_in -= read_size;
                next_in += read_size;
 
+               /* Update output buffer based on number of bytes produced */
+               avail_out -= out_size;
+               next_out += out_size;
                mystreamer->bytes_written += out_size;
 
                /*
                 * If output buffer is full then forward the content to next streamer
-                * and update the output buffer.
+                * and reset the output buffer.
                 */
                if (mystreamer->bytes_written >= mystreamer->base.bbs_buffer.maxlen)
                {
@@ -372,13 +375,8 @@ bbstreamer_lz4_decompressor_content(bbstreamer *streamer,
                                                           context);
 
                        avail_out = mystreamer->base.bbs_buffer.maxlen;
-                       mystreamer->bytes_written = 0;
                        next_out = (uint8 *) mystreamer->base.bbs_buffer.data;
-               }
-               else
-               {
-                       avail_out = mystreamer->base.bbs_buffer.maxlen - mystreamer->bytes_written;
-                       next_out += mystreamer->bytes_written;
+                       mystreamer->bytes_written = 0;
                }
        }
 }