Fix poorly-sized buffers in astreamer compression modules.

astreamer_gzip.c and astreamer_lz4.c left their decompression
output buffers at StringInfo's default allocation, merely 1kB.
This results in a lot of ping-ponging between the decompressor
and the next astreamer filter.  This patch increases these buffer
sizes to 256kB.  In a simple test this had a small but measurable
effect (saving a few percent) on the overall runtime of pg_waldump
for the gzipped-data case; I didn't bother measuring for lz4.

astreamer_zstd.c used ZSTD_DStreamOutSize() to size its
compression output buffer, but the libzstd API says you should use
ZSTD_CStreamOutSize(); ZSTD_DStreamOutSize() is for decompression.
The two functions seem to produce the same value (256kB) here, so
this is just cosmetic, but nonetheless we should play by the rules.

While these issues are old, they don't seem significant enough to
warrant back-patching.

Discussion: https://postgr.es/m/3424809.1774234940@sss.pgh.pa.us

diff --git a/src/fe_utils/astreamer_gzip.c b/src/fe_utils/astreamer_gzip.c
index df392f67cab4706708e9fcc5cca3ad4a5f8f9c9f..5b3c3a175500e5a8493de85f9b4f70c21e664308 100644 (file)
--- a/src/fe_utils/astreamer_gzip.c
+++ b/src/fe_utils/astreamer_gzip.c
@@ -247,6 +247,8 @@ astreamer_gzip_decompressor_new(astreamer *next)
 
        streamer->base.bbs_next = next;
        initStringInfo(&streamer->base.bbs_buffer);
+       /* Use a buffer size comparable to the other decompressors */
+       enlargeStringInfo(&streamer->base.bbs_buffer, 256 * 1024 - 1);
 
        /* Initialize internal stream state for decompression */
        zs = &streamer->zstream;

diff --git a/src/fe_utils/astreamer_lz4.c b/src/fe_utils/astreamer_lz4.c
index 605c188007b05a8202544176d6c809733b040fb5..12dfde2c83702e91b39f2ff42be422c682d88fd0 100644 (file)
--- a/src/fe_utils/astreamer_lz4.c
+++ b/src/fe_utils/astreamer_lz4.c
@@ -288,6 +288,8 @@ astreamer_lz4_decompressor_new(astreamer *next)
 
        streamer->base.bbs_next = next;
        initStringInfo(&streamer->base.bbs_buffer);
+       /* Use a buffer size comparable to the compressor's */
+       enlargeStringInfo(&streamer->base.bbs_buffer, 256 * 1024 - 1);
 
        /* Initialize internal stream state for decompression */
        ctxError = LZ4F_createDecompressionContext(&streamer->dctx, LZ4F_VERSION);

diff --git a/src/fe_utils/astreamer_zstd.c b/src/fe_utils/astreamer_zstd.c
index 4b43ab795e31afa8e71a763776cb619bf935654e..98e8a700efef3ccbe11d35c36fe8f84cd50464f5 100644 (file)
--- a/src/fe_utils/astreamer_zstd.c
+++ b/src/fe_utils/astreamer_zstd.c
@@ -82,7 +82,7 @@ astreamer_zstd_compressor_new(astreamer *next, pg_compress_specification *compre
 
        streamer->base.bbs_next = next;
        initStringInfo(&streamer->base.bbs_buffer);
-       enlargeStringInfo(&streamer->base.bbs_buffer, ZSTD_DStreamOutSize());
+       enlargeStringInfo(&streamer->base.bbs_buffer, ZSTD_CStreamOutSize());
 
        streamer->cctx = ZSTD_createCCtx();
        if (!streamer->cctx)