Skip WAL for unlogged main fork during online checksum enable

ProcessSingleRelationFork() unconditionally generated an FPI WAL
record for every page of every relation when enabling checksums.
Unlogged relations, which by definition never generate WAL for
data changes, were not exempt which generated excessive WAL to
be emitted.

Fix by guarding the FPI WAL record call with RelationNeedsWAL()
to avoid emitting WAL for unlogged main forks.  Unlogged pages
are still dirtied to ensure the checksum is written to disk at
the next checkpoint.  The init fork remains WAL-logged even for
unlogged relations, as it's needed on the standby to materialize
the relation after promotion (see ResetUnloggedRelations()).
Skipping init-fork WAL would leave the standby with a stale init
fork that, once copied to the main fork on promotion, would fail
checksum verification on every read of the unlogged relation.

A test which creates an unlogged table with an index, enables
checksums, promotes the standby, and verifies that the unlogged
relation and its indexes are still readable post-promotion has
been added.

Author: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAHg+QDeGrpZbNZdLjd_T4b43xKEEXZN0HGhkFm-1bkBdyzK7AQ@mail.gmail.com

diff --git a/src/backend/postmaster/datachecksum_state.c b/src/backend/postmaster/datachecksum_state.c
index d0d6acdd6a2d76e7e60085863858a651435d60da..2cfb0450877526f16efd7fbafb6d4d7699134e8f 100644 (file)
--- a/src/backend/postmaster/datachecksum_state.c
+++ b/src/backend/postmaster/datachecksum_state.c
@@ -690,11 +690,22 @@ ProcessSingleRelationFork(Relation reln, ForkNumber forkNum, BufferAccessStrateg
                 * at one point in the past, so only when checksums are first on, then
                 * off, and then turned on again.  TODO: investigate if this could be
                 * avoided if the checksum is calculated to be correct and wal_level
-                * is set to "minimal",
+                * is set to "minimal".
+                *
+                * Unlogged relations don't need WAL since they are reset to their
+                * init fork on recovery.  We still dirty the buffer so that the
+                * checksum is written to disk at the next checkpoint.
+                *
+                * The init fork is an exception: it is WAL-logged so the standby can
+                * materialize the relation after promotion (see
+                * ResetUnloggedRelations()).  Skipping it here would leave the
+                * standby with a stale init fork that, once copied to the main fork
+                * on promotion, would fail checksum verification on every read.
                 */
                START_CRIT_SECTION();
                MarkBufferDirty(buf);
-               log_newpage_buffer(buf, false);
+               if (RelationNeedsWAL(reln) || forkNum == INIT_FORKNUM)
+                       log_newpage_buffer(buf, false);
                END_CRIT_SECTION();
 
                UnlockReleaseBuffer(buf);

diff --git a/src/test/modules/test_checksums/t/003_standby_restarts.pl b/src/test/modules/test_checksums/t/003_standby_restarts.pl
index 11e15c9d73471bf0113f9c63ef797ea2a06ff1f5..5bbf38ed21cbce7a062eb5efa8f6fc057eb3a0d5 100644 (file)
--- a/src/test/modules/test_checksums/t/003_standby_restarts.pl
+++ b/src/test/modules/test_checksums/t/003_standby_restarts.pl
@@ -115,6 +115,115 @@ $result =
   $node_primary->safe_psql('postgres', "SELECT count(a) FROM t WHERE a > 1");
 is($result, "19998", 'ensure we can safely read all data without checksums');
 
-$node_standby->stop;
+# ---------------------------------------------------------------------------
+# Test that enabling checksums does not emit WAL for unlogged relations.
+# Unlogged relations are wiped on recovery, so FPIs for them would be
+# pointless and waste WAL traffic / standby I/O.
+#
+# Additionally, exercise standby promotion to ensure the init fork of an
+# unlogged relation is still WAL-logged during checksum enable -- otherwise
+# the standby keeps a stale init fork and the post-promotion main fork
+# fails verification on every read (see ResetUnloggedRelations()).  Both
+# tables must exist BEFORE enable_data_checksums() so that their init
+# forks get re-checksummed during the enable sweep.
+#
+
+$node_primary->safe_psql('postgres',
+       'CREATE UNLOGGED TABLE unlogged_tbl AS SELECT generate_series(1,1000) AS a;'
+);
+# Use a btree index so the init fork is non-trivial (one metapage).
+$node_primary->safe_psql(
+       'postgres', q[
+       CREATE UNLOGGED TABLE unlogged_promo (id int PRIMARY KEY,
+                                             payload text);
+       INSERT INTO unlogged_promo
+         SELECT g, repeat('x', 100) FROM generate_series(1, 1000) g;
+       CREATE INDEX unlogged_promo_payload_idx ON unlogged_promo (payload);
+]);
+$node_primary->wait_for_catchup($node_standby, 'replay',
+       $node_primary->lsn('insert'));
+
+# Get the relfilenode and database OID so we can inspect the filesystem
+my $unlogged_rfn = $node_primary->safe_psql('postgres',
+       "SELECT relfilenode FROM pg_class WHERE relname = 'unlogged_tbl';");
+my $db_oid = $node_primary->safe_psql('postgres',
+       "SELECT oid FROM pg_database WHERE datname = 'postgres';");
+
+# Verify the standby only has the init fork (no main fork)
+my $standby_datadir = $node_standby->data_dir;
+ok( !-f "$standby_datadir/base/$db_oid/$unlogged_rfn",
+       'standby has no main fork for unlogged table before enable');
+
+# Re-enable data checksums
+enable_data_checksums($node_primary, wait => 'on');
+wait_for_checksum_state($node_standby, 'on');
+
+# After standby replays, the unlogged main file must still not exist.
+# If the bug were present, FPI replay would materialize the full table.
+$node_primary->wait_for_catchup($node_standby, 'replay',
+       $node_primary->lsn('insert'));
+ok( !-f "$standby_datadir/base/$db_oid/$unlogged_rfn",
+       'standby has no main fork for unlogged table after enable');
+
+# Verify unlogged relation size is 0 on the standby (main fork missing)
+my $standby_size = $node_standby->safe_psql('postgres',
+       "SELECT pg_relation_size('unlogged_tbl', 'main');");
+is($standby_size, '0',
+       'unlogged table has zero size on standby after checksum enable');
+
+# Unlogged table should still be readable on primary
+$result =
+  $node_primary->safe_psql('postgres', 'SELECT count(*) FROM unlogged_tbl;');
+is($result, '1000',
+       'unlogged table readable on primary after checksum enable');
+
+# Alter persistence to logged, and make sure we can read it on both the primary
+# and standby without any page verification errors in the logfiles.
+$node_primary->safe_psql('postgres', 'ALTER TABLE unlogged_tbl SET logged;');
+$node_primary->wait_for_catchup($node_standby, 'replay',
+       $node_primary->lsn('insert'));
+
+$result =
+  $node_primary->safe_psql('postgres', 'SELECT sum(a) FROM unlogged_tbl;');
+is($result, '500500', 'previously unlogged table can be read on primary');
+$result =
+  $node_standby->safe_psql('postgres', 'SELECT sum(a) FROM unlogged_tbl;');
+is($result, '500500', 'previously unlogged table can be read on standby');
+
+# ---------------------------------------------------------------------------
+# Promote the standby and verify the unlogged_promo relation (created above
+# before the enable sweep) is still usable.  Without the init-fork WAL fix,
+# every read of the index would fail with
+# "page verification failed, calculated checksum X but expected 0".
+#
 $node_primary->stop;
+$node_standby->promote;
+
+$result =
+  $node_standby->safe_psql('postgres',
+       'SELECT count(*) FROM unlogged_promo;');
+is($result, '0',
+       'unlogged table readable on promoted standby (truncated as expected)');
+
+$node_standby->safe_psql('postgres',
+       "INSERT INTO unlogged_promo SELECT g, repeat('y',100) FROM generate_series(1,100) g;"
+);
+$result = $node_standby->safe_psql('postgres',
+       'SET enable_seqscan = off; SELECT id FROM unlogged_promo WHERE id = 50;');
+is($result, '50', 'indexed lookup on promoted standby returns expected row');
+
+$node_standby->stop;
+
+# Perform one final pass over the logs and hunt for unexpected errors
+my $log = PostgreSQL::Test::Utils::slurp_file($node_primary->logfile, 0);
+unlike(
+       $log,
+       qr/page verification failed,.+\d$/m,
+       "no checksum validation errors in primary log");
+$log = PostgreSQL::Test::Utils::slurp_file($node_standby->logfile, 0);
+unlike(
+       $log,
+       qr/page verification failed,.+\d$/m,
+       "no checksum validation errors in standby log");
+
 done_testing();