Prevent WAL files created by pg_basebackup -x/X from being archived again.

WAL (and timeline history) files created by pg_basebackup did not
maintain the new base backup's archive status. That's currently not a
problem if the new node is used as a standby - but if that node is
promoted all still existing files can get archived again.  With a high
wal_keep_segment settings that can happen a significant time later -
which is quite confusing.

Change both the backend (for the -x/-X fetch case) and pg_basebackup
(for -X stream) itself to always mark WAL/timeline files included in
the base backup as .done. That's in line with walreceiver.c doing so.

The verbosity of the pg_basebackup changes show pretty clearly that it
needs some refactoring, but that'd result in not be backpatchable
changes.

Backpatch to 9.1 where pg_basebackup was introduced.

Discussion: 20141205002854.GE21964@awork2.anarazel.de

diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index ddda0c91416f92d0b9047aa42540cba042c92267..200d4d54f84b57c29222f556fb07481dc4d2bdc3 100644 (file)
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -257,7 +257,9 @@ perform_base_backup(basebackup_options *opt, DIR *tblspcdir)
                while (true)
                {
                        /* Send another xlog segment */
+                       char            sn[MAXPGPATH];
                        char            fn[MAXPGPATH];
+                       char            pathbuf[MAXPGPATH];
                        int                     i;
 
                        XLogFilePath(fn, ThisTimeLineID, logid, logseg);
@@ -290,6 +292,15 @@ perform_base_backup(basebackup_options *opt, DIR *tblspcdir)
                         * boundary, so padding is never necessary.
                         */
 
+                       /*
+                        * Mark file as archived, otherwise files can get archived again
+                        * after promotion of a new node. This is in line with
+                        * walreceiver.c always doing a XLogArchiveForceDone() after a
+                        * complete segment.
+                        */
+                       XLogFileName(sn, ThisTimeLineID, logid, logseg);
+                       StatusFilePath(pathbuf, sn, ".done");
+                       sendFileWithContent(pathbuf, "");
 
                        /* Advance to the next WAL file */
                        NextLogSeg(logid, logseg);
@@ -715,6 +726,15 @@ sendDir(char *path, int basepathlen, bool sizeonly, List *tablespaces)
                                _tarWriteHeader(pathbuf + basepathlen + 1, NULL, &statbuf);
                        }
                        size += 512;            /* Size of the header just added */
+
+                       /*
+                        * Also send archive_status directory (by hackishly reusing
+                        * statbuf from above ...).
+                        */
+                       if (!sizeonly)
+                               _tarWriteHeader("./pg_xlog/archive_status", NULL, &statbuf);
+                       size += 512;            /* Size of the header just added */
+
                        continue;                       /* don't recurse into pg_xlog */
                }