Limit checkpointer requests queue size

If the number of sync requests is big enough, the palloc() call in
AbsorbSyncRequests() will attempt to allocate more than 1 GB of memory,
resulting in failure.  This can lead to an infinite loop in the checkpointer
process, as it repeatedly fails to absorb the pending requests.

This commit limits the checkpointer requests queue size to 10M items. In
addition to preventing the palloc() failure, this change helps to avoid long
queue processing time.

Also, this commit is for backpathing only.  The master branch receives
a more invasive yet comprehensive fix for this problem.

Discussion: https://postgr.es/m/db4534f83a22a29ab5ee2566ad86ca92%40postgrespro.ru
Backpatch-through: 13

diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 199f008bcda81fefe8a005c9f7d5a3971da2df91..c5e86d3317ede47f09136c9e7973b4d706810e5a 100644 (file)
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -130,6 +130,9 @@ static CheckpointerShmemStruct *CheckpointerShmem;
 /* interval for calling AbsorbSyncRequests in CheckpointWriteDelay */
 #define WRITES_PER_ABSORB              1000
 
+/* Max number of requests the checkpointer request queue can hold */
+#define MAX_CHECKPOINT_REQUESTS 10000000
+
 /*
  * GUC parameters
  */
@@ -914,7 +917,7 @@ CheckpointerShmemInit(void)
                 */
                MemSet(CheckpointerShmem, 0, size);
                SpinLockInit(&CheckpointerShmem->ckpt_lck);
-               CheckpointerShmem->max_requests = NBuffers;
+               CheckpointerShmem->max_requests = Min(NBuffers, MAX_CHECKPOINT_REQUESTS);
                ConditionVariableInit(&CheckpointerShmem->start_cv);
                ConditionVariableInit(&CheckpointerShmem->done_cv);
        }