Limit checkpointer requests queue size

If the number of sync requests is big enough, the palloc() call in
AbsorbSyncRequests() will attempt to allocate more than 1 GB of memory,
resulting in failure.  This can lead to an infinite loop in the checkpointer
process, as it repeatedly fails to absorb the pending requests.

This commit limits the checkpointer requests queue size to 10M items. In
addition to preventing the palloc() failure, this change helps to avoid long
queue processing time.

Also, this commit is for backpathing only.  The master branch receives
a more invasive yet comprehensive fix for this problem.

Discussion: https://postgr.es/m/db4534f83a22a29ab5ee2566ad86ca92%40postgrespro.ru
Backpatch-through: 13

diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index fda91ffd1ce2dedb6942af3c74cb8a893c1e4545..903d83e7dea0fdd185b8a3a9f5b374538dfe7d62 100644 (file)
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -135,6 +135,9 @@ static CheckpointerShmemStruct *CheckpointerShmem;
 /* interval for calling AbsorbSyncRequests in CheckpointWriteDelay */
 #define WRITES_PER_ABSORB              1000
 
+/* Max number of requests the checkpointer request queue can hold */
+#define MAX_CHECKPOINT_REQUESTS 10000000
+
 /*
  * GUC parameters
  */
@@ -970,7 +973,7 @@ CheckpointerShmemInit(void)
                 */
                MemSet(CheckpointerShmem, 0, size);
                SpinLockInit(&CheckpointerShmem->ckpt_lck);
-               CheckpointerShmem->max_requests = NBuffers;
+               CheckpointerShmem->max_requests = Min(NBuffers, MAX_CHECKPOINT_REQUESTS);
                ConditionVariableInit(&CheckpointerShmem->start_cv);
                ConditionVariableInit(&CheckpointerShmem->done_cv);
        }