From: Willy Tarreau <w@1wt.eu>
Date: Mon, 24 Jul 2023 13:12:31 +0000 (+0200)
Subject: OPTIM: pools: use exponential back-off on shared pool allocation/release
X-Git-Tag: v2.9-dev3~10
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=feeda4132b5677e01e68eec6eab895a4806105a4;p=thirdparty%2Fhaproxy.git

OPTIM: pools: use exponential back-off on shared pool allocation/release

Running a stick-table stress with -dMglobal under 56 threads shows
extreme contention on the pool's free_list because it has to be
processed in two phases and only used to implement a cpu_relax() on
the retry path.

Let's at least implement exponential back-off here to limit the neighbor's
noise and reduce the time needed to successfully acquire the pointer. Just
doing so shows there's still contention but almost doubled the performance,
from 1.1 to 2.1M req/s.
---

diff --git a/src/pool.c b/src/pool.c
index 292b27d133..a2edaab1fc 100644
--- a/src/pool.c
+++ b/src/pool.c
@@ -632,8 +632,7 @@ void pool_refill_local_from_shared(struct pool_head *pool, struct pool_cache_hea
 	ret = _HA_ATOMIC_LOAD(&pool->free_list);
 	do {
 		while (unlikely(ret == POOL_BUSY)) {
-			__ha_cpu_relax();
-			ret = _HA_ATOMIC_LOAD(&pool->free_list);
+			ret = (void*)pl_wait_new_long((ulong*)&pool->free_list, (ulong)ret);
 		}
 		if (ret == NULL)
 			return;
@@ -678,8 +677,7 @@ void pool_put_to_shared_cache(struct pool_head *pool, struct pool_item *item, ui
 	free_list = _HA_ATOMIC_LOAD(&pool->free_list);
 	do {
 		while (unlikely(free_list == POOL_BUSY)) {
-			__ha_cpu_relax();
-			free_list = _HA_ATOMIC_LOAD(&pool->free_list);
+			free_list = (void*)pl_wait_new_long((ulong*)&pool->free_list, (ulong)free_list);
 		}
 		_HA_ATOMIC_STORE(&item->next, free_list);
 		__ha_barrier_atomic_store();