BUG/MINOR: pool: check one other random bucket on alloc conflict

Since 2.9-dev3 with commit 7bf829ace ("MAJOR: pools: move the shared
pool's free_list over multiple buckets"), the global pool supports
multiple heads to reduce inter-thread contention. However, when
grabbing a freelist head fails because another thread is already
picking from it, we just skip to the next one and try again.

Unfortunately, it still maintains a bit of contention between thread
pairs when for some reasons only a few threads are used. This may
happen for example when running on a 4- or 8- thread system and
the two most active ones end up on adjacent buckets.

A better and much simpler solution consists in visiting a random bucket
instead of the current one. Tests show that the CPU usage spent in
pool_refill_local_from_shared() reduces at low number of connections
(hence threads).

No backport is needed, as the issue is only in 2.9.

diff --git a/src/pool.c b/src/pool.c
index 02b5c6d250ee9d4f9fb854e0109f765be6362c66..75ffb1ec7234580bd4893dc2e79833e2c2088d5d 100644 (file)
--- a/src/pool.c
+++ b/src/pool.c
@@ -675,7 +675,7 @@ void pool_refill_local_from_shared(struct pool_head *pool, struct pool_cache_hea
        do {
                /* look for an apparently non-busy entry */
                while (unlikely(ret == POOL_BUSY)) {
-                       bucket = (bucket + 1) % CONFIG_HAP_POOL_BUCKETS;
+                       bucket = statistical_prng() % CONFIG_HAP_POOL_BUCKETS;
                        ret = _HA_ATOMIC_LOAD(&pool->buckets[bucket].free_list);
                }
                if (ret == NULL)