MEDIUM: peers: move process_peer_sync() to a single thread

The remaining half of the task_queue() and task_wakeup() contention
is caused by this function when peers are in use, because just like
process_table_expire(), it's created using task_new_anywhere() and
is woken up for local updates. Let's turn it to single thread by
rotating the assigned threads during initialization so that a table
only runs on one thread at a time.

Here we go backwards to assign the threads, so that on small setups
they don't end up on the same CPUs as the ones used by the stick-tables.
This way this will make an even better use of large machines. The
performance remains the same as with previous patch, even slightly
better (1-3% on avg).

At this point there's almost no multi-threaded task activity anymore
(only srv_cleanup_idle_server once in a while). This should improve
the situation described by Felipe in issues #3084 and #3101.

This should be backported to 3.2 after some extended checks.

diff --git a/src/peers.c b/src/peers.c
index b13215b5dce8db2080b47ceff46b8c33d14fb8c6..94abe58aa058a57b68f51ddae02983b6e74c7bcd 100644 (file)
--- a/src/peers.c
+++ b/src/peers.c
@@ -3783,13 +3783,18 @@ struct task *process_peer_sync(struct task * task, void *context, unsigned int s
  */
 int peers_init_sync(struct peers *peers)
 {
+       static uint operating_thread = 0;
        struct peer * curpeer;
 
        for (curpeer = peers->remote; curpeer; curpeer = curpeer->next) {
                peers->peers_fe->maxconn += 3;
        }
 
-       peers->sync_task = task_new_anywhere();
+       /* go backwards so as to distribute the load to other threads
+        * than the ones operating the stick-tables for small confs.
+        */
+       operating_thread = (operating_thread - 1) % global.nbthread;
+       peers->sync_task = task_new_on(operating_thread);
        if (!peers->sync_task)
                return 0;