From: Willy Tarreau <w@1wt.eu>
Date: Fri, 31 Jan 2020 16:55:09 +0000 (+0100)
Subject: OPTIM: task: readjust CPU bandwidth distribution since last update
X-Git-Tag: v2.2-dev2~49
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=1dfc9bbdc6ffb986ba9db9ed14684ca93b568598;p=thirdparty%2Fhaproxy.git

OPTIM: task: readjust CPU bandwidth distribution since last update

Now that we can more accurately watch which connection is really
being woken up from itself, it was desirable to re-adjust the CPU BW
thresholds based on measurements. New tests with 60000 concurrent
connections were run at 100 Gbps with unbounded queues and showed
the following distribution:

     scenario           TC0 TC1 TC2   observation
    -------------------+---+---+----+---------------------------
     TCP conn rate     : 32, 51, 17
     HTTP conn rate    : 34, 41, 25
     TCP byte rate     :  2,  3, 95   (2 MB objets)
     splicing byte rate: 11,  6, 83   (2 MB objets)
     H2 10k object     : 44, 23, 33   client-limited
     mixed traffic     : 18, 10, 72   2*1m+1*0: 11kcps, 36 Gbps

The H2 experienced a huge change since it uses a persistent connection
that was accidently flagged in the previous test. The splicing test
exhibits a higher need for short tasklets, so does the mixed traffic
test. Given that latency mainly matters for conn rate and H2 here,
the ratios were readjusted as 33% for TC0, 50% for TC1 and 17% for
TC2, keeping in mind that whatever is not consumed by one class is
automatically shared in equal propertions by the next one(s). This
setting immediately provided a nice improvement as with the default
settings (maxpollevents=200, runqueue-depth=200), the same ratios as
above are still reported, while the time to request "show activity"
on the CLI dropped to 30-50ms. The average loop time is around 5.7ms
on the mixed traffic.

In addition, one extra stress test at 90.5 Gbps with 5100 conn/s shows
70-100ms CLI request time, with an average loop time of 17 ms.
---

diff --git a/src/task.c b/src/task.c
index f7bfd893aa..5921c01fc9 100644
--- a/src/task.c
+++ b/src/task.c
@@ -438,15 +438,15 @@ void process_runnable_tasks()
 	if (likely(niced_tasks))
 		max_processed = (max_processed + 3) / 4;
 
-	/* run up to max_processed/6 urgent tasklets */
-	done = run_tasks_from_list(&tt->tasklets[TL_URGENT], (max_processed + 5) / 6);
+	/* run up to max_processed/3 urgent tasklets */
+	done = run_tasks_from_list(&tt->tasklets[TL_URGENT], (max_processed + 2) / 3);
 	max_processed -= done;
 
-	/* pick up to max_processed/3 (~=0.4*(max_processed-done)) regular tasks from prio-ordered run queues */
+	/* pick up to max_processed/2 (~=3/4*(max_processed-done)) regular tasks from prio-ordered run queues */
 
 	/* Note: the grq lock is always held when grq is not null */
 
-	while (tt->task_list_size < 2 * (max_processed + 4) / 5) {
+	while (tt->task_list_size < (3 * max_processed + 3) / 4) {
 		if ((global_tasks_mask & tid_bit) && !grq) {
 #ifdef USE_THREAD
 			HA_SPIN_LOCK(TASK_RQ_LOCK, &rq_lock);
@@ -508,11 +508,11 @@ void process_runnable_tasks()
 		grq = NULL;
 	}
 
-	/* run between max_processed/3 and max_processed/2 regular tasks */
-	done = run_tasks_from_list(&tt->tasklets[TL_NORMAL], 2 * (max_processed + 4) / 5);
+	/* run between 0.4*max_processed and max_processed/2 regular tasks */
+	done = run_tasks_from_list(&tt->tasklets[TL_NORMAL], (3 * max_processed + 3) / 4);
 	max_processed -= done;
 
-	/* run between max_processed/2 and max_processed bulk tasklets */
+	/* run between max_processed/4 and max_processed bulk tasklets */
 	done = run_tasks_from_list(&tt->tasklets[TL_BULK], max_processed);
 	max_processed -= done;