From 6f95f6e111304c40fd18eee78a7e3372596d599f Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Wed, 4 Mar 2020 18:33:19 +0100 Subject: [PATCH] OPTIM: connection: disable receiving on disabled events when the run queue is too high In order to save a lot on syscalls, we currently don't disable receiving on a file descriptor anymore if its handler was already woken up. But if the run queue is huge and the poller collects a lot of events, this causes excess wakeups which take CPU time which is not used to flush these tasklets. This patch simply considers the run queue size to decide whether or not to stop receiving. Tests show that by stopping receiving when the run queue reaches ~16 times its configured size, we can still hold maximal performance in extreme situations like maxpollevents=20k for runqueue_depth=2, and still totally avoid calling epoll_event under moderate load using default settings on keep-alive connections. --- src/connection.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/connection.c b/src/connection.c index fea092b766..a6e2b3468d 100644 --- a/src/connection.c +++ b/src/connection.c @@ -127,6 +127,15 @@ void conn_fd_handler(int fd) if (!conn->subs->events) conn->subs = NULL; } + else if (tasks_run_queue_cur >= 16*global.tune.runqueue_depth) { + /* In order to save syscalls especially with epoll, we + * prefer *not* to disable receiving and instead let + * the handler do its job. But if the run queue becomes + * high, the excess of events may cause extra wakeups + * and in this case we'd rather flow-control ourselves. + */ + fd_stop_recv(fd); + } } leave: -- 2.39.5