From 065a025610fe86691ea3780330182d502d603ec1 Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Fri, 28 Feb 2020 14:42:26 +0100 Subject: [PATCH] MEDIUM: connection: don't stop receiving events in the FD handler The remaining epoll_ctl() calls are exclusively caused by the disagreement between conn_fd_handler() and the mux receiving the data: the fd handler wants to stop after having woken up the tasklet, then the mux after receiving data wants to receive again. Given that they don't happen in the same poll loop when there are many FDs, this causes a lot of state changes. As suggested by Olivier, if the task is already scheduled for running, we don't need to disable the event because it's in the run queue, poll() cannot stop, and reporting it again will be harmless. What *might* happen however is that a sampling-based poller like epoll() would report many times the same event and has trouble getting others behind. But if it would happen, it would still indicate the run queue has plenty of pending operations, so it would in fact only displace the problem from the poller to the run queue, which doesn't seem to be worse (and in fact we do support priorities while the poller does not). By doing this change, the keep-alive test with 1k conns and 100k reqs completely gets rid of the per-request epoll_ctl changes, while still not causing extra recvfrom() : $ ./h1load -n 100000 -t 4 -c 1000 -T 20 -F 127.0.0.1:8001/?s=1k/t=20 200000 sendto 1 200000 recvfrom 1 10762 epoll_wait 1 3664 epoll_ctl 1 1999 recvfrom -1 In close mode, it didn't change anything, we're still in the optimal case (2 epoll per connection) : $ ./h1load -n 100000 -r 1 -t 4 -c 1000 -T 20 -F 127.0.0.1:8001/?s=1k/t=20 203764 epoll_ctl 1 200000 sendto 1 200000 recvfrom 1 6091 epoll_wait 1 2994 recvfrom -1 --- src/connection.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/connection.c b/src/connection.c index 0a87eb7cf6..ee6818c3d9 100644 --- a/src/connection.c +++ b/src/connection.c @@ -106,7 +106,6 @@ void conn_fd_handler(int fd) conn->subs = NULL; } else io_available = 1; - fd_stop_send(fd); } /* The data transfer starts here and stops on error and handshakes. Note @@ -126,7 +125,6 @@ void conn_fd_handler(int fd) conn->subs = NULL; } else io_available = 1; - fd_stop_recv(fd); } leave: -- 2.39.5