From: Willy Tarreau <w@1wt.eu>
Date: Fri, 30 Jul 2021 08:57:09 +0000 (+0200)
Subject: BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before
X-Git-Tag: v2.5-dev3~14
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=c37ccd70b4f622a4148cb63d3b584357b02cda47;p=thirdparty%2Fhaproxy.git

BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before

A bug was introduced in 2.1-dev2 by commit 305d5ab46 ("MAJOR: fd: Get
rid of the fd cache."). Pollers "poll" and "evport" had the sleeping
bit accidentally removed before the syscall instead of after. This
results in them not being woken up by inter-thread wakeups, which is
particularly visible with the multi-queue accept() and with queues.

As a work-around, when these pollers are used, "nbthread 1" should
be used.

The fact that it has remained broken for 2 years is a great indication
that threads are definitely not enabled outside of epoll and kqueue,
hence why this patch is only tagged medium.

This must be backported as far as 2.2.
---

diff --git a/src/ev_evports.c b/src/ev_evports.c
index 09a48a4d98..9dc5728c64 100644
--- a/src/ev_evports.c
+++ b/src/ev_evports.c
@@ -152,8 +152,6 @@ static void _do_poll(struct poller *p, int exp, int wake)
 	}
 
 	thread_harmless_now();
-	if (sleeping_thread_mask & tid_bit)
-		_HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
 	/*
 	 * Determine how long to wait for events to materialise on the port.
@@ -206,6 +204,8 @@ static void _do_poll(struct poller *p, int exp, int wake)
 	tv_leaving_poll(wait_time, nevlist);
 
 	thread_harmless_end();
+	if (sleeping_thread_mask & tid_bit)
+		_HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
 	if (nevlist > 0)
 		activity[tid].poll_io++;
diff --git a/src/ev_poll.c b/src/ev_poll.c
index 4ed1b9e84d..f973d08bcd 100644
--- a/src/ev_poll.c
+++ b/src/ev_poll.c
@@ -165,8 +165,6 @@ static void _do_poll(struct poller *p, int exp, int wake)
 	} while (!_HA_ATOMIC_CAS(&maxfd, &old_maxfd, new_maxfd));
 
 	thread_harmless_now();
-	if (sleeping_thread_mask & tid_bit)
-		_HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
 	fd_nbupdt = 0;
 
@@ -210,6 +208,8 @@ static void _do_poll(struct poller *p, int exp, int wake)
 	tv_leaving_poll(wait_time, status);
 
 	thread_harmless_end();
+	if (sleeping_thread_mask & tid_bit)
+		_HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
 	if (status > 0)
 		activity[tid].poll_io++;