sched: Clarify wake_up_q()'s write to task->wake_q.next

[ Upstream commit bcc6244e13b4d4903511a1ea84368abf925031c0 ]

Clarify that wake_up_q() does an atomic write to task->wake_q.next, after
which a concurrent __wake_q_add() can immediately overwrite
task->wake_q.next again.

Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20250129-sched-wakeup-prettier-v1-1-2f51f5f663fa@google.com
Signed-off-by: Sasha Levin <sashal@kernel.org>

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 70a7cf563f01e2bcab987a2e3c91626ffd617cbb..380938831b13021c57037013af2aca37953946c1 100644 (file)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -941,9 +941,10 @@ void wake_up_q(struct wake_q_head *head)
                struct task_struct *task;
 
                task = container_of(node, struct task_struct, wake_q);
-               /* Task can safely be re-inserted now: */
                node = node->next;
-               task->wake_q.next = NULL;
+               /* pairs with cmpxchg_relaxed() in __wake_q_add() */
+               WRITE_ONCE(task->wake_q.next, NULL);
+               /* Task can safely be re-inserted now. */
 
                /*
                 * wake_up_process() executes a full barrier, which pairs with