sched_ext: Use dsq->first_task instead of list_empty() in dispatch_enqueue() FIFO...

dispatch_enqueue()'s FIFO-tail path used list_empty(&dsq->list) to decide
whether to set dsq->first_task on enqueue. dsq->list can contain parked BPF
iterator cursors (SCX_DSQ_LNODE_ITER_CURSOR), so list_empty() is not a
reliable "no real task" check. If the last real task is unlinked while a
cursor is parked, first_task becomes NULL; the next FIFO-tail enqueue then
sees list_empty() == false and skips the first_task update, leaving
scx_bpf_dsq_peek() returning NULL for a non-empty DSQ.

Test dsq->first_task directly, which already tracks only real tasks and is
maintained under dsq->lock.

Fixes: 44f5c8ec5b9a ("sched_ext: Add lockless peek operation for DSQs")
Cc: stable@vger.kernel.org # v6.19+
Reported-by: Chris Mason <clm@meta.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Cc: Ryan Newton <newton@meta.com>

diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index cc5df32db8ffff671ec03b4cb8a6b09088c13f01..8a2a90659c65d1f80eb6c667bee09a2530c5d8d5 100644 (file)
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -1495,11 +1495,13 @@ static void dispatch_enqueue(struct scx_sched *sch, struct rq *rq,
                        if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN))
                                rcu_assign_pointer(dsq->first_task, p);
                } else {
-                       bool was_empty;
-
-                       was_empty = list_empty(&dsq->list);
+                       /*
+                        * dsq->list can contain parked BPF iterator cursors, so
+                        * list_empty() here isn't a reliable proxy for "no real
+                        * task in the DSQ". Test dsq->first_task directly.
+                        */
                        list_add_tail(&p->scx.dsq_list.node, &dsq->list);
-                       if (was_empty && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))
+                       if (!dsq->first_task && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))
                                rcu_assign_pointer(dsq->first_task, p);
                }
        }