sched: deadline: Add dl_rq->curr pointer to address issues with Proxy Exec

The DL scheduler keeps the current task in the rbtree, since
the deadline value isn't usually chagned while the task is
runnable.

This results in set_next_task() and put_prev_task() being
simpler, but unfortunately this causes complexity elsewhere.

Specifically when update_curr_dl() updates the deadline, it has
to dequeue and then enqueue the task.

From put_prev_task_dl(), we first call update_curr_dl(), and
then call enqueue_pushable_dl_task().

However, with Proxy Exec this goes awry. Since when a mutex is
released, we might wake the waiting rq->donor. This will cause
put_prev_task() to be called on the donor to take it off the
cpu for return migration.

At that point, from put_prev_task_dl() the update_curr_dl()
logic will dequeue & enqueue the task, and the enqueue function
will call enqueue_pushable_dl_task() (since the task_current()
check won't prevent it). Then back up the callstack in
put_prev_task_dl() we'll end up calling
enqueue_pushable_dl_task() again, tripping the
!RB_EMPTY_NODE(&p->pushable_dl_tasks) warning.

So to avoid this, use Peter's suggested[1] approach, and add a
dl_rq->curr pointer that is set/cleared from set_next_task()/
put_prev_task(), which effectively tracks the rq->donor. We can
then use this to avoid adding the active donor to the pushable
list from enqueue_task_dl().

[1]: https://lore.kernel.org/lkml/20260304095123.GP606826@noisy.programming.kicks-ass.net/

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260512025635.2840817-4-jstultz@google.com

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 0b7ac4c1279704a21cfb53abab6d2843045af919..4754dbe4232d30d28606460cae3c03fd3fb4314b 100644 (file)
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -2541,6 +2541,9 @@ static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags)
        if (task_is_blocked(p))
                return;
 
+       if (dl_rq->curr == dl_se)
+               return;
+
        if (!task_current(rq, p) && !dl_se->dl_throttled && p->nr_cpus_allowed > 1)
                enqueue_pushable_dl_task(rq, p);
 }
@@ -2763,6 +2766,10 @@ static void start_hrtick_dl(struct rq *rq, struct sched_dl_entity *dl_se)
 }
 #endif /* !CONFIG_SCHED_HRTICK */
 
+/*
+ * DL keeps current in tree, because ->deadline is not typically changed while
+ * a task is runnable.
+ */
 static void set_next_task_dl(struct rq *rq, struct task_struct *p, bool first)
 {
        struct sched_dl_entity *dl_se = &p->dl;
@@ -2775,6 +2782,9 @@ static void set_next_task_dl(struct rq *rq, struct task_struct *p, bool first)
        /* You can't push away the running task */
        dequeue_pushable_dl_task(rq, p);
 
+       WARN_ON_ONCE(dl_rq->curr);
+       dl_rq->curr = dl_se;
+
        if (!first)
                return;
 
@@ -2845,6 +2855,9 @@ static void put_prev_task_dl(struct rq *rq, struct task_struct *p, struct task_s
 
        update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1);
 
+       WARN_ON_ONCE(dl_rq->curr != dl_se);
+       dl_rq->curr = NULL;
+
        if (task_is_blocked(p))
                return;
 

diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index ef715f2acbaa20d89750683367e69c989fc00c9d..b3aff26dbb132b030a9c0cebcbf6dde44b9b93c4 100644 (file)
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -893,6 +893,7 @@ struct dl_rq {
 
        bool                    overloaded;
 
+       struct sched_dl_entity  *curr;
        /*
         * Tasks on this rq that can be pushed away. They are kept in
         * an rb-tree, ordered by tasks' deadlines, with caching