sched_ext: Enable tick for finite slices on nohz_full

set_next_task_scx() updates the tick dependency before __schedule()
updates rq->curr. When switching from a non-EXT task, such as idle, to
an EXT task with a finite slice, sched_update_tick_dependency() checks
the outgoing task and can allow the tick to remain stopped.

The dependency can also be lost without a slice-type transition. After a
finite-slice task leaves the CPU idle, the enqueue path can clear the
dependency against the idle rq->curr. SCX_RQ_CAN_STOP_TICK still records
a finite slice, so another finite task skips the transition block and
can run without the ticks needed to expire its slice.

The reverse mismatch can also happen when the last finite-slice EXT task
is dequeued: sub_nr_running() updates the dependency before rq->curr
changes, so the outgoing task state can keep the dependency set after
the CPU goes idle.

Fix this by unconditionally enabling the scheduler tick whenever a
finite-slice EXT task is selected on a nohz_full CPU. Moreover, when the
last runnable EXT task leaves, ignore the outgoing EXT slice state so
the generic scheduler can correctly re-evaluate and clear the tick
dependency.

Fixes: 22a920209ab6 ("sched_ext: Implement tickless support")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index ee66dacfc92c32dcbaa270a99c3f23fe5605356b..2dc6977d7984b1a39694d2edc68ae611e29a33f5 100644 (file)
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -2986,24 +2986,38 @@ static void set_next_task_scx(struct rq *rq, struct task_struct *p, bool first)
 
        /*
         * @p is getting newly scheduled or got kicked after someone updated its
-        * slice. Refresh whether tick can be stopped. See scx_can_stop_tick().
+        * slice. Update SCX_RQ_CAN_STOP_TICK to reflect whether the tick can be
+        * stopped. See scx_can_stop_tick().
+        *
+        * Moreover, refresh the load_avgs just when transitioning in and out of
+        * nohz. In the future, we might want to add a mechanism to update
+        * load_avgs periodically on tick-stopped CPUs.
         */
-       if ((p->scx.slice == SCX_SLICE_INF) !=
-           (bool)(rq->scx.flags & SCX_RQ_CAN_STOP_TICK)) {
-               if (p->scx.slice == SCX_SLICE_INF)
+       if (p->scx.slice == SCX_SLICE_INF) {
+               if (!(rq->scx.flags & SCX_RQ_CAN_STOP_TICK)) {
+                       /*
+                        * Bypass mode always assigns finite slices, so @p
+                        * can't have an infinite slice while bypassing.
+                        * Therefore, sched_update_tick_dependency() can safely
+                        * evaluate the outgoing task.
+                        */
                        rq->scx.flags |= SCX_RQ_CAN_STOP_TICK;
-               else
-                       rq->scx.flags &= ~SCX_RQ_CAN_STOP_TICK;
+                       sched_update_tick_dependency(rq);
 
-               sched_update_tick_dependency(rq);
+                       update_other_load_avgs(rq);
+               }
+       } else {
+               if (rq->scx.flags & SCX_RQ_CAN_STOP_TICK) {
+                       rq->scx.flags &= ~SCX_RQ_CAN_STOP_TICK;
+                       update_other_load_avgs(rq);
+               }
 
                /*
-                * For now, let's refresh the load_avgs just when transitioning
-                * in and out of nohz. In the future, we might want to add a
-                * mechanism which calls the following periodically on
-                * tick-stopped CPUs.
+                * @rq still references the outgoing scheduling context. A finite
+                * slice is sufficient by itself to require the tick.
                 */
-               update_other_load_avgs(rq);
+               if (tick_nohz_full_cpu(cpu_of(rq)))
+                       tick_nohz_dep_set_cpu(cpu_of(rq), TICK_DEP_BIT_SCHED);
        }
 }
 
@@ -4329,6 +4343,15 @@ bool scx_can_stop_tick(struct rq *rq)
        if (p->sched_class != &ext_sched_class)
                return true;
 
+       /*
+        * @rq->curr may still reference an outgoing EXT task after it has been
+        * dequeued. If no EXT tasks are accounted on @rq, ignore its stale
+        * slice state. If another task is dispatched from a DSQ,
+        * set_next_task_scx() will update the dependency for the incoming task.
+        */
+       if (!rq->scx.nr_running)
+               return true;
+
        if (scx_bypassing(sch, cpu_of(rq)))
                return false;