]> git.ipfire.org Git - thirdparty/kernel/linux.git/commit
sched_ext: Implement SCX_ENQ_IMMED
authorTejun Heo <tj@kernel.org>
Fri, 13 Mar 2026 19:43:22 +0000 (09:43 -1000)
committerTejun Heo <tj@kernel.org>
Fri, 13 Mar 2026 19:43:22 +0000 (09:43 -1000)
commit98d709cba3193f0bec54da4cd76ef499ea2f1ef7
treec402f9053b2c3bc28f7132a28b95b88ef12f2972
parentb5b38761b45a6c7d91760d212fda8b46df8c5362
sched_ext: Implement SCX_ENQ_IMMED

Add SCX_ENQ_IMMED enqueue flag for local DSQ insertions. Once a task is
dispatched with IMMED, it either gets on the CPU immediately and stays on it,
or gets reenqueued back to the BPF scheduler. It will never linger on a local
DSQ behind other tasks or on a CPU taken by a higher-priority class.

rq_is_open() uses rq->next_class to determine whether the rq is available,
and wakeup_preempt_scx() triggers reenqueue when a higher-priority class task
arrives. These capture all higher class preemptions. Combined with reenqueue
points in the dispatch path, all cases where an IMMED task would not execute
immediately are covered.

SCX_TASK_IMMED persists in p->scx.flags until the next fresh enqueue, so the
guarantee survives SAVE/RESTORE cycles. If preempted while running,
put_prev_task_scx() reenqueues through ops.enqueue() with
SCX_TASK_REENQ_PREEMPTED instead of silently placing the task back on the
local DSQ.

This enables tighter scheduling latency control by preventing tasks from
piling up on local DSQs. It also enables opportunistic CPU sharing across
sub-schedulers - without this, a sub-scheduler can stuff the local DSQ of a
shared CPU, making it difficult for others to use.

v2: - Rewrite is_curr_done() as rq_is_open() using rq->next_class and
      implement wakeup_preempt_scx() to achieve complete coverage of all
      cases where IMMED tasks could get stranded.
    - Track IMMED persistently in p->scx.flags and reenqueue
      preempted-while-running tasks through ops.enqueue().
    - Bound deferred reenq cycles (SCX_REENQ_LOCAL_MAX_REPEAT).
    - Misc renames, documentation.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
include/linux/sched/ext.h
kernel/sched/ext.c
kernel/sched/ext_internal.h
kernel/sched/sched.h
tools/sched_ext/include/scx/compat.bpf.h