]> git.ipfire.org Git - thirdparty/linux.git/commit
sched/fair: Allocate cfs_tg_state with percpu allocator
authorZecheng Li <zecheng@google.com>
Fri, 22 May 2026 14:15:50 +0000 (10:15 -0400)
committerPeter Zijlstra <peterz@infradead.org>
Tue, 2 Jun 2026 10:26:11 +0000 (12:26 +0200)
commitb8fea7af0e40feb6d9cbbd60b66ff0ec265e868f
treea78769ec4fdaea135d1a78d05199f175f51fd3b5
parent89e1f67186baca353b68115bb98bd0bfed9f80c8
sched/fair: Allocate cfs_tg_state with percpu allocator

To remove the cfs_rq pointer array in task_group, allocate the combined
cfs_rq and sched_entity using the per-cpu allocator.

This patch implements the following:

 - Changes task_group->cfs_rq from 'struct cfs_rq **' to
   'struct cfs_rq __percpu *'.

 - Updates memory allocation in alloc_fair_sched_group() and
   free_fair_sched_group() to use alloc_percpu() and free_percpu()
   respectively.

 - Uses the inline accessor tg_cfs_rq(tg, cpu) with per_cpu_ptr() to retrieve
   the pointer to cfs_rq for the given task group and CPU.

 - Replaces direct accesses tg->cfs_rq[cpu] with calls to the new tg_cfs_rq(tg,
   cpu) helper.

 - Handles the root_task_group: since struct rq is already a per-cpu variable
   (runqueues), its embedded cfs_rq (rq->cfs) is also per-cpu. Therefore, we
   assign root_task_group.cfs_rq = &runqueues.cfs.

 - Cleanup the code in initializing the root task group.

This change places each CPU's cfs_rq and sched_entity in its local per-cpu
memory area to remove the per-task_group pointer arrays.

Signed-off-by: Zecheng Li <zecheng@google.com>
Signed-off-by: Zecheng Li <zli94@ncsu.edu>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Josh Don <joshdon@google.com>
Link: https://patch.msgid.link/20260522141623.600235-4-zli94@ncsu.edu
kernel/sched/core.c
kernel/sched/fair.c
kernel/sched/sched.h