]> git.ipfire.org Git - thirdparty/kernel/linux.git/commit
sched/fair: Co-locate cfs_rq and sched_entity in cfs_tg_state
authorZecheng Li <zecheng@google.com>
Fri, 22 May 2026 14:15:48 +0000 (10:15 -0400)
committerPeter Zijlstra <peterz@infradead.org>
Tue, 2 Jun 2026 10:26:10 +0000 (12:26 +0200)
commitdfcfc97b6df0ea8e1b7d3b590022782abbec3389
treec3ff207e28f3e327c3b732e0719896efb2999465
parent63c1a12bc0e09af7dee919c4fb4a300a719d5125
sched/fair: Co-locate cfs_rq and sched_entity in cfs_tg_state

Improve data locality and reduce pointer chasing by allocating struct
cfs_rq and struct sched_entity together for non-root task groups. This
is achieved by introducing a new combined struct cfs_tg_state that
holds both objects in a single allocation.

This patch:

 - Introduces struct cfs_tg_state that embeds cfs_rq, sched_entity, and
   sched_statistics together in a single structure.

 - Updates __schedstats_from_se() in stats.h to use cfs_tg_state for accessing
   sched_statistics from a group sched_entity.

 - Modifies alloc_fair_sched_group() and free_fair_sched_group() to allocate
   and free the new struct as a single unit.

 - Modifies the per-CPU pointers in task_group->se and task_group->cfs_rq to
   point to the members in the new combined structure.

Signed-off-by: Zecheng Li <zecheng@google.com>
Signed-off-by: Zecheng Li <zli94@ncsu.edu>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Josh Don <joshdon@google.com>
Link: https://patch.msgid.link/20260522141623.600235-2-zli94@ncsu.edu
kernel/sched/fair.c
kernel/sched/sched.h
kernel/sched/stats.h