sched/cache: Fix cache aware scheduling enabling for multi LLCs system

If there are multiple LLCs in the system, cache aware scheduling
should be enabled. However, there is a corner case where, if there
is a single NUMA node and a single LLC per node, cache aware
scheduling will be turned on in the current implementation -
because at this moment, the parent domain has not yet been
degenerated, and it is possible that the current domain has the
same cpu span as its parent. There is no need to turn cache aware
scheduling on in this scenario.

Fix it by iterating the parent domains to find a domain that is
a superset of the current sd_llc, so that later, after the duplicated
parent domains have been degenerated, cache aware scheduling will
take effect.

For example, the expected behavior would be:
2 sockets, 1 LLC per socket: MC span=0-3, PKG span=0-7, has_multi_llcs=true
1 socket, 2 LLCs per socket: MC span=0-3, PKG span=0-7, has_multi_llcs=true
2 sockets, 2 LLCs per socket: MC span=0-3, PKG span=0-7, has_multi_llcs=true
1 socket, 1 LLC per socket: MC span=0-3, PKG span=0-3, has_multi_llcs=false

This bug was reported by sashiko.

Fixes: d59f4fd1d303 ("sched/cache: Enable cache aware scheduling for multi LLCs NUMA node")
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Co-developed-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/6328a8a7f40925cec2a712d81ee58128a4c4444a.1778703694.git.tim.c.chen@linux.intel.com

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index c257134f613d938deb5a3c7135a0ed227cc11e2e..4b7c64cbe8540fd11029596caf932773f4567173 100644 (file)
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1008,6 +1008,37 @@ static bool alloc_sd_llc(const struct cpumask *cpu_map,
 }
 #endif
 
+/*
+ * Return true if @sd belongs to an LLC group whose enclosing
+ * partition spans more than one LLC. @sd must be the topmost
+ * SD_SHARE_LLC domain.
+ *
+ * Any duplicated parent domains with the same span as @sd are
+ * skipped: before cpu_attach_domain() degeneration these still
+ * exist, after degeneration the loop is a no-op. This makes the
+ * helper usable both during sched domain build and against an
+ * already-attached domain tree.
+ *
+ * Note: For systems with a single LLC per node, cache-aware
+ * scheduling is still enabled when multiple nodes exist.
+ * However, NUMA balancing decisions take precedence over
+ * cache-aware scheduling. Conversely, if there is only one
+ * LLC per partition, cache-aware scheduling should be disabled.
+ */
+static bool sd_in_multi_llcs(struct sched_domain *sd)
+{
+       struct sched_domain *sdp = sd->parent;
+
+       /* it does not make sense to aggregate to 1 CPU */
+       if (sd->span_weight == 1)
+               return false;
+
+       while (sdp && sdp->span_weight == sd->span_weight)
+               sdp = sdp->parent;
+
+       return !!sdp;
+}
+
 /*
  * Return the canonical balance CPU for this group, this is the first CPU
  * of this group that's also in the balance mask.
@@ -3017,9 +3048,11 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
                         * NUMA imbalance stats for the hierarchy.
                         */
                        if (sd->parent) {
-                           if (IS_ENABLED(CONFIG_NUMA))
-                                   adjust_numa_imbalance(sd);
-                           has_multi_llcs = true;
+                               if (IS_ENABLED(CONFIG_NUMA))
+                                       adjust_numa_imbalance(sd);
+
+                               if (sd_in_multi_llcs(sd))
+                                       has_multi_llcs = true;
                        }
                }
        }