PM: EM: Switch to rcu_dereference_all() in wakeup path

em_cpu_energy() is part of the EAS (Fair) task wakeup path. Now that
rcu_read_{,un}lock() have been removed from find_energy_efficient_cpu()
switch to rcu_dereference_all() and check for rcu_read_lock_any_held()
in em_cpu_energy() as well.
In EAS (Fair) task wakeup path is a preempt/IRQ disabled region, so
rcu_read_{,un}lock() can be removed.

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Link: https://patch.msgid.link/5b1228b7-5949-4a45-9f62-e8ce936de694@arm.com

diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h
index e7497f8046442dabb5463914fd0be42a8ba1f4c2..c909a8ba22e8320f5670b39f7d05be26d51b27ff 100644 (file)
--- a/include/linux/energy_model.h
+++ b/include/linux/energy_model.h
@@ -248,7 +248,7 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd,
        struct em_perf_state *ps;
        int i;
 
-       WARN_ONCE(!rcu_read_lock_held(), "EM: rcu read lock needed\n");
+       lockdep_assert(rcu_read_lock_any_held());
 
        if (!sum_util)
                return 0;
@@ -267,7 +267,7 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd,
         * Find the lowest performance state of the Energy Model above the
         * requested performance.
         */
-       em_table = rcu_dereference(pd->em_table);
+       em_table = rcu_dereference_all(pd->em_table);
        i = em_pd_get_efficient_state(em_table->state, pd, max_util);
        ps = &em_table->state[i];