From: Paul E. McKenney Date: Tue, 3 Jun 2025 20:49:53 +0000 (-0700) Subject: srcu: Expedite SRCU-fast grace periods X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=3aea745a2a82e8397d2f30326f0dcf5f375dd7c8;p=thirdparty%2Fkernel%2Fstable.git srcu: Expedite SRCU-fast grace periods Currently, SRCU-fast grace periods use synchronize_rcu() to provide the needed ordering with readers, even given an expedited SRCU-fast grace period, which isn't all that expedited. This commit therefore instead uses synchronize_rcu_expedited() if there is an expedited SRCU-fast grace period in flight. Of course, given an non-expedited SRCU-fast grace period blocked in synchronize_rcu(), a later request for an expedited SRCU-fast grace period will wait for that synchronize_rcu() to return before switching to use of synchronize_rcu_expedited(). If this turns out to be a real problem for a production workload, we can increase the complexity (but likely also degrade the energy efficiency) to speed things up further. Signed-off-by: Paul E. McKenney Cc: Andrii Nakryiko Cc: Alexei Starovoitov Signed-off-by: Neeraj Upadhyay (AMD) --- diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 48047260697e..c5e8ebc493d5 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -502,6 +502,8 @@ static bool srcu_readers_active_idx_check(struct srcu_struct *ssp, int idx) */ if (!did_gp) smp_mb(); /* A */ + else if (srcu_gp_is_expedited(ssp)) + synchronize_rcu_expedited(); /* X */ else synchronize_rcu(); /* X */