]> git.ipfire.org Git - thirdparty/kernel/stable.git/commit
srcu: Make SRCU-fast also be NMI-safe
authorPaul E. McKenney <paulmck@kernel.org>
Wed, 29 Jan 2025 02:32:49 +0000 (18:32 -0800)
committerBoqun Feng <boqun.feng@gmail.com>
Wed, 5 Feb 2025 15:12:06 +0000 (07:12 -0800)
commit3cec27453db49a176e688b7721c3cd26be5ef835
tree93c9f7472e8c5bf00677dd1bff96bab0e6e16321
parentf8b8df19b2dcb528e97395fdb74ca18b61db5207
srcu: Make SRCU-fast also be NMI-safe

BPF uses rcu_read_lock_trace() in NMI context, so srcu_read_lock_fast()
must be NMI-safe if it is to have any chance of addressing RCU Tasks
Trace use cases.  This commit therefore causes srcu_read_lock_fast()
and srcu_read_unlock_fast() to use atomic_long_inc() instead of
this_cpu_inc() on architectures that support NMIs but do not have
NMI-safe implementations of this_cpu_inc().  Note that both x86 and
arm64 have NMI-safe implementations of this_cpu_inc(), and thus do not
pay the performance penalty inherent in atomic_inc_long().

It is tempting to use this trick to fold srcu_read_lock_nmisafe()
into srcu_read_lock(), but this would need careful thought, review,
and performance analysis.  Though those smp_mb() calls might well make
performance a non-issue.

Reported-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
include/linux/srcutree.h