rqspinlock: Disable spinning for trylock fallback

The original trylock fallback was inherited from qspinlock, and then
reused for the reentrant NMIs while the slow path is active. However,
under contention, it is very unlikely for the trylock to succeed in
taking the lock. In addition, a trylock also has no fairness guarantees,
and thus is prone to starvation issues under extreme scenarios.

The original qspinlock had no choice in terms of returning an error the
caller; if the node count was breached, it had to fall back to trylock
to attempt to take the lock. In case of rqspinlock, we do have the
option of returning to the user. Thus, simply attempt the trylock once,
and instead of spinning, return an error in case the lock cannot be
taken.

This ends up significantly reducing the time spent in the trylock
fallback, since we no longer wait for the timeout duration trying to
aimlessly acquire the lock when there's a high-probability that under
contention, it won't be available to us anyway.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20251128232802.1031906-5-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

diff --git a/kernel/bpf/rqspinlock.c b/kernel/bpf/rqspinlock.c
index e602cbbbd0295b918b8a88cee685672cba9de0a6..e35b06fcf9eef9c2d3b8fa205f11f7cc5c42c7b8 100644 (file)
--- a/kernel/bpf/rqspinlock.c
+++ b/kernel/bpf/rqspinlock.c
@@ -450,19 +450,17 @@ queue:
         * not be nested NMIs taking spinlocks. That may not be true in
         * some architectures even though the chance of needing more than
         * 4 nodes will still be extremely unlikely. When that happens,
-        * we fall back to spinning on the lock directly without using
-        * any MCS node. This is not the most elegant solution, but is
-        * simple enough.
+        * we fall back to attempting a trylock operation without using
+        * any MCS node. Unlike qspinlock which cannot fail, we have the
+        * option of failing the slow path, and under contention, such a
+        * trylock spinning will likely be treated unfairly due to lack of
+        * queueing, hence do not spin.
         */
        if (unlikely(idx >= _Q_MAX_NODES || (in_nmi() && idx > 0))) {
                lockevent_inc(lock_no_node);
-               RES_RESET_TIMEOUT(ts, RES_DEF_TIMEOUT);
-               while (!queued_spin_trylock(lock)) {
-                       if (RES_CHECK_TIMEOUT(ts, ret, ~0u)) {
-                               lockevent_inc(rqspinlock_lock_timeout);
-                               goto err_release_node;
-                       }
-                       cpu_relax();
+               if (!queued_spin_trylock(lock)) {
+                       ret = -EDEADLK;
+                       goto err_release_node;
                }
                goto release;
        }