futex: Verify under the lock if hash can be replaced

Once the global hash is requested there is no way back to switch back to
the per-task private hash. This is checked at the begin of the function.

It is possible that two threads simultaneously request the global hash
and both pass the initial check and block later on the
mm::futex_hash_lock. In this case the first thread performs the switch
to the global hash. The second thread will also attempt to switch to the
global hash and while doing so, accessing the nonexisting slot 1 of the
struct futex_private_hash.
The same applies if the hash is made immutable: There is no reference
counting and the hash must not be replaced.

Verify under mm_struct::futex_phash that neither the global hash nor an
immutable hash in use.

Tested-by: "Lai, Yi" <yi1.lai@linux.intel.com>
Reported-by: "Lai, Yi" <yi1.lai@linux.intel.com>
Closes: https://lore.kernel.org/all/aDwDw9Aygqo6oAx+@ly-workstation/
Fixes: bd54df5ea7cad ("futex: Allow to resize the private local hash")
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/all/20250610104400.1077266-5-bigeasy@linutronix.de/

diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index b652d2f60c4097c09015a409dadd6a10ce7a4eeb..90d53fb0ee9e1563c355ef0499df441367e1a46c 100644 (file)
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -1629,6 +1629,16 @@ again:
                mm->futex_phash_new = NULL;
 
                if (fph) {
+                       if (cur && (!cur->hash_mask || cur->immutable)) {
+                               /*
+                                * If two threads simultaneously request the global
+                                * hash then the first one performs the switch,
+                                * the second one returns here.
+                                */
+                               free = fph;
+                               mm->futex_phash_new = new;
+                               return -EBUSY;
+                       }
                        if (cur && !new) {
                                /*
                                 * If we have an existing hash, but do not yet have