ARM: fix hash_name() fault

Zizhi Wo reports:

"During the execution of hash_name()->load_unaligned_zeropad(), a
 potential memory access beyond the PAGE boundary may occur. For
 example, when the filename length is near the PAGE_SIZE boundary.
 This triggers a page fault, which leads to a call to
 do_page_fault()->mmap_read_trylock(). If we can't acquire the lock,
 we have to fall back to the mmap_read_lock() path, which calls
 might_sleep(). This breaks RCU semantics because path lookup occurs
 under an RCU read-side critical section."

This is seen with CONFIG_DEBUG_ATOMIC_SLEEP=y and CONFIG_KFENCE=y.

Kernel addresses (with the exception of the vectors/kuser helper
page) do not have VMAs associated with them. If the vectors/kuser
helper page faults, then there are two possibilities:

1. if the fault happened while in kernel mode, then we're basically
   dead, because the CPU won't be able to vector through this page
   to handle the fault.
2. if the fault happened while in user mode, that means the page was
   protected from user access, and we want to fault anyway.

Thus, we can handle kernel addresses from any context entirely
separately without going anywhere near the mmap lock. This gives us
an entirely non-sleeping path for all kernel mode kernel address
faults.

As we handle the kernel address faults before interrupts are enabled,
this change has the side effect of improving the branch predictor
hardening, but does not completely solve the issue.

Reported-by: Zizhi Wo <wozizhi@huaweicloud.com>
Reported-by: Xie Yuanbin <xieyuanbin1@huawei.com>
Link: https://lore.kernel.org/r/20251126090505.3057219-1-wozizhi@huaweicloud.com
Reviewed-by: Xie Yuanbin <xieyuanbin1@huawei.com>
Tested-by: Xie Yuanbin <xieyuanbin1@huawei.com>
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>

diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index 192c8ab196dbaba3c4b4d8b473c545f6d2b1065c..0e5b4bc7b217603d3a6c7cd494117d3c7c250b37 100644 (file)
--- a/arch/arm/mm/fault.c
+++ b/arch/arm/mm/fault.c
@@ -261,6 +261,35 @@ static inline bool ttbr0_usermode_access_allowed(struct pt_regs *regs)
 }
 #endif
 
+static int __kprobes
+do_kernel_address_page_fault(struct mm_struct *mm, unsigned long addr,
+                            unsigned int fsr, struct pt_regs *regs)
+{
+       if (user_mode(regs)) {
+               /*
+                * Fault from user mode for a kernel space address. User mode
+                * should not be faulting in kernel space, which includes the
+                * vector/khelper page. Send a SIGSEGV.
+                */
+               __do_user_fault(addr, fsr, SIGSEGV, SEGV_MAPERR, regs);
+       } else {
+               /*
+                * Fault from kernel mode. Enable interrupts if they were
+                * enabled in the parent context. Section (upper page table)
+                * translation faults are handled via do_translation_fault(),
+                * so we will only get here for a non-present kernel space
+                * PTE or PTE permission fault. This may happen in exceptional
+                * circumstances and need the fixup tables to be walked.
+                */
+               if (interrupts_enabled(regs))
+                       local_irq_enable();
+
+               __do_kernel_fault(mm, addr, fsr, regs);
+       }
+
+       return 0;
+}
+
 static int __kprobes
 do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 {
@@ -274,6 +303,12 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
        if (kprobe_page_fault(regs, fsr))
                return 0;
 
+       /*
+        * Handle kernel addresses faults separately, which avoids touching
+        * the mmap lock from contexts that are not able to sleep.
+        */
+       if (addr >= TASK_SIZE)
+               return do_kernel_address_page_fault(mm, addr, fsr, regs);
 
        /* Enable interrupts if they were enabled in the parent context. */
        if (interrupts_enabled(regs))