KVM: s390: Fix unlikely race in try_get_locked_pte()

Fix an unlikely race in try_get_locked_pte(), which could have happened
if puds or pmds get unmapped between the p?dp_get() and p?d_offset()
functions.

Fixes: 89fa757931dc ("KVM: s390: Avoid potentially sleeping while atomic when zapping pages")
CC: stable@vger.kernel.org # 7.1
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260611104850.110313-3-imbrenda@linux.ibm.com>

diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c
index 1cfe4724fbe2a69fd4a038059eea9034899741ac..ee3f37af8aee4aa386d6dffafbf338908d559c87 100644 (file)
--- a/arch/s390/mm/gmap_helpers.c
+++ b/arch/s390/mm/gmap_helpers.c
@@ -51,15 +51,15 @@ pte_t *try_get_locked_pte(struct mm_struct *mm, unsigned long vmaddr, spinlock_t
        pgd = pgdp_get(pgdp);
        if (pgd_none(pgd) || !pgd_present(pgd))
                return NULL;
-       p4dp = p4d_offset(pgdp, vmaddr);
+       p4dp = p4d_offset_lockless(pgdp, pgd, vmaddr);
        p4d = p4dp_get(p4dp);
        if (p4d_none(p4d) || !p4d_present(p4d))
                return NULL;
-       pudp = pud_offset(p4dp, vmaddr);
+       pudp = pud_offset_lockless(p4dp, p4d, vmaddr);
        pud = pudp_get(pudp);
        if (pud_none(pud) || pud_leaf(pud) || !pud_present(pud))
                return NULL;
-       pmdp = pmd_offset(pudp, vmaddr);
+       pmdp = pmd_offset_lockless(pudp, pud, vmaddr);
        pmd = pmdp_get_lockless(pmdp);
        if (pmd_none(pmd) || pmd_leaf(pmd) || !pmd_present(pmd))
                return NULL;