KVM: riscv: Add a G-stage PTE cmpxchg helper

Permission-only G-stage PTE updates can run in parallel once they are
moved to the read side of mmu_lock. Plain set_pte() is not enough for
that case because another CPU may update the same PTE first.

x86 handles the same class of SPTE races with cmpxchg-based updates in
its fast page fault and TDP MMU paths. Add a small RISC-V helper for
atomic G-stage PTE updates. The helper reports contention to the caller
and flushes the target range only when the PTE value actually changes.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260517153427.94889-4-tjytimi@163.com
Signed-off-by: Anup Patel <anup@brainfault.org>

diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index f820c6783e16302c4872bdce876c2b30e3c4bae6..21e2019df0cf5773193793482aae958c8f76f4b7 100644 (file)
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -54,6 +54,10 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
                             struct kvm_mmu_memory_cache *pcache,
                             const struct kvm_gstage_mapping *map);
 
+bool kvm_riscv_gstage_try_update_pte(struct kvm_gstage *gstage, u32 level,
+                                    gpa_t addr, pte_t *ptep,
+                                    pte_t old_pte, pte_t new_pte);
+
 int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
                              struct kvm_mmu_memory_cache *pcache,
                              gpa_t gpa, phys_addr_t hpa, unsigned long page_size,

diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index d4ce5e76989c85fd5327655b890886596daa80a0..d878100aeb353dde63c7b805332ff986183a2fe1 100644 (file)
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -123,6 +123,20 @@ static void gstage_tlb_flush(struct kvm_gstage *gstage, u32 level, gpa_t addr)
                                               gstage->vmid);
 }
 
+bool kvm_riscv_gstage_try_update_pte(struct kvm_gstage *gstage, u32 level,
+                                    gpa_t addr, pte_t *ptep,
+                                    pte_t old_pte, pte_t new_pte)
+{
+       if (cmpxchg(&ptep->pte, pte_val(old_pte), pte_val(new_pte)) !=
+           pte_val(old_pte))
+               return false;
+
+       if (pte_val(old_pte) != pte_val(new_pte))
+               gstage_tlb_flush(gstage, level, addr);
+
+       return true;
+}
+
 int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
                             struct kvm_mmu_memory_cache *pcache,
                             const struct kvm_gstage_mapping *map)