riscv/mm: update write protect to work on shadow stacks

'fork' implements copy-on-write (COW) by making pages readonly in both
child and parent.

ptep_set_wrprotect() and pte_wrprotect() clear _PAGE_WRITE in PTE.
The assumption is that the page is readable and, on a fault,
copy-on-write happens.

To implement COW on shadow stack pages, clearing the W bit makes them
XWR = 000. This will result in the wrong PTE setting, which allows no
permissions, but with V=1 and the PFN field pointing to the final
page. Instead, the desired behavior is to turn it into a readable
page, take an access (load/store) fault on sspush/sspop (shadow stack)
and then perform COW on such pages. This way regular reads would still
be allowed and not lead to COW maintaining current behavior of COW on
non-shadow stack but writeable memory.

On the other hand, this doesn't interfere with existing COW for
read-write memory.  The assumption is always that _PAGE_READ must have
been set, and thus, setting _PAGE_READ is harmless.

Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Tested-by: Andreas Korb <andreas.korb@aisec.fraunhofer.de> # QEMU, custom CVA6
Tested-by: Valentin Haudiquet <valentin.haudiquet@canonical.com>
Link: https://patch.msgid.link/20251112-v5_user_cfi_series-v23-9-b55691eacf4f@rivosinc.com
[pjw@kernel.org: clarify patch description]
Signed-off-by: Paul Walmsley <pjw@kernel.org>

diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 7f81eecac243c649391f95deb83f63d387d51e5b..1340aa398a74cb52a52eec8b3e5cfbc4edf347ac 100644 (file)
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -411,7 +411,7 @@ static inline int pte_special(pte_t pte)
 
 static inline pte_t pte_wrprotect(pte_t pte)
 {
-       return __pte(pte_val(pte) & ~(_PAGE_WRITE));
+       return __pte((pte_val(pte) & ~(_PAGE_WRITE)) | (_PAGE_READ));
 }
 
 #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP
@@ -683,7 +683,15 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
 static inline void ptep_set_wrprotect(struct mm_struct *mm,
                                      unsigned long address, pte_t *ptep)
 {
-       atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep);
+       pte_t read_pte = READ_ONCE(*ptep);
+       /*
+        * ptep_set_wrprotect can be called for shadow stack ranges too.
+        * shadow stack memory is XWR = 010 and thus clearing _PAGE_WRITE will lead to
+        * encoding 000b which is wrong encoding with V = 1. This should lead to page fault
+        * but we dont want this wrong configuration to be set in page tables.
+        */
+       atomic_long_set((atomic_long_t *)ptep,
+                       ((pte_val(read_pte) & ~(unsigned long)_PAGE_WRITE) | _PAGE_READ));
 }
 
 #define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH