From: Palmer Dabbelt <palmer@rivosinc.com>
Date: Mon, 16 Sep 2024 03:16:12 +0000 (-0700)
Subject: Merge patch series "Svvptc extension to remove preventive sfence.vma"
X-Git-Tag: v6.12-rc1~76^2~14
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=7e340f4fad46b766705be96f5d1c764a397a7a36;p=thirdparty%2Fkernel%2Flinux.git

Merge patch series "Svvptc extension to remove preventive sfence.vma"

Alexandre Ghiti <alexghiti@rivosinc.com> says:

In RISC-V, after a new mapping is established, a sfence.vma needs to be
emitted for different reasons:

- if the uarch caches invalid entries, we need to invalidate it otherwise
  we would trap on this invalid entry,
- if the uarch does not cache invalid entries, a reordered access could fail
  to see the new mapping and then trap (sfence.vma acts as a fence).

We can actually avoid emitting those (mostly) useless and costly sfence.vma
by handling the traps instead:

- for new kernel mappings: only vmalloc mappings need to be taken care of,
  other new mapping are rare and already emit the required sfence.vma if
  needed.
  That must be achieved very early in the exception path as explained in
  patch 3, and this also fixes our fragile way of dealing with vmalloc faults.

- for new user mappings: Svvptc makes update_mmu_cache() a no-op but we can
  take some gratuitous page faults (which are very unlikely though).

Patch 1 and 2 introduce Svvptc extension probing.

On our uarch that does not cache invalid entries and a 6.5 kernel, the
gains are measurable:

* Kernel boot:                  6%
* ltp - mmapstress01:           8%
* lmbench - lat_pagefault:      20%
* lmbench - lat_mmap:           5%

Here are the corresponding numbers of sfence.vma emitted:

* Ubuntu boot to login:
Before: ~630k sfence.vma
After:  ~200k sfence.vma

* ltp - mmapstress01
Before: ~45k
After:  ~6.3k

* lmbench - lat_pagefault
Before: ~665k
After:   832 (!)

* lmbench - lat_mmap
Before: ~546k
After:   718 (!)

Thanks to Ved and Matt Evans for triggering the discussion that led to
this patchset!

* b4-shazam-merge:
  riscv: Stop emitting preventive sfence.vma for new userspace mappings with Svvptc
  riscv: Stop emitting preventive sfence.vma for new vmalloc mappings
  dt-bindings: riscv: Add Svvptc ISA extension description
  riscv: Add ISA extension parsing for Svvptc

Link: https://lore.kernel.org/r/20240717060125.139416-1-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
---

7e340f4fad46b766705be96f5d1c764a397a7a36
diff --cc arch/riscv/include/asm/hwcap.h
index 5a0bd27fd11a6,6dd0dd8beb303..46d9de54179ed
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@@ -80,18 -80,8 +80,19 @@@
  #define RISCV_ISA_EXT_ZFA		71
  #define RISCV_ISA_EXT_ZTSO		72
  #define RISCV_ISA_EXT_ZACAS		73
 -#define RISCV_ISA_EXT_XANDESPMU		74
 -#define RISCV_ISA_EXT_SVVPTC		75
 +#define RISCV_ISA_EXT_ZVE32X		74
 +#define RISCV_ISA_EXT_ZVE32F		75
 +#define RISCV_ISA_EXT_ZVE64X		76
 +#define RISCV_ISA_EXT_ZVE64F		77
 +#define RISCV_ISA_EXT_ZVE64D		78
 +#define RISCV_ISA_EXT_ZIMOP		79
 +#define RISCV_ISA_EXT_ZCA		80
 +#define RISCV_ISA_EXT_ZCB		81
 +#define RISCV_ISA_EXT_ZCD		82
 +#define RISCV_ISA_EXT_ZCF		83
 +#define RISCV_ISA_EXT_ZCMOP		84
 +#define RISCV_ISA_EXT_ZAWRS		85
++#define RISCV_ISA_EXT_SVVPTC		86
  
  #define RISCV_ISA_EXT_XLINUXENVCFG	127
  
diff --cc arch/riscv/kernel/cpufeature.c
index b427188b28fc1,60780d246743e..3a8eeaa9310c3
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@@ -381,6 -304,8 +381,7 @@@ const struct riscv_isa_ext_data riscv_i
  	__RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
  	__RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT),
  	__RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT),
 -	__RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_EXT_XANDESPMU),
+ 	__RISCV_ISA_EXT_DATA(svvptc, RISCV_ISA_EXT_SVVPTC),
  };
  
  const size_t riscv_isa_ext_count = ARRAY_SIZE(riscv_isa_ext);