KVM: arm64: Check for SYSREGS_ON_CPU before accessing the CPU state

Mark Brown reports that since we commit to making exceptions
visible without the vcpu being loaded, the external abort selftest
fails.

Upon investigation, it turns out that the code that makes registers
affected by an exception visible to the guest is completely broken
on VHE, as we don't check whether the system registers are loaded
on the CPU at this point. We managed to get away with this so far,
but that's obviously as bad as it gets,

Add the required checksm and document the absolute need to check
for the SYSREGS_ON_CPU flag before calling into any of the
__vcpu_write_sys_reg_to_cpu()__vcpu_read_sys_reg_from_cpu() helpers.

Reported-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/18535df8-e647-4643-af9a-bb780af03a70@sirena.org.uk
Link: https://lore.kernel.org/r/20250720102229.179114-1-maz@kernel.org
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e54d29feb469bf07448349df55a79759bae6574a..d373d555a69ba304393889613c3daf7eebefefcc 100644 (file)
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1169,6 +1169,8 @@ static inline bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val)
         * System registers listed in the switch are not saved on every
         * exit from the guest but are only saved on vcpu_put.
         *
+        * SYSREGS_ON_CPU *MUST* be checked before using this helper.
+        *
         * Note that MPIDR_EL1 for the guest is set by KVM via VMPIDR_EL2 but
         * should never be listed below, because the guest cannot modify its
         * own MPIDR_EL1 and MPIDR_EL1 is accessed for VCPU A from VCPU B's
@@ -1221,6 +1223,8 @@ static inline bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg)
         * System registers listed in the switch are not restored on every
         * entry to the guest but are only restored on vcpu_load.
         *
+        * SYSREGS_ON_CPU *MUST* be checked before using this helper.
+        *
         * Note that MPIDR_EL1 for the guest is set by KVM via VMPIDR_EL2 but
         * should never be listed below, because the MPIDR should only be set
         * once, before running the VCPU, and never changed later.

diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 7dafd10e52e8c6e2b0b77ded7ee13d4127a4c985..95d186e0bf54f8398a829035a514ee5db909d454 100644 (file)
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -26,7 +26,8 @@ static inline u64 __vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg)
 
        if (unlikely(vcpu_has_nv(vcpu)))
                return vcpu_read_sys_reg(vcpu, reg);
-       else if (__vcpu_read_sys_reg_from_cpu(reg, &val))
+       else if (vcpu_get_flag(vcpu, SYSREGS_ON_CPU) &&
+                __vcpu_read_sys_reg_from_cpu(reg, &val))
                return val;
 
        return __vcpu_sys_reg(vcpu, reg);
@@ -36,7 +37,8 @@ static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg)
 {
        if (unlikely(vcpu_has_nv(vcpu)))
                vcpu_write_sys_reg(vcpu, val, reg);
-       else if (!__vcpu_write_sys_reg_to_cpu(val, reg))
+       else if (!vcpu_get_flag(vcpu, SYSREGS_ON_CPU) ||
+                !__vcpu_write_sys_reg_to_cpu(val, reg))
                __vcpu_assign_sys_reg(vcpu, reg, val);
 }