KVM: SVM: Add support for expedited writes to the fast MMIO bus

Wire up SVM's #NPF handler to fast MMIO.  While SVM doesn't provide a
dedicated exit reason, it's trivial to key off PFERR_RSVD_MASK.  Like VMX,
restrict the fast path to L1 to avoid having to deal with nGPA=>GPA
translations.

For simplicity, use the fast path if and only if the next RIP is known.
While KVM could utilize EMULTYPE_SKIP, doing so would require additional
logic to deal with SEV guests, e.g. to go down the slow path if the
instruction buffer is empty.  All modern CPUs support next RIP, and in
practice the next RIP will be available for any guest fast path.

Copy+paste the kvm_io_bus_write() + trace_kvm_fast_mmio() logic even
though KVM would ideally provide a small helper, as such a helper would
need to either be a macro or non-inline to avoid including trace.h in a
header (trace.h must not be included by x86.c prior to CREATE_TRACE_POINTS
being defined).

Link: https://patch.msgid.link/20251113221642.1673023-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index af018c1196b5a15aeb44b06ebb15afb6a7768f07..d1ff23e02ecd90d3de9a0eaae61fd45306946751 100644 (file)
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -1862,6 +1862,9 @@ static int pf_interception(struct kvm_vcpu *vcpu)
                        svm->vmcb->control.insn_len);
 }
 
+static int svm_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type,
+                                        void *insn, int insn_len);
+
 static int npf_interception(struct kvm_vcpu *vcpu)
 {
        struct vcpu_svm *svm = to_svm(vcpu);
@@ -1879,6 +1882,24 @@ static int npf_interception(struct kvm_vcpu *vcpu)
        if (WARN_ON_ONCE(error_code & PFERR_SYNTHETIC_MASK))
                error_code &= ~PFERR_SYNTHETIC_MASK;
 
+       /*
+        * Expedite fast MMIO kicks if the next RIP is known and KVM is allowed
+        * emulate a page fault, e.g. skipping the current instruction is wrong
+        * if the #NPF occurred while vectoring an event.
+        */
+       if ((error_code & PFERR_RSVD_MASK) && !is_guest_mode(vcpu)) {
+               const int emul_type = EMULTYPE_PF | EMULTYPE_NO_DECODE;
+
+               if (svm_check_emulate_instruction(vcpu, emul_type, NULL, 0))
+                       return 1;
+
+               if (nrips && svm->vmcb->control.next_rip &&
+                   !kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+                       trace_kvm_fast_mmio(gpa);
+                       return kvm_skip_emulated_instruction(vcpu);
+               }
+       }
+
        if (sev_snp_guest(vcpu->kvm) && (error_code & PFERR_GUEST_ENC_MASK))
                error_code |= PFERR_PRIVATE_ACCESS;