KVM: SVM: Flush the current TLB when transitioning from xAVIC => x2AVIC

Flush the current TLB when xAVIC *or* x2AVIC is activated, as KVM is
(apparently) responsible for purging TLB entries when transitioning from
xAVIC to x2AVIC.  The APM says a whole lot of nothing about TLB flushing
with respect to (x2)AVIC, but empirical data strongly suggests hardware
also does a whole lot of nothing.

Failure to flush the TLB when enabling x2AVIC can lead to guest accesses
to the APIC base address getting incorrectly redirected to the virtual
APIC page.  The flaw most visibly manifests as failures in KVM-Unit-Test's
verify_disabled_apic_mmio() testcase when x2APIC is enabled (though for
reasons unknown, the test only reliably fails with EFI builds).

Fixes: 0ccf3e7cb95a ("KVM: SVM: Flush the "current" TLB when activating AVIC")
Fixes: 4d1d7942e36a ("KVM: SVM: Introduce logic to (de)activate x2AVIC mode")
Cc: stable@vger.kernel.org
Cc: Naveen N Rao (AMD) <naveen@kernel.org>
Link: https://patch.msgid.link/20260515171536.1841645-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
index adf211860949a205820f73fe023fa01c3d86f205..e8bd60156941bb019430988c9c5f808c104747b7 100644 (file)
--- a/arch/x86/kvm/svm/avic.c
+++ b/arch/x86/kvm/svm/avic.c
@@ -206,6 +206,35 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
 
        svm_clr_intercept(svm, INTERCEPT_CR8_WRITE);
 
+       /*
+        * Flush the TLB when enabling (x2)AVIC and when transitioning between
+        * xAVIC and x2AVIC, as the CPU may have inserted a TLB entry for the
+        * "wrong" mapping.
+        *
+        * KVM uses a per-VM "scratch" page to back the APIC memslot, because
+        * KVM also uses per-VM page tables *and* maintains the page table (NPT
+        * or shadow page) mappings for said memslot even if one or more vCPUs
+        * have their local APIC hardware-disabled or are in x2APIC mode, i.e.
+        * even if one or more vCPUs' APIC MMIO BAR is effectively disabled.
+        *
+        * If xAVIC is fully enabled, hardware ignores the physical address in
+        * KVM's page tables, i.e. in the leaf SPTE for the APIC memslot, and
+        * instead redirects the access to the AVIC backing page, i.e. to the
+        * vCPU's virtual APIC page.  If xAVIC is not enabled (APIC is either
+        * hardware-disabled or in x2APIC mode), then guest accesses will use
+        * the page table mapping verbatim, i.e. will access the per-VM scratch
+        * page, as normal memory.
+        *
+        * In both cases, the CPU is allowed to cache TLB entries for the APIC
+        * base GPA.  So, KVM needs to flush the TLB when enabling xAVIC, as
+        * accesses need to be redirected to the virtual APIC page, but the TLB
+        * may contain entries pointing at the scratch page.  KVM also needs to
+        * flush the TLB when enabling x2AVIC, as accesses need to go to the
+        * scratch page, but the TLB may contain entries tagged as xAVIC, i.e.
+        * entries pointing to the vCPU's virtual APIC page.
+        */
+       kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu);
+
        /*
         * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR
         * accesses, while interrupt injection to a running vCPU can be
@@ -219,12 +248,6 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
                /* Disabling MSR intercept for x2APIC registers */
                avic_set_x2apic_msr_interception(svm, false);
        } else {
-               /*
-                * Flush the TLB, the guest may have inserted a non-APIC
-                * mapping into the TLB while AVIC was disabled.
-                */
-               kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu);
-
                /* Enabling MSR intercept for x2APIC registers */
                avic_set_x2apic_msr_interception(svm, true);
        }