KVM: SVM: Filter out 64-bit exit codes when invoking exit handlers on bare metal

Explicitly filter out 64-bit exit codes when invoking exit handlers, as
svm_exit_handlers[] will never be sized with entries that use bits 63:32.

Processing the non-failing exit code as a 32-bit value will allow tracking
exit_code as a single 64-bit value (which it is, architecturally).  This
will also allow hardening KVM against Spectre-like attacks without needing
to do silly things to avoid build failures on 32-bit kernels
(array_index_nospec() rightly asserts that the index fits in an "unsigned
long").

Omit the check when running as a VM, as KVM has historically failed to set
bits 63:32 appropriately when synthesizing VM-Exits, i.e. KVM could get
false positives when running as a VM on an older, broken KVM/kernel.  From
a functional perspective, omitting the check is "fine", as any unwanted
collision between e.g. VMEXIT_INVALID and a 32-bit exit code will be
fatal to KVM-on-KVM regardless of what KVM-as-L1 does.

Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>
Link: https://patch.msgid.link/20251230211347.4099600-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index d2f997965a965a29c93315f8f048d0baab1776f6..3caf7a21679fc366646afc7a0a76f30e1ebed90e 100644 (file)
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -3467,8 +3467,22 @@ no_vmsa:
                sev_free_decrypted_vmsa(vcpu, save);
 }
 
-int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
+int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 __exit_code)
 {
+       u32 exit_code = __exit_code;
+
+       /*
+        * SVM uses negative values, i.e. 64-bit values, to indicate that VMRUN
+        * failed.  Report all such errors to userspace (note, VMEXIT_INVALID,
+        * a.k.a. SVM_EXIT_ERR, is special cased by svm_handle_exit()).  Skip
+        * the check when running as a VM, as KVM has historically left garbage
+        * in bits 63:32, i.e. running KVM-on-KVM would hit false positives if
+        * the underlying kernel is buggy.
+        */
+       if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR) &&
+           (u64)exit_code != __exit_code)
+               goto unexpected_vmexit;
+
 #ifdef CONFIG_MITIGATION_RETPOLINE
        if (exit_code == SVM_EXIT_MSR)
                return msr_interception(vcpu);
@@ -3495,7 +3509,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
 
 unexpected_vmexit:
        dump_vmcb(vcpu);
-       kvm_prepare_unexpected_reason_exit(vcpu, exit_code);
+       kvm_prepare_unexpected_reason_exit(vcpu, __exit_code);
        return 0;
 }