KVM: VMX: add memory clobber to asm for VMX instructions

VMCLEAR/VMREAD/VMWRITE/VMPTRLD access the internal VMCS cache, which
is not visible to the compiler; without a memory clobber, the compiler
can reorder them in troublesome ways because "asm volatile" and "asm goto"
only protect against removal of the asm.  For example, placing a VMWRITE
before the corresponding VMCS pointer is loaded can lead to corruption.
While none of this has been observed, it is better to prevent than cure.

Likewise, INVEPT and INVVPID access the TLB and, even though in their
case the effect is only visible to the next VMLAUNCH/VMRESUME, it is
technically correct to add the clobber there too.  So avoid any urge to
special case them, and simply hardcode "memory" into the clobber list
of vmx_asm1() and vmx_asm2().  __vmcs_readl() open-codes its own asm,
so add the clobber there as well.

Link: https://lore.kernel.org/kvm/CABgObfbL3t21yVeSwiLSjjOUER+rTYDPHYAH9YU4TWGRjx6XHg@mail.gmail.com/
Cc: Sean Christopherson <seanjc@google.com>
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

diff --git a/arch/x86/kvm/vmx/vmx_ops.h b/arch/x86/kvm/vmx/vmx_ops.h
index 81784befaaf4989d31dcf8fe0d61018c070a9841..9ffe03a0c223c4822916c2cdaa2cef35083d4dd2 100644 (file)
--- a/arch/x86/kvm/vmx/vmx_ops.h
+++ b/arch/x86/kvm/vmx/vmx_ops.h
@@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field)
 
                          : [output] "=r" (value)
                          : [field] "r" (field)
-                         : "cc"
+                         : "cc", "memory"
                          : do_fail, do_exception);
 
        return value;
@@ -145,7 +145,7 @@ do_exception:
 
                     : ASM_CALL_CONSTRAINT, [output] "=&r" (value)
                     : [field] "r" (field)
-                    : "cc");
+                    : "cc", "memory");
        return value;
 
 #endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
@@ -192,7 +192,7 @@ do {                                                                        \
        asm goto("1: " __stringify(insn) " %0\n\t"                      \
                          "jna %l[error]\n\t"                           \
                          _ASM_EXTABLE(1b, %l[fault])                   \
-                         : : op1 : "cc" : error, fault);               \
+                         : : op1 : "cc", "memory" : error, fault);     \
        return;                                                         \
 error:                                                                 \
        instrumentation_begin();                                        \
@@ -208,7 +208,7 @@ do {                                                                        \
        asm goto("1: "  __stringify(insn) " %1, %0\n\t"                 \
                          "jna %l[error]\n\t"                           \
                          _ASM_EXTABLE(1b, %l[fault])                   \
-                         : : op1, op2 : "cc" : error, fault);          \
+                         : : op1, op2 : "cc", "memory" : error, fault);\
        return;                                                         \
 error:                                                                 \
        instrumentation_begin();                                        \