KVM: arm64: nv: Fix incorrect VNCR invalidation range calculation

The code for invalidating VNCR entries in both kvm_invalidate_vncr_ipa()
and invalidate_vncr_va() incorrectly uses a bitwise AND with `(size - 1)`
instead of `~(size - 1)` to align the start address. This results
in masking the address bits instead of aligning them down to the start
of the block.

This bug may cause stale VNCR TLB entries to remain valid even after a
TLBI or MMU notifier, leading to incorrect memory translation and
unexpected guest behavior.

Credit to Team 0xB6 in bob14: DongHa Lee, Gyujeong Jin, Daehyeon Ko,
Geonha Lee, Hyungyu Oh, and Jaewon Yang.

Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Dongha Lee <p@sswd.pw>
Link: https://lore.kernel.org/r/20250906040724.72960-1-p@sswd.pw
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 24eab94d7d7f4cd15104ade6fe081f5971f23543..50d559248a1f0f91e8caf1ca31fa9f79b53d1da7 100644 (file)
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -847,7 +847,7 @@ static void kvm_invalidate_vncr_ipa(struct kvm *kvm, u64 start, u64 end)
 
                ipa_size = ttl_to_size(pgshift_level_to_ttl(vt->wi.pgshift,
                                                            vt->wr.level));
-               ipa_start = vt->wr.pa & (ipa_size - 1);
+               ipa_start = vt->wr.pa & ~(ipa_size - 1);
                ipa_end = ipa_start + ipa_size;
 
                if (ipa_end <= start || ipa_start >= end)
@@ -887,7 +887,7 @@ static void invalidate_vncr_va(struct kvm *kvm,
 
                va_size = ttl_to_size(pgshift_level_to_ttl(vt->wi.pgshift,
                                                           vt->wr.level));
-               va_start = vt->gva & (va_size - 1);
+               va_start = vt->gva & ~(va_size - 1);
                va_end = va_start + va_size;
 
                switch (scope->type) {