iommu/vt-d: Fix missing PASID in dev TLB flush with cache_tag_flush_all

The function cache_tag_flush_all() was originally implemented with
incorrect device TLB invalidation logic that does not handle PASID, in
commit c4d27ffaa8eb ("iommu/vt-d: Add cache tag invalidation helpers")

This causes regressions where full address space TLB invalidations occur
with a PASID attached, such as during transparent hugepage unmapping in
SVA configurations or when calling iommu_flush_iotlb_all(). In these
cases, the device receives a TLB invalidation that lacks PASID.

This incorrect logic was later extracted into
cache_tag_flush_devtlb_all(), in commit 3297d047cd7f ("iommu/vt-d:
Refactor IOTLB and Dev-IOTLB flush for batching")

The fix replaces the call to cache_tag_flush_devtlb_all() with
cache_tag_flush_devtlb_psi(), which properly handles PASID.

Fixes: 4f609dbff51b ("iommu/vt-d: Use cache helpers in arch_invalidate_secondary_tlbs")
Fixes: 4e589a53685c ("iommu/vt-d: Use cache_tag_flush_all() in flush_iotlb_all")
Signed-off-by: Ethan Milon <ethan.milon@eviden.com>
Link: https://lore.kernel.org/r/20250708214821.30967-1-ethan.milon@eviden.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Link: https://lore.kernel.org/r/20250714045028.958850-11-baolu.lu@linux.intel.com
Signed-off-by: Will Deacon <will@kernel.org>

diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/cache.c
index f8bcc5f674630d2412294256ad5d762b834dbbc6..ff45c4c9609d2cbccbaf715be05307df602ddb2d 100644 (file)
--- a/drivers/iommu/intel/cache.c
+++ b/drivers/iommu/intel/cache.c
@@ -423,22 +423,6 @@ static void cache_tag_flush_devtlb_psi(struct dmar_domain *domain, struct cache_
                                             domain->qi_batch);
 }
 
-static void cache_tag_flush_devtlb_all(struct dmar_domain *domain, struct cache_tag *tag)
-{
-       struct intel_iommu *iommu = tag->iommu;
-       struct device_domain_info *info;
-       u16 sid;
-
-       info = dev_iommu_priv_get(tag->dev);
-       sid = PCI_DEVID(info->bus, info->devfn);
-
-       qi_batch_add_dev_iotlb(iommu, sid, info->pfsid, info->ats_qdep, 0,
-                              MAX_AGAW_PFN_WIDTH, domain->qi_batch);
-       if (info->dtlb_extra_inval)
-               qi_batch_add_dev_iotlb(iommu, sid, info->pfsid, info->ats_qdep, 0,
-                                      MAX_AGAW_PFN_WIDTH, domain->qi_batch);
-}
-
 /*
  * Invalidates a range of IOVA from @start (inclusive) to @end (inclusive)
  * when the memory mappings in the target domain have been modified.
@@ -509,7 +493,7 @@ void cache_tag_flush_all(struct dmar_domain *domain)
                        break;
                case CACHE_TAG_DEVTLB:
                case CACHE_TAG_NESTING_DEVTLB:
-                       cache_tag_flush_devtlb_all(domain, tag);
+                       cache_tag_flush_devtlb_psi(domain, tag, 0, MAX_AGAW_PFN_WIDTH);
                        break;
                }