iommu: Ensure .iotlb_sync is called correctly

Many drivers have no reason to use the iotlb_gather mechanism, but do
still depend on .iotlb_sync being called to properly complete an unmap.
Since the core code is now relying on the gather to detect when there
is legitimately something to sync, it should also take care of encoding
a successful unmap when the driver does not touch the gather itself.

Fixes: 90c5def10bea ("iommu: Do not call drivers for empty gathers")
Reported-by: Jon Hunter <jonathanh@nvidia.com>
Closes: https://lore.kernel.org/r/8800a38b-8515-4bbe-af15-0dae81274bf7@nvidia.com
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Will Deacon <will@kernel.org>

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 50718ab810a413e0ffd929c1d430ffda2a7cfa46..ee83850c70605e4008a5f66de4950619966338c1 100644 (file)
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2717,6 +2717,12 @@ static size_t __iommu_unmap(struct iommu_domain *domain,
 
                pr_debug("unmapped: iova 0x%lx size 0x%zx\n",
                         iova, unmapped_page);
+               /*
+                * If the driver itself isn't using the gather, make sure
+                * it looks non-empty so iotlb_sync will still be called.
+                */
+               if (iotlb_gather->start >= iotlb_gather->end)
+                       iommu_iotlb_gather_add_range(iotlb_gather, iova, size);
 
                iova += unmapped_page;
                unmapped += unmapped_page;