mm/khugepaged: fix PMD collapse swap PTE accounting

mthp_collapse() uses mthp_present_ptes to decide whether a range has
enough occupied PTEs to try collapse.  Swap PTEs accepted by
collapse_scan_pmd() are counted in unmapped, but are not represented in
mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none
so the scan can cover the whole PMD and build the bitmap.  mthp_collapse()
then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap
PTE no longer reaches collapse_huge_page(), even though PMD collapse can
handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order.  PMD collapse supports swap PTEs
through max_ptes_swap, while lower-order mTHP collapse does not currently
support non-present PTEs.  Keep non-present PTEs out of the lower-order
eligibility check.

Link: https://lore.kernel.org/20260609120443.71864-1-lance.yang@linux.dev
Fixes: 90ed32d00054 ("mm/khugepaged: introduce mTHP collapse support")
Signed-off-by: Lance Yang <lance.yang@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Nico Pache <npache@redhat.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Wei Yang <richard.weiyang@gmail.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 6de1a148222a0f303e41c1b42738ad2a12d431ef..33ab5d12e1bc7f280b3e9daf2d8992ea6a708c7d 100644 (file)
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1502,6 +1502,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
                nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
                                                      offset + nr_ptes);
 
+               /*
+                * Swap PTEs accepted during the scan are counted in @unmapped,
+                * not in the present-PTE bitmap. Account them for the PMD-order
+                * candidate.
+                */
+               if (is_pmd_order(order))
+                       nr_occupied_ptes += unmapped;
+
                if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
                        enum scan_result ret;