mm/khugepaged: add per-order mTHP collapse failure statistics

Add three new mTHP statistics to track collapse failures for different
orders when encountering swap PTEs, excessive none PTEs, and shared PTEs:

- collapse_exceed_swap_pte: Increment when mTHP collapse fails due to
	encountering a swap PTE.

- collapse_exceed_none_pte: Counts when mTHP collapse fails due to
  	exceeding the none PTE threshold for the given order

- collapse_exceed_shared_pte: Counts when mTHP collapse fails due to
	encountering a shared PTE.

These statistics complement the existing THP_SCAN_EXCEED_* events by
providing per-order granularity for mTHP collapse attempts.  The stats are
exposed via sysfs under
`/sys/kernel/mm/transparent_hugepage/hugepages-*/stats/` for each
supported hugepage size.

As we currently do not support collapsing mTHPs that contain a swap or
shared entry, those statistics keep track of how often we are encountering
failed mTHP collapses due to these restrictions.

We will add support for mTHP collapse for anonymous pages next; lets also
track when this happens at the PMD level within the per-mTHP stats.

Link: https://lore.kernel.org/20260605161422.213817-9-npache@redhat.com
Signed-off-by: Nico Pache <npache@redhat.com>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Bagas Sanjaya <bagasdotme@gmail.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nanyong Sun <sunnanyong@huawei.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Peter Xu <peterx@redhat.com>
Cc: Rafael Aquini <raquini@redhat.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shivank Garg <shivankg@amd.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Takashi Iwai (SUSE) <tiwai@suse.de>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Usama Arif <usamaarif642@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vishal Moola (Oracle) <vishal.moola@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Wei Yang <richard.weiyang@gmail.com>
Cc: Will Deacon <will@kernel.org>
Cc: Yang Shi <yang@os.amperecomputing.com>
Cc: Zach O'Keefe <zokeefe@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
index a74844e01f1e86e982f36da5994f52fa949db63a..b98e18c80185062125d9b16970233fc795ffaf71 100644 (file)
--- a/Documentation/admin-guide/mm/transhuge.rst
+++ b/Documentation/admin-guide/mm/transhuge.rst
@@ -714,6 +714,20 @@ nr_anon_partially_mapped
        an anonymous THP as "partially mapped" and count it here, even though it
        is not actually partially mapped anymore.
 
+collapse_exceed_none_pte
+       The number of collapse attempts that failed due to exceeding the
+       max_ptes_none threshold.
+
+collapse_exceed_swap_pte
+       The number of collapse attempts that failed due to exceeding the
+       max_ptes_swap threshold. For non-PMD orders this occurs if a mTHP range
+       contains at least one swap PTE.
+
+collapse_exceed_shared_pte
+       The number of collapse attempts that failed due to exceeding the
+       max_ptes_shared threshold. For non-PMD orders this occurs if a mTHP range
+       contains at least one shared PTE.
+
 As the system ages, allocating huge pages may be expensive as the
 system uses memory compaction to copy data around memory to free a
 huge page for use. There are some counters in ``/proc/vmstat`` to help

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index e225d1d4a3fef61c7ab9b43cb12e3087f7b3a923..89f4c22d7d62e6f8673e183173ec4576ed4ccf0d 100644 (file)
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -144,6 +144,9 @@ enum mthp_stat_item {
        MTHP_STAT_SPLIT_DEFERRED,
        MTHP_STAT_NR_ANON,
        MTHP_STAT_NR_ANON_PARTIALLY_MAPPED,
+       MTHP_STAT_COLLAPSE_EXCEED_SWAP,
+       MTHP_STAT_COLLAPSE_EXCEED_NONE,
+       MTHP_STAT_COLLAPSE_EXCEED_SHARED,
        __MTHP_STAT_COUNT
 };
 

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index c8b851dd518afdc878f9c5af528a800778014ab9..b4062396ce87742347df3f33796c9d140ac1477f 100644 (file)
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -720,6 +720,10 @@ DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED);
 DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED);
 DEFINE_MTHP_STAT_ATTR(nr_anon, MTHP_STAT_NR_ANON);
 DEFINE_MTHP_STAT_ATTR(nr_anon_partially_mapped, MTHP_STAT_NR_ANON_PARTIALLY_MAPPED);
+DEFINE_MTHP_STAT_ATTR(collapse_exceed_swap_pte, MTHP_STAT_COLLAPSE_EXCEED_SWAP);
+DEFINE_MTHP_STAT_ATTR(collapse_exceed_none_pte, MTHP_STAT_COLLAPSE_EXCEED_NONE);
+DEFINE_MTHP_STAT_ATTR(collapse_exceed_shared_pte, MTHP_STAT_COLLAPSE_EXCEED_SHARED);
+
 
 static struct attribute *anon_stats_attrs[] = {
        &anon_fault_alloc_attr.attr,
@@ -736,6 +740,9 @@ static struct attribute *anon_stats_attrs[] = {
        &split_deferred_attr.attr,
        &nr_anon_attr.attr,
        &nr_anon_partially_mapped_attr.attr,
+       &collapse_exceed_swap_pte_attr.attr,
+       &collapse_exceed_none_pte_attr.attr,
+       &collapse_exceed_shared_pte_attr.attr,
        NULL,
 };
 

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index c7819a854f14cb543f1b2abfce6d7210a723fb9f..a496484157a17e75dcdcf25523cc78e38edb5aad 100644 (file)
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -651,7 +651,9 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
                if (pte_none_or_zero(pteval)) {
                        if (++none_or_zero > max_ptes_none) {
                                result = SCAN_EXCEED_NONE_PTE;
-                               count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
+                               if (is_pmd_order(order))
+                                       count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
+                               count_mthp_stat(order, MTHP_STAT_COLLAPSE_EXCEED_NONE);
                                goto out;
                        }
                        continue;
@@ -693,7 +695,9 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
                         */
                        if (++shared > max_ptes_shared) {
                                result = SCAN_EXCEED_SHARED_PTE;
-                               count_vm_event(THP_SCAN_EXCEED_SHARED_PTE);
+                               if (is_pmd_order(order))
+                                       count_vm_event(THP_SCAN_EXCEED_SHARED_PTE);
+                               count_mthp_stat(order, MTHP_STAT_COLLAPSE_EXCEED_SHARED);
                                goto out;
                        }
                }
@@ -1152,6 +1156,7 @@ static enum scan_result __collapse_huge_page_swapin(struct mm_struct *mm,
                 * range.
                 */
                if (!is_pmd_order(order)) {
+                       count_mthp_stat(order, MTHP_STAT_COLLAPSE_EXCEED_SWAP);
                        pte_unmap(pte);
                        mmap_read_unlock(mm);
                        result = SCAN_EXCEED_SWAP_PTE;
@@ -1464,6 +1469,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
                        if (++none_or_zero > max_ptes_none) {
                                result = SCAN_EXCEED_NONE_PTE;
                                count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
+                               count_mthp_stat(HPAGE_PMD_ORDER,
+                                               MTHP_STAT_COLLAPSE_EXCEED_NONE);
                                goto out_unmap;
                        }
                        continue;
@@ -1472,6 +1479,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
                        if (++unmapped > max_ptes_swap) {
                                result = SCAN_EXCEED_SWAP_PTE;
                                count_vm_event(THP_SCAN_EXCEED_SWAP_PTE);
+                               count_mthp_stat(HPAGE_PMD_ORDER,
+                                               MTHP_STAT_COLLAPSE_EXCEED_SWAP);
                                goto out_unmap;
                        }
                        /*
@@ -1529,6 +1538,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
                        if (++shared > max_ptes_shared) {
                                result = SCAN_EXCEED_SHARED_PTE;
                                count_vm_event(THP_SCAN_EXCEED_SHARED_PTE);
+                               count_mthp_stat(HPAGE_PMD_ORDER,
+                                               MTHP_STAT_COLLAPSE_EXCEED_SHARED);
                                goto out_unmap;
                        }
                }