mm/rmap: improve mlock tracking for large folios

The kernel currently does not mlock large folios when adding them to rmap,
stating that it is difficult to confirm that the folio is fully mapped and
safe to mlock it.

This leads to a significant undercount of Mlocked in /proc/meminfo,
causing problems in production where the stat was used to estimate system
utilization and determine if load shedding is required.

However, nowadays the caller passes a number of pages of the folio that
are getting mapped, making it easy to check if the entire folio is mapped
to the VMA.

mlock the folio on rmap if it is fully mapped to the VMA.

Mlocked in /proc/meminfo can still undercount, but the value is closer the
truth and is useful for userspace.

Link: https://lkml.kernel.org/r/20250923110711.690639-7-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/rmap.c b/mm/rmap.c
index 92eeb3866494247256143c10093b73d4ae94aa20..ac4f783d6ec2f50c281cdc0221c1918ed4f1a17b 100644 (file)
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1463,12 +1463,12 @@ static __always_inline void __folio_add_anon_rmap(struct folio *folio,
        }
 
        /*
-        * For large folio, only mlock it if it's fully mapped to VMA. It's
-        * not easy to check whether the large folio is fully mapped to VMA
-        * here. Only mlock normal 4K folio and leave page reclaim to handle
-        * large folio.
+        * Only mlock it if the folio is fully mapped to the VMA.
+        *
+        * Partially mapped folios can be split on reclaim and part outside
+        * of mlocked VMA can be evicted or freed.
         */
-       if (!folio_test_large(folio))
+       if (folio_nr_pages(folio) == nr_pages)
                mlock_vma_folio(folio, vma);
 }
 
@@ -1601,8 +1601,13 @@ static __always_inline void __folio_add_file_rmap(struct folio *folio,
 
        __folio_add_rmap(folio, page, nr_pages, vma, level);
 
-       /* See comments in folio_add_anon_rmap_*() */
-       if (!folio_test_large(folio))
+       /*
+        * Only mlock it if the folio is fully mapped to the VMA.
+        *
+        * Partially mapped folios can be split on reclaim and part outside
+        * of mlocked VMA can be evicted or freed.
+        */
+       if (folio_nr_pages(folio) == nr_pages)
                mlock_vma_folio(folio, vma);
 }