mm/damon: adjust isolated pages stat for DAMOS_MIGRATE_{HOT,COLD}

Callers of migrate_pages() should adjust NR_MIGRATED_{ANON,FILE} for
isolations and putback of the folios.  That for migration succeeded folios
is done by migrate_pages(), in migrate_folio_done().  That for MR_DEMOTION
reason is an exception though.

DAMOS_MIGRATE_{HOT,COLD} call migrate_pages() but mistakenly not doing the
stat adjustment.  As a result, use of DAMOS_MIGRATE_{HOT,COLD} could
corrupt the stat.  It could confuse too_many_isolated(), make compaction
and reclaim to behave in unexpected ways.  The stat corruption can be
reproduced and confirmed using DAMON user-space tool [1] on NUMA systems,
like below.

    $ numactl --hardware
    available: 2 nodes (0-1)
    [...]
    $ sudo ./damo start --damos_action migrate_hot 1
    $ sudo cat /proc/sys/vm/stat_refresh
    $ sudo dmesg
    [...]
    [   80.215554] vmstat_refresh: nr_isolated_anon -5578
    [   80.216842] vmstat_refresh: nr_isolated_file -34400

This issue was discovered [2] by Sashiko.

Link: https://lore.kernel.org/20260728140404.94476-1-sj@kernel.org
Link: https://github.com/damonitor/damo
Link: https://lore.kernel.org/20260726164356.87940-1-sj@kernel.org
Fixes: b51820ebea65 ("mm/damon/paddr: introduce DAMOS_MIGRATE_COLD action for demotion")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Honggyu Kim <honggyu.kim@sk.com>
Cc: Hyeongtak Ji <hyeongtak.ji@sk.com>
Cc: <stable@vger.kernel.org> # 6.11.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 9c178c175832fde9be6bf6a2489fa0a88fffe420..0bcad6b1e5b9e97adfc63db32365224618d6bbd9 100644 (file)
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -375,6 +375,8 @@ keep:
        while (!list_empty(folio_list)) {
                folio = lru_to_folio(folio_list);
                list_del(&folio->lru);
+               node_stat_sub_folio(folio, NR_ISOLATED_ANON +
+                               folio_is_file_lru(folio));
                folio_putback_lru(folio);
        }
 
@@ -397,6 +399,8 @@ unsigned long damon_migrate_pages(struct list_head *folio_list, int target_nid)
                        struct folio *folio = lru_to_folio(folio_list);
 
                        list_del(&folio->lru);
+                       node_stat_sub_folio(folio, NR_ISOLATED_ANON +
+                                       folio_is_file_lru(folio));
                        folio_putback_lru(folio);
                }
                return nr_migrated;

diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
index 5c2da45f988cf9cc982d08da34844c35b4c4df3d..f7613ce279a24c8c475ea75ada8b1aa4c28fa923 100644 (file)
--- a/mm/damon/paddr.c
+++ b/mm/damon/paddr.c
@@ -350,6 +350,8 @@ static unsigned long damon_pa_migrate(struct damon_region *r,
 
                if (!folio_isolate_lru(folio))
                        goto put_folio;
+               node_stat_add_folio(folio, NR_ISOLATED_ANON +
+                               folio_is_file_lru(folio));
                list_add(&folio->lru, &folio_list);
 put_folio:
                addr += folio_size(folio);

diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index e73ec1ce016e5b7f6456713873b8960dec1b21ae..2c1c1952c008d0334abfa500620f0d5ad9fab44e 100644 (file)
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -649,7 +649,8 @@ static void damos_va_migrate_dests_add(struct folio *folio,
 isolate:
        if (!folio_isolate_lru(folio))
                return;
-
+       node_stat_add_folio(folio, NR_ISOLATED_ANON +
+                       folio_is_file_lru(folio));
        list_add(&folio->lru, &migration_lists[i]);
 }