mm/kasan: fix incorrect unpoisoning in vrealloc for KASAN

commit 007f5da43b3d0ecff972e2616062b8da1f862f5e upstream.

Patch series "kasan: vmalloc: Fixes for the percpu allocator and
vrealloc", v3.

Patches fix two issues related to KASAN and vmalloc.

The first one, a KASAN tag mismatch, possibly resulting in a kernel panic,
can be observed on systems with a tag-based KASAN enabled and with
multiple NUMA nodes.  Initially it was only noticed on x86 [1] but later a
similar issue was also reported on arm64 [2].

Specifically the problem is related to how vm_structs interact with
pcpu_chunks - both when they are allocated, assigned and when pcpu_chunk
addresses are derived.

When vm_structs are allocated they are unpoisoned, each with a different
random tag, if vmalloc support is enabled along the KASAN mode.  Later
when first pcpu chunk is allocated it gets its 'base_addr' field set to
the first allocated vm_struct.  With that it inherits that vm_struct's
tag.

When pcpu_chunk addresses are later derived (by pcpu_chunk_addr(), for
example in pcpu_alloc_noprof()) the base_addr field is used and offsets
are added to it.  If the initial conditions are satisfied then some of the
offsets will point into memory allocated with a different vm_struct.  So
while the lower bits will get accurately derived the tag bits in the top
of the pointer won't match the shadow memory contents.

The solution (proposed at v2 of the x86 KASAN series [3]) is to unpoison
the vm_structs with the same tag when allocating them for the per cpu
allocator (in pcpu_get_vm_areas()).

The second one reported by syzkaller [4] is related to vrealloc and
happens because of random tag generation when unpoisoning memory without
allocating new pages.  This breaks shadow memory tracking and needs to
reuse the existing tag instead of generating a new one.  At the same time
an inconsistency in used flags is corrected.

This patch (of 3):

Syzkaller reported a memory out-of-bounds bug [4].  This patch fixes two
issues:

1. In vrealloc the KASAN_VMALLOC_VM_ALLOC flag is missing when
   unpoisoning the extended region. This flag is required to correctly
   associate the allocation with KASAN's vmalloc tracking.

   Note: In contrast, vzalloc (via __vmalloc_node_range_noprof)
   explicitly sets KASAN_VMALLOC_VM_ALLOC and calls
   kasan_unpoison_vmalloc() with it.  vrealloc must behave consistently --
   especially when reusing existing vmalloc regions -- to ensure KASAN can
   track allocations correctly.

2. When vrealloc reuses an existing vmalloc region (without allocating
   new pages) KASAN generates a new tag, which breaks tag-based memory
   access tracking.

Introduce KASAN_VMALLOC_KEEP_TAG, a new KASAN flag that allows reusing the
tag already attached to the pointer, ensuring consistent tag behavior
during reallocation.

Pass KASAN_VMALLOC_KEEP_TAG and KASAN_VMALLOC_VM_ALLOC to the
kasan_unpoison_vmalloc inside vrealloc_node_align_noprof().

Link: https://lkml.kernel.org/r/cover.1765978969.git.m.wieczorretman@pm.me
Link: https://lkml.kernel.org/r/38dece0a4074c43e48150d1e242f8242c73bf1a5.1764874575.git.m.wieczorretman@pm.me
Link: https://lore.kernel.org/all/e7e04692866d02e6d3b32bb43b998e5d17092ba4.1738686764.git.maciej.wieczor-retman@intel.com/
Link: https://lore.kernel.org/all/aMUrW1Znp1GEj7St@MiWiFi-R3L-srv/
Link: https://lore.kernel.org/all/CAPAsAGxDRv_uFeMYu9TwhBVWHCCtkSxoWY4xmFB_vowMbi8raw@mail.gmail.com/
Link: https://syzkaller.appspot.com/bug?extid=997752115a851cb0cf36
Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Co-developed-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
Signed-off-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
Reported-by: syzbot+997752115a851cb0cf36@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/68e243a2.050a0220.1696c6.007d.GAE@google.com/T/
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Dmitriy Vyukov <dvyukov@google.com>
Cc: Kees Cook <kees@kernel.org>
Cc: Marco Elver <elver@google.com>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 6bbfc8aa42e8f47b25e7886f540574c87e02989b..dcbd8b09680e05da3b9b66296103219176539c45 100644 (file)
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -28,6 +28,7 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;
 #define KASAN_VMALLOC_INIT             ((__force kasan_vmalloc_flags_t)0x01u)
 #define KASAN_VMALLOC_VM_ALLOC         ((__force kasan_vmalloc_flags_t)0x02u)
 #define KASAN_VMALLOC_PROT_NORMAL      ((__force kasan_vmalloc_flags_t)0x04u)
+#define KASAN_VMALLOC_KEEP_TAG         ((__force kasan_vmalloc_flags_t)0x08u)
 
 #define KASAN_VMALLOC_PAGE_RANGE 0x1 /* Apply exsiting page range */
 #define KASAN_VMALLOC_TLB_FLUSH  0x2 /* TLB flush */

diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c
index 9958ebc15d38309955cd12184c4f431bca8c7d2a..faf0d954a296e2155a00c628027acf46ce4c06e2 100644 (file)
--- a/mm/kasan/hw_tags.c
+++ b/mm/kasan/hw_tags.c
@@ -345,7 +345,7 @@ void *__kasan_unpoison_vmalloc(const void *start, unsigned long size,
                return (void *)start;
        }
 
-       tag = kasan_random_tag();
+       tag = (flags & KASAN_VMALLOC_KEEP_TAG) ? get_tag(start) : kasan_random_tag();
        start = set_tag(start, tag);
 
        /* Unpoison and initialize memory up to size. */

diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index 88d1c9dcb5072158852fc0a9b77d2fc0c6a9a6c8..b18cfa5e2cb79ba7dc50a6e8407806e71f649942 100644 (file)
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -561,7 +561,9 @@ void *__kasan_unpoison_vmalloc(const void *start, unsigned long size,
            !(flags & KASAN_VMALLOC_PROT_NORMAL))
                return (void *)start;
 
-       start = set_tag(start, kasan_random_tag());
+       if (unlikely(!(flags & KASAN_VMALLOC_KEEP_TAG)))
+               start = set_tag(start, kasan_random_tag());
+
        kasan_unpoison(start, size, false);
        return (void *)start;
 }

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 3519c4e4f841dd610076ade6ede0522016f7baf6..6870e030f9a78dc19e168c75527583187948028b 100644 (file)
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -4118,7 +4118,9 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
         */
        if (size <= alloced_size) {
                kasan_unpoison_vmalloc(p + old_size, size - old_size,
-                                      KASAN_VMALLOC_PROT_NORMAL);
+                                      KASAN_VMALLOC_PROT_NORMAL |
+                                      KASAN_VMALLOC_VM_ALLOC |
+                                      KASAN_VMALLOC_KEEP_TAG);
                /*
                 * No need to zero memory here, as unused memory will have
                 * already been zeroed at initial allocation time or during