git.ipfire.org Git - thirdparty/kernel/stable-queue.git/blob

   1 From d4322d88f5fdf92729dd40f923013414fbb2184d Mon Sep 17 00:00:00 2001
   2 From: Catalin Marinas <catalin.marinas@arm.com>
   3 Date: Thu, 5 Nov 2015 18:45:54 -0800
   4 Subject: mm: slab: only move management objects off-slab for sizes larger than KMALLOC_MIN_SIZE
   5
   6 From: Catalin Marinas <catalin.marinas@arm.com>
   7
   8 commit d4322d88f5fdf92729dd40f923013414fbb2184d upstream.
   9
  10 On systems with a KMALLOC_MIN_SIZE of 128 (arm64, some mips and powerpc
  11 configurations defining ARCH_DMA_MINALIGN to 128), the first
  12 kmalloc_caches[] entry to be initialised after slab_early_init = 0 is
  13 "kmalloc-128" with index 7.  Depending on the debug kernel configuration,
  14 sizeof(struct kmem_cache) can be larger than 128 resulting in an
  15 INDEX_NODE of 8.
  16
  17 Commit 8fc9cf420b36 ("slab: make more slab management structure off the
  18 slab") enables off-slab management objects for sizes starting with
  19 PAGE_SIZE >> 5 (128 bytes for a 4KB page configuration) and the creation
  20 of the "kmalloc-128" cache would try to place the management objects
  21 off-slab.  However, since KMALLOC_MIN_SIZE is already 128 and
  22 freelist_size == 32 in __kmem_cache_create(), kmalloc_slab(freelist_size)
  23 returns NULL (kmalloc_caches[7] not populated yet).  This triggers the
  24 following bug on arm64:
  25
  26   kernel BUG at /work/Linux/linux-2.6-aarch64/mm/slab.c:2283!
  27   Internal error: Oops - BUG: 0 [#1] SMP
  28   Modules linked in:
  29   CPU: 0 PID: 0 Comm: swapper Not tainted 4.3.0-rc4+ #540
  30   Hardware name: Juno (DT)
  31   PC is at __kmem_cache_create+0x21c/0x280
  32   LR is at __kmem_cache_create+0x210/0x280
  33   [...]
  34   Call trace:
  35     __kmem_cache_create+0x21c/0x280
  36     create_boot_cache+0x48/0x80
  37     create_kmalloc_cache+0x50/0x88
  38     create_kmalloc_caches+0x4c/0xf4
  39     kmem_cache_init+0x100/0x118
  40     start_kernel+0x214/0x33c
  41
  42 This patch introduces an OFF_SLAB_MIN_SIZE definition to avoid off-slab
  43 management objects for sizes equal to or smaller than KMALLOC_MIN_SIZE.
  44
  45 Fixes: 8fc9cf420b36 ("slab: make more slab management structure off the slab")
  46 Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
  47 Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
  48 Acked-by: Christoph Lameter <cl@linux.com>
  49 Cc: Pekka Enberg <penberg@kernel.org>
  50 Cc: David Rientjes <rientjes@google.com>
  51 Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
  52 Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
  53 Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
  54 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  55
  56 ---
  57  mm/slab.c |    5 +++--
  58  1 file changed, 3 insertions(+), 2 deletions(-)
  59
  60 --- a/mm/slab.c
  61 +++ b/mm/slab.c
  62 @@ -282,6 +282,7 @@ static void kmem_cache_node_init(struct
  63
  64  #define CFLGS_OFF_SLAB         (0x80000000UL)
  65  #define        OFF_SLAB(x)     ((x)->flags & CFLGS_OFF_SLAB)
  66 +#define OFF_SLAB_MIN_SIZE (max_t(size_t, PAGE_SIZE >> 5, KMALLOC_MIN_SIZE + 1))
  67
  68  #define BATCHREFILL_LIMIT      16
  69  /*
  70 @@ -2212,7 +2213,7 @@ __kmem_cache_create (struct kmem_cache *
  71          * it too early on. Always use on-slab management when
  72          * SLAB_NOLEAKTRACE to avoid recursive calls into kmemleak)
  73          */
  74 -       if ((size >= (PAGE_SIZE >> 5)) && !slab_early_init &&
  75 +       if (size >= OFF_SLAB_MIN_SIZE && !slab_early_init &&
  76             !(flags & SLAB_NOLEAKTRACE))
  77                 /*
  78                  * Size is large, assume best to place the slab management obj
  79 @@ -2276,7 +2277,7 @@ __kmem_cache_create (struct kmem_cache *
  80                 /*
  81                  * This is a possibility for one of the kmalloc_{dma,}_caches.
  82                  * But since we go off slab only for object size greater than
  83 -                * PAGE_SIZE/8, and kmalloc_{dma,}_caches get created
  84 +                * OFF_SLAB_MIN_SIZE, and kmalloc_{dma,}_caches get created
  85                  * in ascending order,this should not happen at all.
  86                  * But leave a BUG_ON for some lucky dude.
  87                  */