releases/5.0.6/mm-add-support-for-kmem-caches-in-dma32-zone.patch

   1 From 6d6ea1e967a246f12cfe2f5fb743b70b2e608d4a Mon Sep 17 00:00:00 2001
   2 From: Nicolas Boichat <drinkcat@chromium.org>
   3 Date: Thu, 28 Mar 2019 20:43:42 -0700
   4 Subject: mm: add support for kmem caches in DMA32 zone
   5
   6 From: Nicolas Boichat <drinkcat@chromium.org>
   7
   8 commit 6d6ea1e967a246f12cfe2f5fb743b70b2e608d4a upstream.
   9
  10 Patch series "iommu/io-pgtable-arm-v7s: Use DMA32 zone for page tables",
  11 v6.
  12
  13 This is a followup to the discussion in [1], [2].
  14
  15 IOMMUs using ARMv7 short-descriptor format require page tables (level 1
  16 and 2) to be allocated within the first 4GB of RAM, even on 64-bit
  17 systems.
  18
  19 For L1 tables that are bigger than a page, we can just use
  20 __get_free_pages with GFP_DMA32 (on arm64 systems only, arm would still
  21 use GFP_DMA).
  22
  23 For L2 tables that only take 1KB, it would be a waste to allocate a full
  24 page, so we considered 3 approaches:
  25  1. This series, adding support for GFP_DMA32 slab caches.
  26  2. genalloc, which requires pre-allocating the maximum number of L2 page
  27     tables (4096, so 4MB of memory).
  28  3. page_frag, which is not very memory-efficient as it is unable to reuse
  29     freed fragments until the whole page is freed. [3]
  30
  31 This series is the most memory-efficient approach.
  32
  33 stable@ note:
  34   We confirmed that this is a regression, and IOMMU errors happen on 4.19
  35   and linux-next/master on MT8173 (elm, Acer Chromebook R13). The issue
  36   most likely starts from commit ad67f5a6545f ("arm64: replace ZONE_DMA
  37   with ZONE_DMA32"), i.e. 4.15, and presumably breaks a number of Mediatek
  38   platforms (and maybe others?).
  39
  40 [1] https://lists.linuxfoundation.org/pipermail/iommu/2018-November/030876.html
  41 [2] https://lists.linuxfoundation.org/pipermail/iommu/2018-December/031696.html
  42 [3] https://patchwork.codeaurora.org/patch/671639/
  43
  44 This patch (of 3):
  45
  46 IOMMUs using ARMv7 short-descriptor format require page tables to be
  47 allocated within the first 4GB of RAM, even on 64-bit systems.  On arm64,
  48 this is done by passing GFP_DMA32 flag to memory allocation functions.
  49
  50 For IOMMU L2 tables that only take 1KB, it would be a waste to allocate
  51 a full page using get_free_pages, so we considered 3 approaches:
  52  1. This patch, adding support for GFP_DMA32 slab caches.
  53  2. genalloc, which requires pre-allocating the maximum number of L2
  54     page tables (4096, so 4MB of memory).
  55  3. page_frag, which is not very memory-efficient as it is unable
  56     to reuse freed fragments until the whole page is freed.
  57
  58 This change makes it possible to create a custom cache in DMA32 zone using
  59 kmem_cache_create, then allocate memory using kmem_cache_alloc.
  60
  61 We do not create a DMA32 kmalloc cache array, as there are currently no
  62 users of kmalloc(..., GFP_DMA32).  These calls will continue to trigger a
  63 warning, as we keep GFP_DMA32 in GFP_SLAB_BUG_MASK.
  64
  65 This implies that calls to kmem_cache_*alloc on a SLAB_CACHE_DMA32
  66 kmem_cache must _not_ use GFP_DMA32 (it is anyway redundant and
  67 unnecessary).
  68
  69 Link: http://lkml.kernel.org/r/20181210011504.122604-2-drinkcat@chromium.org
  70 Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
  71 Acked-by: Vlastimil Babka <vbabka@suse.cz>
  72 Acked-by: Will Deacon <will.deacon@arm.com>
  73 Cc: Robin Murphy <robin.murphy@arm.com>
  74 Cc: Joerg Roedel <joro@8bytes.org>
  75 Cc: Christoph Lameter <cl@linux.com>
  76 Cc: Pekka Enberg <penberg@kernel.org>
  77 Cc: David Rientjes <rientjes@google.com>
  78 Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
  79 Cc: Michal Hocko <mhocko@suse.com>
  80 Cc: Mel Gorman <mgorman@techsingularity.net>
  81 Cc: Sasha Levin <Alexander.Levin@microsoft.com>
  82 Cc: Huaisheng Ye <yehs1@lenovo.com>
  83 Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
  84 Cc: Yong Wu <yong.wu@mediatek.com>
  85 Cc: Matthias Brugger <matthias.bgg@gmail.com>
  86 Cc: Tomasz Figa <tfiga@google.com>
  87 Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
  88 Cc: Christoph Hellwig <hch@infradead.org>
  89 Cc: Matthew Wilcox <willy@infradead.org>
  90 Cc: Hsin-Yi Wang <hsinyi@chromium.org>
  91 Cc: <stable@vger.kernel.org>
  92 Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
  93 Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
  94 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  95
  96 ---
  97  include/linux/slab.h |    2 ++
  98  mm/slab.c            |    2 ++
  99  mm/slab.h            |    3 ++-
 100  mm/slab_common.c     |    2 +-
 101  mm/slub.c            |    5 +++++
 102  5 files changed, 12 insertions(+), 2 deletions(-)
 103
 104 --- a/include/linux/slab.h
 105 +++ b/include/linux/slab.h
 106 @@ -32,6 +32,8 @@
 107  #define SLAB_HWCACHE_ALIGN     ((slab_flags_t __force)0x00002000U)
 108  /* Use GFP_DMA memory */
 109  #define SLAB_CACHE_DMA         ((slab_flags_t __force)0x00004000U)
 110 +/* Use GFP_DMA32 memory */
 111 +#define SLAB_CACHE_DMA32       ((slab_flags_t __force)0x00008000U)
 112  /* DEBUG: Store the last owner for bug hunting */
 113  #define SLAB_STORE_USER                ((slab_flags_t __force)0x00010000U)
 114  /* Panic if kmem_cache_create() fails */
 115 --- a/mm/slab.c
 116 +++ b/mm/slab.c
 117 @@ -2111,6 +2111,8 @@ done:
 118         cachep->allocflags = __GFP_COMP;
 119         if (flags & SLAB_CACHE_DMA)
 120                 cachep->allocflags |= GFP_DMA;
 121 +       if (flags & SLAB_CACHE_DMA32)
 122 +               cachep->allocflags |= GFP_DMA32;
 123         if (flags & SLAB_RECLAIM_ACCOUNT)
 124                 cachep->allocflags |= __GFP_RECLAIMABLE;
 125         cachep->size = size;
 126 --- a/mm/slab.h
 127 +++ b/mm/slab.h
 128 @@ -127,7 +127,8 @@ static inline slab_flags_t kmem_cache_fl
 129
 130
 131  /* Legal flag mask for kmem_cache_create(), for various configurations */
 132 -#define SLAB_CORE_FLAGS (SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA | SLAB_PANIC | \
 133 +#define SLAB_CORE_FLAGS (SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA | \
 134 +                        SLAB_CACHE_DMA32 | SLAB_PANIC | \
 135                          SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS )
 136
 137  #if defined(CONFIG_DEBUG_SLAB)
 138 --- a/mm/slab_common.c
 139 +++ b/mm/slab_common.c
 140 @@ -53,7 +53,7 @@ static DECLARE_WORK(slab_caches_to_rcu_d
 141                 SLAB_FAILSLAB | SLAB_KASAN)
 142
 143  #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
 144 -                        SLAB_ACCOUNT)
 145 +                        SLAB_CACHE_DMA32 | SLAB_ACCOUNT)
 146
 147  /*
 148   * Merge control. If this is set then no merging of slab caches will occur.
 149 --- a/mm/slub.c
 150 +++ b/mm/slub.c
 151 @@ -3591,6 +3591,9 @@ static int calculate_sizes(struct kmem_c
 152         if (s->flags & SLAB_CACHE_DMA)
 153                 s->allocflags |= GFP_DMA;
 154
 155 +       if (s->flags & SLAB_CACHE_DMA32)
 156 +               s->allocflags |= GFP_DMA32;
 157 +
 158         if (s->flags & SLAB_RECLAIM_ACCOUNT)
 159                 s->allocflags |= __GFP_RECLAIMABLE;
 160
 161 @@ -5681,6 +5684,8 @@ static char *create_unique_id(struct kme
 162          */
 163         if (s->flags & SLAB_CACHE_DMA)
 164                 *p++ = 'd';
 165 +       if (s->flags & SLAB_CACHE_DMA32)
 166 +               *p++ = 'D';
 167         if (s->flags & SLAB_RECLAIM_ACCOUNT)
 168                 *p++ = 'a';
 169         if (s->flags & SLAB_CONSISTENCY_CHECKS)