]> git.ipfire.org Git - thirdparty/linux.git/commit - mm/mmap.c
mm/mmap: free vm_area_struct without call_rcu in exit_mmap
authorSuren Baghdasaryan <surenb@google.com>
Mon, 27 Feb 2023 17:36:31 +0000 (09:36 -0800)
committerAndrew Morton <akpm@linux-foundation.org>
Thu, 6 Apr 2023 03:03:02 +0000 (20:03 -0700)
commit0d2ebf9c3f7822e7ba3e4792ea3b6b19aa2da34a
tree238c51ee2bbbe47ded831d364d90b56ad8b9b7a3
parent70d4cbc80c88251de0a5b3e8df3275901f1fa99a
mm/mmap: free vm_area_struct without call_rcu in exit_mmap

call_rcu() can take a long time when callback offloading is enabled.  Its
use in the vm_area_free can cause regressions in the exit path when
multiple VMAs are being freed.

Because exit_mmap() is called only after the last mm user drops its
refcount, the page fault handlers can't be racing with it.  Any other
possible user like oom-reaper or process_mrelease are already synchronized
using mmap_lock.  Therefore exit_mmap() can free VMAs directly, without
the use of call_rcu().

Expose __vm_area_free() and use it from exit_mmap() to avoid possible
call_rcu() floods and performance regressions caused by it.

Link: https://lkml.kernel.org/r/20230227173632.3292573-33-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
include/linux/mm.h
kernel/fork.c
mm/mmap.c