]> git.ipfire.org Git - thirdparty/linux.git/commit - arch/x86/mm/tlb.c
x86/mm/tlb: Skip atomic operations for 'init_mm' in switch_mm_irqs_off()
authorRik van Riel <riel@surriel.com>
Mon, 16 Jul 2018 19:03:37 +0000 (15:03 -0400)
committerIngo Molnar <mingo@kernel.org>
Tue, 17 Jul 2018 07:35:34 +0000 (09:35 +0200)
commite9d8c61557687b7126101e9550bdf243223f0d8f
tree2d467669575e3dbecc972a3c4922f6a615686acb
parent95b0e6357d3e4e05349668940d7ff8f3b7e7e11e
x86/mm/tlb: Skip atomic operations for 'init_mm' in switch_mm_irqs_off()

Song Liu noticed switch_mm_irqs_off() taking a lot of CPU time in recent
kernels,using 1.8% of a 48 CPU system during a netperf to localhost run.
Digging into the profile, we noticed that cpumask_clear_cpu and
cpumask_set_cpu together take about half of the CPU time taken by
switch_mm_irqs_off().

However, the CPUs running netperf end up switching back and forth
between netperf and the idle task, which does not require changes
to the mm_cpumask. Furthermore, the init_mm cpumask ends up being
the most heavily contended one in the system.

Simply skipping changes to mm_cpumask(&init_mm) reduces overhead.

Reported-and-tested-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Rik van Riel <riel@surriel.com>
Acked-by: Dave Hansen <dave.hansen@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: efault@gmx.de
Cc: kernel-team@fb.com
Cc: luto@kernel.org
Link: http://lkml.kernel.org/r/20180716190337.26133-8-riel@surriel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
arch/x86/mm/tlb.c