]> git.ipfire.org Git - thirdparty/kernel/stable.git/commit
powerpc/32s: Do kuep_lock() and kuep_unlock() in assembly
authorChristophe Leroy <christophe.leroy@csgroup.eu>
Tue, 19 Oct 2021 07:29:17 +0000 (09:29 +0200)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Wed, 25 Oct 2023 09:58:59 +0000 (11:58 +0200)
commit558ee0fafd404dfbcef864945310b9a98d459678
tree3754e668a9dd3ea73ed68a5dfe078f0b66e12924
parentd00f4ae3accfed3fb5763dd600efc6d26bbadabe
powerpc/32s: Do kuep_lock() and kuep_unlock() in assembly

[ Upstream commit 526d4a4c77aedf1b7df1133e5cced29c70232e6e ]

When interrupt and syscall entries where converted to C, KUEP locking
and unlocking was also converted. It improved performance by unrolling
the loop, and allowed easily implementing boot time deactivation of
KUEP.

However, null_syscall selftest shows that KUEP is still heavy
(361 cycles with KUEP, 212 cycles without).

A way to improve more is to group 'mtsr's together, instead of
repeating 'addi' + 'mtsr' several times.

In order to do that, more registers need to be available. In C, GCC
will always be able to provide the requested number of registers, but
at the cost of saving some data on the stack, which is counter
performant here.

So let's do it in assembly, when we have full control of which
register can be used. It also has the advantage of locking earlier
and unlocking later and it helps GCC generating less tricky code.
The only drawback is to make boot time deactivation less straight
forward and require 'hand' instruction patching.

Group 'mtsr's by 4.

With this change, null_syscall selftest reports 336 cycles. Without
the change it was 361 cycles, that's a 7% reduction.

Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/115cb279e9b9948dfd93a065e047081c59e3a2a6.1634627931.git.christophe.leroy@csgroup.eu
Stable-dep-of: f0eee815babe ("powerpc/47x: Fix 47x syscall return crash")
Signed-off-by: Sasha Levin <sashal@kernel.org>
arch/powerpc/include/asm/book3s/32/kup.h
arch/powerpc/include/asm/book3s/32/mmu-hash.h
arch/powerpc/include/asm/interrupt.h
arch/powerpc/include/asm/kup.h
arch/powerpc/kernel/entry_32.S
arch/powerpc/kernel/head_32.h
arch/powerpc/kernel/head_book3s_32.S
arch/powerpc/kernel/interrupt.c
arch/powerpc/mm/book3s32/kuep.c