]> git.ipfire.org Git - thirdparty/linux.git/commit
lib/crypto: x86/ghash: Migrate optimized code into library
authorEric Biggers <ebiggers@kernel.org>
Thu, 19 Mar 2026 06:17:15 +0000 (23:17 -0700)
committerEric Biggers <ebiggers@kernel.org>
Mon, 23 Mar 2026 23:44:29 +0000 (16:44 -0700)
commit3e79c8ec49596288c4460029c4971b9c838103b9
tree91cd30ff4d393636bd3c67fe2cfc439a5c15f780
parentefd1d2c8f3c073c43d5616d0c2d698cbe8a3ecde
lib/crypto: x86/ghash: Migrate optimized code into library

Remove the "ghash-pclmulqdqni" crypto_shash algorithm.  Move the
corresponding assembly code into lib/crypto/, and wire it up to the
GHASH library.

This makes the GHASH library be optimized with x86's carryless
multiplication instructions.  It also greatly reduces the amount of
x86-specific glue code that is needed, and it fixes the issue where this
GHASH optimization was disabled by default.

Rename and adjust the prototypes of the assembly functions to make them
fit better with the library.  Remove the byte-swaps (pshufb
instructions) that are no longer necessary because the library keeps the
accumulator in POLYVAL format rather than GHASH format.

Rename clmul_ghash_mul() to polyval_mul_pclmul() to reflect that it
really does a POLYVAL style multiplication.  Wire it up to both
ghash_mul_arch() and polyval_mul_arch().

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260319061723.1140720-15-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
arch/x86/crypto/Kconfig
arch/x86/crypto/Makefile
arch/x86/crypto/ghash-clmulni-intel_glue.c [deleted file]
lib/crypto/Makefile
lib/crypto/x86/gf128hash.h
lib/crypto/x86/ghash-pclmul.S [moved from arch/x86/crypto/ghash-clmulni-intel_asm.S with 54% similarity]