disable 5x interleave on buffers shorter than 512 bytes: 3% speedup on Graviton2

d6e4287c9726691e800bff221be71edd894a3c6a introduced 5x interleaving as an
optimization for ThunderX2, and that leads to some performance degradation on
when encoding short buffers.  We found this performance degradation by measuring
the performance of nginx on Ubuntu 20.04 that comes with OpenSSL 1.1.1f and
Ubuntu 22.04 with OpenSSL 3.0.1.

This patch limits the 5x interleave to buffers larger than 512 bytes.
On Graviton2 we see the following performance with this patch:

$ openssl speed -evp aes-128-gcm -bytes 128

AES-128-GCM   64 bytes     79 bytes     80 bytes     128 bytes    256 bytes    511 bytes    512 bytes    1024 bytes
master        1062564.71k  775113.11k   1069959.33k  1411716.28k  1653114.86k  1585981.16k  1973683.03k  2203214.08k
master+patch  1062729.28k  771915.11k   1103883.42k  1458665.43k  1708701.20k  1647060.84k  1975571.80k  2204038.42k
diff          0%           0%           3%           3%           3%           4%           0%           0%
revert d6e428 1055290.03k  773448.92k   1117411.97k  1441478.57k  1695698.52k  1634598.04k  1981851.65k  2196680.36k

CLA: trivial

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17984)

diff --git a/crypto/aes/asm/aesv8-armx.pl b/crypto/aes/asm/aesv8-armx.pl
index da10c44030c2bf4cb3f0f1e0085e61391875f327..57e7f5146fad1d28fa4b2636dae965badb31be53 100755 (executable)
--- a/crypto/aes/asm/aesv8-armx.pl
+++ b/crypto/aes/asm/aesv8-armx.pl
@@ -1822,7 +1822,7 @@ $code.=<<___;
        vorr            $dat2,$ivec,$ivec
 ___
 $code.=<<___   if ($flavour =~ /64/);
-       cmp             $len,#2
+       cmp             $len,#32
        b.lo            .Loop3x_ctr32
 
        add             w13,$ctr,#1