mirror of
https://github.com/torvalds/linux.git
synced 2024-09-20 23:13:00 +00:00
86cd97ec4b
The current NEON based ChaCha implementation for ARM is optimized for multiples of 4x the ChaCha block size (64 bytes). This makes sense for block encryption, but given that ChaCha is also often used in the context of networking, it makes sense to consider arbitrary length inputs as well. For example, WireGuard typically uses 1420 byte packets, and performing ChaCha encryption involves 5 invocations of chacha_4block_xor_neon() and 3 invocations of chacha_block_xor_neon(), where the last one also involves a memcpy() using a buffer on the stack to process the final chunk of 1420 % 64 == 12 bytes. Let's optimize for this case as well, by letting chacha_4block_xor_neon() deal with any input size between 64 and 256 bytes, using NEON permutation instructions and overlapping loads and stores. This way, the 140 byte tail of a 1420 byte input buffer can simply be processed in one go. This results in the following performance improvements for 1420 byte blocks, without significant impact on power-of-2 input sizes. (Note that Raspberry Pi is widely used in combination with a 32-bit kernel, even though the core is 64-bit capable) Cortex-A8 (BeagleBone) : 7% Cortex-A15 (Calxeda Midway) : 21% Cortex-A53 (Raspberry Pi 3) : 3% Cortex-A72 (Raspberry Pi 4) : 19% Cc: Eric Biggers <ebiggers@google.com> Cc: "Jason A . Donenfeld" <Jason@zx2c4.com> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| aes-ce-core.S | ||
| aes-ce-glue.c | ||
| aes-cipher-core.S | ||
| aes-cipher-glue.c | ||
| aes-neonbs-core.S | ||
| aes-neonbs-glue.c | ||
| chacha-glue.c | ||
| chacha-neon-core.S | ||
| chacha-scalar-core.S | ||
| crc32-ce-core.S | ||
| crc32-ce-glue.c | ||
| crct10dif-ce-core.S | ||
| crct10dif-ce-glue.c | ||
| curve25519-core.S | ||
| curve25519-glue.c | ||
| ghash-ce-core.S | ||
| ghash-ce-glue.c | ||
| Kconfig | ||
| Makefile | ||
| nh-neon-core.S | ||
| nhpoly1305-neon-glue.c | ||
| poly1305-armv4.pl | ||
| poly1305-core.S_shipped | ||
| poly1305-glue.c | ||
| sha1_glue.c | ||
| sha1_neon_glue.c | ||
| sha1-armv4-large.S | ||
| sha1-armv7-neon.S | ||
| sha1-ce-core.S | ||
| sha1-ce-glue.c | ||
| sha1.h | ||
| sha2-ce-core.S | ||
| sha2-ce-glue.c | ||
| sha256_glue.c | ||
| sha256_glue.h | ||
| sha256_neon_glue.c | ||
| sha256-armv4.pl | ||
| sha256-core.S_shipped | ||
| sha512-armv4.pl | ||
| sha512-core.S_shipped | ||
| sha512-glue.c | ||
| sha512-neon-glue.c | ||
| sha512.h | ||