- 13 Dec, 2018 15 commits
-
-
Eric Biggers authored
To improve responsiveness, yield the FPU (temporarily re-enabling preemption) every 4 KiB encrypted/decrypted, rather than keeping preemption disabled during the entire encryption/decryption operation. Alternatively we could do this for every skcipher_walk step, but steps may be small in some cases, and yielding the FPU is expensive on x86. Suggested-by:
Martin Willi <martin@strongswan.org> Signed-off-by:
Eric Biggers <ebiggers@google.com> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Eric Biggers authored
Now that the x86_64 SIMD implementations of ChaCha20 and XChaCha20 have been refactored to support varying the number of rounds, add support for XChaCha12. This is identical to XChaCha20 except for the number of rounds, which is 12 instead of 20. This can be used by Adiantum. Reviewed-by:
-
Eric Biggers authored
In preparation for adding XChaCha12 support, rename/refactor the x86_64 SIMD implementations of ChaCha20 to support different numbers of rounds. Reviewed-by:
-
Eric Biggers authored
Add an XChaCha20 implementation that is hooked up to the x86_64 SIMD implementations of ChaCha20. This can be used by Adiantum. An SSSE3 implementation of single-block HChaCha20 is also added so that XChaCha20 can use it rather than the generic implementation. This required refactoring the ChaCha permutation into its own function. Signed-off-by:
-
Eric Biggers authored
Add a 64-bit AVX2 implementation of NHPoly1305, an ε-almost-∆-universal hash function used in the Adiantum encryption mode. For now, only the NH portion is actually AVX2-accelerated; the Poly1305 part is less performance-critical so is just implemented in C. Signed-off-by:
-
Eric Biggers authored
Add a 64-bit SSE2 implementation of NHPoly1305, an ε-almost-∆-universal hash function used in the Adiantum encryption mode. For now, only the NH portion is actually SSE2-accelerated; the Poly1305 part is less performance-critical so is just implemented in C. Signed-off-by:
-
Eric Biggers authored
If the stream cipher implementation is asynchronous, then the Adiantum instance must be flagged as asynchronous as well. Otherwise someone asking for a synchronous algorithm can get an asynchronous algorithm. There are no asynchronous xchacha12 or xchacha20 implementations yet which makes this largely a theoretical issue, but it should be fixed. Fixes: 059c2a4d ("crypto: adiantum - add Adiantum support") Signed-off-by:
-
Ard Biesheuvel authored
To some degree, most known AArch64 micro-architectures appear to be able to issue ALU instructions in parellel to SIMD instructions without affecting the SIMD throughput. This means we can use the ALU to process a fifth ChaCha block while the SIMD is processing four blocks in parallel. Signed-off-by:
Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by:
-
Ard Biesheuvel authored
Update the 4-way NEON ChaCha routine so it can handle input of any length >64 bytes in its entirety, rather than having to call into the 1-way routine and/or memcpy()s via temp buffers to handle the tail of a ChaCha invocation that is not a multiple of 256 bytes. On inputs that are a multiple of 256 bytes (and thus in tcrypt benchmarks), performance drops by around 1% on Cortex-A57, while performance for inputs drawn randomly from the range [64, 1024) increases by around 30%. Signed-off-by:
-
Ard Biesheuvel authored
In order to have better coverage of algorithms operating on block sizes that are in the ballpark of a VPN packet, add 1472 to the block_sizes array. Signed-off-by:
-
Srikanth, Jampala authored
Enabled the PF->VF Mailbox support. Mailbox message are interpreted as {type, opcode, data}. Supported message types are REQ, ACK and NACK. Signed-off-by:Srikanth Jampala <Jampala.Srikanth@cavium.com> Signed-off-by:
-
Eric Biggers authored
Now that the ARM64 NEON implementation of ChaCha20 and XChaCha20 has been refactored to support varying the number of rounds, add support for XChaCha12. This is identical to XChaCha20 except for the number of rounds, which is 12 instead of 20. This can be used by Adiantum. Reviewed-by:
-
Eric Biggers authored
In preparation for adding XChaCha12 support, rename/refactor the ARM64 NEON implementation of ChaCha20 to support different numbers of rounds. Reviewed-by:
-
Eric Biggers authored
Add an XChaCha20 implementation that is hooked up to the ARM64 NEON implementation of ChaCha20. This can be used by Adiantum. A NEON implementation of single-block HChaCha20 is also added so that XChaCha20 can use it rather than the generic implementation. This required refactoring the ChaCha20 permutation into its own function. Signed-off-by:
-
Eric Biggers authored
Add an ARM64 NEON implementation of NHPoly1305, an ε-almost-∆-universal hash function used in the Adiantum encryption mode. For now, only the NH portion is actually NEON-accelerated; the Poly1305 part is less performance-critical so is just implemented in C. Reviewed-by:
-
- 07 Dec, 2018 20 commits
-
-
Yangtao Li authored
Use DEFINE_SHOW_ATTRIBUTE macro to simplify the code. Signed-off-by:
Yangtao Li <tiny.windzz@gmail.com> Signed-off-by:
-
Atul Gupta authored
Send SPI, 64b seq nos and 64b IV with aadiv drop for inline crypto. This information is added in outgoing packet after the CPL TX PKT XT and removed by hardware. The aad, auth and cipher offsets are then adjusted for ESN enabled tunnel. Signed-off-by:
Atul Gupta <atul.gupta@chelsio.com> Signed-off-by:
-
Atul Gupta authored
Immediate packets sent to hardware should include the work request length in calculating the flits. WR occupy one flit and if not accounted result in invalid request which stalls the HW queue. Cc: stable@vger.kernel.org Signed-off-by:
-
Corentin Labbe authored
This patch add the crypto_stats_init() function. This will permit to remove some ifdef from __crypto_register_alg(). Signed-off-by:
Corentin Labbe <clabbe@baylibre.com> Signed-off-by:
-
Corentin Labbe authored
Since now all crypto stats are on their own structures, it is now useless to have the algorithm name in the err_cnt member. Signed-off-by:
-
Corentin Labbe authored
Like for userspace, this patch splits stats into multiple structures, one for each algorithm class. Signed-off-by:
-
Corentin Labbe authored
The use of the v64 intermediate variable is useless, and removing it bring to much readable code. Signed-off-by:
-
Corentin Labbe authored
Some error count use the wrong name for getting this data. But this had not caused any reporting problem, since all error count are shared in the same union. Signed-off-by:
-
Corentin Labbe authored
All crypto_stats functions use the struct xxx_request for feeding stats, but in some case this structure could already be freed. For fixing this, the needed parameters (len and alg) will be stored before the request being executed. Fixes: cac5818c ("crypto: user - Implement a generic crypto statistics") Reported-by:
syzbot <syzbot+6939a606a5305e9e9799@syzkaller.appspotmail.com> Signed-off-by:
-
Corentin Labbe authored
This patch converts the getstat example tool to the recent changes done in crypto_user_stat - changed all stats to u64 - separated struct stats for each crypto alg Signed-off-by:
-
Corentin Labbe authored
It is cleaner to have each stat in their own structures. Signed-off-by:
-
Corentin Labbe authored
All the 32-bit fields need to be 64-bit. In some cases, UINT32_MAX crypto operations can be done in seconds. Reported-by:
Eric Biggers <ebiggers@kernel.org> Signed-off-by:
-
Corentin Labbe authored
CRYPTO_STATS is using CRYPTO_USER stuff, so it should depends on it. Signed-off-by:
-
Corentin Labbe authored
Even if CRYPTO_STATS is set to n, some part of CRYPTO_STATS are compiled. This patch made all part of crypto_user_stat uncompiled in that case. Signed-off-by:
-
Paulo Flabiano Smorigo authored
Add Breno and Nayna as NX/VMX crypto driver maintainers. Also change my email address to my personal account and remove Leonidas since he's not working with the driver anymore. Signed-off-by:
Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com> Signed-off-by:
-
Gilad Ben-Yossef authored
Add Yael Chemla as co-maintainer of Arm TrustZone CryptoCell REE driver. Signed-off-by:
Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by:
-
Gilad Ben-Yossef authored
Add device tree bindings associating Arm TrustZone CryptoCell 703 with the ccree driver. Signed-off-by:
Rob Herring <robh@kernel.org> Signed-off-by:
-
Gilad Ben-Yossef authored
Add support for Arm TrustZone CryptoCell 703. The 703 is a variant of the CryptoCell 713 that supports only algorithms certified by the Chinesse Office of the State Commercial Cryptography Administration (OSCCA). Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6Herbert Xu authored
Merge crypto tree to pick up crypto stats API revert.
-
Herbert Xu authored
Since this user-space API is still undergoing significant changes, this patch disables it for the current merge window. Signed-off-by:
-
- 29 Nov, 2018 5 commits
-
-
Srikanth, Jampala authored
Enable the available interrupt vectors for PF in SR-IOV Mode. Only single vector entry 192 is valid of PF. This is used to notify any hardware errors and mailbox messages from VF(s). Signed-off-by:
-
Nagadheeraj, Rottela authored
nitrox_skcipher_crypt() will do the necessary formatting/ordering of input and output sglists based on the algorithm requirements. It will also accommodate the mandatory output buffers required for NITROX hardware like Output request headers (ORH) and Completion headers. Signed-off-by:
Nagadheeraj Rottela <rottela.nagadheeraj@cavium.com> Reviewed-by:
-
Martin Willi authored
This version uses the same principle as the AVX2 version by scheduling the operations for two block pairs in parallel. It benefits from the AVX-512VL rotate instructions and the more efficient partial block handling using "vmovdqu8", resulting in a speedup of the raw block function of ~20%. Signed-off-by:
-
Martin Willi authored
This version uses the same principle as the AVX2 version. It benefits from the AVX-512VL rotate instructions and the more efficient partial block handling using "vmovdqu8", resulting in a speedup of ~20%. Unlike the AVX2 version, it is faster than the single block SSSE3 version to process a single block. Hence we engage that function for (partial) single block lengths as well. Signed-off-by:
-
Martin Willi authored
This variant is similar to the AVX2 version, but benefits from the AVX-512 rotate instructions and the additional registers, so it can operate without any data on the stack. It uses ymm registers only to avoid the massive core throttling on Skylake-X platforms. Nontheless does it bring a ~30% speed improvement compared to the AVX2 variant for random encryption lengths. The AVX2 version uses "rep movsb" for partial block XORing via the stack. With AVX-512, the new "vmovdqu8" can do this much more efficiently. The associated "kmov" instructions to work with dynamic masks is not part of the AVX-512VL instruction set, hence we depend on AVX-512BW as well. Given that the major AVX-512VL architectures provide AVX-512BW and this extension does not affect core clocking, this seems to be no problem at least for now. Signed-off-by:
-
