- 20 Nov, 2020 2 commits
-
-
Ard Biesheuvel authored
Instead of copying the calculated authentication tag to memory and calling crypto_memneq() to verify it, use vector bytewise compare and min across vector instructions to decide whether the tag is valid. This is more efficient, and given that the tag is only transiently held in a NEON register, it is also safer, given that calculated tags for failed decryptions should be withheld. Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Kai Ye authored
Fix aead auth setting key process error. if use soft shash function, driver need to use digest size replace of the user input key length. Signed-off-by:
Kai Ye <yekai13@huawei.com> Signed-off-by:
-
- 13 Nov, 2020 38 commits
-
-
Ard Biesheuvel authored
Based on lessons learnt from optimizing the 32-bit version of this driver, we can simplify the arm64 version considerably, by reordering the final two stores when the last block is not a multiple of 64 bytes. This removes the need to use permutation instructions to calculate the elements that are clobbered by the final overlapping store, given that the store of the penultimate block now follows it, and that one carries the correct values for those elements already. While at it, simplify the overlapping loads as well, by calculating the address of the final overlapping load upfront, and switching to this address for every load that would otherwise extend past the end of the source buffer. There is no impact on performance, but the resulting code is substantially smaller and easier to follow. Cc: Eric Biggers <ebiggers@google.com> Cc: "Jason A . Donenfeld" <Jason@zx2c4.com> Signed-off-by:
-
Jack Xu authored
Add support for the QAT gen4 devices in the firmware loader. Signed-off-by:
Jack Xu <jack.xu@intel.com> Reviewed-by:
Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by:
-
Jack Xu authored
Add support for broadcasting mode in firmware loader to enable the next generation of QAT devices. Signed-off-by:
Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by:
-
Jack Xu authored
Add support for shared ustore mode support. This is required by the next generation of QAT devices to share the same fw image across engines. Signed-off-by:
-
Jack Xu authored
Introduce new API, qat_uclo_set_cfg_ae_mask(), to allow the load of the firmware image to a subset of Acceleration Engines (AEs). This is required by the next generation of QAT devices to be able to load different firmware images to the device. Signed-off-by:
-
Jack Xu authored
Add firmware control unit (FCU) CSRs to chip info so the firmware authentication code is common between all devices. Signed-off-by:
-
Jack Xu authored
Add support for CSS3K, which uses RSA3K as image signature algorithm, to support the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Use ae_mask to decide which Accelerator Engine (AE) to target in AE related operations, instead of a sequential loop, to skip AEs that are fused out. Signed-off-by:
-
Jack Xu authored
Add null pointer check when freeing the memory for firmware. Signed-off-by:
-
Jack Xu authored
Add misc control CSR to chip info since the CSR offset will be different in the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Add the wake up event to chip info since this value will be different in the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Add global clock enable CSR to the chip info since the CSR offset will be different in the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Add reset CSR offset and mask to chip info since they are different in new QAT devices. This also simplifies the reset/clrReset functions by using the reset mask. Signed-off-by:
-
Jack Xu authored
Add the local memory size to the chip info since the size of this memory will be different in the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Add support for local memory lm2 and lm3 which is introduced in the next generation of QAT devices. Signed-off-by:
-
Jack Xu authored
Introduce the next neighbor (NN) capability in chip_info as NN registers are not supported in certain SKUs of QAT. Signed-off-by:
-
Jack Xu authored
Modify condition in qat_uclo_wr_mimage() to use a capability of the device (sram_visible), rather than the device ID, so the check is not specific to devices of the same type. Signed-off-by:
-
Jack Xu authored
Introduce the chip info structure which contains device specific information. The initialization path has been split between common and hardware specific in order to facilitate the introduction of the next generation hardware. Signed-off-by:
-
Jack Xu authored
Replace long expressions with local variables in the functions qat_uclo_wr_uimage_page(), qat_uclo_init_globals() and qat_uclo_init_umem_seg() to improve readability. Signed-off-by:
-
Jack Xu authored
Refactor qat_uclo_set_ae_mode() by moving the logic that sets the AE modes to a separate function, qat_hal_set_modes(). Signed-off-by:
-
Jack Xu authored
Move the definition of ICP_QAT_AE_OFFSET, ICP_QAT_CAP_OFFSET, LOCAL_TO_XFER_REG_OFFSET and ICP_QAT_EP_OFFSET from qat_hal.c to icp_qat_hal.h to avoid the definition of generation specific constants in qat_hal.c. Signed-off-by:
-
Jack Xu authored
Include the offset of GLOBAL_CSR directly into the enum hal_global_csr and remove the macros SET_GLB_CSR/GET_GLB_CSR to simplify the global CSR access. Signed-off-by:
-
Jack Xu authored
Change the API and the behaviour of the qat_hal_start() function. With this change, the function starts under the hood all acceleration engines (AEs) and there is no longer need to call it for each engine. Signed-off-by:
-
Jack Xu authored
Change micro word data mask since the Acceleration Engine (AE) instruction codes have been changed in the new generation QAT devices. Signed-off-by:
-
Jack Xu authored
Change type for ctx_mask from unsigned char to unsigned long to avoid type casting. Signed-off-by:
-
Jack Xu authored
Improve the way micro instructions (FW code) are uploaded to Accelerator Engines (AEs). If code starts at PC zero (absolute addressing), read uwords with no relative address. Otherwise, use relative addressing to the page region. Signed-off-by:
-
Jack Xu authored
Rename the function qat_uclo_del_uof_obj() in qat_uclo_del_obj() since it frees the memory allocated for all firmware objects. Signed-off-by:
-
Jack Xu authored
Introduce additional parenthesis to resolve a warninga reported by checkpatch. Signed-off-by:
-
Jack Xu authored
Remove unnecessary parenthesis across the firmware loader. Signed-off-by:
-
Jack Xu authored
Change message in error path of qat_uclo_check_image_compat() to report an incompatible firmware image that contains a neighbor register table. Signed-off-by:
-
Jack Xu authored
Do not mask the AE number with the AE mask when accessing the AE local CSRs. Bit 12 of the local CSR address is the start of AE number so just take out the AE mask here. Signed-off-by:
-
Jack Xu authored
The return value of qat_hal_rd_ae_csr() is always a CSR value and never a status and should not be stored in the status variable of qat_hal_put_rel_rd_xfer(). This removes the assignment as qat_hal_rd_ae_csr() is not expected to fail. A more comprehensive handling of the theoretical corner case which could result in a fail will be submitted in a separate patch. Fixes: 8c9478a4 ("crypto: qat - reduce stack size with KASAN") Signed-off-by:
Fiona Trahe <fiona.trahe@intel.com> Signed-off-by:
-
Giovanni Cabiddu authored
Implement infrastructure for the Multiple Object File (MOF) format in the firmware loader. This will allow to load a specific firmware image contained inside an MOF file. This patch is based on earlier work done by Pingchao Yang. Signed-off-by:
-
Herbert Xu authored
This patch fixes all the sparse warnings in cavium/nitrox: - Fix endianness warnings by adding the correct markers to unions. - Add missing header inclusions for prototypes. - Move nitrox_sriov_configure prototype into the isr header file. Signed-off-by: -
Adam Guerin authored
Change all lower case pci in comments to be upper case PCI. Suggested-by:
Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by:
Adam Guerin <adam.guerin@intel.com> Reviewed-by:
-
Ard Biesheuvel authored
The current NEON based ChaCha implementation for ARM is optimized for multiples of 4x the ChaCha block size (64 bytes). This makes sense for block encryption, but given that ChaCha is also often used in the context of networking, it makes sense to consider arbitrary length inputs as well. For example, WireGuard typically uses 1420 byte packets, and performing ChaCha encryption involves 5 invocations of chacha_4block_xor_neon() and 3 invocations of chacha_block_xor_neon(), where the last one also involves a memcpy() using a buffer on the stack to process the final chunk of 1420 % 64 == 12 bytes. Let's optimize for this case as well, by letting chacha_4block_xor_neon() deal with any input size between 64 and 256 bytes, using NEON permutation instructions and overlapping loads and stores. This way, the 140 byte tail of a 1420 byte input buffer can simply be processed in one go. This results in the following performance improvements for 1420 byte blocks, without significant impact on power-of-2 input sizes. (Note that Raspberry Pi is widely used in combination with a 32-bit kernel, even though the core is 64-bit capable) Cortex-A8 (BeagleBone) : 7% Cortex-A15 (Calxeda Midway) : 21% Cortex-A53 (Raspberry Pi 3) : 3% Cortex-A72 (Raspberry Pi 4) : 19% Cc: Eric Biggers <ebiggers@google.com> Cc: "Jason A . Donenfeld" <Jason@zx2c4.com> Signed-off-by:
-
Adam Guerin authored
Remove cast for mailbox CSR in adf_admin.c as it is not needed. Suggested-by:
-
Jason A. Donenfeld authored
The extra tests in the manager actually require the manager to be selected too. Otherwise the linker gives errors like: ld: arch/x86/crypto/chacha_glue.o: in function `chacha_simd_stream_xor': chacha_glue.c:(.text+0x422): undefined reference to `crypto_simd_disabled_for_test' Fixes: 2343d152 ("crypto: Kconfig - allow tests to be disabled when manager is disabled") Signed-off-by:
Jason A. Donenfeld <Jason@zx2c4.com> Signed-off-by:
-
