- 21 Aug, 2021 2 commits
-
-
Ben Hutchings authored
lockdep complains that in omap-aes, the list_lock is taken both with softirqs enabled at probe time, and also in softirq context, which could lead to a deadlock: ================================ WARNING: inconsistent lock state 5.14.0-rc1-00035-gc836005b01c5-dirty #69 Not tainted -------------------------------- inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. ksoftirqd/0/7 [HC0[0]:SC1[3]:HE1:SE0] takes: bf00e014 (list_lock){+.?.}-{2:2}, at: omap_aes_find_dev+0x18/0x54 [omap_aes_driver] {SOFTIRQ-ON-W} state was registered at: _raw_spin_lock+0x40/0x50 omap_aes_probe+0x1d4/0x664 [omap_aes_driver] platform_probe+0x58/0xb8 really_probe+0xbc/0x314 __driver_probe_device+0x80/0xe4 driver_probe_device+0x30/0xc8 __driver_attach+0x70/0xf4 bus_for_each_dev+0x70/0xb4 bus_add_driver+0xf0/0x1d4 driver_register+0x74/0x108 do_one_initcall+0x84/0x2e4 do_init_module+0x5c/0x240 load_module+0x221c/0x2584 sys_finit_module+0xb0/0xec ret_fast_syscall+0x0/0x2c 0xbed90b30 irq event stamp: 111800 hardirqs last enabled at (111800): [<c02a21e4>] __kmalloc+0x484/0x5ec hardirqs last disabled at (111799): [<c02a21f0>] __kmalloc+0x490/0x5ec softirqs last enabled at (111776): [<c01015f0>] __do_softirq+0x2b8/0x4d0 softirqs last disabled at (111781): [<c0135948>] run_ksoftirqd+0x34/0x50 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(list_lock); <Interrupt> lock(list_lock); *** DEADLOCK *** 2 locks held by ksoftirqd/0/7: #0: c0f5e8c8 (rcu_read_lock){....}-{1:2}, at: netif_receive_skb+0x6c/0x260 #1: c0f5e8c8 (rcu_read_lock){....}-{1:2}, at: ip_local_deliver_finish+0x2c/0xdc stack backtrace: CPU: 0 PID: 7 Comm: ksoftirqd/0 Not tainted 5.14.0-rc1-00035-gc836005b01c5-dirty #69 Hardware name: Generic AM43 (Flattened Device Tree) [<c010e6e0>] (unwind_backtrace) from [<c010b9d0>] (show_stack+0x10/0x14) [<c010b9d0>] (show_stack) from [<c017c640>] (mark_lock.part.17+0x5bc/0xd04) [<c017c640>] (mark_lock.part.17) from [<c017d9e4>] (__lock_acquire+0x960/0x2fa4) [<c017d9e4>] (__lock_acquire) from [<c0180980>] (lock_acquire+0x10c/0x358) [<c0180980>] (lock_acquire) from [<c093d324>] (_raw_spin_lock_bh+0x44/0x58) [<c093d324>] (_raw_spin_lock_bh) from [<bf00b258>] (omap_aes_find_dev+0x18/0x54 [omap_aes_driver]) [<bf00b258>] (omap_aes_find_dev [omap_aes_driver]) from [<bf00b328>] (omap_aes_crypt+0x94/0xd4 [omap_aes_driver]) [<bf00b328>] (omap_aes_crypt [omap_aes_driver]) from [<c08ac6d0>] (esp_input+0x1b0/0x2c8) [<c08ac6d0>] (esp_input) from [<c08c9e90>] (xfrm_input+0x410/0x1290) [<c08c9e90>] (xfrm_input) from [<c08b6374>] (xfrm4_esp_rcv+0x54/0x11c) [<c08b6374>] (xfrm4_esp_rcv) from [<c0838840>] (ip_protocol_deliver_rcu+0x48/0x3bc) [<c0838840>] (ip_protocol_deliver_rcu) from [<c0838c50>] (ip_local_deliver_finish+0x9c/0xdc) [<c0838c50>] (ip_local_deliver_finish) from [<c0838dd8>] (ip_local_deliver+0x148/0x1b0) [<c0838dd8>] (ip_local_deliver) from [<c0838f5c>] (ip_rcv+0x11c/0x180) [<c0838f5c>] (ip_rcv) from [<c077e3a4>] (__netif_receive_skb_one_core+0x54/0x74) [<c077e3a4>] (__netif_receive_skb_one_core) from [<c077e588>] (netif_receive_skb+0xa8/0x260) [<c077e588>] (netif_receive_skb) from [<c068d6d4>] (cpsw_rx_handler+0x224/0x2fc) [<c068d6d4>] (cpsw_rx_handler) from [<c0688ccc>] (__cpdma_chan_process+0xf4/0x188) [<c0688ccc>] (__cpdma_chan_process) from [<c068a0c0>] (cpdma_chan_process+0x3c/0x5c) [<c068a0c0>] (cpdma_chan_process) from [<c0690e14>] (cpsw_rx_mq_poll+0x44/0x98) [<c0690e14>] (cpsw_rx_mq_poll) from [<c0780810>] (__napi_poll+0x28/0x268) [<c0780810>] (__napi_poll) from [<c0780c64>] (net_rx_action+0xcc/0x204) [<c0780c64>] (net_rx_action) from [<c0101478>] (__do_softirq+0x140/0x4d0) [<c0101478>] (__do_softirq) from [<c0135948>] (run_ksoftirqd+0x34/0x50) [<c0135948>] (run_ksoftirqd) from [<c01583b8>] (smpboot_thread_fn+0xf4/0x1d8) [<c01583b8>] (smpboot_thread_fn) from [<c01546dc>] (kthread+0x14c/0x174) [<c01546dc>] (kthread) from [<c010013c>] (ret_from_fork+0x14/0x38) ... The omap-des and omap-sham drivers appear to have a similar issue. Fix this by using spin_{,un}lock_bh() around device list access in all the probe and remove functions. Signed-off-by:Ben Hutchings <ben.hutchings@mind.be> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Ben Hutchings authored
omap_crypto_cleanup() currently copies data from sg to orig if either copy flag is set. However OMAP_CRYPTO_SG_COPIED means that sg refers to the same pages as orig, truncated to len bytes. There is no need to copy in this case. Only copy data if the OMAP_CRYPTO_DATA_COPIED flag is set. Fixes: 74ed87e7 ("crypto: omap - add base support library for common ...") Signed-off-by:
-
- 12 Aug, 2021 8 commits
-
-
Randy Dunlap authored
Don't use "/**" to begin a comment that is not kernel-doc notation. crypto/wp512.c:779: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst * The core Whirlpool transform. Signed-off-by:
Randy Dunlap <rdunlap@infradead.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: linux-crypto@vger.kernel.org Signed-off-by:
-
Weili Qian authored
Kunpeng930 hpre device supports dynamic clock gating. When doing tasks, the algorithm core is opened, and when idle, the algorithm core is closed. This patch enables hpre dynamic clock gating by writing hardware registers. Signed-off-by:
Weili Qian <qianweili@huawei.com> Signed-off-by:
-
Weili Qian authored
Kunpeng930 sec device supports dynamic clock gating. When doing tasks, the algorithm core is opened, and when idle, the algorithm core is closed. This patch enables sec dynamic clock gating by writing hardware registers. Signed-off-by:
-
Weili Qian authored
Kunpeng930 zip device supports dynamic clock gating. When executing tasks, the algorithm core is opened, and when idle, the algorithm core is closed. This patch enables zip dynamic clock gating by writing hardware registers. Signed-off-by:
-
Hongbo Li authored
We should set the additional space to 0 in mpi_resize(). So use kcalloc() instead of kmalloc_array(). In lib/mpi/ec.c: /**************** * Resize the array of A to NLIMBS. the additional space is cleared * (set to 0) [done by m_realloc()] */ int mpi_resize(MPI a, unsigned nlimbs) Like the comment of kernel's mpi_resize() said, the additional space need to be set to 0, but when a->d is not NULL, it does not set. The kernel's mpi lib is from libgcrypt, the mpi resize in libgcrypt is _gcry_mpi_resize() which set the additional space to 0. This bug may cause mpi api which use mpi_resize() get wrong result under the condition of using the additional space without initiation. If this condition is not met, the bug would not be triggered. Currently in kernel, rsa, sm2 and dh use mpi lib, and they works well, so the bug is not triggered in these cases. add_points_edwards() use the additional space directly, so it will get a wrong result. Fixes: cdec9cb5 ("crypto: GnuPG based MPI lib - source files (part 1)") Signed-off-by:
Hongbo Li <herberthbli@tencent.com> Signed-off-by:
-
Sebastian Andrzej Siewior authored
The functions get_online_cpus() and put_online_cpus() have been deprecated during the CPU hotplug rework. They map directly to cpus_read_lock() and cpus_read_unlock(). Replace deprecated CPU-hotplug functions with the official version. The behavior remains unchanged. Cc: Steffen Klassert <steffen.klassert@secunet.com> Cc: Daniel Jordan <daniel.m.jordan@oracle.com> Cc: linux-crypto@vger.kernel.org Signed-off-by:
Sebastian Andrzej Siewior <bigeasy@linutronix.de> Acked-by:
Daniel Jordan <daniel.m.jordan@oracle.com> Signed-off-by:
-
Sebastian Andrzej Siewior authored
The functions get_online_cpus() and put_online_cpus() have been deprecated during the CPU hotplug rework. They map directly to cpus_read_lock() and cpus_read_unlock(). Replace deprecated CPU-hotplug functions with the official version. The behavior remains unchanged. Cc: Gonglei <arei.gonglei@huawei.com> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: virtualization@lists.linux-foundation.org Cc: linux-crypto@vger.kernel.org Signed-off-by:
-
Jason Wang authored
The kfree_sensitive is a kernel API to clear sensitive information that should not be leaked to other future users of the same memory objects and free the memory. Its function is the same as the combination of memzero_explicit and kfree. Thus, we can replace the combination APIs with the single kfree_sensitive API. Signed-off-by:
Jason Wang <wangborong@cdjrlc.com> Signed-off-by:
-
- 06 Aug, 2021 9 commits
-
-
Andre Przywara authored
The "Arm True Random Number Generator Firmware Interface"[1] provides an SMCCC based interface to a true hardware random number generator. So far we are using that in arch_get_random_seed(), but it might be useful to expose the entropy through the /dev/hwrng device as well. This allows to assess the quality of the implementation, by using "rngtest" from the rng-tools package, for example. Add a simple platform driver implementing the hw_random interface. The corresponding platform device is created by the SMCCC core code, we just match it here by name and provide a module alias. Since the firmware takes care about serialisation, this can happily coexist with the arch_get_random_seed() bits. [1] https://developer.arm.com/documentation/den0098/latest/Signed-off-by:
Andre Przywara <andre.przywara@arm.com> Reviewed-by:
Ard Biesheuvel <ardb@kernel.org> Reviewed-by:
Mark Brown <broonie@kernel.org> Signed-off-by:
-
Andre Przywara authored
At the moment we probe for the Random Number Generator SMCCC service, and use that in the core code (arch_get_random). However the hardware entropy can also be useful to access from userland, and be it to assess its quality. Register a platform device when the SMCCC TRNG service is detected, to allow a hw_random driver to hook onto this. The function registering the device is deliberately made in a way which allows expansion, so other services that could be exposed via a platform device (or some other interface), can be added here easily. Signed-off-by:
-
Brijesh Singh authored
The commit 97f9ac3d ("crypto: ccp - Add support for SEV-ES to the PSP driver") added support to allocate Trusted Memory Region (TMR) used during the SEV-ES firmware initialization. The TMR gets locked during the firmware initialization and unlocked during the shutdown. While the TMR is locked, access to it is disallowed. Currently, the CCP driver does not shutdown the firmware during the kexec reboot, leaving the TMR memory locked. Register a callback to shutdown the SEV firmware on the kexec boot. Fixes: 97f9ac3d ("crypto: ccp - Add support for SEV-ES to the PSP driver") Reported-by:
Lucas Nussbaum <lucas.nussbaum@inria.fr> Tested-by:
Brijesh Singh <brijesh.singh@amd.com> Acked-by:
Tom Lendacky <thomas.lendacky@gmail.com> Signed-off-by:
-
Tony Lindgren authored
Commit b0a3d898 ("crypto: omap-sham - Use pm_runtime_irq_safe()") added the use of pm_runtime_irq_safe() as pm_runtime_get_sync() was called from a tasklet. We now use the crypto engine queue instead of a custom queue since commit 33c3d434d91 ("crypto: omap-sham - convert to use crypto engine"). We want to drop the use of pm_runtime_irq_safe() in general as it takes a permanent usage count on the parent device causing issues for power management. Based on testing with CONFIG_DEBUG_ATOMIC_SLEEP=y, modprobe omap-sham, followed by modprobe tcrypt sec=1 mode=423, I have not been able to reproduce the scheduling while atomic issue seen earlier with current kernels and we can just drop the call to pm_runtime_irq_safe(). Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
Tony Lindgren <tony@atomide.com> Signed-off-by:
-
Tony Lindgren authored
Let's get rid of the suspend and resume calls to runtime PM as these calls do not idle the hardware. The runtime suspend has been disabled for system suspend since commit 88d26136 ("PM: Prevent runtime suspend during system resume"). Instead of runtime PM, the system suspend and resume functions should call driver internal shared functions to idle the hardware as needed. Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
-
Tony Lindgren authored
FLAGS_INIT is now unused and we can just use standard runtime PM functions instead. Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
-
Tony Lindgren authored
We should pair the usage of pm_runtime_use_autosuspend() with pm_runtime_dont_use_autosuspend(). Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
-
Tony Lindgren authored
Let's only initialize dd->req after omap_sham_hw_init() in case of errors. Looks like leaving dd->req initialized on omap_sham_hw_init() errors is is not causing issues though as we return on errors. So this patch can be applied as clean-up. Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
-
Tony Lindgren authored
We should not clear FLAGS_DMA_ACTIVE before omap_sham_update_dma_stop() is done calling dma_unmap_sg(). We already clear FLAGS_DMA_ACTIVE at the end of omap_sham_update_dma_stop(). The early clearing of FLAGS_DMA_ACTIVE is not causing issues as we do not need to defer anything based on FLAGS_DMA_ACTIVE currently. So this can be applied as clean-up. Cc: Lokesh Vutla <lokeshvutla@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by:
-
- 30 Jul, 2021 17 commits
-
-
Salah Triki authored
Use swap() instead of implementing it in order to make code more clean. Signed-off-by:
Salah Triki <salah.triki@gmail.com> Reviewed-by:
Tudor Ambarus <tudor.ambarus@microchip.com> Signed-off-by:
-
Mian Yousaf Kaukab authored
ecdsa_set_pub_key() makes an u64 pointer at 1 byte offset of the key. This results in an unaligned u64 pointer. This pointer is passed to ecc_swap_digits() which assumes natural alignment. This causes a kernel crash on an armv7 platform: [ 0.409022] Unhandled fault: alignment exception (0x001) at 0xc2a0a6a9 ... [ 0.416982] PC is at ecdsa_set_pub_key+0xdc/0x120 ... [ 0.491492] Backtrace: [ 0.492059] [<c07c266c>] (ecdsa_set_pub_key) from [<c07c75d4>] (test_akcipher_one+0xf4/0x6c0) Handle unaligned input buffer in ecc_swap_digits() by replacing be64_to_cpu() to get_unaligned_be64(). Change type of in pointer to void to reflect it doesn’t necessarily need to be aligned. Fixes: 4e660291 ("crypto: ecdsa - Add support for ECDSA signature verification") Reported-by:
Guillaume Gardet <guillaume.gardet@arm.com> Suggested-by:
Takashi Iwai <tiwai@suse.de> Signed-off-by:
Mian Yousaf Kaukab <ykaukab@suse.de> Tested-by:
Stefan Berger <stefanb@linux.ibm.com> Signed-off-by:
-
Xiyu Yang authored
refcount_t type and corresponding API can protect refcounters from accidental underflow and overflow and further use-after-free situations. Signed-off-by:
Xiyu Yang <xiyuyang19@fudan.edu.cn> Signed-off-by:
Xin Tan <tanxin.ctf@gmail.com> Acked-by:
-
Jason Wang authored
The kfree_sensitive is a kernel API to clear sensitive information that should not be leaked to other future users of the same memory objects and free the memory. Its function is the same as the combination of memzero_explicit and kfree. Thus, we can replace the combination APIs with the single kfree_sensitive API. Signed-off-by:
-
Tudor Ambarus authored
Allocate the atmel_aes_dev data at tfm init time, and not for each crypt request. There's a single AES IP per SoC, clarify that in the code. Signed-off-by:
-
Tudor Ambarus authored
XTS is supported just for input lengths with data units of 128-bit blocks. Add a fallback to software implementation when the last block is shorter than 128 bits. Signed-off-by:
-
Tudor Ambarus authored
Set cra_blocksize to 1 to indicate OFB is a stream cipher. Signed-off-by:
-
Tudor Ambarus authored
FIPS81 requires for the ECB, CBC, CFB, and OFB modes that the plaintext and ciphertext to have a positive integer length. Add this constraint and just return 0 for a zero length cryptlen. Signed-off-by:
-
Tudor Ambarus authored
NIST 800-38A requires for the ECB, CBC, CFB, OFB and CTR modes that the plaintext and ciphertext to have a positive integer length. Signed-off-by:
-
Tudor Ambarus authored
Input length smaller than block size does not make sense for XTS. Signed-off-by:
-
Tudor Ambarus authored
NIST 800-38A requires for the ECB and CBC modes that the total number of bits in the plaintext to be a multiple of the block cipher. Signed-off-by:
-
Tudor Ambarus authored
Downgrade all runtime error messages to dev_dbg so that we don't pollute the console. All probe error messages are kept with dev_err. Get rid of pr_err and use dev_dbg instead, so that we know from which device the error comes. dma_mapping_error() return code was overwritten, use the error code that the function returns. Signed-off-by:
-
Tudor Ambarus authored
The tdes dev gets allocated to the tfm at alg->init time, there's no need to overwrite the pointer to tdes_dd afterwards. There's a single IP per SoC anyway, the first entry from the atmel_tdes.dev_list is chosen without counting for tfms for example, in case one thinks of an even distribution of tfms across the TDES IPs: there's only one. At alg->init time the ctx->dd should already be NULL, there's no need to check its value before requesting for a tdes dev. Signed-off-by:
-
Tianjia Zhang authored
tcrypt supports testing of SM4 cipher algorithms that use avx instruction set acceleration. The implementation of sm4 instruction set acceleration supports up to 8 blocks in parallel encryption and decryption, which is 128 bytes. Therefore, the 128-byte block size is also added to block_sizes. Signed-off-by:
Tianjia Zhang <tianjia.zhang@linux.alibaba.com> Signed-off-by:
-
Tianjia Zhang authored
This patch adds AES-NI/AVX/x86_64 assembler implementation of SM4 block cipher. Through two affine transforms, we can use the AES S-Box to simulate the SM4 S-Box to achieve the effect of instruction acceleration. The main algorithm implementation comes from SM4 AES-NI work by libgcrypt and Markku-Juhani O. Saarinen at: https://github.com/mjosaarinen/sm4ni This optimization supports the four modes of SM4, ECB, CBC, CFB, and CTR. Since CBC and CFB do not support multiple block parallel encryption, the optimization effect is not obvious. Benchmark on Intel Xeon Cascadelake, the data comes from the 218 mode and 518 mode of tcrypt. The abscissas are blocks of different lengths. The data is tabulated and the unit is Mb/s: sm4-generic | 16 64 128 256 1024 1420 4096 ECB enc | 40.99 46.50 48.05 48.41 49.20 49.25 49.28 ECB dec | 41.07 46.99 48.15 48.67 49.20 49.25 49.29 CBC enc | 37.71 45.28 46.77 47.60 48.32 48.37 48.40 CBC dec | 36.48 44.82 46.43 47.45 48.23 48.30 48.36 CFB enc | 37.94 44.84 46.12 46.94 47.57 47.46 47.68 CFB dec | 37.50 42.84 43.74 44.37 44.85 44.80 44.96 CTR enc | 39.20 45.63 46.75 47.49 48.09 47.85 48.08 CTR dec | 39.64 45.70 46.72 47.47 47.98 47.88 48.06 sm4-aesni-avx ECB enc | 33.75 134.47 221.64 243.43 264.05 251.58 258.13 ECB dec | 34.02 134.92 223.11 245.14 264.12 251.04 258.33 CBC enc | 38.85 46.18 47.67 48.34 49.00 48.96 49.14 CBC dec | 33.54 131.29 223.88 245.27 265.50 252.41 263.78 CFB enc | 38.70 46.10 47.58 48.29 49.01 48.94 49.19 CFB dec | 32.79 128.40 223.23 244.87 265.77 253.31 262.79 CTR enc | 32.58 122.23 220.29 241.16 259.57 248.32 256.69 CTR dec | 32.81 122.47 218.99 241.54 258.42 248.58 256.61 Signed-off-by:
-
Tianjia Zhang authored
SM4 library is abstracted from sm4-generic algorithm, sm4-ce can depend on the SM4 library instead of sm4-generic, and some functions in sm4-generic do not need to be exported. Signed-off-by:
-
Tianjia Zhang authored
Take the existing small footprint and mostly time invariant C code and turn it into a SM4 library that can be used for non-performance critical, casual use of SM4, and as a fallback for, e.g., SIMD code that needs a secondary path that can be taken in contexts where the SIMD unit is off limits. Secondly, some codes have been optimized, such as unrolling small times loop, removing unnecessary memory shifts, exporting sbox, fk, ck arrays, and basic encryption and decryption functions. Signed-off-by:
-
- 23 Jul, 2021 1 commit
-
-
Ard Biesheuvel authored
The updated XTS code fails to check the return code of skcipher_walk_virt, which may lead to skcipher_walk_abort() or skcipher_walk_done() being called while the walk argument is in an inconsistent state. So check the return value after each such call, and bail on errors. Fixes: 2481104f ("crypto: x86/aes-ni-xts - rewrite and drop indirections via glue helper") Reported-by:
Dave Hansen <dave.hansen@intel.com> Reported-by:
syzbot <syzbot+5d1bad8042a8f0e8117a@syzkaller.appspotmail.com> Signed-off-by:
Eric Biggers <ebiggers@google.com> Signed-off-by:
-
- 16 Jul, 2021 3 commits
-
-
Randy Dunlap authored
Rename module_init & module_exit functions that are named "mod_init" and "mod_exit" so that they are unique in both the System.map file and in initcall_debug output instead of showing up as almost anonymous "mod_init". This is helpful for debugging and in determining how long certain module_init calls take to execute. Signed-off-by:
-
Randy Dunlap authored
Rename module_init & module_exit functions that are named "mod_init" and "mod_exit" so that they are unique in both the System.map file and in initcall_debug output instead of showing up as almost anonymous "mod_init". This is helpful for debugging and in determining how long certain module_init calls take to execute. Signed-off-by:
-
Randy Dunlap authored
Rename module_init & module_exit functions that are named "mod_init" and "mod_exit" so that they are unique in both the System.map file and in initcall_debug output instead of showing up as almost anonymous "mod_init". This is helpful for debugging and in determining how long certain module_init calls take to execute. Signed-off-by:
Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Signed-off-by:
-
