- 24 Jan, 2024 9 commits
-
-
Clément Léger authored
Adding kprobes on some assembly functions (mainly exception handling) will result in crashes (either recursive trap or panic). To avoid such errors, add ASM_NOKPROBE() macro which allow adding specific symbols into the __kprobe_blacklist section and use to blacklist the following symbols that showed to be problematic: - handle_exception() - ret_from_exception() - handle_kernel_stack_overflow() Signed-off-by:
Clément Léger <cleger@rivosinc.com> Reviewed-by:
Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20231004131009.409193-1-cleger@rivosinc.comSigned-off-by:
Palmer Dabbelt <palmer@rivosinc.com>
-
Palmer Dabbelt authored
Jisheng Zhang <jszhang@kernel.org> says: This series adds fast gup support to riscv. The First patch fixes a bug in __p*d_free_tlb(). Per the riscv privileged spec, if non-leaf PTEs I.E pmd, pud or p4d is modified, a sfence.vma is a must. The 2nd patch is a preparation patch. The last two patches do the real work: In order to implement fast gup we need to ensure that the page table walker is protected from page table pages being freed from under it. riscv situation is more complicated than other architectures: some riscv platforms may use IPI to perform TLB shootdown, for example, those platforms which support AIA, usually the riscv_ipi_for_rfence is true on these platforms; some riscv platforms may rely on the SBI to perform TLB shootdown, usually the riscv_ipi_for_rfence is false on these platforms. To keep software pagetable walkers safe in this case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h for more details. This patch enables MMU_GATHER_RCU_TABLE_FREE, then use *tlb_remove_page_ptdesc() for those platforms which use IPI to perform TLB shootdown; *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB shootdown; Both case mean that disabling interrupts will block the free and protect the fast gup page walker. So after the 3rd patch, everything is well prepared, let's select HAVE_FAST_GUP if MMU. * b4-shazam-merge: riscv: enable HAVE_FAST_GUP if MMU riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU riscv: tlb: convert __p*d_free_tlb() to inline functions riscv: tlb: fix __p*d_free_tlb() Link: https://lore.kernel.org/r/20231219175046.2496-1-jszhang@kernel.orgSigned-off-by:
-
Jisheng Zhang authored
Activate the fast gup for riscv mmu platforms. Here are some GUP_FAST_BENCHMARK performance numbers: Before the patch: GUP_FAST_BENCHMARK: Time: get:53203 put:5085 us After the patch: GUP_FAST_BENCHMARK: Time: get:17711 put:5060 us The get time is reduced by 66.7%! IOW, 3x get speed! Signed-off-by:
Jisheng Zhang <jszhang@kernel.org> Reviewed-by:
Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20231219175046.2496-5-jszhang@kernel.orgSigned-off-by:
-
Jisheng Zhang authored
In order to implement fast gup we need to ensure that the page table walker is protected from page table pages being freed from under it. riscv situation is more complicated than other architectures: some riscv platforms may use IPI to perform TLB shootdown, for example, those platforms which support AIA, usually the riscv_ipi_for_rfence is true on these platforms; some riscv platforms may rely on the SBI to perform TLB shootdown, usually the riscv_ipi_for_rfence is false on these platforms. To keep software pagetable walkers safe in this case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h for more details. This patch enables MMU_GATHER_RCU_TABLE_FREE, then use *tlb_remove_page_ptdesc() for those platforms which use IPI to perform TLB shootdown; *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB shootdown; Both case mean that disabling interrupts will block the free and protect the fast gup page walker. Signed-off-by:
-
Jisheng Zhang authored
This is to prepare for enabling MMU_GATHER_RCU_TABLE_FREE. No functionality changes. Signed-off-by:
-
Jisheng Zhang authored
If non-leaf PTEs I.E pmd, pud or p4d is modified, a sfence.vma is a must for safe, imagine if an implementation caches the non-leaf translation in TLB, although I didn't meet this HW so far, but it's possible in theory. Signed-off-by:
-
Palmer Dabbelt authored
Sami Tolvanen <samitolvanen@google.com> says: We noticed that 64-bit RISC-V kernels limit mmap_rnd_bits to 24 even if the hardware supports a larger virtual address space size [1]. These two patches allow mmap_rnd_bits_max to be changed during init, and bumps up the maximum randomness if we end up setting up 4/5-level paging at boot. * b4-shazam-merge: riscv: mm: Update mmap_rnd_bits_max mm: Change mmap_rnd_bits_max to __ro_after_init Link: https://lore.kernel.org/r/20230929211155.3910949-4-samitolvanen@google.comSigned-off-by:
-
Sami Tolvanen authored
ARCH_MMAP_RND_BITS_MAX is based on Sv39, which leaves a few potential bits of mmap randomness on the table if we end up enabling 4/5-level paging. Update mmap_rnd_bits_max to take the final address space size into account. This increases mmap_rnd_bits_max from 24 to 33 with Sv48/57. Signed-off-by:
Sami Tolvanen <samitolvanen@google.com> Reviewed-by:
Kees Cook <keescook@chromium.org> Reviewed-by:
-
Sami Tolvanen authored
Allow mmap_rnd_bits_max to be updated on architectures that determine virtual address space size at runtime instead of relying on Kconfig options by changing it from const to __ro_after_init. Signed-off-by:
-
- 23 Jan, 2024 13 commits
-
-
Conor Dooley authored
Revert commit ed309ce5 ("RISC-V: mark hibernation as nonportable") as it appears the broken versions of OpenSBI have not made it to production on any systems that support hibernation. Signed-off-by:
Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230802-chef-throng-d9de8b672a49@wendySigned-off-by:
-
Vincent Chen authored
When registering the riscv-timer or clint-timer as a clock_event device, the driver needs to specify the value of max_delta_ticks. This value directly influences the max_delta_ns, which represents the maximum time interval for configuring subsequent clock events. Currently, both riscv-timer and clint-timer are set with a max_delta_ticks value of 0x7fff_ffff. When the timer operates at a high frequency, this values limists the system to sleep only for a short time. For the 1GHz case, the sleep cannot exceed two seconds. To address this limitation, refer to other timer implementations to extend it to 2^(bit-width of the timer) - 1. Because the bit-width of $mtimecmp is 64bit, this value becomes ULONG_MAX (0xffff_ffff_ffff_ffff). Signed-off-by:
Vincent Chen <vincent.chen@sifive.com> Link: https://lore.kernel.org/r/20230905070945.404653-1-vincent.chen@sifive.comSigned-off-by:
-
Palmer Dabbelt authored
Eric Biggers <ebiggers@kernel.org> says: This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions (https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf) and RISC-V vector extension (https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf). The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4. In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, little endian byte order, and vector unaligned access support. The ECB, CTR, XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. Building the assembly code requires tip-of-tree binutils (future 2.42) or tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923. It's been cleaned up for integration with the kernel, e.g. reducing code duplication, eliminating use of .inst and perlasm, and fixing a few bugs. This patchset incorporates the work of multiple people, including Jerry Shih, Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and myself. This patchset went through several versions from Heiko (last version https://lore.kernel.org/linux-crypto/20230711153743.1970625-1-heiko@sntech.de), then several versions from Jerry (last version: https://lore.kernel.org/linux-crypto/20231231152743.6304-1-jerry.shih@sifive.com), then finally several versions from me. Thanks to everyone who has contributed to this patchset or its prerequisites. * b4-shazam-merge: crypto: riscv - add vector crypto accelerated SM4 crypto: riscv - add vector crypto accelerated SM3 crypto: riscv - add vector crypto accelerated SHA-{512,384} crypto: riscv - add vector crypto accelerated SHA-{256,224} crypto: riscv - add vector crypto accelerated GHASH crypto: riscv - add vector crypto accelerated ChaCha20 crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} RISC-V: hook new crypto subdir into build-system RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO RISC-V: add helper function to read the vector VLEN Link: https://lore.kernel.org/r/20240122002024.27477-1-ebiggers@kernel.orgSigned-off-by:
-
Jerry Shih authored
Add an implementation of SM4 using the Zvksed extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by:
Christoph Müllner <christoph.muellner@vrull.eu> Signed-off-by:
Heiko Stuebner <heiko.stuebner@vrull.eu> Signed-off-by:
Jerry Shih <jerry.shih@sifive.com> Co-developed-by:
Eric Biggers <ebiggers@google.com> Signed-off-by:
-
Jerry Shih authored
Add an implementation of SM3 using the Zvksh extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by:
-
Jerry Shih authored
Add an implementation of SHA-512 and SHA-384 using the Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by:
Charalampos Mitrodimas <charalampos.mitrodimas@vrull.eu> Signed-off-by:
Phoebe Chen <phoebe.chen@sifive.com> Signed-off-by:
-
Jerry Shih authored
Add an implementation of SHA-256 and SHA-224 using the Zvknha or Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by:
-
Jerry Shih authored
Add an implementation of GHASH using the zvkg extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', reducing code duplication, and eliminating unnecessary endianness conversions. Co-developed-by:
-
Jerry Shih authored
Add an implementation of ChaCha20 using the Zvkb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and reducing code duplication. Signed-off-by:
-
Jerry Shih authored
Add implementations of AES-ECB, AES-CBC, AES-CTR, and AES-XTS, as well as bare (single-block) AES, using the RISC-V vector crypto extensions. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using regular .S files instead of the so-called perlasm, using the assembler instead of bare '.inst', greatly reducing code duplication, supporting AES-192, and making the code use the same AES key structure as the C code. Co-developed-by:
-
Heiko Stuebner authored
Create a crypto subdirectory for added accelerated cryptography routines and hook it into the riscv Kbuild and the main crypto Kconfig. Signed-off-by:
-
Eric Biggers authored
Add a kconfig symbol that indicates whether the toolchain supports the vector crypto extensions. This is needed by the RISC-V crypto code. Signed-off-by:
-
Heiko Stuebner authored
VLEN describes the length of each vector register and some instructions need specific minimal VLENs to work correctly. The vector code already includes a variable riscv_v_vsize that contains the value of "32 vector registers with vlenb length" that gets filled during boot. vlenb is the value contained in the CSR_VLENB register and the value represents "VLEN / 8". So add riscv_vector_vlen() to return the actual VLEN value for in-kernel users when they need to check the available VLEN. Signed-off-by:
-
- 22 Jan, 2024 2 commits
-
-
Wende Tan authored
Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an issue [1] in prior LLD versions that prevents LLD to generate proper machine code for RISC-V when writing `nop`s. To avoid boot failures in QEMU [2], '-mattr=+c' and '-mattr=+relax' need to be passed via '-mllvm' to ld.lld, as there appears to be an issue with LLVM's target-features and LTO [3], which can result in incorrect relocations to branch targets [4]. Once this is fixed in LLVM, it can be made conditional on affected ld.lld versions. Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF object file when manipulating the files in that subfolder, rather than LLVM bitcode. [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops") [2] https://github.com/ClangBuiltLinux/linux/issues/1942 [3] https://github.com/llvm/llvm-project/issues/59350 [4] https://github.com/llvm/llvm-project/issues/65090Tested-by:
Wende Tan <twd2.me@gmail.com> Signed-off-by:
Nathan Chancellor <nathan@kernel.org> Signed-off-by:
-
Masahiro Yamada authored
Commit 62694797 ("use linux/export.h rather than asm-generic/export.h") replaced deprecated <asm-generic/export.h> inclusions. Commit c2a658d4 ("riscv: lib: vectorize copy_to_user/copy_from_user") introduced a new instance of #include <asm-generic/export.h>. arch/riscv/lib/uaccess_vector.S does not use EXPORT_SYMBOL, hence this include directive is unneeded. Signed-off-by:
Masahiro Yamada <masahiroy@kernel.org> Link: https://lore.kernel.org/r/20240120213312.3033528-1-masahiroy@kernel.orgSigned-off-by:
-
- 21 Jan, 2024 16 commits
-
-
Linus Torvalds authored
-
https://evilpiepirate.org/git/bcachefsLinus Torvalds authored
Pull more bcachefs updates from Kent Overstreet: "Some fixes, Some refactoring, some minor features: - Assorted prep work for disk space accounting rewrite - BTREE_TRIGGER_ATOMIC: after combining our trigger callbacks, this makes our trigger context more explicit - A few fixes to avoid excessive transaction restarts on multithreaded workloads: fstests (in addition to ktest tests) are now checking slowpath counters, and that's shaking out a few bugs - Assorted tracepoint improvements - Starting to break up bcachefs_format.h and move on disk types so they're with the code they belong to; this will make room to start documenting the on disk format better. - A few minor fixes" * tag 'bcachefs-2024-01-21' of https://evilpiepirate.org/git/bcachefs: (46 commits) bcachefs: Improve inode_to_text() bcachefs: logged_ops_format.h bcachefs: reflink_format.h bcachefs; extents_format.h bcachefs: ec_format.h bcachefs: subvolume_format.h bcachefs: snapshot_format.h bcachefs: alloc_background_format.h bcachefs: xattr_format.h bcachefs: dirent_format.h bcachefs: inode_format.h bcachefs; quota_format.h bcachefs: sb-counters_format.h bcachefs: counters.c -> sb-counters.c bcachefs: comment bch_subvolume bcachefs: bch_snapshot::btime bcachefs: add missing __GFP_NOWARN bcachefs: opts->compression can now also be applied in the background bcachefs: Prep work for variable size btree node buffers bcachefs: grab s_umount only if snapshotting ... -
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull timer updates from Thomas Gleixner: "Updates for time and clocksources: - A fix for the idle and iowait time accounting vs CPU hotplug. The time is reset on CPU hotplug which makes the accumulated systemwide time jump backwards. - Assorted fixes and improvements for clocksource/event drivers" * tag 'timers-core-2024-01-21' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: tick-sched: Fix idle and iowait sleeptime accounting vs CPU hotplug clocksource/drivers/ep93xx: Fix error handling during probe clocksource/drivers/cadence-ttc: Fix some kernel-doc warnings clocksource/drivers/timer-ti-dm: Fix make W=n kerneldoc warnings clocksource/timer-riscv: Add riscv_clock_shutdown callback dt-bindings: timer: Add StarFive JH8100 clint dt-bindings: timer: thead,c900-aclint-mtimer: separate mtime and mtimecmp regs -
git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linuxLinus Torvalds authored
Pull powerpc fixes from Aneesh Kumar: - Increase default stack size to 32KB for Book3S Thanks to Michael Ellerman. * tag 'powerpc-6.8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: powerpc/64s: Increase default stack size to 32KB
-
Kent Overstreet authored
Add line breaks - inode_to_text() is now much easier to read. Signed-off-by:Kent Overstreet <kent.overstreet@linux.dev>
-
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by: -
Kent Overstreet authored
Signed-off-by:
-
