An error occurred fetching the project authors.
- 21 Oct, 2021 7 commits
-
-
Mark Brown authored
Currently when restoring the SVE state we supply the SVE vector length as an argument to sve_load_state() and the underlying macros. This becomes inconvenient with the addition of SME since we may need to restore any combination of SVE and SME vector lengths, and we already separately restore the vector length in the KVM code. We don't need to know the vector length during the actual register load since the SME load instructions can index into the data array for us. Refactor the interface so we explicitly set the vector length separately to restoring the SVE registers in preparation for adding SME support, no functional change should be involved. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-9-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
With the introduction of SME we will have a second vector length in the system, enumerated and configured in a very similar fashion to the existing SVE vector length. While there are a few differences in how things are handled this is a relatively small portion of the overall code so in order to avoid code duplication we factor out We create two structs, one vl_info for the static hardware properties and one vl_config for the runtime configuration, with an array instantiated for each and update all the users to reference these. Some accessor functions are provided where helpful for readability, and the write to set the vector length is put into a function since the system register being updated needs to be chosen at compile time. This is a mostly mechanical replacement, further work will be required to actually make things generic, ensuring that we handle those places where there are differences properly. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-8-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
In a system with SME there are parallel vector length controls for SVE and SME vectors which function in much the same way so it is desirable to share the code for handling them as much as possible. In order to prepare for doing this add a layer of accessor functions for the various VL related operations on tasks. Since almost all current interactions are actually via task->thread rather than directly with the thread_info the accessors use that. Accessors are provided for both generic and SVE specific usage, the generic accessors should be used for cases where register state is being manipulated since the registers are shared between streaming and regular SVE so we know that when SME support is implemented we will always have to be in the appropriate mode already and hence can generalise now. Since we are using task_struct and we don't want to cause widespread inclusion of sched.h the acessors are all out of line, it is hoped that none of the uses are in a sufficiently critical path for this to be an issue. Those that are most likely to present an issue are in the same translation unit so hopefully the compiler may be able to inline anyway. This is purely adding the layer of abstraction, additional work will be needed to support tasks using SME. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-7-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
The function has SVE specific checks in it and it will be more trouble to add conditional code for SME than it is to simply rename it to be SVE specific. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-6-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
SME introduces streaming SVE mode in which FFR is not present and the instructions for accessing it UNDEF. In preparation for handling this update the low level SVE state access functions to take a flag specifying if FFR should be handled. When saving the register state we store a zero for FFR to guard against uninitialized data being read. No behaviour change should be introduced by this patch. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-5-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
There are no users outside fpsimd.c so make sve_state_size() static. KVM open codes an equivalent. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-4-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
Mark Brown authored
Currently all the active code in fpsimd_save() is inside a check for TIF_FOREIGN_FPSTATE. Reduce the indentation level by changing to return from the function if TIF_FOREIGN_FPSTATE is set. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20211019172247.3045838-2-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
- 16 Sep, 2021 1 commit
-
-
Mark Brown authored
When we need a buffer for SVE register state we call sve_alloc() to make sure that one is there. In order to avoid repeated allocations and frees we keep the buffer around unless we change vector length and just memset() it to ensure a clean register state. The function that deals with this takes the task to operate on as an argument, however in the case where we do a memset() we initialise using the SVE state size for the current task rather than the task passed as an argument. This is only an issue in the case where we are setting the register state for a task via ptrace and the task being configured has a different vector length to the task tracing it. In the case where the buffer is larger in the traced process we will leak old state from the traced process to itself, in the case where the buffer is smaller in the traced process we will overflow the buffer and corrupt memory. Fixes: bc0ee476 ("arm64/sve: Core task context handling") Cc: <stable@vger.kernel.org> # 4.15.x Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210909165356.10675-1-broonie@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 24 Aug, 2021 1 commit
-
-
Mark Brown authored
Currently we "handle" failure to allocate the SVE register storage by doing a BUG_ON() and hoping for the best. This is obviously not great and the memory allocation failure will already be loud enough without the BUG_ON(). As the comment says it is a corner case but let's try to do a bit better, remove the BUG_ON() and add code to handle the failure in the callers. For the ptrace and signal code we can return -ENOMEM gracefully however we have no real error reporting path available to us for the SVE access trap so instead generate a SIGKILL if the allocation fails there. This at least means that we won't try to soldier on and end up trying to access the nonexistant state and while it's obviously not ideal for userspace SIGKILL doesn't allow any handling so minimises the ABI impact, making it easier to improve the interface later if we come up with a better idea. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210824153417.18371-1-broonie@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 02 Aug, 2021 1 commit
-
-
Mark Brown authored
This function is not referenced outside fpsimd.c so can be static, making it that little bit easier to follow what is called from where. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210730165846.18558-1-broonie@kernel.orgReviewed-by:
Dave Martin <Dave.Martin@arm.com> Signed-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 26 May, 2021 1 commit
-
-
Mark Brown authored
When the SVE vector length is 128 bits then there are no bits in the Z registers which are not shared with the V registers so we can skip them when zeroing state not shared with FPSIMD, this results in a minor performance improvement. Signed-off-by:
Mark Brown <broonie@kernel.org> Reviewed-by:
Dave Martin <Dave.Martin@arm.com> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/20210512151131.27877-4-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
- 15 Apr, 2021 1 commit
-
-
Mark Brown authored
The FPSIMD code was relying on IS_ENABLED() checks in system_suppors_sve() to cause the compiler to delete references to SVE functions in some places, add explicit IS_ENABLED() checks back. Fixes: ef9c5d09 ("arm64/sve: Remove redundant system_supports_sve() tests") Reported-by:
kernel test robot <lkp@intel.com> Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210415121742.36628-1-broonie@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 13 Apr, 2021 1 commit
-
-
Mark Brown authored
Currently there are a number of places in the SVE code where we check both system_supports_sve() and TIF_SVE. This is a bit redundant given that we should never get into a situation where we have set TIF_SVE without having SVE support and it is not clear that silently ignoring a mistakenly set TIF_SVE flag is the most sensible error handling approach. For now let's just drop the system_supports_sve() checks since this will at least reduce overhead a little. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210412172320.3315-1-broonie@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 12 Apr, 2021 1 commit
-
-
Ard Biesheuvel authored
Kernel mode NEON can be used in task or softirq context, but only in a non-nesting manner, i.e., softirq context is only permitted if the interrupt was not taken at a point where the kernel was using the NEON in task context. This means all users of kernel mode NEON have to be aware of this limitation, and either need to provide scalar fallbacks that may be much slower (up to 20x for AES instructions) and potentially less safe, or use an asynchronous interface that defers processing to a later time when the NEON is guaranteed to be available. Given that grabbing and releasing the NEON is cheap, we can relax this restriction, by increasing the granularity of kernel mode NEON code, and always disabling softirq processing while the NEON is being used in task context. Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Acked-by:
Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20210302090118.30666-4-ardb@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 08 Apr, 2021 1 commit
-
-
Mark Brown authored
When we enable SVE usage in userspace after taking a SVE access trap we need to ensure that the portions of the register state that are not shared with the FPSIMD registers are zeroed. Currently we do this by forcing the FPSIMD registers to be saved to the task struct and converting them there. This is wasteful in the common case where the task state is loaded into the registers and we will immediately return to userspace since we can initialise the SVE state directly in registers instead of accessing multiple copies of the register state in memory. Instead in that common case do the conversion in the registers and update the task metadata so that we can return to userspace without spilling the register state to memory unless there is some other reason to do so. Signed-off-by:
Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20210312190313.24598-1-broonie@kernel.orgSigned-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 15 Oct, 2020 1 commit
-
-
Peter Collingbourne authored
It doesn't make sense to issue prctl(PR_PAC_RESET_KEYS) on a compat task because the 32-bit instruction set does not offer PAuth instructions. For consistency with other 64-bit only prctls such as {SET,GET}_TAGGED_ADDR_CTRL, reject the prctl on compat tasks. Although this is a userspace-visible change, maybe it isn't too late to make this change given that the hardware isn't available yet and it's very unlikely that anyone has 32-bit software that actually depends on this succeeding. Signed-off-by:
Peter Collingbourne <pcc@google.com> Link: https://linux-review.googlesource.com/id/Ie885a1ff84ab498cc9f62d6451e9f2cfd4b1d06a Link: https://lore.kernel.org/r/20201014052430.11630-1-pcc@google.com [will: Do the same for the SVE prctl()s] Signed-off-by:
Will Deacon <will@kernel.org>
-
- 21 Sep, 2020 1 commit
-
-
Julien Grall authored
fpsimd_restore_current_state() enables and disables the SVE access trap based on TIF_SVE, not task_fpsimd_load(). Update the documentation of do_sve_acc to reflect this behavior. Signed-off-by:
Julien Grall <julien.grall@arm.com> Signed-off-by:
Mark Brown <broonie@kernel.org> Reviewed-by:
Dave Martin <Dave.Martin@arm.com> Link: https://lore.kernel.org/r/20200828181155.17745-2-broonie@kernel.orgSigned-off-by:
Will Deacon <will@kernel.org>
-
- 18 Sep, 2020 1 commit
-
-
Tian Tao authored
Fix the following warnings. arch/arm64/kernel/fpsimd.c:935:6: warning: no previous prototype for ‘do_sve_acc’ [-Wmissing-prototypes] arch/arm64/kernel/fpsimd.c:962:6: warning: no previous prototype for ‘do_fpsimd_acc’ [-Wmissing-prototypes] arch/arm64/kernel/fpsimd.c:971:6: warning: no previous prototype for ‘do_fpsimd_exc’ [-Wmissing-prototypes] arch/arm64/kernel/fpsimd.c:1266:6: warning: no previous prototype for ‘kernel_neon_begin’ [-Wmissing-prototypes] arch/arm64/kernel/fpsimd.c:1292:6: warning: no previous prototype for ‘kernel_neon_end’ [-Wmissing-prototypes] Signed-off-by:
Tian Tao <tiantao6@hisilicon.com> Reviewed-by:
Dave Martin <Dave.Martin@arm.com> Link: https://lore.kernel.org/r/1600157999-14802-1-git-send-email-tiantao6@hisilicon.comSigned-off-by:
Will Deacon <will@kernel.org>
-
- 14 Sep, 2020 1 commit
-
-
Amit Daniel Kachhap authored
Some error signal need to pass proper ARM esr error code to userspace to better identify the cause of the signal. So the function force_signal_inject is extended to pass this as a parameter. The existing code is not affected by this change. Signed-off-by:
Amit Daniel Kachhap <amit.kachhap@arm.com> Reviewed-by:
Dave Martin <Dave.Martin@arm.com> Link: https://lore.kernel.org/r/20200914083656.21428-3-amit.kachhap@arm.comSigned-off-by:
Will Deacon <will@kernel.org>
-
- 16 Jun, 2020 1 commit
-
-
Will Deacon authored
When I squashed the 'allnoconfig' compiler warning about the set_sve_default_vl() function being defined but not used in commit 1e570f51 ("arm64/sve: Eliminate data races on sve_default_vl"), I accidentally broke the build for configs where ARM64_SVE is enabled, but SYSCTL is not. Fix this by only compiling the SVE sysctl support if both CONFIG_SVE=y and CONFIG_SYSCTL=y. Cc: Dave Martin <Dave.Martin@arm.com> Reported-by:
Qian Cai <cai@lca.pw> Link: https://lore.kernel.org/r/20200616131808.GA1040@lca.pwSigned-off-by:
Will Deacon <will@kernel.org>
-
- 15 Jun, 2020 1 commit
-
-
Dave Martin authored
sve_default_vl can be modified via the /proc/sys/abi/sve_default_vl sysctl concurrently with use, and modified concurrently by multiple threads. Adding a lock for this seems overkill, and I don't want to think any more than necessary, so just define wrappers using READ_ONCE()/ WRITE_ONCE(). This will avoid the possibility of torn accesses and repeated loads and stores. There's no evidence yet that this is going wrong in practice: this is just hygiene. For generic sysctl users, it would be better to build this kind of thing into the sysctl common code somehow. Reported-by:
Will Deacon <will@kernel.org> Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Link: https://lore.kernel.org/r/1591808590-20210-3-git-send-email-Dave.Martin@arm.com [will: move set_sve_default_vl() inside #ifdef to squash allnoconfig warning] Signed-off-by:
Will Deacon <will@kernel.org>
-
- 27 Apr, 2020 1 commit
-
-
Christoph Hellwig authored
Instead of having all the sysctl handlers deal with user pointers, which is rather hairy in terms of the BPF interaction, copy the input to and from userspace in common code. This also means that the strings are always NUL-terminated by the common code, making the API a little bit safer. As most handler just pass through the data to one of the common handlers a lot of the changes are mechnical. Signed-off-by:
Christoph Hellwig <hch@lst.de> Acked-by:
Andrey Ignatov <rdna@fb.com> Signed-off-by:
Al Viro <viro@zeniv.linux.org.uk>
-
- 14 Jan, 2020 1 commit
-
-
Suzuki K Poulose authored
We detect the absence of FP/SIMD after an incapable CPU is brought up, and by then we have kernel threads running already with TIF_FOREIGN_FPSTATE set which could be set for early userspace applications (e.g, modprobe triggered from initramfs) and init. This could cause the applications to loop forever in do_nofity_resume() as we never clear the TIF flag, once we now know that we don't support FP. Fix this by making sure that we clear the TIF_FOREIGN_FPSTATE flag for tasks which may have them set, as we would have done in the normal case, but avoiding touching the hardware state (since we don't support any). Also to make sure we handle the cases seemlessly we categorise the helper functions to two : 1) Helpers for common core code, which calls into take appropriate actions without knowing the current FPSIMD state of the CPU/task. e.g fpsimd_restore_current_state(), fpsimd_flush_task_state(), fpsimd_save_and_flush_cpu_state(). We bail out early for these functions, taking any appropriate actions (e.g, clearing the TIF flag) where necessary to hide the handling from core code. 2) Helpers used when the presence of FP/SIMD is apparent. i.e, save/restore the FP/SIMD register state, modify the CPU/task FP/SIMD state. e.g, fpsimd_save(), task_fpsimd_load() - save/restore task FP/SIMD registers fpsimd_bind_task_to_cpu() \ - Update the "state" metadata for CPU/task. fpsimd_bind_state_to_cpu() / fpsimd_update_current_state() - Update the fp/simd state for the current task from memory. These must not be called in the absence of FP/SIMD. Put in a WARNING to make sure they are not invoked in the absence of FP/SIMD. KVM also uses the TIF_FOREIGN_FPSTATE flag to manage the FP/SIMD state on the CPU. However, without FP/SIMD support we trap all accesses and inject undefined instruction. Thus we should never "load" guest state. Add a sanity check to make sure this is valid. Fixes: 82e0191a ("arm64: Support systems without FP/ASIMD") Cc: Will Deacon <will@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Reviewed-by:
Ard Biesheuvel <ardb@kernel.org> Reviewed-by:
Catalin Marinas <catalin.marinas@arm.com> Acked-by:
Marc Zyngier <maz@kernel.org> Signed-off-by:
Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by:
Will Deacon <will@kernel.org>
-
- 28 Oct, 2019 1 commit
-
-
James Morse authored
Now that the callers of these functions have moved into C, they no longer need the asmlinkage annotation. Remove it. Signed-off-by:
James Morse <james.morse@arm.com> Acked-by:
Mark Rutland <mark.rutland@arm.com> Signed-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 22 Jul, 2019 2 commits
-
-
Dave Martin authored
There are some hand-written instances of "32" to express the number of SVE Z-registers. Since this code was written a #define was added for this, so convert trivial instances of this magic number as appropriate. No functional change. Reviewed-by:
Julien Grall <julien.grall@arm.com> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Signed-off-by:
Will Deacon <will@kernel.org>
-
Dave Martin authored
Currently we convert from FPSIMD to SVE register state in memory in two places. To ease future maintenance, let's consolidate this in one place. Reviewed-by:
Julien Grall <julien.grall@arm.com> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Signed-off-by:
Will Deacon <will@kernel.org>
-
- 19 Jun, 2019 1 commit
-
-
Thomas Gleixner authored
Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not see http www gnu org licenses extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 503 file(s). Signed-off-by:
Thomas Gleixner <tglx@linutronix.de> Reviewed-by:
Alexios Zavras <alexios.zavras@intel.com> Reviewed-by:
Allison Randal <allison@lohutok.net> Reviewed-by:
Enrico Weigelt <info@metux.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.deSigned-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-
- 13 Jun, 2019 1 commit
-
-
Dave Martin authored
The in-memory representation of SVE and FPSIMD registers is different: the FPSIMD V-registers are stored as single 128-bit host-endian values, whereas SVE registers are stored in an endianness-invariant byte order. This means that the two representations differ when running on a big-endian host. But we blindly copy data from one representation to another when converting between the two, resulting in the register contents being unintentionally byteswapped in certain situations. Currently this can be triggered by the first SVE instruction after a syscall, for example (though the potential trigger points may vary in future). So, fix the conversion functions fpsimd_to_sve(), sve_to_fpsimd() and sve_sync_from_fpsimd_zeropad() to swab where appropriate. There is no common swahl128() or swab128() that we could use here. Maybe it would be worth making this generic, but for now add a simple local hack. Since the byte order differences are exposed in ABI, also clarify the documentation. Cc: Alex Bennée <alex.bennee@linaro.org> Cc: Peter Maydell <peter.maydell@linaro.org> Cc: Alan Hayward <alan.hayward@arm.com> Cc: Julien Grall <julien.grall@arm.com> Fixes: bc0ee476 ("arm64/sve: Core task context handling") Fixes: 8cd969d2 ("arm64/sve: Signal handling support") Fixes: 43d4da2c ("arm64/sve: ptrace and ELF coredump support") Signed-off-by:
Dave Martin <Dave.Martin@arm.com> [will: Fix typos in comments and docs spotted by Julien] Signed-off-by:
Will Deacon <will.deacon@arm.com>
-
- 04 Jun, 2019 2 commits
-
-
Julien Grall authored
When the kernel is compiled with CONFIG_KERNEL_MODE_NEON, some part of the kernel may be able to use FPSIMD/SVE. This is for instance the case for crypto code. Any use of FPSIMD/SVE in the kernel are clearly marked by using the function kernel_neon_{begin, end}. Furthermore, this can only be used when may_use_simd() returns true. The current implementation of may_use_simd() allows softirq to use FPSIMD/SVE unless it is currently in use (i.e kernel_neon_busy is true). When in use, softirqs usually fall back to a software method. At the moment, as a softirq may use FPSIMD/SVE, softirqs are disabled when touching the FPSIMD/SVE context. This has the drawback to disable all softirqs even if they are not using FPSIMD/SVE. Since a softirq is supposed to check may_use_simd() anyway before attempting to use FPSIMD/SVE, there is limited reason to keep softirq disabled when touching the FPSIMD/SVE context. Instead, we can simply disable preemption and mark the FPSIMD/SVE context as in use by setting CPU's fpsimd_context_busy flag. Two new helpers {get, put}_cpu_fpsimd_context are introduced to mark the area using FPSIMD/SVE context and they are used to replace local_bh_{disable, enable}. The functions kernel_neon_{begin, end} are also re-implemented to use the new helpers. Additionally, double-underscored versions of the helpers are provided to called when preemption is already disabled. These are only relevant on paths where irqs are disabled anyway, so they are not needed for correctness in the current code. Let's use them anyway though: this marks critical sections clearly and will help to avoid mistakes during future maintenance. The change has been benchmarked on Linux 5.1-rc4 with defconfig. On Juno2: * hackbench 100 process 1000 (10 times) * .7% quicker On ThunderX 2: * hackbench 1000 process 1000 (20 times) * 3.4% quicker Reviewed-by:
Dave Martin <dave.martin@arm.com> Acked-by:
Marc Zyngier <marc.zyngier@arm.com> Signed-off-by:
Julien Grall <julien.grall@arm.com> Signed-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
Julien Grall authored
The only external user of fpsimd_save() and fpsimd_flush_cpu_state() is the KVM FPSIMD code. A following patch will introduce a mechanism to acquire owernship of the FPSIMD/SVE context for performing context management operations. Rather than having to export the new helpers to get/put the context, we can just introduce a new function to combine fpsimd_save() and fpsimd_flush_cpu_state(). This has also the advantage to remove any external call of fpsimd_save() and fpsimd_flush_cpu_state(), so they can be turned static. Lastly, the new function can also be used in the PM notifier. Reviewed-by:
Dave Martin <Dave.Martin@arm.com> Acked-by:
Marc Zyngier <marc.zyngier@arm.com> Signed-off-by:
Julien Grall <julien.grall@arm.com> Signed-off-by:
Catalin Marinas <catalin.marinas@arm.com>
-
- 18 Apr, 2019 1 commit
-
-
Dave Martin authored
Currently the meanings of sve_vq_map and the ancillary helpers __bit_to_vq() and __vq_to_bit() are not clearly explained. This patch makes the explanatory comment clearer, and removes the duplicate comment from fpsimd.h. The WARN_ON() currently present in __bit_to_vq() confuses the intended use of this helper. Since these are low-level helpers not intended for general-purpose use anyway, it is better not to make guesses about how these functions will be used: rather, this patch removes the WARN_ON() and relies on callers to use the helpers sensibly. Suggested-by:
Andrew Jones <drjones@redhat.com> Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Andrew Jones <drjones@redhat.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
- 16 Apr, 2019 1 commit
-
-
Andrew Murray authored
As we will exhaust the first 32 bits of AT_HWCAP let's start exposing AT_HWCAP2 to userspace to give us up to 64 caps. Whilst it's possible to use the remaining 32 bits of AT_HWCAP, we prefer to expand into AT_HWCAP2 in order to provide a consistent view to userspace between ILP32 and LP64. However internal to the kernel we prefer to continue to use the full space of elf_hwcap. To reduce complexity and allow for future expansion, we now represent hwcaps in the kernel as ordinals and use a KERNEL_HWCAP_ prefix. This allows us to support automatic feature based module loading for all our hwcaps. We introduce cpu_set_feature to set hwcaps which complements the existing cpu_have_feature helper. These helpers allow us to clean up existing direct uses of elf_hwcap and reduce any future effort required to move beyond 64 caps. For convenience we also introduce cpu_{have,set}_named_feature which makes use of the cpu_feature macro to allow providing a hwcap name without a {KERNEL_}HWCAP_ prefix. Signed-off-by:
Andrew Murray <andrew.murray@arm.com> [will: use const_ilog2() and tweak documentation] Signed-off-by:
Will Deacon <will.deacon@arm.com>
-
- 29 Mar, 2019 5 commits
-
-
Dave Martin authored
KVM will need to interrogate the set of SVE vector lengths available on the system. This patch exposes the relevant bits to the kernel, along with a sve_vq_available() helper to check whether a particular vector length is supported. __vq_to_bit() and __bit_to_vq() are not intended for use outside these functions: now that these are exposed outside fpsimd.c, they are prefixed with __ in order to provide an extra hint that they are not intended for general-purpose use. Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Alex Bennée <alex.bennee@linaro.org> Tested-by:
zhang.lei <zhang.lei@jp.fujitsu.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
Dave Martin authored
The current FPSIMD/SVE context handling support for non-task (i.e., KVM vcpu) contexts does not take SVE into account. This means that only task contexts can safely use SVE at present. In preparation for enabling KVM guests to use SVE, it is necessary to keep track of SVE state for non-task contexts too. This patch adds the necessary support, removing assumptions from the context switch code about the location of the SVE context storage. When binding a vcpu context, its vector length is arbitrarily specified as SVE_VL_MIN for now. In any case, because TIF_SVE is presently cleared at vcpu context bind time, the specified vector length will not be used for anything yet. In later patches TIF_SVE will be set here as appropriate, and the appropriate maximum vector length for the vcpu will be passed when binding. Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Alex Bennée <alex.bennee@linaro.org> Reviewed-by:
Julien Grall <julien.grall@arm.com> Tested-by:
zhang.lei <zhang.lei@jp.fujitsu.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
Dave Martin authored
Due to the way the effective SVE vector length is controlled and trapped at different exception levels, certain mismatches in the sets of vector lengths supported by different physical CPUs in the system may prevent straightforward virtualisation of SVE at parity with the host. This patch analyses the extent to which SVE can be virtualised safely without interfering with migration of vcpus between physical CPUs, and rejects late secondary CPUs that would erode the situation further. It is left up to KVM to decide what to do with this information. Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Julien Thierry <julien.thierry@arm.com> Tested-by:
zhang.lei <zhang.lei@jp.fujitsu.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
Dave Martin authored
The roles of sve_init_vq_map(), sve_update_vq_map() and sve_verify_vq_map() are highly non-obvious to anyone who has not dug through cpufeatures.c in detail. Since the way these functions interact with each other is more important here than a full understanding of the cpufeatures code, this patch adds comments to make the functions' roles clearer. No functional change. Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Julien Thierry <julien.thierry@arm.com> Reviewed-by:
Julien Grall <julien.grall@arm.com> Tested-by:
zhang.lei <zhang.lei@jp.fujitsu.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
Dave Martin authored
This patch updates fpsimd_flush_task_state() to mirror the new semantics of fpsimd_flush_cpu_state() introduced by commit d8ad71fa ("arm64: fpsimd: Fix TIF_FOREIGN_FPSTATE after invalidating cpu regs"). Both functions now implicitly set TIF_FOREIGN_FPSTATE to indicate that the task's FPSIMD state is not loaded into the cpu. As a side-effect, fpsimd_flush_task_state() now sets TIF_FOREIGN_FPSTATE even for non-running tasks. In the case of non-running tasks this is not useful but also harmless, because the flag is live only while the corresponding task is running. This function is not called from fast paths, so special-casing this for the task == current case is not really worth it. Compiler barriers previously present in restore_sve_fpsimd_context() are pulled into fpsimd_flush_task_state() so that it can be safely called with preemption enabled if necessary. Explicit calls to set TIF_FOREIGN_FPSTATE that accompany fpsimd_flush_task_state() calls and are now redundant are removed as appropriate. fpsimd_flush_task_state() is used to get exclusive access to the representation of the task's state via task_struct, for the purpose of replacing the state. Thus, the call to this function should happen before manipulating fpsimd_state or sve_state etc. in task_struct. Anomalous cases are reordered appropriately in order to make the code more consistent, although there should be no functional difference since these cases are protected by local_bh_disable() anyway. Signed-off-by:
Dave Martin <Dave.Martin@arm.com> Reviewed-by:
Alex Bennée <alex.bennee@linaro.org> Reviewed-by:
Julien Grall <julien.grall@arm.com> Tested-by:
zhang.lei <zhang.lei@jp.fujitsu.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com>
-
- 27 Sep, 2018 1 commit
-
-
Eric W. Biederman authored
Reviewed-by:
Catalin Marinas <catalin.marinas@arm.com> Tested-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
"Eric W. Biederman" <ebiederm@xmission.com>
-
- 12 Jul, 2018 2 commits
-
-
Mark Rutland authored
In subsequent patches, we'll want to make use of sve_user_enable() and sve_user_disable() outside of kernel/fpsimd.c. Let's move these to <asm/fpsimd.h> where we can make use of them. To avoid ifdeffery in sequences like: if (system_supports_sve() && some_condition) sve_user_disable(); ... empty stubs are provided when support for SVE is not enabled. Note that system_supports_sve() contains as IS_ENABLED(CONFIG_ARM64_SVE), so the sve_user_disable() call should be optimized away entirely when CONFIG_ARM64_SVE is not selected. To ensure that this is the case, the stub definitions contain a BUILD_BUG(), as we do for other stubs for which calls should always be optimized away when the relevant config option is not selected. At the same time, the include list of <asm/fpsimd.h> is sorted while adding <asm/sysreg.h>. Signed-off-by:
Mark Rutland <mark.rutland@arm.com> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Reviewed-by:
Dave Martin <dave.martin@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by:
Will Deacon <will.deacon@arm.com>
-
Mark Rutland authored
Now that we have sysreg_clear_set(), we can use this instead of change_cpacr(). Note that the order of the set and clear arguments differs between change_cpacr() and sysreg_clear_set(), so these are flipped as part of the conversion. Also, sve_user_enable() redundantly clears CPACR_EL1_ZEN_EL0EN before setting it; this is removed for clarity. Signed-off-by:
Mark Rutland <mark.rutland@arm.com> Reviewed-by:
Dave Martin <dave.martin@arm.com> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by:
Will Deacon <will.deacon@arm.com>
-