An error occurred fetching the project authors.
  1. 21 Oct, 2021 7 commits
  2. 16 Sep, 2021 1 commit
    • Mark Brown's avatar
      arm64/sve: Use correct size when reinitialising SVE state · e35ac9d0
      Mark Brown authored
      When we need a buffer for SVE register state we call sve_alloc() to make
      sure that one is there. In order to avoid repeated allocations and frees
      we keep the buffer around unless we change vector length and just memset()
      it to ensure a clean register state. The function that deals with this
      takes the task to operate on as an argument, however in the case where we
      do a memset() we initialise using the SVE state size for the current task
      rather than the task passed as an argument.
      
      This is only an issue in the case where we are setting the register state
      for a task via ptrace and the task being configured has a different vector
      length to the task tracing it. In the case where the buffer is larger in
      the traced process we will leak old state from the traced process to
      itself, in the case where the buffer is smaller in the traced process we
      will overflow the buffer and corrupt memory.
      
      Fixes: bc0ee476 ("arm64/sve: Core task context handling")
      Cc: <stable@vger.kernel.org> # 4.15.x
      Signed-off-by: default avatarMark Brown <broonie@kernel.org>
      Link: https://lore.kernel.org/r/20210909165356.10675-1-broonie@kernel.orgSigned-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      e35ac9d0
  3. 24 Aug, 2021 1 commit
    • Mark Brown's avatar
      arm64/sve: Better handle failure to allocate SVE register storage · 7559b7d7
      Mark Brown authored
      Currently we "handle" failure to allocate the SVE register storage by
      doing a BUG_ON() and hoping for the best. This is obviously not great and
      the memory allocation failure will already be loud enough without the
      BUG_ON(). As the comment says it is a corner case but let's try to do a bit
      better, remove the BUG_ON() and add code to handle the failure in the
      callers.
      
      For the ptrace and signal code we can return -ENOMEM gracefully however
      we have no real error reporting path available to us for the SVE access
      trap so instead generate a SIGKILL if the allocation fails there. This
      at least means that we won't try to soldier on and end up trying to
      access the nonexistant state and while it's obviously not ideal for
      userspace SIGKILL doesn't allow any handling so minimises the ABI
      impact, making it easier to improve the interface later if we come up
      with a better idea.
      Signed-off-by: default avatarMark Brown <broonie@kernel.org>
      Link: https://lore.kernel.org/r/20210824153417.18371-1-broonie@kernel.orgSigned-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      7559b7d7
  4. 02 Aug, 2021 1 commit
  5. 26 May, 2021 1 commit
  6. 15 Apr, 2021 1 commit
  7. 13 Apr, 2021 1 commit
  8. 12 Apr, 2021 1 commit
  9. 08 Apr, 2021 1 commit
    • Mark Brown's avatar
      arm64/sve: Rework SVE access trap to convert state in registers · cccb78ce
      Mark Brown authored
      When we enable SVE usage in userspace after taking a SVE access trap we
      need to ensure that the portions of the register state that are not
      shared with the FPSIMD registers are zeroed. Currently we do this by
      forcing the FPSIMD registers to be saved to the task struct and converting
      them there. This is wasteful in the common case where the task state is
      loaded into the registers and we will immediately return to userspace
      since we can initialise the SVE state directly in registers instead of
      accessing multiple copies of the register state in memory.
      
      Instead in that common case do the conversion in the registers and
      update the task metadata so that we can return to userspace without
      spilling the register state to memory unless there is some other reason
      to do so.
      Signed-off-by: default avatarMark Brown <broonie@kernel.org>
      Link: https://lore.kernel.org/r/20210312190313.24598-1-broonie@kernel.orgSigned-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      cccb78ce
  10. 15 Oct, 2020 1 commit
  11. 21 Sep, 2020 1 commit
  12. 18 Sep, 2020 1 commit
  13. 14 Sep, 2020 1 commit
  14. 16 Jun, 2020 1 commit
  15. 15 Jun, 2020 1 commit
  16. 27 Apr, 2020 1 commit
  17. 14 Jan, 2020 1 commit
    • Suzuki K Poulose's avatar
      arm64: nofpsmid: Handle TIF_FOREIGN_FPSTATE flag cleanly · 52f73c38
      Suzuki K Poulose authored
      We detect the absence of FP/SIMD after an incapable CPU is brought up,
      and by then we have kernel threads running already with TIF_FOREIGN_FPSTATE set
      which could be set for early userspace applications (e.g, modprobe triggered
      from initramfs) and init. This could cause the applications to loop forever in
      do_nofity_resume() as we never clear the TIF flag, once we now know that
      we don't support FP.
      
      Fix this by making sure that we clear the TIF_FOREIGN_FPSTATE flag
      for tasks which may have them set, as we would have done in the normal
      case, but avoiding touching the hardware state (since we don't support any).
      
      Also to make sure we handle the cases seemlessly we categorise the
      helper functions to two :
       1) Helpers for common core code, which calls into take appropriate
          actions without knowing the current FPSIMD state of the CPU/task.
      
          e.g fpsimd_restore_current_state(), fpsimd_flush_task_state(),
              fpsimd_save_and_flush_cpu_state().
      
          We bail out early for these functions, taking any appropriate actions
          (e.g, clearing the TIF flag) where necessary to hide the handling
          from core code.
      
       2) Helpers used when the presence of FP/SIMD is apparent.
          i.e, save/restore the FP/SIMD register state, modify the CPU/task
          FP/SIMD state.
          e.g,
      
          fpsimd_save(), task_fpsimd_load() - save/restore task FP/SIMD registers
      
          fpsimd_bind_task_to_cpu()  \
                                      - Update the "state" metadata for CPU/task.
          fpsimd_bind_state_to_cpu() /
      
          fpsimd_update_current_state() - Update the fp/simd state for the current
                                          task from memory.
      
          These must not be called in the absence of FP/SIMD. Put in a WARNING
          to make sure they are not invoked in the absence of FP/SIMD.
      
      KVM also uses the TIF_FOREIGN_FPSTATE flag to manage the FP/SIMD state
      on the CPU. However, without FP/SIMD support we trap all accesses and
      inject undefined instruction. Thus we should never "load" guest state.
      Add a sanity check to make sure this is valid.
      
      Fixes: 82e0191a ("arm64: Support systems without FP/ASIMD")
      Cc: Will Deacon <will@kernel.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Reviewed-by: default avatarArd Biesheuvel <ardb@kernel.org>
      Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Acked-by: default avatarMarc Zyngier <maz@kernel.org>
      Signed-off-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
      Signed-off-by: default avatarWill Deacon <will@kernel.org>
      52f73c38
  18. 28 Oct, 2019 1 commit
  19. 22 Jul, 2019 2 commits
  20. 19 Jun, 2019 1 commit
  21. 13 Jun, 2019 1 commit
    • Dave Martin's avatar
      arm64/sve: Fix missing SVE/FPSIMD endianness conversions · 41040cf7
      Dave Martin authored
      The in-memory representation of SVE and FPSIMD registers is
      different: the FPSIMD V-registers are stored as single 128-bit
      host-endian values, whereas SVE registers are stored in an
      endianness-invariant byte order.
      
      This means that the two representations differ when running on a
      big-endian host.  But we blindly copy data from one representation
      to another when converting between the two, resulting in the
      register contents being unintentionally byteswapped in certain
      situations.  Currently this can be triggered by the first SVE
      instruction after a syscall, for example (though the potential
      trigger points may vary in future).
      
      So, fix the conversion functions fpsimd_to_sve(), sve_to_fpsimd()
      and sve_sync_from_fpsimd_zeropad() to swab where appropriate.
      
      There is no common swahl128() or swab128() that we could use here.
      Maybe it would be worth making this generic, but for now add a
      simple local hack.
      
      Since the byte order differences are exposed in ABI, also clarify
      the documentation.
      
      Cc: Alex Bennée <alex.bennee@linaro.org>
      Cc: Peter Maydell <peter.maydell@linaro.org>
      Cc: Alan Hayward <alan.hayward@arm.com>
      Cc: Julien Grall <julien.grall@arm.com>
      Fixes: bc0ee476 ("arm64/sve: Core task context handling")
      Fixes: 8cd969d2 ("arm64/sve: Signal handling support")
      Fixes: 43d4da2c ("arm64/sve: ptrace and ELF coredump support")
      Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
      [will: Fix typos in comments and docs spotted by Julien]
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      41040cf7
  22. 04 Jun, 2019 2 commits
    • Julien Grall's avatar
      arm64/fpsimd: Don't disable softirq when touching FPSIMD/SVE state · 6dcdefcd
      Julien Grall authored
      When the kernel is compiled with CONFIG_KERNEL_MODE_NEON, some part of
      the kernel may be able to use FPSIMD/SVE. This is for instance the case
      for crypto code.
      
      Any use of FPSIMD/SVE in the kernel are clearly marked by using the
      function kernel_neon_{begin, end}. Furthermore, this can only be used
      when may_use_simd() returns true.
      
      The current implementation of may_use_simd() allows softirq to use
      FPSIMD/SVE unless it is currently in use (i.e kernel_neon_busy is true).
      When in use, softirqs usually fall back to a software method.
      
      At the moment, as a softirq may use FPSIMD/SVE, softirqs are disabled
      when touching the FPSIMD/SVE context. This has the drawback to disable
      all softirqs even if they are not using FPSIMD/SVE.
      
      Since a softirq is supposed to check may_use_simd() anyway before
      attempting to use FPSIMD/SVE, there is limited reason to keep softirq
      disabled when touching the FPSIMD/SVE context. Instead, we can simply
      disable preemption and mark the FPSIMD/SVE context as in use by setting
      CPU's fpsimd_context_busy flag.
      
      Two new helpers {get, put}_cpu_fpsimd_context are introduced to mark
      the area using FPSIMD/SVE context and they are used to replace
      local_bh_{disable, enable}. The functions kernel_neon_{begin, end} are
      also re-implemented to use the new helpers.
      
      Additionally, double-underscored versions of the helpers are provided to
      called when preemption is already disabled. These are only relevant on
      paths where irqs are disabled anyway, so they are not needed for
      correctness in the current code. Let's use them anyway though: this
      marks critical sections clearly and will help to avoid mistakes during
      future maintenance.
      
      The change has been benchmarked on Linux 5.1-rc4 with defconfig.
      
      On Juno2:
          * hackbench 100 process 1000 (10 times)
          * .7% quicker
      
      On ThunderX 2:
          * hackbench 1000 process 1000 (20 times)
          * 3.4% quicker
      Reviewed-by: default avatarDave Martin <dave.martin@arm.com>
      Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJulien Grall <julien.grall@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      6dcdefcd
    • Julien Grall's avatar
      arm64/fpsimd: Introduce fpsimd_save_and_flush_cpu_state() and use it · 54b8c7cb
      Julien Grall authored
      The only external user of fpsimd_save() and fpsimd_flush_cpu_state() is
      the KVM FPSIMD code.
      
      A following patch will introduce a mechanism to acquire owernship of the
      FPSIMD/SVE context for performing context management operations. Rather
      than having to export the new helpers to get/put the context, we can just
      introduce a new function to combine fpsimd_save() and
      fpsimd_flush_cpu_state().
      
      This has also the advantage to remove any external call of fpsimd_save()
      and fpsimd_flush_cpu_state(), so they can be turned static.
      
      Lastly, the new function can also be used in the PM notifier.
      Reviewed-by: default avatarDave Martin <Dave.Martin@arm.com>
      Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJulien Grall <julien.grall@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      54b8c7cb
  23. 18 Apr, 2019 1 commit
    • Dave Martin's avatar
      arm64/sve: Clarify vq map semantics · 624835ab
      Dave Martin authored
      Currently the meanings of sve_vq_map and the ancillary helpers
      __bit_to_vq() and __vq_to_bit() are not clearly explained.
      
      This patch makes the explanatory comment clearer, and removes the
      duplicate comment from fpsimd.h.
      
      The WARN_ON() currently present in __bit_to_vq() confuses the
      intended use of this helper.  Since these are low-level helpers not
      intended for general-purpose use anyway, it is better not to make
      guesses about how these functions will be used: rather, this patch
      removes the WARN_ON() and relies on callers to use the helpers
      sensibly.
      Suggested-by: default avatarAndrew Jones <drjones@redhat.com>
      Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
      Reviewed-by: default avatarAndrew Jones <drjones@redhat.com>
      Signed-off-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      624835ab
  24. 16 Apr, 2019 1 commit
    • Andrew Murray's avatar
      arm64: HWCAP: add support for AT_HWCAP2 · aaba098f
      Andrew Murray authored
      As we will exhaust the first 32 bits of AT_HWCAP let's start
      exposing AT_HWCAP2 to userspace to give us up to 64 caps.
      
      Whilst it's possible to use the remaining 32 bits of AT_HWCAP, we
      prefer to expand into AT_HWCAP2 in order to provide a consistent
      view to userspace between ILP32 and LP64. However internal to the
      kernel we prefer to continue to use the full space of elf_hwcap.
      
      To reduce complexity and allow for future expansion, we now
      represent hwcaps in the kernel as ordinals and use a
      KERNEL_HWCAP_ prefix. This allows us to support automatic feature
      based module loading for all our hwcaps.
      
      We introduce cpu_set_feature to set hwcaps which complements the
      existing cpu_have_feature helper. These helpers allow us to clean
      up existing direct uses of elf_hwcap and reduce any future effort
      required to move beyond 64 caps.
      
      For convenience we also introduce cpu_{have,set}_named_feature which
      makes use of the cpu_feature macro to allow providing a hwcap name
      without a {KERNEL_}HWCAP_ prefix.
      Signed-off-by: default avatarAndrew Murray <andrew.murray@arm.com>
      [will: use const_ilog2() and tweak documentation]
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      aaba098f
  25. 29 Mar, 2019 5 commits
  26. 27 Sep, 2018 1 commit
  27. 12 Jul, 2018 2 commits
    • Mark Rutland's avatar
      arm64: move sve_user_{enable,disable} to <asm/fpsimd.h> · f9209e26
      Mark Rutland authored
      In subsequent patches, we'll want to make use of sve_user_enable() and
      sve_user_disable() outside of kernel/fpsimd.c. Let's move these to
      <asm/fpsimd.h> where we can make use of them.
      
      To avoid ifdeffery in sequences like:
      
      if (system_supports_sve() && some_condition)
      	sve_user_disable();
      
      ... empty stubs are provided when support for SVE is not enabled. Note
      that system_supports_sve() contains as IS_ENABLED(CONFIG_ARM64_SVE), so
      the sve_user_disable() call should be optimized away entirely when
      CONFIG_ARM64_SVE is not selected.
      
      To ensure that this is the case, the stub definitions contain a
      BUILD_BUG(), as we do for other stubs for which calls should always be
      optimized away when the relevant config option is not selected.
      
      At the same time, the include list of <asm/fpsimd.h> is sorted while
      adding <asm/sysreg.h>.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Reviewed-by: default avatarDave Martin <dave.martin@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      f9209e26
    • Mark Rutland's avatar
      arm64: kill change_cpacr() · 8d370933
      Mark Rutland authored
      Now that we have sysreg_clear_set(), we can use this instead of
      change_cpacr().
      
      Note that the order of the set and clear arguments differs between
      change_cpacr() and sysreg_clear_set(), so these are flipped as part of
      the conversion. Also, sve_user_enable() redundantly clears
      CPACR_EL1_ZEN_EL0EN before setting it; this is removed for clarity.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Reviewed-by: default avatarDave Martin <dave.martin@arm.com>
      Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      8d370933