1. 06 Feb, 2018 23 commits
  2. 26 Jan, 2018 1 commit
  3. 23 Jan, 2018 4 commits
  4. 19 Jan, 2018 1 commit
  5. 16 Jan, 2018 11 commits
    • Catalin Marinas's avatar
      arm64: kpti: Fix the interaction between ASID switching and software PAN · 6b88a32c
      Catalin Marinas authored
      With ARM64_SW_TTBR0_PAN enabled, the exception entry code checks the
      active ASID to decide whether user access was enabled (non-zero ASID)
      when the exception was taken. On return from exception, if user access
      was previously disabled, it re-instates TTBR0_EL1 from the per-thread
      saved value (updated in switch_mm() or efi_set_pgd()).
      
      Commit 7655abb9 ("arm64: mm: Move ASID from TTBR0 to TTBR1") makes a
      TTBR0_EL1 + ASID switching non-atomic. Subsequently, commit 27a921e7
      ("arm64: mm: Fix and re-enable ARM64_SW_TTBR0_PAN") changes the
      __uaccess_ttbr0_disable() function and asm macro to first write the
      reserved TTBR0_EL1 followed by the ASID=0 update in TTBR1_EL1. If an
      exception occurs between these two, the exception return code will
      re-instate a valid TTBR0_EL1. Similar scenario can happen in
      cpu_switch_mm() between setting the reserved TTBR0_EL1 and the ASID
      update in cpu_do_switch_mm().
      
      This patch reverts the entry.S check for ASID == 0 to TTBR0_EL1 and
      disables the interrupts around the TTBR0_EL1 and ASID switching code in
      __uaccess_ttbr0_disable(). It also ensures that, when returning from the
      EFI runtime services, efi_set_pgd() doesn't leave a non-zero ASID in
      TTBR1_EL1 by using uaccess_ttbr0_{enable,disable}.
      
      The accesses to current_thread_info()->ttbr0 are updated to use
      READ_ONCE/WRITE_ONCE.
      
      As a safety measure, __uaccess_ttbr0_enable() always masks out any
      existing non-zero ASID TTBR1_EL1 before writing in the new ASID.
      
      Fixes: 27a921e7 ("arm64: mm: Fix and re-enable ARM64_SW_TTBR0_PAN")
      Acked-by: default avatarWill Deacon <will.deacon@arm.com>
      Reported-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
      Tested-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
      Reviewed-by: default avatarJames Morse <james.morse@arm.com>
      Tested-by: default avatarJames Morse <james.morse@arm.com>
      Co-developed-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      6b88a32c
    • Dongjiu Geng's avatar
      KVM: arm64: Emulate RAS error registers and set HCR_EL2's TERR & TEA · 558daf69
      Dongjiu Geng authored
      ARMv8.2 adds a new bit HCR_EL2.TEA which routes synchronous external
      aborts to EL2, and adds a trap control bit HCR_EL2.TERR which traps
      all Non-secure EL1&0 error record accesses to EL2.
      
      This patch enables the two bits for the guest OS, guaranteeing that
      KVM takes external aborts and traps attempts to access the physical
      error registers.
      
      ERRIDR_EL1 advertises the number of error records, we return
      zero meaning we can treat all the other registers as RAZ/WI too.
      Signed-off-by: default avatarDongjiu Geng <gengdongjiu@huawei.com>
      [removed specific emulation, use trap_raz_wi() directly for everything,
       rephrased parts of the commit message]
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
      Reviewed-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      558daf69
    • James Morse's avatar
      KVM: arm64: Handle RAS SErrors from EL2 on guest exit · 0067df41
      James Morse authored
      We expect to have firmware-first handling of RAS SErrors, with errors
      notified via an APEI method. For systems without firmware-first, add
      some minimal handling to KVM.
      
      There are two ways KVM can take an SError due to a guest, either may be a
      RAS error: we exit the guest due to an SError routed to EL2 by HCR_EL2.AMO,
      or we take an SError from EL2 when we unmask PSTATE.A from __guest_exit.
      
      The current SError from EL2 code unmasks SError and tries to fence any
      pending SError into a single instruction window. It then leaves SError
      unmasked.
      
      With the v8.2 RAS Extensions we may take an SError for a 'corrected'
      error, but KVM is only able to handle SError from EL2 if they occur
      during this single instruction window...
      
      The RAS Extensions give us a new instruction to synchronise and
      consume SErrors. The RAS Extensions document (ARM DDI0587),
      '2.4.1 ESB and Unrecoverable errors' describes ESB as synchronising
      SError interrupts generated by 'instructions, translation table walks,
      hardware updates to the translation tables, and instruction fetches on
      the same PE'. This makes ESB equivalent to KVMs existing
      'dsb, mrs-daifclr, isb' sequence.
      
      Use the alternatives to synchronise and consume any SError using ESB
      instead of unmasking and taking the SError. Set ARM_EXIT_WITH_SERROR_BIT
      in the exit_code so that we can restart the vcpu if it turns out this
      SError has no impact on the vcpu.
      Reviewed-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      0067df41
    • James Morse's avatar
      KVM: arm64: Handle RAS SErrors from EL1 on guest exit · 3368bd80
      James Morse authored
      We expect to have firmware-first handling of RAS SErrors, with errors
      notified via an APEI method. For systems without firmware-first, add
      some minimal handling to KVM.
      
      There are two ways KVM can take an SError due to a guest, either may be a
      RAS error: we exit the guest due to an SError routed to EL2 by HCR_EL2.AMO,
      or we take an SError from EL2 when we unmask PSTATE.A from __guest_exit.
      
      For SError that interrupt a guest and are routed to EL2 the existing
      behaviour is to inject an impdef SError into the guest.
      
      Add code to handle RAS SError based on the ESR. For uncontained and
      uncategorized errors arm64_is_fatal_ras_serror() will panic(), these
      errors compromise the host too. All other error types are contained:
      For the fatal errors the vCPU can't make progress, so we inject a virtual
      SError. We ignore contained errors where we can make progress as if
      we're lucky, we may not hit them again.
      
      If only some of the CPUs support RAS the guest will see the cpufeature
      sanitised version of the id registers, but we may still take RAS SError
      on this CPU. Move the SError handling out of handle_exit() into a new
      handler that runs before we can be preempted. This allows us to use
      this_cpu_has_cap(), via arm64_is_ras_serror().
      Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      3368bd80
    • James Morse's avatar
      KVM: arm64: Save ESR_EL2 on guest SError · c60590b5
      James Morse authored
      When we exit a guest due to an SError the vcpu fault info isn't updated
      with the ESR. Today this is only done for traps.
      
      The v8.2 RAS Extensions define ISS values for SError. Update the vcpu's
      fault_info with the ESR on SError so that handle_exit() can determine
      if this was a RAS SError and decode its severity.
      Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      c60590b5
    • James Morse's avatar
      KVM: arm64: Save/Restore guest DISR_EL1 · c773ae2b
      James Morse authored
      If we deliver a virtual SError to the guest, the guest may defer it
      with an ESB instruction. The guest reads the deferred value via DISR_EL1,
      but the guests view of DISR_EL1 is re-mapped to VDISR_EL2 when HCR_EL2.AMO
      is set.
      
      Add the KVM code to save/restore VDISR_EL2, and make it accessible to
      userspace as DISR_EL1.
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      c773ae2b
    • James Morse's avatar
      KVM: arm64: Set an impdef ESR for Virtual-SError using VSESR_EL2. · 4715c14b
      James Morse authored
      Prior to v8.2's RAS Extensions, the HCR_EL2.VSE 'virtual SError' feature
      generated an SError with an implementation defined ESR_EL1.ISS, because we
      had no mechanism to specify the ESR value.
      
      On Juno this generates an all-zero ESR, the most significant bit 'ISV'
      is clear indicating the remainder of the ISS field is invalid.
      
      With the RAS Extensions we have a mechanism to specify this value, and the
      most significant bit has a new meaning: 'IDS - Implementation Defined
      Syndrome'. An all-zero SError ESR now means: 'RAS error: Uncategorized'
      instead of 'no valid ISS'.
      
      Add KVM support for the VSESR_EL2 register to specify an ESR value when
      HCR_EL2.VSE generates a virtual SError. Change kvm_inject_vabt() to
      specify an implementation-defined value.
      
      We only need to restore the VSESR_EL2 value when HCR_EL2.VSE is set, KVM
      save/restores this bit during __{,de}activate_traps() and hardware clears the
      bit once the guest has consumed the virtual-SError.
      
      Future patches may add an API (or KVM CAP) to pend a virtual SError with
      a specified ESR.
      
      Cc: Dongjiu Geng <gengdongjiu@huawei.com>
      Reviewed-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      4715c14b
    • James Morse's avatar
      KVM: arm/arm64: mask/unmask daif around VHE guests · 4f5abad9
      James Morse authored
      Non-VHE systems take an exception to EL2 in order to world-switch into the
      guest. When returning from the guest KVM implicitly restores the DAIF
      flags when it returns to the kernel at EL1.
      
      With VHE none of this exception-level jumping happens, so KVMs
      world-switch code is exposed to the host kernel's DAIF values, and KVM
      spills the guest-exit DAIF values back into the host kernel.
      On entry to a guest we have Debug and SError exceptions unmasked, KVM
      has switched VBAR but isn't prepared to handle these. On guest exit
      Debug exceptions are left disabled once we return to the host and will
      stay this way until we enter user space.
      
      Add a helper to mask/unmask DAIF around VHE guests. The unmask can only
      happen after the hosts VBAR value has been synchronised by the isb in
      __vhe_hyp_call (via kvm_call_hyp()). Masking could be as late as
      setting KVMs VBAR value, but is kept here for symmetry.
      Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      4f5abad9
    • James Morse's avatar
      arm64: kernel: Prepare for a DISR user · 68ddbf09
      James Morse authored
      KVM would like to consume any pending SError (or RAS error) after guest
      exit. Today it has to unmask SError and use dsb+isb to synchronise the
      CPU. With the RAS extensions we can use ESB to synchronise any pending
      SError.
      
      Add the necessary macros to allow DISR to be read and converted to an
      ESR.
      
      We clear the DISR register when we enable the RAS cpufeature, and the
      kernel has not executed any ESB instructions. Any value we find in DISR
      must have belonged to firmware. Executing an ESB instruction is the
      only way to update DISR, so we can expect firmware to have handled
      any deferred SError. By the same logic we clear DISR in the idle path.
      Reviewed-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      68ddbf09
    • James Morse's avatar
      arm64: Unconditionally enable IESB on exception entry/return for firmware-first · f751daa4
      James Morse authored
      ARM v8.2 has a feature to add implicit error synchronization barriers
      whenever the CPU enters or returns from an exception level. Add this to the
      features we always enable. CPUs that don't support this feature will treat
      the bit as RES0.
      
      This feature causes RAS errors that are not yet visible to software to
      become pending SErrors. We expect to have firmware-first RAS support
      so synchronised RAS errors will be take immediately to EL3.
      Any system without firmware-first handling of errors will take the SError
      either immediatly after exception return, or when we unmask SError after
      entry.S's work.
      
      Adding IESB to the ELx flags causes it to be enabled by KVM and kexec
      too.
      
      Platform level RAS support may require additional firmware support.
      
      Cc: Christoffer Dall <christoffer.dall@linaro.org>
      Suggested-by: default avatarWill Deacon <will.deacon@arm.com>
      Link: https://www.spinics.net/lists/kvm-arm/msg28192.htmlAcked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      f751daa4
    • James Morse's avatar
      arm64: kernel: Survive corrected RAS errors notified by SError · 6bf0dcfd
      James Morse authored
      Prior to v8.2, SError is an uncontainable fatal exception. The v8.2 RAS
      extensions use SError to notify software about RAS errors, these can be
      contained by the Error Syncronization Barrier.
      
      An ACPI system with firmware-first may use SError as its 'SEI'
      notification. Future patches may add code to 'claim' this SError as a
      notification.
      
      Other systems can distinguish these RAS errors from the SError ESR and
      use the AET bits and additional data from RAS-Error registers to handle
      the error. Future patches may add this kernel-first handling.
      
      Without support for either of these we will panic(), even if we received
      a corrected error. Add code to decode the severity of RAS errors. We can
      safely ignore contained errors where the CPU can continue to make
      progress. For all other errors we continue to panic().
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      6bf0dcfd