• Mark Rutland's avatar
    arm64: entry: fix NMI {user, kernel}->kernel transitions · f0cd5ac1
    Mark Rutland authored
    Exceptions which can be taken at (almost) any time are consdiered to be
    NMIs. On arm64 that includes:
    
    * SDEI events
    * GICv3 Pseudo-NMIs
    * Kernel stack overflows
    * Unexpected/unhandled exceptions
    
    ... but currently debug exceptions (BRKs, breakpoints, watchpoints,
    single-step) are not considered NMIs.
    
    As these can be taken at any time, kernel features (lockdep, RCU,
    ftrace) may not be in a consistent kernel state. For example, we may
    take an NMI from the idle code or partway through an entry/exit path.
    
    While nmi_enter() and nmi_exit() handle most of this state, notably they
    don't save/restore the lockdep state across an NMI being taken and
    handled. When interrupts are enabled and an NMI is taken, lockdep may
    see interrupts become disabled within the NMI code, but not see
    interrupts become enabled when returning from the NMI, leaving lockdep
    believing interrupts are disabled when they are actually disabled.
    
    The x86 code handles this in idtentry_{enter,exit}_nmi(), which will
    shortly be moved to the generic entry code. As we can't use either yet,
    we copy the x86 approach in arm64-specific helpers. All the NMI
    entrypoints are marked as noinstr to prevent any instrumentation
    handling code being invoked before the state has been corrected.
    Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
    Cc: Catalin Marinas <catalin.marinas@arm.com>
    Cc: James Morse <james.morse@arm.com>
    Cc: Will Deacon <will@kernel.org>
    Link: https://lore.kernel.org/r/20201130115950.22492-11-mark.rutland@arm.comSigned-off-by: default avatarWill Deacon <will@kernel.org>
    f0cd5ac1
traps.c 25.1 KB