• Nicholas Piggin's avatar
    powerpc/64s: Implement interrupt exit logic in C · 6cc0c16d
    Nicholas Piggin authored
    Implement the bulk of interrupt return logic in C. The asm return code
    must handle a few cases: restoring full GPRs, and emulating stack
    store.
    
    The stack store emulation is significantly simplfied, rather than
    creating a new return frame and switching to that before performing
    the store, it uses the PACA to keep a scratch register around to
    perform the store.
    
    The asm return code is moved into 64e for now. The new logic has made
    allowance for 64e, but I don't have a full environment that works well
    to test it, and even booting in emulated qemu is not great for stress
    testing. 64e shouldn't be too far off working with this, given a bit
    more testing and auditing of the logic.
    
    This is slightly faster on a POWER9 (page fault speed increases about
    1.1%), probably due to reduced mtmsrd.
    
    mpe: Includes fixes from Nick for _TIF_EMULATE_STACK_STORE
    handling (including the fast_interrupt_return path), to remove
    trace_hardirqs_on(), and fixes the interrupt-return part of the
    MSR_VSX restore bug caught by tm-unavailable selftest.
    
    mpe: Incorporate fix from Nick:
    
    The return-to-kernel path has to replay any soft-pending interrupts if
    it is returning to a context that had interrupts soft-enabled. It has
    to do this carefully and avoid plain enabling interrupts if this is an
    irq context, which can cause multiple nesting of interrupts on the
    stack, and other unexpected issues.
    
    The code which avoided this case got the soft-mask state wrong, and
    marked interrupts as enabled before going around again to retry. This
    seems to be mostly harmless except when PREEMPT=y, this calls
    preempt_schedule_irq with irqs apparently enabled and runs into a BUG
    in kernel/sched/core.c
    Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
    Signed-off-by: default avatarMichal Suchanek <msuchanek@suse.de>
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Link: https://lore.kernel.org/r/20200225173541.1549955-29-npiggin@gmail.com
    6cc0c16d
exceptions-64e.S 50 KB