- 03 Jul, 2019 6 commits
-
-
Nicholas Piggin authored
ISA v3.0 radix modes provide SLBIA variants which can invalidate ERAT for effPID!=0 or for effLPID!=0, which allows user and guest invalidations to retain kernel/host ERAT entries. Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au>
-
Nicholas Piggin authored
This makes it clear to the caller that it can only be used on POWER9 and later CPUs. Signed-off-by:
-
Nicholas Piggin authored
Branch to the relocated 0xc000 address early (still in real mode), to simplify subsequent branches. Have the virt mode handler avoid just 'windup' and redo the exception from scratch, rather than branching back to the trampoline. Rearrange the stack setup instruction location to match the system reset handler (e.g., right before EXCEPTION_PROLOG_COMMON). Signed-off-by:
-
Nicholas Piggin authored
No code change. Signed-off-by:
-
Nicholas Piggin authored
Follow convention and move tramp ahead of common. Signed-off-by:
-
Nicholas Piggin authored
The idle wake up code in the system reset interrupt is not very optimal. There are two requirements: perform idle wake up quickly; and save everything including CFAR for non-idle interrupts, with no performance requirement. The problem with placing the idle test in the middle of the handler and using the normal handler code to save CFAR, is that it's quite costly (e.g., mfcfar is serialising, speculative workarounds get applied, SRR1 has to be reloaded, etc). It also prevents the standard interrupt handler boilerplate being used. This pain can be avoided by using a dedicated idle interrupt handler at the start of the interrupt handler, which restores all registers back to the way they were in case it was not an idle wake up. CFAR is preserved without saving it before the non-idle case by making that the fall-through, and idle is a taken branch. Performance seems to be in the noise, but possibly around 0.5% faster, the executed instructions certainly look better. The bigger benefit is being able to drop in standard interrupt handlers after the idle code, which helps with subsequent cleanup and consolidation. Signed-off-by:
-
- 02 Jul, 2019 34 commits
-
-
Nicholas Piggin authored
The bad stack test in interrupt handlers has a few problems. For performance it is taken in the common case, which is a fetch bubble and a waste of i-cache. For code development and maintainence, it requires yet another stack frame setup routine, and that constrains all exception handlers to follow the same register save pattern which inhibits future optimisation. Remove the test/branch and replace it with a trap. Teach the program check handler to use the emergency stack for this case. This does not result in quite so nice a message, however the SRR0 and SRR1 of the crashed interrupt can be seen in r11 and r12, as is the original r1 (adjusted by INT_FRAME_SIZE). These are the most important parts to debugging the issue. The original r9-12 and cr0 is lost, which is the main downside. kernel BUG at linux/arch/powerpc/kernel/exceptions-64s.S:847! Oops: Exception in kernel mode, sig: 5 [#1] BE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted NIP: c000000000009108 LR: c000000000cadbcc CTR: c0000000000090f0 REGS: c0000000fffcbd70 TRAP: 0700 Not tainted MSR: 9000000000021032 <SF,HV,ME,IR,DR,RI> CR: 28222448 XER: 20040000 CFAR: c000000000009100 IRQMASK: 0 GPR00: 000000000000003d fffffffffffffd00 c0000000018cfb00 c0000000f02b3166 GPR04: fffffffffffffffd 0000000000000007 fffffffffffffffb 0000000000000030 GPR08: 0000000000000037 0000000028222448 0000000000000000 c000000000ca8de0 GPR12: 9000000002009032 c000000001ae0000 c000000000010a00 0000000000000000 GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 GPR20: c0000000f00322c0 c000000000f85200 0000000000000004 ffffffffffffffff GPR24: fffffffffffffffe 0000000000000000 0000000000000000 000000000000000a GPR28: 0000000000000000 0000000000000000 c0000000f02b391c c0000000f02b3167 NIP [c000000000009108] decrementer_common+0x18/0x160 LR [c000000000cadbcc] .vsnprintf+0x3ec/0x4f0 Call Trace: Instruction dump: 996d098a 994d098b 38610070 480246ed 48005518 60000000 38200000 718a4000 7c2a0b78 3821fd00 41c20008 e82d0970 <0981fd00> f92101a0 f9610170 f9810178 Signed-off-by:
-
Nicholas Piggin authored
Since the system reset interrupt began to use its own stack, and machine check interrupts have done so for some time, r1 can be changed without clearing MSR[RI], provided no other interrupts (including SLB misses) are taken. MSR[RI] does have to be cleared when using SCRATCH0, however. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Although the 0x1500 interrupt only applies to bare metal, it is better to just use the standard macro for scratch save. Runtime code path remains unchanged (due to instruction patching). Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
Some exception entry requires DAR and/or DSISR to be saved into the paca exception save area. Add options to the standard exception macros for these. Generated code changes slightly due to code structure. - 554: a6 02 72 7d mfdsisr r11 - 558: a8 00 4d f9 std r10,168(r13) - 55c: b0 00 6d 91 stw r11,176(r13) + 554: a8 00 4d f9 std r10,168(r13) + 558: a6 02 52 7d mfdsisr r10 + 55c: b0 00 4d 91 stw r10,176(r13) Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Restore all SPRs and CR up-front, these are longer latency instructions. Move register restore around to maximise pairs of adjacent loads (e.g., restore r0 next to r1). Signed-off-by:
-
Nicholas Piggin authored
Duplicate the hmi windup code for both cases, rather than to put a special case branch in the middle of it. Remove unused label. This helps with later code consolidation. Signed-off-by:
-
Nicholas Piggin authored
Move in_mce decrement earlier before registers are restored (but still after RI=0). This helps with later consolidation. Signed-off-by:
-
Nicholas Piggin authored
Trivial code change, r3->r9. Signed-off-by:
-
Nicholas Piggin authored
All supported 64s CPUs support mtmsrd L=1 instruction, so a cleanup can be made in sreset and mce handlers. Signed-off-by:
-
Nicholas Piggin authored
Move SPR reads ahead of writes. Real mode entry that is not a KVM guest is rare these days, but bad practice propagates. Signed-off-by:
-
Nicholas Piggin authored
syscall / hcall entry unnecessarily differs between KVM and non-KVM builds. Move the SMT priority instruction to the same location (after INTERRUPT_TO_KERNEL). Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Final vmlinux is changed only due to change in bug table line numbers. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Generally, macros that result in instructions being expanded are indented by a tab, and those that don't have no indent. Fix the obvious cases that go contrary to style. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
These are only called in one place each. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
After the previous cleanup, it becomes possible to consolidate some common code outside the runtime alternate patching. Also remove unused labels. This results in some code change, but unchanged runtime instruction sequence. Signed-off-by:
-
Nicholas Piggin authored
Many of these macros just specify 1-4 lines which are only called a few times each at most, and often just once. Remove this indirection. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
More cases of code insertion via macros that does not add a great deal. All the additions have to be specified in the macro arguments, so they can just as well go after the macro. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
The aim is to reduce the amount of indirection it takes to get through the exception handler macros, particularly where it provides little code sharing. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Move the KVM trap HSRR bit into the KVM handler, which can be conditionally applied when hsrr parameter is set. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Conditionally expand the skip case if it is specified. No generated code change. Signed-off-by:
-
Nicholas Piggin authored
Conditionally expand the soft-masking test if a mask is passed in. No generated code change. Signed-off-by:
-
