• Thomas Gleixner's avatar
    genirq/cpuhotplug: Enforce affinity setting on startup of managed irqs · e43b3b58
    Thomas Gleixner authored
    Managed interrupts can end up in a stale state on CPU hotplug. If the
    interrupt is not targeting a single CPU, i.e. the affinity mask spawns
    multiple CPUs then the following can happen:
    
    After boot:
    
    dstate:   0x01601200
                IRQD_ACTIVATED
                IRQD_IRQ_STARTED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_SET
                IRQD_AFFINITY_MANAGED
    node:     0
    affinity: 24-31
    effectiv: 24
    pending:  0
    
    After offlining CPU 31 - 24
    
    dstate:   0x01a31000
                IRQD_IRQ_DISABLED
                IRQD_IRQ_MASKED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_SET
                IRQD_AFFINITY_MANAGED
                IRQD_MANAGED_SHUTDOWN
    node:     0
    affinity: 24-31
    effectiv: 24
    pending:  0
    
    Now CPU 25 gets onlined again, so it should get the effective interrupt
    affinity for this interruopt, but due to the x86 interrupt affinity setter
    restrictions this ends up after restarting the interrupt with:
    
    dstate:   0x01601300
                IRQD_ACTIVATED
                IRQD_IRQ_STARTED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_SET
                IRQD_SETAFFINITY_PENDING
                IRQD_AFFINITY_MANAGED
    node:     0
    affinity: 24-31
    effectiv: 24
    pending:  24-31
    
    So the interrupt is still affine to CPU 24, which was the last CPU to go
    offline of that affinity set and the move to an online CPU within 24-31,
    in this case 25, is pending. This mechanism is x86/ia64 specific as those
    architectures cannot move interrupts from thread context and do this when
    an interrupt is actually handled. So the move is set to pending.
    
    Whats worse is that offlining CPU 25 again results in:
    
    dstate:   0x01601300
                IRQD_ACTIVATED
                IRQD_IRQ_STARTED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_SET
                IRQD_SETAFFINITY_PENDING
                IRQD_AFFINITY_MANAGED
    node:     0
    affinity: 24-31
    effectiv: 24
    pending:  24-31
    
    This means the interrupt has not been shut down, because the outgoing CPU
    is not in the effective affinity mask, but of course nothing notices that
    the effective affinity mask is pointing at an offline CPU.
    
    In the case of restarting a managed interrupt the move restriction does not
    apply, so the affinity setting can be made unconditional. This needs to be
    done _before_ the interrupt is started up as otherwise the condition for
    moving it from thread context would not longer be fulfilled.
    
    With that change applied onlining CPU 25 after offlining 31-24 results in:
    
    dstate:   0x01600200
                IRQD_ACTIVATED
                IRQD_IRQ_STARTED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_MANAGED
    node:     0
    affinity: 24-31
    effectiv: 25
    pending:  
    
    And after offlining CPU 25:
    
    dstate:   0x01a30000
                IRQD_IRQ_DISABLED
                IRQD_IRQ_MASKED
                IRQD_SINGLE_TARGET
                IRQD_AFFINITY_MANAGED
                IRQD_MANAGED_SHUTDOWN
    node:     0
    affinity: 24-31
    effectiv: 25
    pending:  
    
    which is the correct and expected result.
    
    Fixes: 761ea388 ("genirq: Handle managed irqs gracefully in irq_startup()")
    Reported-by: default avatarYASUAKI ISHIMATSU <yasu.isimatu@gmail.com>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Cc: axboe@kernel.dk
    Cc: linux-scsi@vger.kernel.org
    Cc: Sumit Saxena <sumit.saxena@broadcom.com>
    Cc: Marc Zyngier <marc.zyngier@arm.com>
    Cc: mpe@ellerman.id.au
    Cc: Shivasharan Srikanteshwara <shivasharan.srikanteshwara@broadcom.com>
    Cc: Kashyap Desai <kashyap.desai@broadcom.com>
    Cc: keith.busch@intel.com
    Cc: peterz@infradead.org
    Cc: Hannes Reinecke <hare@suse.de>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: stable@vger.kernel.org
    Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1710042208400.2406@nanos
    e43b3b58
manage.c 56.7 KB