• Thomas Gleixner's avatar
    x86/entry: Force rcu_irq_enter() when in idle task · 0bf3924b
    Thomas Gleixner authored
    The idea of conditionally calling into rcu_irq_enter() only when RCU is
    not watching turned out to be not completely thought through.
    
    Paul noticed occasional premature end of grace periods in RCU torture
    testing. Bisection led to the commit which made the invocation of
    rcu_irq_enter() conditional on !rcu_is_watching().
    
    It turned out that this conditional breaks RCU assumptions about the idle
    task when the scheduler tick happens to be a nested interrupt. Nested
    interrupts can happen when the first interrupt invokes softirq processing
    on return which enables interrupts.
    
    If that nested tick interrupt does not invoke rcu_irq_enter() then the
    RCU's irq-nesting checks will believe that this interrupt came directly
    from idle, which will cause RCU to report a quiescent state.  Because this
    interrupt instead came from a softirq handler which might have been
    executing an RCU read-side critical section, this can cause the grace
    period to end prematurely.
    
    Change the condition from !rcu_is_watching() to is_idle_task(current) which
    enforces that interrupts in the idle task unconditionally invoke
    rcu_irq_enter() independent of the RCU state.
    
    This is also correct vs. user mode entries in NOHZ full scenarios because
    user mode entries bring RCU out of EQS and force the RCU irq nesting state
    accounting to nested. As only the first interrupt can enter from user mode
    a nested tick interrupt will enter from kernel mode and as the nesting
    state accounting is forced to nesting it will not do anything stupid even
    if rcu_irq_enter() has not been invoked.
    
    Fixes: 3eeec385 ("x86/entry: Provide idtentry_entry/exit_cond_rcu()")
    Reported-by: default avatar"Paul E. McKenney" <paulmck@kernel.org>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Tested-by: default avatar"Paul E. McKenney" <paulmck@kernel.org>
    Reviewed-by: default avatar"Paul E. McKenney" <paulmck@kernel.org>
    Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
    Acked-by: default avatarFrederic Weisbecker <frederic@kernel.org>
    Link: https://lkml.kernel.org/r/87wo4cxubv.fsf@nanos.tec.linutronix.de
    0bf3924b
common.c 21 KB