• Paul E. McKenney's avatar
    rcu: Confine ->core_needs_qs accesses to the corresponding CPU · ed93dfc6
    Paul E. McKenney authored
    Commit 671a6351 ("rcu: Avoid unnecessary softirq when system
    is idle") fixed a bug that could result in an indefinite number of
    unnecessary invocations of the RCU_SOFTIRQ handler at the trailing edge
    of a scheduler-clock interrupt.  However, the fix introduced off-CPU
    stores to ->core_needs_qs.  These writes did not conflict with the
    on-CPU stores because the CPU's leaf rcu_node structure's ->lock was
    held across all such stores.  However, the loads from ->core_needs_qs
    were not promoted to READ_ONCE() and, worse yet, the code loading from
    ->core_needs_qs was written assuming that it was only ever updated by
    the corresponding CPU.  So operation has been robust, but only by luck.
    This situation is therefore an accident waiting to happen.
    
    This commit therefore takes a different approach.  Instead of clearing
    ->core_needs_qs from the grace-period kthread's force-quiescent-state
    processing, it modifies the rcu_pending() function to suppress the
    rcu_sched_clock_irq() function's call to invoke_rcu_core() if there is no
    grace period in progress.  This avoids the infinite needless RCU_SOFTIRQ
    handlers while still keeping all accesses to ->core_needs_qs local to
    the corresponding CPU.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    ed93dfc6
tree.c 115 KB