• Boqun Feng's avatar
    rcu: exp: Protect all sync_rcu_preempt_exp_done() with rcu_node lock · 55ebfce0
    Boqun Feng authored
    Currently some callsites of sync_rcu_preempt_exp_done() are not called
    with the corresponding rcu_node's ->lock held, which could introduces
    bugs as per Paul:
    
    o	CPU 0 in sync_rcu_preempt_exp_done() reads ->exp_tasks and
    	sees that it is NULL.
    
    o	CPU 1 blocks within an RCU read-side critical section, so
    	it enqueues the task and points ->exp_tasks at it and
    	clears CPU 1's bit in ->expmask.
    
    o	All other CPUs clear their bits in ->expmask.
    
    o	CPU 0 reads ->expmask, sees that it is zero, so incorrectly
    	concludes that all quiescent states have completed, despite
    	the fact that ->exp_tasks is non-NULL.
    
    To fix this, sync_rcu_preempt_exp_unlocked() is introduced to replace
    lockless callsites of sync_rcu_preempt_exp_done().
    
    Further, a lockdep annotation is added into sync_rcu_preempt_exp_done()
    to prevent mis-use in the future.
    Signed-off-by: default avatarBoqun Feng <boqun.feng@gmail.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Tested-by: default avatarNicholas Piggin <npiggin@gmail.com>
    55ebfce0
tree_exp.h 24.4 KB