• Paul E. McKenney's avatar
    rcu: Add funnel locking to rcu_start_this_gp() · 360e0da6
    Paul E. McKenney authored
    The rcu_start_this_gp() function had a simple form of funnel locking that
    used only the leaves and root of the rcu_node tree, which is fine for
    systems with only a few hundred CPUs, but sub-optimal for systems having
    thousands of CPUs.  This commit therefore adds full-tree funnel locking.
    
    This variant of funnel locking is unusual in the following ways:
    
    1.	The leaf-level rcu_node structure's ->lock is held throughout.
    	Other funnel-locking implementations drop the leaf-level lock
    	before progressing to the next level of the tree.
    
    2.	Funnel locking can be started at the root, which is convenient
    	for code that already holds the root rcu_node structure's ->lock.
    	Other funnel-locking implementations start at the leaves.
    
    3.	If an rcu_node structure other than the initial one believes
    	that a grace period is in progress, it is not necessary to
    	go further up the tree.  This is because grace-period cleanup
    	scans the full tree, so that marking the need for a subsequent
    	grace period anywhere in the tree suffices -- but only if
    	a grace period is currently in progress.
    
    4.	It is possible that the RCU grace-period kthread has not yet
    	started, and this case must be handled appropriately.
    
    However, the general approach of using a tree to control lock contention
    is still in place.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Tested-by: default avatarNicholas Piggin <npiggin@gmail.com>
    360e0da6
tree.c 129 KB