• Paul E. McKenney's avatar
    rcu: Fix TREE_PREEMPT_RCU CPU_HOTPLUG bad-luck hang · 237c80c5
    Paul E. McKenney authored
    If the following sequence of events occurs, then
    TREE_PREEMPT_RCU will hang waiting for a grace period to
    complete, eventually OOMing the system:
    
    o	A TREE_PREEMPT_RCU build of the kernel is booted on a system
    	with more than 64 physical CPUs present (32 on a 32-bit system).
    	Alternatively, a TREE_PREEMPT_RCU build of the kernel is booted
    	with RCU_FANOUT set to a sufficiently small value that the
    	physical CPUs populate two or more leaf rcu_node structures.
    
    o	A task is preempted in an RCU read-side critical section
    	while running on a CPU corresponding to a given leaf rcu_node
    	structure.
    
    o	All CPUs corresponding to this same leaf rcu_node structure
    	record quiescent states for the current grace period.
    
    o	All of these same CPUs go offline (hence the need for enough
    	physical CPUs to populate more than one leaf rcu_node structure).
    	This causes the preempted task to be moved to the root rcu_node
    	structure.
    
    At this point, there is nothing left to cause the quiescent
    state to be propagated up the rcu_node tree, so the current
    grace period never completes.
    
    The simplest fix, especially after considering the deadlock
    possibilities, is to detect this situation when the last CPU is
    offlined, and to set that CPU's ->qsmask bit in its leaf
    rcu_node structure.  This will cause the next invocation of
    force_quiescent_state() to end the grace period.
    
    Without this fix, this hang can be triggered in an hour or so on
    some machines with rcutorture and random CPU onlining/offlining.
    With this fix, these same machines pass a full 10 hours of this
    sort of abuse.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: laijs@cn.fujitsu.com
    Cc: dipankar@in.ibm.com
    Cc: mathieu.desnoyers@polymtl.ca
    Cc: josh@joshtriplett.org
    Cc: dvhltc@us.ibm.com
    Cc: niv@us.ibm.com
    Cc: peterz@infradead.org
    Cc: rostedt@goodmis.org
    Cc: Valdis.Kletnieks@vt.edu
    Cc: dhowells@redhat.com
    LKML-Reference: <20091015162614.GA19131@linux.vnet.ibm.com>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    237c80c5
rcutree_plugin.h 17.2 KB