• Paul E. McKenney's avatar
    rcu: Make cond_resched_rcu_qs() apply to normal RCU flavors · 5cd37193
    Paul E. McKenney authored
    Although cond_resched_rcu_qs() only applies to TASKS_RCU, it is used
    in places where it would be useful for it to apply to the normal RCU
    flavors, rcu_preempt, rcu_sched, and rcu_bh.  This is especially the
    case for workloads that aggressively overload the system, particularly
    those that generate large numbers of RCU updates on systems running
    NO_HZ_FULL CPUs.  This commit therefore communicates quiescent states
    from cond_resched_rcu_qs() to the normal RCU flavors.
    
    Note that it is unfortunately necessary to leave the old ->passed_quiesce
    mechanism in place to allow quiescent states that apply to only one
    flavor to be recorded.  (Yes, we could decrement ->rcu_qs_ctr_snap in
    that case, but that is not so good for debugging of RCU internals.)
    In addition, if one of the RCU flavor's grace period has stalled, this
    will invoke rcu_momentary_dyntick_idle(), resulting in a heavy-weight
    quiescent state visible from other CPUs.
    Reported-by: default avatarSasha Levin <sasha.levin@oracle.com>
    Reported-by: default avatarDave Jones <davej@redhat.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    [ paulmck: Merge commit from Sasha Levin fixing a bug where __this_cpu()
      was used in preemptible code. ]
    5cd37193
rcutree.h 3.38 KB