• Zqiang's avatar
    rcu: Add QS check in rcu_exp_handler() for non-preemptible kernels · fcb42c9a
    Zqiang authored
    Kernels built with CONFIG_PREEMPTION=n and CONFIG_PREEMPT_COUNT=y maintain
    preempt_count() state.  Because such kernels map __rcu_read_lock()
    and __rcu_read_unlock() to preempt_disable() and preempt_enable(),
    respectively, this allows the expedited grace period's !CONFIG_PREEMPT_RCU
    version of the rcu_exp_handler() IPI handler function to use
    preempt_count() to detect quiescent states.
    
    This preempt_count() usage might seem to risk failures due to
    use of implicit RCU readers in portions of the kernel under #ifndef
    CONFIG_PREEMPTION, except that rcu_core() already disallows such implicit
    RCU readers.  The moral of this story is that you must use explicit
    read-side markings such as rcu_read_lock() or preempt_disable() even if
    the code knows that this kernel does not support preemption.
    
    This commit therefore adds a preempt_count()-based check for a quiescent
    state in the !CONFIG_PREEMPT_RCU version of the rcu_exp_handler()
    function for kernels built with CONFIG_PREEMPT_COUNT=y, reporting an
    immediate quiescent state when the interrupted code had both preemption
    and softirqs enabled.
    
    This change results in about a 2% reduction in expedited grace-period
    latency in kernels built with both CONFIG_PREEMPT_RCU=n and
    CONFIG_PREEMPT_COUNT=y.
    Signed-off-by: default avatarZqiang <qiang1.zhang@intel.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    Link: https://lore.kernel.org/all/20220622103549.2840087-1-qiang1.zhang@intel.com/
    fcb42c9a
tree_exp.h 31.5 KB