• Paul E. McKenney's avatar
    rcu: Tag callback lists with corresponding grace-period number · dc35c893
    Paul E. McKenney authored
    Currently, callbacks are advanced each time the corresponding CPU
    notices a change in its leaf rcu_node structure's ->completed value
    (this value counts grace-period completions).  This approach has worked
    quite well, but with the advent of RCU_FAST_NO_HZ, we cannot count on
    a given CPU seeing all the grace-period completions.  When a CPU misses
    a grace-period completion that occurs while it is in dyntick-idle mode,
    this will delay invocation of its callbacks.
    
    In addition, acceleration of callbacks (when RCU realizes that a given
    callback need only wait until the end of the next grace period, rather
    than having to wait for a partial grace period followed by a full
    grace period) must be carried out extremely carefully.  Insufficient
    acceleration will result in unnecessarily long grace-period latencies,
    while excessive acceleration will result in premature callback invocation.
    Changes that involve this tradeoff are therefore among the most
    nerve-wracking changes to RCU.
    
    This commit therefore explicitly tags groups of callbacks with the
    number of the grace period that they are waiting for.  This means that
    callback-advancement and callback-acceleration functions are idempotent,
    so that excessive acceleration will merely waste a few CPU cycles.  This
    also allows a CPU to take full advantage of any grace periods that have
    elapsed while it has been in dyntick-idle mode.  It should also enable
    simulataneous simplifications to and optimizations of RCU_FAST_NO_HZ.
    Signed-off-by: default avatarPaul E. McKenney <paul.mckenney@linaro.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    dc35c893
rcutree.c 99 KB