• Paul E. McKenney's avatar
    refperf: Hoist function-pointer calls out of the loop · 75dd8efe
    Paul E. McKenney authored
    Current runs show PREEMPT=n rcu_read_lock()/rcu_read_unlock() pairs
    consuming between 20 and 30 nanoseconds, when in fact the actual value is
    zero, give or take the barrier() asm's effect on compiler optimizations.
    The additional overhead is caused by function calls through pointers
    (especially in these days of Spectre mitigations) and perhaps also
    needless argument passing, a non-const loop limit, and an upcounting loop.
    
    This commit therefore combines the ->readlock() and ->readunlock()
    function pointers into a single ->readsection() function pointer that
    takes the loop count as a const parameter and keeps any data passed
    from the read-lock to the read-unlock internal to this new function.
    
    These changes reduce the measured overhead of the aforementioned
    PREEMPT=n rcu_read_lock()/rcu_read_unlock() pairs from between 20 and
    30 nanoseconds to somewhere south of 500 picoseconds.
    
    Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    75dd8efe
refperf.c 12.4 KB