1. 30 Apr, 2012 2 commits
    • Paul E. McKenney's avatar
      rcu: Direct algorithmic SRCU implementation · cef50120
      Paul E. McKenney authored
      The current implementation of synchronize_srcu_expedited() can cause
      severe OS jitter due to its use of synchronize_sched(), which in turn
      invokes try_stop_cpus(), which causes each CPU to be sent an IPI.
      This can result in severe performance degradation for real-time workloads
      and especially for short-interation-length HPC workloads.  Furthermore,
      because only one instance of try_stop_cpus() can be making forward progress
      at a given time, only one instance of synchronize_srcu_expedited() can
      make forward progress at a time, even if they are all operating on
      distinct srcu_struct structures.
      
      This commit, inspired by an earlier implementation by Peter Zijlstra
      (https://lkml.org/lkml/2012/1/31/211) and by further offline discussions,
      takes a strictly algorithmic bits-in-memory approach.  This has the
      disadvantage of requiring one explicit memory-barrier instruction in
      each of srcu_read_lock() and srcu_read_unlock(), but on the other hand
      completely dispenses with OS jitter and furthermore allows SRCU to be
      used freely by CPUs that RCU believes to be idle or offline.
      
      The update-side implementation handles the single read-side memory
      barrier by rechecking the per-CPU counters after summing them and
      by running through the update-side state machine twice.
      
      This implementation has passed moderate rcutorture testing on both
      x86 and Power.  Also updated to use this_cpu_ptr() instead of per_cpu_ptr(),
      as suggested by Peter Zijlstra.
      Reported-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Signed-off-by: default avatarPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Acked-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      Reviewed-by: default avatarLai Jiangshan <laijs@cn.fujitsu.com>
      cef50120
    • Paul E. McKenney's avatar
      rcu: Introduce rcutorture testing for rcu_barrier() · fae4b54f
      Paul E. McKenney authored
      Although rcutorture does invoke rcu_barrier() and friends, it cannot
      really be called a torture test given that it invokes them only once
      at the end of the test.  This commit therefore introduces heavy-duty
      rcutorture testing for rcu_barrier(), which may be carried out
      concurrently with normal rcutorture testing.
      Signed-off-by: default avatarPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      fae4b54f
  2. 25 Apr, 2012 1 commit
  3. 21 Apr, 2012 23 commits
  4. 20 Apr, 2012 14 commits