• Benjamin Herrenschmidt's avatar
    powerpc: Fix atomic_xxx_return barrier semantics · b97021f8
    Benjamin Herrenschmidt authored
    The Documentation/memory-barriers.txt document requires that atomic
    operations that return a value act as a memory barrier both before
    and after the actual atomic operation.
    
    Our current implementation doesn't guarantee this. More specifically,
    while a load following the isync can not be issued before stwcx. has
    completed, that completion doesn't architecturally means that the
    result of stwcx. is visible to other processors (or any previous stores
    for that matter) (typically, the other processors L1 caches can still
    hold the old value).
    
    This has caused an actual crash in RCU torture testing on Power 7
    
    This fixes it by changing those atomic ops to use new macros instead
    of RELEASE/ACQUIRE barriers, called ATOMIC_ENTRY and ATMOIC_EXIT barriers,
    which are then defined respectively to lwsync and sync.
    
    I haven't had a chance to measure the performance impact (or rather
    what I measured with kernel compiles is in the noise, I yet have to
    find a more precise benchmark)
    Signed-off-by: default avatarBenjamin Herrenschmidt <benh@kernel.crashing.org>
    Acked-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    b97021f8
bitops.h 9.1 KB