• Peter Zijlstra's avatar
    locking/qspinlock: Fix spin_unlock_wait() some more · 2c610022
    Peter Zijlstra authored
    While this prior commit:
    
      54cf809b ("locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait()")
    
    ... fixes spin_is_locked() and spin_unlock_wait() for the usage
    in ipc/sem and netfilter, it does not in fact work right for the
    usage in task_work and futex.
    
    So while the 2 locks crossed problem:
    
    	spin_lock(A)		spin_lock(B)
    	if (!spin_is_locked(B)) spin_unlock_wait(A)
    	  foo()			foo();
    
    ... works with the smp_mb() injected by both spin_is_locked() and
    spin_unlock_wait(), this is not sufficient for:
    
    	flag = 1;
    	smp_mb();		spin_lock()
    	spin_unlock_wait()	if (!flag)
    				  // add to lockless list
    	// iterate lockless list
    
    ... because in this scenario, the store from spin_lock() can be delayed
    past the load of flag, uncrossing the variables and loosing the
    guarantee.
    
    This patch reworks spin_is_locked() and spin_unlock_wait() to work in
    both cases by exploiting the observation that while the lock byte
    store can be delayed, the contender must have registered itself
    visibly in other state contained in the word.
    
    It also allows for architectures to override both functions, as PPC
    and ARM64 have an additional issue for which we currently have no
    generic solution.
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Boqun Feng <boqun.feng@gmail.com>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Giovanni Gherdovich <ggherdovich@suse.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Pan Xinhui <xinhui.pan@linux.vnet.ibm.com>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Waiman Long <waiman.long@hpe.com>
    Cc: Will Deacon <will.deacon@arm.com>
    Cc: stable@vger.kernel.org # v4.2 and later
    Fixes: 54cf809b ("locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait()")
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    2c610022
qspinlock.c 15.7 KB