• Prateek Sood's avatar
    locking/rwsem-xadd: Fix missed wakeup due to reordering of load · 9c29c318
    Prateek Sood authored
    If a spinner is present, there is a chance that the load of
    rwsem_has_spinner() in rwsem_wake() can be reordered with
    respect to decrement of rwsem count in __up_write() leading
    to wakeup being missed:
    
     spinning writer                  up_write caller
     ---------------                  -----------------------
     [S] osq_unlock()                 [L] osq
      spin_lock(wait_lock)
      sem->count=0xFFFFFFFF00000001
                +0xFFFFFFFF00000000
      count=sem->count
      MB
                                       sem->count=0xFFFFFFFE00000001
                                                 -0xFFFFFFFF00000001
                                       spin_trylock(wait_lock)
                                       return
     rwsem_try_write_lock(count)
     spin_unlock(wait_lock)
     schedule()
    
    Reordering of atomic_long_sub_return_release() in __up_write()
    and rwsem_has_spinner() in rwsem_wake() can cause missing of
    wakeup in up_write() context. In spinning writer, sem->count
    and local variable count is 0XFFFFFFFE00000001. It would result
    in rwsem_try_write_lock() failing to acquire rwsem and spinning
    writer going to sleep in rwsem_down_write_failed().
    
    The smp_rmb() will make sure that the spinner state is
    consulted after sem->count is updated in up_write context.
    Signed-off-by: default avatarPrateek Sood <prsood@codeaurora.org>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: dave@stgolabs.net
    Cc: longman@redhat.com
    Cc: parri.andrea@gmail.com
    Cc: sramana@codeaurora.org
    Link: http://lkml.kernel.org/r/1504794658-15397-1-git-send-email-prsood@codeaurora.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    9c29c318
rwsem-xadd.c 19.5 KB