• Bart Van Assche's avatar
    blk-mq: Fix a race between bt_clear_tag() and bt_get() · 6148ec1c
    Bart Van Assche authored
    commit c38d185d upstream.
    
    What we need is the following two guarantees:
    * Any thread that observes the effect of the test_and_set_bit() by
      __bt_get_word() also observes the preceding addition of 'current'
      to the appropriate wait list. This is guaranteed by the semantics
      of the spin_unlock() operation performed by prepare_and_wait().
      Hence the conversion of test_and_set_bit_lock() into
      test_and_set_bit().
    * The wait lists are examined by bt_clear() after the tag bit has
      been cleared. clear_bit_unlock() guarantees that any thread that
      observes that the bit has been cleared also observes the store
      operations preceding clear_bit_unlock(). However,
      clear_bit_unlock() does not prevent that the wait lists are examined
      before that the tag bit is cleared. Hence the addition of a memory
      barrier between clear_bit() and the wait list examination.
    Signed-off-by: default avatarBart Van Assche <bvanassche@acm.org>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Robert Elliott <elliott@hp.com>
    Cc: Ming Lei <ming.lei@canonical.com>
    Cc: Alexander Gordeev <agordeev@redhat.com>
    Signed-off-by: default avatarJens Axboe <axboe@fb.com>
    Signed-off-by: default avatarLuis Henriques <luis.henriques@canonical.com>
    6148ec1c
blk-mq-tag.c 13.7 KB