• Alexander Gordeev's avatar
    blk-mq: bitmap tag: fix race on blk_mq_bitmap_tags::wake_cnt · 2971c35f
    Alexander Gordeev authored
    This piece of code in bt_clear_tag() function is racy:
    
    	bs = bt_wake_ptr(bt);
    	if (bs && atomic_dec_and_test(&bs->wait_cnt)) {
    		atomic_set(&bs->wait_cnt, bt->wake_cnt);
     		wake_up(&bs->wait);
    	}
    
    Since nothing prevents bt_wake_ptr() from returning the very
    same 'bs' address on multiple CPUs, the following scenario is
    possible:
    
        CPU1                                CPU2
        ----                                ----
    
    0.  bs = bt_wake_ptr(bt);               bs = bt_wake_ptr(bt);
    1.  atomic_dec_and_test(&bs->wait_cnt)
    2.                                      atomic_dec_and_test(&bs->wait_cnt)
    3.  atomic_set(&bs->wait_cnt, bt->wake_cnt);
    
    If the decrement in [1] yields zero then for some amount of time
    the decrement in [2] results in a negative/overflow value, which
    is not expected. The follow-up assignment in [3] overwrites the
    invalid value with the batch value (and likely prevents the issue
    from being severe) which is still incorrect and should be a lesser.
    
    Cc: Ming Lei <tom.leiming@gmail.com>
    Cc: Jens Axboe <axboe@kernel.dk>
    Signed-off-by: default avatarAlexander Gordeev <agordeev@redhat.com>
    Signed-off-by: default avatarJens Axboe <axboe@fb.com>
    2971c35f
blk-mq-tag.c 13.7 KB