• Jens Axboe's avatar
    blk-mq: fix false negative out-of-tags condition · 0bf36498
    Jens Axboe authored
    The blk-mq tagging tries to maintain some locality between CPUs and
    the tags issued. The tags are split into groups of words, and the
    words may not be fully populated. When searching for a new free tag,
    blk-mq may look at partial words, hence it passes in an offset/size
    to find_next_zero_bit(). However, it does that wrong, the size must
    always be the full length of the number of tags in that word,
    otherwise we'll potentially miss some near the end.
    
    Another issue is when __bt_get() goes from one word set to the next.
    It bumps the index, but not the last_tag associated with the
    previous index. Bump that to be in the range of the new word.
    
    Finally, clean up __bt_get() and __bt_get_word() a bit and get
    rid of the goto in there, and the unnecessary 'wrap' variable.
    Signed-off-by: default avatarJens Axboe <axboe@fb.com>
    0bf36498
blk-mq-tag.c 14.9 KB