• Pavel Begunkov's avatar
    io_uring: fix concurrent parking · 9e138a48
    Pavel Begunkov authored
    If io_sq_thread_park() of one task got rescheduled right after
    set_bit(), before it gets back to mutex_lock() there can happen
    park()/unpark() by another task with SQPOLL locking again and
    continuing running never seeing that first set_bit(SHOULD_PARK),
    so won't even try to put the mutex down for parking.
    
    It will get parked eventually when SQPOLL drops the lock for reschedule,
    but may be problematic and will get in the way of further fixes.
    
    Account number of tasks waiting for parking with a new atomic variable
    park_pending and adjust SHOULD_PARK accordingly. It doesn't entirely
    replaces SHOULD_PARK bit with this atomic var because it's convenient
    to have it as a bit in the state and will help to do optimisations
    later.
    Signed-off-by: default avatarPavel Begunkov <asml.silence@gmail.com>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    9e138a48
io_uring.c 240 KB