• Kent Overstreet's avatar
    aio: make aio_put_req() lockless · 11599eba
    Kent Overstreet authored
    Freeing a kiocb needed to touch the kioctx for three things:
    
     * Pull it off the reqs_active list
     * Decrementing reqs_active
     * Issuing a wakeup, if the kioctx was in the process of being freed.
    
    This patch moves these to aio_complete(), for a couple reasons:
    
     * aio_complete() already has to issue the wakeup, so if we drop the
       kioctx refcount before aio_complete does its wakeup we don't have to
       do it twice.
     * aio_complete currently has to take the kioctx lock, so it makes sense
       for it to pull the kiocb off the reqs_active list too.
     * A later patch is going to change reqs_active to include unreaped
       completions - this will mean allocating a kiocb doesn't have to look
       at the ringbuffer. So taking the decrement of reqs_active out of
       kiocb_free() is useful prep work for that patch.
    
    This doesn't really affect cancellation, since existing (usb) code that
    implements a cancel function still calls aio_complete() - we just have
    to make sure that aio_complete does the necessary teardown for cancelled
    kiocbs.
    
    It does affect code paths where we free kiocbs that were never
    submitted; they need to decrement reqs_active and pull the kiocb off the
    reqs_active list.  This occurs in two places: kiocb_batch_free(), which
    is going away in a later patch, and the error path in io_submit_one.
    
    [akpm@linux-foundation.org: coding-style fixes]
    Signed-off-by: default avatarKent Overstreet <koverstreet@google.com>
    Cc: Zach Brown <zab@redhat.com>
    Cc: Felipe Balbi <balbi@ti.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Mark Fasheh <mfasheh@suse.com>
    Cc: Joel Becker <jlbec@evilplan.org>
    Cc: Rusty Russell <rusty@rustcorp.com.au>
    Cc: Jens Axboe <axboe@kernel.dk>
    Cc: Asai Thambi S P <asamymuthupa@micron.com>
    Cc: Selvan Mani <smani@micron.com>
    Cc: Sam Bradshaw <sbradshaw@micron.com>
    Acked-by: default avatarJeff Moyer <jmoyer@redhat.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: Benjamin LaHaise <bcrl@kvack.org>
    Reviewed-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    11599eba
aio.c 38.1 KB