• Oleg Nesterov's avatar
    percpu_rw_semaphore: reimplement to not block the readers unnecessarily · a1fd3e24
    Oleg Nesterov authored
    Currently the writer does msleep() plus synchronize_sched() 3 times to
    acquire/release the semaphore, and during this time the readers are
    blocked completely.  Even if the "write" section was not actually started
    or if it was already finished.
    
    With this patch down_write/up_write does synchronize_sched() twice and
    down_read/up_read are still possible during this time, just they use the
    slow path.
    
    percpu_down_write() first forces the readers to use rw_semaphore and
    increment the "slow" counter to take the lock for reading, then it
    takes that rw_semaphore for writing and blocks the readers.
    
    Also.  With this patch the code relies on the documented behaviour of
    synchronize_sched(), it doesn't try to pair synchronize_sched() with
    barrier.
    Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
    Reviewed-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Mikulas Patocka <mpatocka@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
    Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
    Cc: Anton Arapov <anton@redhat.com>
    Cc: Jens Axboe <axboe@kernel.dk>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    a1fd3e24
percpu-rwsem.c 4.44 KB