• Linus Torvalds's avatar
    mm: avoid repeated anon_vma lock/unlock sequences in anon_vma_clone() · bb4aa396
    Linus Torvalds authored
    In anon_vma_clone() we traverse the vma->anon_vma_chain of the source
    vma, locking the anon_vma for each entry.
    
    But they are all going to have the same root entry, which means that
    we're locking and unlocking the same lock over and over again.  Which is
    expensive in locked operations, but can get _really_ expensive when that
    root entry sees any kind of lock contention.
    
    In fact, Tim Chen reports a big performance regression due to this: when
    we switched to use a mutex instead of a spinlock, the contention case
    gets much worse.
    
    So to alleviate this all, this commit creates a small helper function
    (lock_anon_vma_root()) that can be used to take the lock just once
    rather than taking and releasing it over and over again.
    
    We still have the same "take the lock and release" it behavior in the
    exit path (in unlink_anon_vmas()), but that one is a bit harder to fix
    since we're actually freeing the anon_vma entries as we go, and that
    will touch the lock too.
    Reported-and-tested-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
    Tested-by: default avatarHugh Dickins <hughd@google.com>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Andi Kleen <ak@linux.intel.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    bb4aa396
rmap.c 49.8 KB