• Nick Piggin's avatar
    mm: introduce get_user_pages_fast · 21cc199b
    Nick Piggin authored
    Introduce a new get_user_pages_fast mm API, which is basically a
    get_user_pages with a less general API (but still tends to be suited to
    the common case):
    
    - task and mm are always current and current->mm
    - force is always 0
    - pages is always non-NULL
    - don't pass back vmas
    
    This restricted API can be implemented in a much more scalable way on many
    architectures when the ptes are present, by walking the page tables
    locklessly (no mmap_sem or page table locks).  When the ptes are not
    populated, get_user_pages_fast() could be slower.
    
    This is implemented locklessly on x86, and used in some key direct IO call
    sites, in later patches, which provides nearly 10% performance improvement
    on a threaded database workload.
    
    Lots of other code could use this too, depending on use cases (eg.  grep
    drivers/).  And it might inspire some new and clever ways to use it.
    
    [akpm@linux-foundation.org: build fix]
    [akpm@linux-foundation.org: coding-style fixes]
    Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
    Cc: Dave Kleikamp <shaggy@austin.ibm.com>
    Cc: Andy Whitcroft <apw@shadowen.org>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Andi Kleen <andi@firstfloor.org>
    Cc: Dave Kleikamp <shaggy@austin.ibm.com>
    Cc: Badari Pulavarty <pbadari@us.ibm.com>
    Cc: Zach Brown <zach.brown@oracle.com>
    Cc: Jens Axboe <jens.axboe@oracle.com>
    Reviewed-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    21cc199b
mm.h 44.3 KB