• John Hubbard's avatar
    mm/gup: refactor and de-duplicate gup_fast() code · 376a34ef
    John Hubbard authored
    There were two nearly identical sets of code for gup_fast() style of
    walking the page tables with interrupts disabled.  This has lead to the
    usual maintenance problems that arise from having duplicated code.
    
    There is already a core internal routine in gup.c for gup_fast(), so just
    enhance it very slightly: allow skipping the fall-back to "slow" (regular)
    get_user_pages(), via the new FOLL_FAST_ONLY flag.  Then, just call
    internal_get_user_pages_fast() from __get_user_pages_fast(), and adjust
    the API to match pre-existing API behavior.
    
    There is a change in behavior from this refactoring: the nested form of
    interrupt disabling is used in all gup_fast() variants now.  That's
    because there is only one place that interrupt disabling for page walking
    is done, and so the safer form is required.  This should, if anything,
    eliminate possible (rare) bugs, because the non-nested form of enabling
    interrupts was fragile at best.
    
    [jhubbard@nvidia.com: fixup]
      Link: http://lkml.kernel.org/r/20200521233841.1279742-1-jhubbard@nvidia.comSigned-off-by: default avatarJohn Hubbard <jhubbard@nvidia.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Reviewed-by: default avatarChris Wilson <chris@chris-wilson.co.uk>
    Cc: Daniel Vetter <daniel@ffwll.ch>
    Cc: David Airlie <airlied@linux.ie>
    Cc: Jani Nikula <jani.nikula@linux.intel.com>
    Cc: "Joonas Lahtinen" <joonas.lahtinen@linux.intel.com>
    Cc: Matthew Auld <matthew.auld@intel.com>
    Cc: Matthew Wilcox <willy@infradead.org>
    Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
    Cc: Souptick Joarder <jrdr.linux@gmail.com>
    Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
    Link: http://lkml.kernel.org/r/20200519002124.2025955-3-jhubbard@nvidia.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    376a34ef
gup.c 85 KB