• Vlastimil Babka's avatar
    mm, compaction: reduce zone checking frequency in the migration scanner · 7d49d886
    Vlastimil Babka authored
    The unification of the migrate and free scanner families of function has
    highlighted a difference in how the scanners ensure they only isolate
    pages of the intended zone.  This is important for taking zone lock or lru
    lock of the correct zone.  Due to nodes overlapping, it is however
    possible to encounter a different zone within the range of the zone being
    compacted.
    
    The free scanner, since its inception by commit 748446bb ("mm:
    compaction: memory compaction core"), has been checking the zone of the
    first valid page in a pageblock, and skipping the whole pageblock if the
    zone does not match.
    
    This checking was completely missing from the migration scanner at first,
    and later added by commit dc908600 ("mm: compaction: check for
    overlapping nodes during isolation for migration") in a reaction to a bug
    report.  But the zone comparison in migration scanner is done once per a
    single scanned page, which is more defensive and thus more costly than a
    check per pageblock.
    
    This patch unifies the checking done in both scanners to once per
    pageblock, through a new pageblock_pfn_to_page() function, which also
    includes pfn_valid() checks.  It is more defensive than the current free
    scanner checks, as it checks both the first and last page of the
    pageblock, but less defensive by the migration scanner per-page checks.
    It assumes that node overlapping may result (on some architecture) in a
    boundary between two nodes falling into the middle of a pageblock, but
    that there cannot be a node0 node1 node0 interleaving within a single
    pageblock.
    
    The result is more code being shared and a bit less per-page CPU cost in
    the migration scanner.
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Acked-by: default avatarMel Gorman <mgorman@suse.de>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Michal Nazarewicz <mina86@mina86.com>
    Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Rik van Riel <riel@redhat.com>
    Acked-by: default avatarDavid Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    7d49d886
compaction.c 36.7 KB