• Mel Gorman's avatar
    mm, compaction: use free lists to quickly locate a migration source · 70b44595
    Mel Gorman authored
    The migration scanner is a linear scan of a zone with a potentiall large
    search space.  Furthermore, many pageblocks are unusable such as those
    filled with reserved pages or partially filled with pages that cannot
    migrate.  These still get scanned in the common case of allocating a THP
    and the cost accumulates.
    
    The patch uses a partial search of the free lists to locate a migration
    source candidate that is marked as MOVABLE when allocating a THP.  It
    prefers picking a block with a larger number of free pages already on
    the basis that there are fewer pages to migrate to free the entire
    block.  The lowest PFN found during searches is tracked as the basis of
    the start for the linear search after the first search of the free list
    fails.  After the search, the free list is shuffled so that the next
    search will not encounter the same page.  If the search fails then the
    subsequent searches will be shorter and the linear scanner is used.
    
    If this search fails, or if the request is for a small or
    unmovable/reclaimable allocation then the linear scanner is still used.
    It is somewhat pointless to use the list search in those cases.  Small
    free pages must be used for the search and there is no guarantee that
    movable pages are located within that block that are contiguous.
    
                                         5.0.0-rc1              5.0.0-rc1
                                     noboost-v3r10          findmig-v3r15
    Amean     fault-both-3      3771.41 (   0.00%)     3390.40 (  10.10%)
    Amean     fault-both-5      5409.05 (   0.00%)     5082.28 (   6.04%)
    Amean     fault-both-7      7040.74 (   0.00%)     7012.51 (   0.40%)
    Amean     fault-both-12    11887.35 (   0.00%)    11346.63 (   4.55%)
    Amean     fault-both-18    16718.19 (   0.00%)    15324.19 (   8.34%)
    Amean     fault-both-24    21157.19 (   0.00%)    16088.50 *  23.96%*
    Amean     fault-both-30    21175.92 (   0.00%)    18723.42 *  11.58%*
    Amean     fault-both-32    21339.03 (   0.00%)    18612.01 *  12.78%*
    
                                    5.0.0-rc1              5.0.0-rc1
                                noboost-v3r10          findmig-v3r15
    Percentage huge-3        86.50 (   0.00%)       89.83 (   3.85%)
    Percentage huge-5        92.52 (   0.00%)       91.96 (  -0.61%)
    Percentage huge-7        92.44 (   0.00%)       92.85 (   0.44%)
    Percentage huge-12       92.98 (   0.00%)       92.74 (  -0.25%)
    Percentage huge-18       91.70 (   0.00%)       91.71 (   0.02%)
    Percentage huge-24       91.59 (   0.00%)       92.13 (   0.60%)
    Percentage huge-30       90.14 (   0.00%)       93.79 (   4.04%)
    Percentage huge-32       90.03 (   0.00%)       91.27 (   1.37%)
    
    This shows an improvement in allocation latencies with similar
    allocation success rates.  While not presented, there was a 31%
    reduction in migration scanning and a 8% reduction on system CPU usage.
    A 2-socket machine showed similar benefits.
    
    [mgorman@techsingularity.net: several fixes]
      Link: http://lkml.kernel.org/r/20190204120111.GL9565@techsingularity.net
    [vbabka@suse.cz: migrate block that was found-fast, some optimisations]
    Link: http://lkml.kernel.org/r/20190118175136.31341-10-mgorman@techsingularity.netSigned-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Acked-by: default avatarVlastimil Babka <Vbabka@suse.cz>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Dan Carpenter <dan.carpenter@oracle.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: YueHaibing <yuehaibing@huawei.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    70b44595
compaction.c 63.2 KB