• Mel Gorman's avatar
    mm, compaction: capture a page under direct compaction · 5e1f0f09
    Mel Gorman authored
    Compaction is inherently race-prone as a suitable page freed during
    compaction can be allocated by any parallel task.  This patch uses a
    capture_control structure to isolate a page immediately when it is freed
    by a direct compactor in the slow path of the page allocator.  The
    intent is to avoid redundant scanning.
    
                                         5.0.0-rc1              5.0.0-rc1
                                   selective-v3r17          capture-v3r19
    Amean     fault-both-1         0.00 (   0.00%)        0.00 *   0.00%*
    Amean     fault-both-3      2582.11 (   0.00%)     2563.68 (   0.71%)
    Amean     fault-both-5      4500.26 (   0.00%)     4233.52 (   5.93%)
    Amean     fault-both-7      5819.53 (   0.00%)     6333.65 (  -8.83%)
    Amean     fault-both-12     9321.18 (   0.00%)     9759.38 (  -4.70%)
    Amean     fault-both-18     9782.76 (   0.00%)    10338.76 (  -5.68%)
    Amean     fault-both-24    15272.81 (   0.00%)    13379.55 *  12.40%*
    Amean     fault-both-30    15121.34 (   0.00%)    16158.25 (  -6.86%)
    Amean     fault-both-32    18466.67 (   0.00%)    18971.21 (  -2.73%)
    
    Latency is only moderately affected but the devil is in the details.  A
    closer examination indicates that base page fault latency is reduced but
    latency of huge pages is increased as it takes creater care to succeed.
    Part of the "problem" is that allocation success rates are close to 100%
    even when under pressure and compaction gets harder
    
                                    5.0.0-rc1              5.0.0-rc1
                              selective-v3r17          capture-v3r19
    Percentage huge-3        96.70 (   0.00%)       98.23 (   1.58%)
    Percentage huge-5        96.99 (   0.00%)       95.30 (  -1.75%)
    Percentage huge-7        94.19 (   0.00%)       97.24 (   3.24%)
    Percentage huge-12       94.95 (   0.00%)       97.35 (   2.53%)
    Percentage huge-18       96.74 (   0.00%)       97.30 (   0.58%)
    Percentage huge-24       97.07 (   0.00%)       97.55 (   0.50%)
    Percentage huge-30       95.69 (   0.00%)       98.50 (   2.95%)
    Percentage huge-32       96.70 (   0.00%)       99.27 (   2.65%)
    
    And scan rates are reduced as expected by 6% for the migration scanner
    and 29% for the free scanner indicating that there is less redundant
    work.
    
    Compaction migrate scanned    20815362    19573286
    Compaction free scanned       16352612    11510663
    
    [mgorman@techsingularity.net: remove redundant check]
      Link: http://lkml.kernel.org/r/20190201143853.GH9565@techsingularity.net
    Link: http://lkml.kernel.org/r/20190118175136.31341-23-mgorman@techsingularity.netSigned-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Dan Carpenter <dan.carpenter@oracle.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: YueHaibing <yuehaibing@huawei.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    5e1f0f09
page_alloc.c 232 KB