• Vlastimil Babka's avatar
    mm, compaction: always update cached scanner positions · 6bace090
    Vlastimil Babka authored
    Compaction caches the migration and free scanner positions between
    compaction invocations, so that the whole zone gets eventually scanned and
    there is no bias towards the initial scanner positions at the
    beginning/end of the zone.
    
    The cached positions are continuously updated as scanners progress and the
    updating stops as soon as a page is successfully isolated.  The reasoning
    behind this is that a pageblock where isolation succeeded is likely to
    succeed again in near future and it should be worth revisiting it.
    
    However, the downside is that potentially many pages are rescanned without
    successful isolation.  At worst, there might be a page where isolation
    from LRU succeeds but migration fails (potentially always).  So upon
    encountering this page, cached position would always stop being updated
    for no good reason.  It might have been useful to let such page be
    rescanned with sync compaction after async one failed, but this is now
    handled by caching scanner position for async and sync mode separately
    since commit 35979ef3 ("mm, compaction: add per-zone migration pfn
    cache for async compaction").
    
    After this patch, cached positions are updated unconditionally.  In
    stress-highalloc benchmark, this has decreased the numbers of scanned
    pages by few percent, without affecting allocation success rates.
    
    To prevent free scanner from leaving free pages behind after they are
    returned due to page migration failure, the cached scanner pfn is changed
    to point to the pageblock of the returned free page with the highest pfn,
    before leaving compact_zone().
    
    [akpm@linux-foundation.org: coding-style fixes]
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Michal Nazarewicz <mina86@mina86.com>
    Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Christoph Lameter <cl@linux.com>
    Acked-by: default avatarRik van Riel <riel@redhat.com>
    Cc: David Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    6bace090
compaction.c 42.3 KB