• Vlastimil Babka's avatar
    mm, compaction: always skip all compound pages by order in migrate scanner · 29c0dde8
    Vlastimil Babka authored
    The compaction migrate scanner tries to skip THP pages by their order,
    to reduce number of iterations for pages it cannot isolate.  The check
    is only done if PageLRU() is true, which means it applies to THP pages,
    but not e.g.  hugetlbfs pages or any other non-LRU compound pages, which
    we have to iterate by base pages.
    
    This limitation comes from the assumption that it's only safe to read
    compound_order() when we have the zone's lru_lock and THP cannot be
    split under us.  But the only danger (after filtering out order values
    that are not below MAX_ORDER, to prevent overflows) is that we skip too
    much or too little after reading a bogus compound_order() due to a rare
    race.  This is the same reasoning as patch 99c0fd5e ("mm,
    compaction: skip buddy pages by their order in the migrate scanner")
    introduced for unsafely reading PageBuddy() order.
    
    After this patch, all pages are tested for PageCompound() and we skip
    them by compound_order().  The test is done after the test for
    balloon_page_movable() as we don't want to assume if balloon pages (or
    other pages with own isolation and migration implementation if a generic
    API gets implemented) are compound or not.
    
    When tested with stress-highalloc from mmtests on 4GB system with 1GB
    hugetlbfs pages, the vmstat compact_migrate_scanned count decreased by
    15%.
    
    [kirill.shutemov@linux.intel.com: change PageTransHuge checks to PageCompound for different series was squashed here]
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Acked-by: default avatarMel Gorman <mgorman@suse.de>
    Acked-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Acked-by: default avatarMichal Nazarewicz <mina86@mina86.com>
    Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: David Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    29c0dde8
compaction.c 47.7 KB