• Joonsoo Kim's avatar
    mm/slub: don't wait for high-order page allocation · 45eb00cd
    Joonsoo Kim authored
    Description is almost copied from commit fb05e7a8 ("net: don't wait
    for order-3 page allocation").
    
    I saw excessive direct memory reclaim/compaction triggered by slub.  This
    causes performance issues and add latency.  Slub uses high-order
    allocation to reduce internal fragmentation and management overhead.  But,
    direct memory reclaim/compaction has high overhead and the benefit of
    high-order allocation can't compensate the overhead of both work.
    
    This patch makes auxiliary high-order allocation atomic.  If there is no
    memory pressure and memory isn't fragmented, the alloction will still
    success, so we don't sacrifice high-order allocation's benefit here.  If
    the atomic allocation fails, direct memory reclaim/compaction will not be
    triggered, allocation fallback to low-order immediately, hence the direct
    memory reclaim/compaction overhead is avoided.  In the allocation failure
    case, kswapd is waken up and trying to make high-order freepages, so
    allocation could success next time.
    
    Following is the test to measure effect of this patch.
    
    System: QEMU, CPU 8, 512 MB
    Mem: 25% memory is allocated at random position to make fragmentation.
     Memory-hogger occupies 150 MB memory.
    Workload: hackbench -g 20 -l 1000
    
    Average result by 10 runs (Base va Patched)
    
    elapsed_time(s): 4.3468 vs 2.9838
    compact_stall: 461.7 vs 73.6
    pgmigrate_success: 28315.9 vs 7256.1
    Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Acked-by: default avatarDavid Rientjes <rientjes@google.com>
    Cc: Shaohua Li <shli@fb.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Eric Dumazet <edumazet@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    45eb00cd
slub.c 130 KB