• Johannes Weiner's avatar
    mm: memcontrol: fix transparent huge page allocations under pressure · b70a2a21
    Johannes Weiner authored
    In a memcg with even just moderate cache pressure, success rates for
    transparent huge page allocations drop to zero, wasting a lot of effort
    that the allocator puts into assembling these pages.
    
    The reason for this is that the memcg reclaim code was never designed for
    higher-order charges.  It reclaims in small batches until there is room
    for at least one page.  Huge page charges only succeed when these batches
    add up over a series of huge faults, which is unlikely under any
    significant load involving order-0 allocations in the group.
    
    Remove that loop on the memcg side in favor of passing the actual reclaim
    goal to direct reclaim, which is already set up and optimized to meet
    higher-order goals efficiently.
    
    This brings memcg's THP policy in line with the system policy: if the
    allocator painstakingly assembles a hugepage, memcg will at least make an
    honest effort to charge it.  As a result, transparent hugepage allocation
    rates amid cache activity are drastically improved:
    
                                          vanilla                 patched
    pgalloc                 4717530.80 (  +0.00%)   4451376.40 (  -5.64%)
    pgfault                  491370.60 (  +0.00%)    225477.40 ( -54.11%)
    pgmajfault                    2.00 (  +0.00%)         1.80 (  -6.67%)
    thp_fault_alloc               0.00 (  +0.00%)       531.60 (+100.00%)
    thp_fault_fallback          749.00 (  +0.00%)       217.40 ( -70.88%)
    
    [ Note: this may in turn increase memory consumption from internal
      fragmentation, which is an inherent risk of transparent hugepages.
      Some setups may have to adjust the memcg limits accordingly to
      accomodate this - or, if the machine is already packed to capacity,
      disable the transparent huge page feature. ]
    Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Reviewed-by: default avatarVladimir Davydov <vdavydov@parallels.com>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Dave Hansen <dave@sr71.net>
    Cc: Greg Thelen <gthelen@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    b70a2a21
memcontrol.c 174 KB