• Feng Tang's avatar
    mm: adjust vm_committed_as_batch according to vm overcommit policy · 56f3547b
    Feng Tang authored
    When checking a performance change for will-it-scale scalability mmap test
    [1], we found very high lock contention for spinlock of percpu counter
    'vm_committed_as':
    
        94.14%     0.35%  [kernel.kallsyms]         [k] _raw_spin_lock_irqsave
        48.21% _raw_spin_lock_irqsave;percpu_counter_add_batch;__vm_enough_memory;mmap_region;do_mmap;
        45.91% _raw_spin_lock_irqsave;percpu_counter_add_batch;__do_munmap;
    
    Actually this heavy lock contention is not always necessary.  The
    'vm_committed_as' needs to be very precise when the strict
    OVERCOMMIT_NEVER policy is set, which requires a rather small batch number
    for the percpu counter.
    
    So keep 'batch' number unchanged for strict OVERCOMMIT_NEVER policy, and
    lift it to 64X for OVERCOMMIT_ALWAYS and OVERCOMMIT_GUESS policies.  Also
    add a sysctl handler to adjust it when the policy is reconfigured.
    
    Benchmark with the same testcase in [1] shows 53% improvement on a 8C/16T
    desktop, and 2097%(20X) on a 4S/72C/144T server.  We tested with test
    platforms in 0day (server, desktop and laptop), and 80%+ platforms shows
    improvements with that test.  And whether it shows improvements depends on
    if the test mmap size is bigger than the batch number computed.
    
    And if the lift is 16X, 1/3 of the platforms will show improvements,
    though it should help the mmap/unmap usage generally, as Michal Hocko
    mentioned:
    
    : I believe that there are non-synthetic worklaods which would benefit from
    : a larger batch.  E.g.  large in memory databases which do large mmaps
    : during startups from multiple threads.
    
    [1] https://lore.kernel.org/lkml/20200305062138.GI5972@shao2-debian/Signed-off-by: default avatarFeng Tang <feng.tang@intel.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Acked-by: default avatarMichal Hocko <mhocko@suse.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Qian Cai <cai@lca.pw>
    Cc: Kees Cook <keescook@chromium.org>
    Cc: Andi Kleen <andi.kleen@intel.com>
    Cc: Tim Chen <tim.c.chen@intel.com>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Huang Ying <ying.huang@intel.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Dennis Zhou <dennis@kernel.org>
    Cc: Haiyang Zhang <haiyangz@microsoft.com>
    Cc: kernel test robot <rong.a.chen@intel.com>
    Cc: "K. Y. Srinivasan" <kys@microsoft.com>
    Cc: Tejun Heo <tj@kernel.org>
    Link: http://lkml.kernel.org/r/1589611660-89854-4-git-send-email-feng.tang@intel.com
    Link: http://lkml.kernel.org/r/1592725000-73486-4-git-send-email-feng.tang@intel.com
    Link: http://lkml.kernel.org/r/1594389708-60781-5-git-send-email-feng.tang@intel.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    56f3547b
sysctl.c 79.8 KB