• Shaohua Li's avatar
    mm: batch activate_page() to reduce lock contention · 744ed144
    Shaohua Li authored
    The zone->lru_lock is heavily contented in workload where activate_page()
    is frequently used.  We could do batch activate_page() to reduce the lock
    contention.  The batched pages will be added into zone list when the pool
    is full or page reclaim is trying to drain them.
    
    For example, in a 4 socket 64 CPU system, create a sparse file and 64
    processes, processes shared map to the file.  Each process read access the
    whole file and then exit.  The process exit will do unmap_vmas() and cause
    a lot of activate_page() call.  In such workload, we saw about 58% total
    time reduction with below patch.  Other workloads with a lot of
    activate_page also benefits a lot too.
    
    I tested some microbenchmarks:
    case-anon-cow-rand-mt		0.58%
    case-anon-cow-rand		-3.30%
    case-anon-cow-seq-mt		-0.51%
    case-anon-cow-seq		-5.68%
    case-anon-r-rand-mt		0.23%
    case-anon-r-rand		0.81%
    case-anon-r-seq-mt		-0.71%
    case-anon-r-seq			-1.99%
    case-anon-rx-rand-mt		2.11%
    case-anon-rx-seq-mt		3.46%
    case-anon-w-rand-mt		-0.03%
    case-anon-w-rand		-0.50%
    case-anon-w-seq-mt		-1.08%
    case-anon-w-seq			-0.12%
    case-anon-wx-rand-mt		-5.02%
    case-anon-wx-seq-mt		-1.43%
    case-fork			1.65%
    case-fork-sleep			-0.07%
    case-fork-withmem		1.39%
    case-hugetlb			-0.59%
    case-lru-file-mmap-read-mt	-0.54%
    case-lru-file-mmap-read		0.61%
    case-lru-file-mmap-read-rand	-2.24%
    case-lru-file-readonce		-0.64%
    case-lru-file-readtwice		-11.69%
    case-lru-memcg			-1.35%
    case-mmap-pread-rand-mt		1.88%
    case-mmap-pread-rand		-15.26%
    case-mmap-pread-seq-mt		0.89%
    case-mmap-pread-seq		-69.72%
    case-mmap-xread-rand-mt		0.71%
    case-mmap-xread-seq-mt		0.38%
    
    The most significent are:
    case-lru-file-readtwice		-11.69%
    case-mmap-pread-rand		-15.26%
    case-mmap-pread-seq		-69.72%
    
    which use activate_page a lot.  others are basically variations because
    each run has slightly difference.
    
    [akpm@linux-foundation.org: coding-style fixes]
    Signed-off-by: default avatarShaohua Li <shaohua.li@intel.com>
    Cc: Andi Kleen <andi@firstfloor.org>
    Cc: Minchan Kim <minchan.kim@gmail.com>
    Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Cc: Rik van Riel <riel@redhat.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    744ed144
internal.h 7.47 KB