Commit d778df51 authored by Johannes Weiner's avatar Johannes Weiner Committed by Linus Torvalds

mm: vmscan: save work scanning (almost) empty LRU lists

In certain cases (kswapd reclaim, memcg target reclaim), a fixed minimum
amount of pages is scanned from the LRU lists on each iteration, to make
progress.

Do not make this minimum bigger than the respective LRU list size,
however, and save some busy work trying to isolate and reclaim pages
that are not there.

Empty LRU lists are quite common with memory cgroups in NUMA
environments because there exists a set of LRU lists for each zone for
each memory cgroup, while the memory of a single cgroup is expected to
stay on just one node.  The number of expected empty LRU lists is thus

  memcgs * (nodes - 1) * lru types

Each attempt to reclaim from an empty LRU list does expensive size
comparisons between lists, acquires the zone's lru lock etc.  Avoid
that.
Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
Reviewed-by: default avatarRik van Riel <riel@redhat.com>
Acked-by: default avatarMel Gorman <mgorman@suse.de>
Reviewed-by: default avatarMichal Hocko <mhocko@suse.cz>
Cc: Hugh Dickins <hughd@google.com>
Cc: Satoru Moriya <satoru.moriya@hds.com>
Cc: Simon Jeons <simon.jeons@gmail.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 7c5bd705
...@@ -156,7 +156,7 @@ enum { ...@@ -156,7 +156,7 @@ enum {
SWP_SCANNING = (1 << 8), /* refcount in scan_swap_map */ SWP_SCANNING = (1 << 8), /* refcount in scan_swap_map */
}; };
#define SWAP_CLUSTER_MAX 32 #define SWAP_CLUSTER_MAX 32UL
#define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX #define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX
/* /*
......
...@@ -1761,15 +1761,17 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, ...@@ -1761,15 +1761,17 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc,
out: out:
for_each_evictable_lru(lru) { for_each_evictable_lru(lru) {
int file = is_file_lru(lru); int file = is_file_lru(lru);
unsigned long size;
unsigned long scan; unsigned long scan;
scan = get_lru_size(lruvec, lru); size = get_lru_size(lruvec, lru);
if (sc->priority || noswap || !vmscan_swappiness(sc)) { if (sc->priority || noswap || !vmscan_swappiness(sc)) {
scan >>= sc->priority; scan = size >> sc->priority;
if (!scan && force_scan) if (!scan && force_scan)
scan = SWAP_CLUSTER_MAX; scan = min(size, SWAP_CLUSTER_MAX);
scan = div64_u64(scan * fraction[file], denominator); scan = div64_u64(scan * fraction[file], denominator);
} } else
scan = size;
nr[lru] = scan; nr[lru] = scan;
} }
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment