Commit e2b19197 authored by Mel Gorman's avatar Mel Gorman Committed by Linus Torvalds

mm, page_alloc: remove unnecessary parameter from zone_watermark_ok_safe

Overall, the intent of this series is to remove the zonelist cache which
was introduced to avoid high overhead in the page allocator.  Once this is
done, it is necessary to reduce the cost of watermark checks.

The series starts with minor micro-optimisations.

Next it notes that GFP flags that affect watermark checks are abused.
__GFP_WAIT historically identified callers that could not sleep and could
access reserves.  This was later abused to identify callers that simply
prefer to avoid sleeping and have other options.  A patch distinguishes
between atomic callers, high-priority callers and those that simply wish
to avoid sleep.

The zonelist cache has been around for a long time but it is of dubious
merit with a lot of complexity and some issues that are explained.  The
most important issue is that a failed THP allocation can cause a zone to
be treated as "full".  This potentially causes unnecessary stalls, reclaim
activity or remote fallbacks.  The issues could be fixed but it's not
worth it.  The series places a small number of other micro-optimisations
on top before examining GFP flags watermarks.

High-order watermarks enforcement can cause high-order allocations to fail
even though pages are free.  The watermark checks both protect high-order
atomic allocations and make kswapd aware of high-order pages but there is
a much better way that can be handled using migrate types.  This series
uses page grouping by mobility to reserve pageblocks for high-order
allocations with the size of the reservation depending on demand.  kswapd
awareness is maintained by examining the free lists.  By patch 12 in this
series, there are no high-order watermark checks while preserving the
properties that motivated the introduction of the watermark checks.

This patch (of 10):

No user of zone_watermark_ok_safe() specifies alloc_flags.  This patch
removes the unnecessary parameter.
Signed-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
Acked-by: default avatarDavid Rientjes <rientjes@google.com>
Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
Acked-by: default avatarMichal Hocko <mhocko@suse.com>
Reviewed-by: default avatarChristoph Lameter <cl@linux.com>
Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent db2a0dd7
...@@ -817,7 +817,7 @@ void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx); ...@@ -817,7 +817,7 @@ void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
bool zone_watermark_ok(struct zone *z, unsigned int order, bool zone_watermark_ok(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx, int alloc_flags); unsigned long mark, int classzone_idx, int alloc_flags);
bool zone_watermark_ok_safe(struct zone *z, unsigned int order, bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx, int alloc_flags); unsigned long mark, int classzone_idx);
enum memmap_context { enum memmap_context {
MEMMAP_EARLY, MEMMAP_EARLY,
MEMMAP_HOTPLUG, MEMMAP_HOTPLUG,
......
...@@ -2249,6 +2249,7 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order, ...@@ -2249,6 +2249,7 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
min -= min / 2; min -= min / 2;
if (alloc_flags & ALLOC_HARDER) if (alloc_flags & ALLOC_HARDER)
min -= min / 4; min -= min / 4;
#ifdef CONFIG_CMA #ifdef CONFIG_CMA
/* If allocation can't use CMA areas don't use free CMA pages */ /* If allocation can't use CMA areas don't use free CMA pages */
if (!(alloc_flags & ALLOC_CMA)) if (!(alloc_flags & ALLOC_CMA))
...@@ -2278,14 +2279,14 @@ bool zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark, ...@@ -2278,14 +2279,14 @@ bool zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
} }
bool zone_watermark_ok_safe(struct zone *z, unsigned int order, bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx, int alloc_flags) unsigned long mark, int classzone_idx)
{ {
long free_pages = zone_page_state(z, NR_FREE_PAGES); long free_pages = zone_page_state(z, NR_FREE_PAGES);
if (z->percpu_drift_mark && free_pages < z->percpu_drift_mark) if (z->percpu_drift_mark && free_pages < z->percpu_drift_mark)
free_pages = zone_page_state_snapshot(z, NR_FREE_PAGES); free_pages = zone_page_state_snapshot(z, NR_FREE_PAGES);
return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags, return __zone_watermark_ok(z, order, mark, classzone_idx, 0,
free_pages); free_pages);
} }
......
...@@ -2477,7 +2477,7 @@ static inline bool compaction_ready(struct zone *zone, int order) ...@@ -2477,7 +2477,7 @@ static inline bool compaction_ready(struct zone *zone, int order)
balance_gap = min(low_wmark_pages(zone), DIV_ROUND_UP( balance_gap = min(low_wmark_pages(zone), DIV_ROUND_UP(
zone->managed_pages, KSWAPD_ZONE_BALANCE_GAP_RATIO)); zone->managed_pages, KSWAPD_ZONE_BALANCE_GAP_RATIO));
watermark = high_wmark_pages(zone) + balance_gap + (2UL << order); watermark = high_wmark_pages(zone) + balance_gap + (2UL << order);
watermark_ok = zone_watermark_ok_safe(zone, 0, watermark, 0, 0); watermark_ok = zone_watermark_ok_safe(zone, 0, watermark, 0);
/* /*
* If compaction is deferred, reclaim up to a point where * If compaction is deferred, reclaim up to a point where
...@@ -2960,7 +2960,7 @@ static bool zone_balanced(struct zone *zone, int order, ...@@ -2960,7 +2960,7 @@ static bool zone_balanced(struct zone *zone, int order,
unsigned long balance_gap, int classzone_idx) unsigned long balance_gap, int classzone_idx)
{ {
if (!zone_watermark_ok_safe(zone, order, high_wmark_pages(zone) + if (!zone_watermark_ok_safe(zone, order, high_wmark_pages(zone) +
balance_gap, classzone_idx, 0)) balance_gap, classzone_idx))
return false; return false;
if (IS_ENABLED(CONFIG_COMPACTION) && order && compaction_suitable(zone, if (IS_ENABLED(CONFIG_COMPACTION) && order && compaction_suitable(zone,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment