Commit 44b7a8d3 authored by Johannes Weiner's avatar Johannes Weiner Committed by Linus Torvalds

mm: memcontrol: do not uncharge old page in page cache replacement

Changing page->mem_cgroup of a live page is tricky and fragile.  In
particular, the memcg writeback code relies on that mapping being stable
and users of mem_cgroup_replace_page() not overlapping with dirtyable
inodes.

Page cache replacement doesn't have to do that, though.  Instead of being
clever and transferring the charge from the old page to the new,
force-charge the new page and leave the old page alone.  A temporary
overcharge won't matter in practice, and the old page is going to be freed
shortly after this anyway.  And this is not performance critical.
Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Vladimir Davydov <vdavydov@virtuozzo.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 3e24b19d
...@@ -366,13 +366,6 @@ mem_cgroup_zone_zoneinfo(struct mem_cgroup *memcg, struct zone *zone) ...@@ -366,13 +366,6 @@ mem_cgroup_zone_zoneinfo(struct mem_cgroup *memcg, struct zone *zone)
* *
* If memcg is bound to a traditional hierarchy, the css of root_mem_cgroup * If memcg is bound to a traditional hierarchy, the css of root_mem_cgroup
* is returned. * is returned.
*
* XXX: The above description of behavior on the default hierarchy isn't
* strictly true yet as replace_page_cache_page() can modify the
* association before @page is released even on the default hierarchy;
* however, the current and planned usages don't mix the the two functions
* and replace_page_cache_page() will soon be updated to make the invariant
* actually true.
*/ */
struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page) struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page)
{ {
...@@ -5464,7 +5457,8 @@ void mem_cgroup_uncharge_list(struct list_head *page_list) ...@@ -5464,7 +5457,8 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
void mem_cgroup_replace_page(struct page *oldpage, struct page *newpage) void mem_cgroup_replace_page(struct page *oldpage, struct page *newpage)
{ {
struct mem_cgroup *memcg; struct mem_cgroup *memcg;
int isolated; unsigned int nr_pages;
bool compound;
VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage); VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
VM_BUG_ON_PAGE(!PageLocked(newpage), newpage); VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);
...@@ -5484,11 +5478,21 @@ void mem_cgroup_replace_page(struct page *oldpage, struct page *newpage) ...@@ -5484,11 +5478,21 @@ void mem_cgroup_replace_page(struct page *oldpage, struct page *newpage)
if (!memcg) if (!memcg)
return; return;
lock_page_lru(oldpage, &isolated); /* Force-charge the new page. The old one will be freed soon */
oldpage->mem_cgroup = NULL; compound = PageTransHuge(newpage);
unlock_page_lru(oldpage, isolated); nr_pages = compound ? hpage_nr_pages(newpage) : 1;
page_counter_charge(&memcg->memory, nr_pages);
if (do_memsw_account())
page_counter_charge(&memcg->memsw, nr_pages);
css_get_many(&memcg->css, nr_pages);
commit_charge(newpage, memcg, true); commit_charge(newpage, memcg, true);
local_irq_disable();
mem_cgroup_charge_statistics(memcg, newpage, compound, nr_pages);
memcg_check_events(memcg, newpage);
local_irq_enable();
} }
DEFINE_STATIC_KEY_FALSE(memcg_sockets_enabled_key); DEFINE_STATIC_KEY_FALSE(memcg_sockets_enabled_key);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment