• Johannes Weiner's avatar
    mm: memcontrol: decouple reference counting from page accounting · 1a3e1f40
    Johannes Weiner authored
    The reference counting of a memcg is currently coupled directly to how
    many 4k pages are charged to it.  This doesn't work well with Roman's new
    slab controller, which maintains pools of objects and doesn't want to keep
    an extra balance sheet for the pages backing those objects.
    
    This unusual refcounting design (reference counts usually track pointers
    to an object) is only for historical reasons: memcg used to not take any
    css references and simply stalled offlining until all charges had been
    reparented and the page counters had dropped to zero.  When we got rid of
    the reparenting requirement, the simple mechanical translation was to take
    a reference for every charge.
    
    More historical context can be found in commit e8ea14cc ("mm:
    memcontrol: take a css reference for each charged page"), commit
    64f21993 ("mm: memcontrol: remove obsolete kmemcg pinning tricks") and
    commit b2052564 ("mm: memcontrol: continue cache reclaim from offlined
    groups").
    
    The new slab controller exposes the limitations in this scheme, so let's
    switch it to a more idiomatic reference counting model based on actual
    kernel pointers to the memcg:
    
    - The per-cpu stock holds a reference to the memcg its caching
    
    - User pages hold a reference for their page->mem_cgroup. Transparent
      huge pages will no longer acquire tail references in advance, we'll
      get them if needed during the split.
    
    - Kernel pages hold a reference for their page->mem_cgroup
    
    - Pages allocated in the root cgroup will acquire and release css
      references for simplicity. css_get() and css_put() optimize that.
    
    - The current memcg_charge_slab() already hacked around the per-charge
      references; this change gets rid of that as well.
    
    - tcp accounting will handle reference in mem_cgroup_sk_{alloc,free}
    
    Roman:
    1) Rebased on top of the current mm tree: added css_get() in
       mem_cgroup_charge(), dropped mem_cgroup_try_charge() part
    2) I've reformatted commit references in the commit log to make
       checkpatch.pl happy.
    
    [hughd@google.com: remove css_put_many() from __mem_cgroup_clear_mc()]
      Link: http://lkml.kernel.org/r/alpine.LSU.2.11.2007302011450.2347@eggly.anvilsSigned-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
    Signed-off-by: default avatarHugh Dickins <hughd@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Reviewed-by: default avatarShakeel Butt <shakeelb@google.com>
    Acked-by: default avatarRoman Gushchin <guro@fb.com>
    Acked-by: default avatarMichal Hocko <mhocko@suse.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Link: http://lkml.kernel.org/r/20200623174037.3951353-6-guro@fb.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    1a3e1f40
memcontrol.c 186 KB