• Roman Gushchin's avatar
    mm: memcg/slab: obj_cgroup API · bf4f0599
    Roman Gushchin authored
    Obj_cgroup API provides an ability to account sub-page sized kernel
    objects, which potentially outlive the original memory cgroup.
    
    The top-level API consists of the following functions:
      bool obj_cgroup_tryget(struct obj_cgroup *objcg);
      void obj_cgroup_get(struct obj_cgroup *objcg);
      void obj_cgroup_put(struct obj_cgroup *objcg);
    
      int obj_cgroup_charge(struct obj_cgroup *objcg, gfp_t gfp, size_t size);
      void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size);
    
      struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg);
      struct obj_cgroup *get_obj_cgroup_from_current(void);
    
    Object cgroup is basically a pointer to a memory cgroup with a per-cpu
    reference counter.  It substitutes a memory cgroup in places where it's
    necessary to charge a custom amount of bytes instead of pages.
    
    All charged memory rounded down to pages is charged to the corresponding
    memory cgroup using __memcg_kmem_charge().
    
    It implements reparenting: on memcg offlining it's getting reattached to
    the parent memory cgroup.  Each online memory cgroup has an associated
    active object cgroup to handle new allocations and the list of all
    attached object cgroups.  On offlining of a cgroup this list is reparented
    and for each object cgroup in the list the memcg pointer is swapped to the
    parent memory cgroup.  It prevents long-living objects from pinning the
    original memory cgroup in the memory.
    
    The implementation is based on byte-sized per-cpu stocks.  A sub-page
    sized leftover is stored in an atomic field, which is a part of obj_cgroup
    object.  So on cgroup offlining the leftover is automatically reparented.
    
    memcg->objcg is rcu protected.  objcg->memcg is a raw pointer, which is
    always pointing at a memory cgroup, but can be atomically swapped to the
    parent memory cgroup.  So a user must ensure the lifetime of the
    cgroup, e.g.  grab rcu_read_lock or css_set_lock.
    Suggested-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Reviewed-by: default avatarShakeel Butt <shakeelb@google.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Link: http://lkml.kernel.org/r/20200623174037.3951353-7-guro@fb.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    bf4f0599
memcontrol.c 193 KB