• Roman Gushchin's avatar
    mm: memcg/percpu: account percpu memory to memory cgroups · 3c7be18a
    Roman Gushchin authored
    Percpu memory is becoming more and more widely used by various subsystems,
    and the total amount of memory controlled by the percpu allocator can make
    a good part of the total memory.
    
    As an example, bpf maps can consume a lot of percpu memory, and they are
    created by a user.  Also, some cgroup internals (e.g.  memory controller
    statistics) can be quite large.  On a machine with many CPUs and big
    number of cgroups they can consume hundreds of megabytes.
    
    So the lack of memcg accounting is creating a breach in the memory
    isolation.  Similar to the slab memory, percpu memory should be accounted
    by default.
    
    To implement the perpcu accounting it's possible to take the slab memory
    accounting as a model to follow.  Let's introduce two types of percpu
    chunks: root and memcg.  What makes memcg chunks different is an
    additional space allocated to store memcg membership information.  If
    __GFP_ACCOUNT is passed on allocation, a memcg chunk should be be used.
    If it's possible to charge the corresponding size to the target memory
    cgroup, allocation is performed, and the memcg ownership data is recorded.
    System-wide allocations are performed using root chunks, so there is no
    additional memory overhead.
    
    To implement a fast reparenting of percpu memory on memcg removal, we
    don't store mem_cgroup pointers directly: instead we use obj_cgroup API,
    introduced for slab accounting.
    
    [akpm@linux-foundation.org: fix CONFIG_MEMCG_KMEM=n build errors and warning]
    [akpm@linux-foundation.org: move unreachable code, per Roman]
    [cuibixuan@huawei.com: mm/percpu: fix 'defined but not used' warning]
      Link: http://lkml.kernel.org/r/6d41b939-a741-b521-a7a2-e7296ec16219@huawei.comSigned-off-by: default avatarRoman Gushchin <guro@fb.com>
    Signed-off-by: default avatarBixuan Cui <cuibixuan@huawei.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Reviewed-by: default avatarShakeel Butt <shakeelb@google.com>
    Acked-by: default avatarDennis Zhou <dennis@kernel.org>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Tobin C. Harding <tobin@kernel.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Waiman Long <longman@redhat.com>
    Cc: Bixuan Cui <cuibixuan@huawei.com>
    Cc: Michal Koutný <mkoutny@suse.com>
    Cc: Stephen Rothwell <sfr@canb.auug.org.au>
    Link: http://lkml.kernel.org/r/20200623184515.4132564-3-guro@fb.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    3c7be18a
percpu-km.c 3 KB