• Roman Gushchin's avatar
    mm: memcontrol: Use helpers to read page's memcg data · bcfe06bf
    Roman Gushchin authored
    Patch series "mm: allow mapping accounted kernel pages to userspace", v6.
    
    Currently a non-slab kernel page which has been charged to a memory cgroup
    can't be mapped to userspace.  The underlying reason is simple: PageKmemcg
    flag is defined as a page type (like buddy, offline, etc), so it takes a
    bit from a page->mapped counter.  Pages with a type set can't be mapped to
    userspace.
    
    But in general the kmemcg flag has nothing to do with mapping to
    userspace.  It only means that the page has been accounted by the page
    allocator, so it has to be properly uncharged on release.
    
    Some bpf maps are mapping the vmalloc-based memory to userspace, and their
    memory can't be accounted because of this implementation detail.
    
    This patchset removes this limitation by moving the PageKmemcg flag into
    one of the free bits of the page->mem_cgroup pointer.  Also it formalizes
    accesses to the page->mem_cgroup and page->obj_cgroups using new helpers,
    adds several checks and removes a couple of obsolete functions.  As the
    result the code became more robust with fewer open-coded bit tricks.
    
    This patch (of 4):
    
    Currently there are many open-coded reads of the page->mem_cgroup pointer,
    as well as a couple of read helpers, which are barely used.
    
    It creates an obstacle on a way to reuse some bits of the pointer for
    storing additional bits of information.  In fact, we already do this for
    slab pages, where the last bit indicates that a pointer has an attached
    vector of objcg pointers instead of a regular memcg pointer.
    
    This commits uses 2 existing helpers and introduces a new helper to
    converts all read sides to calls of these helpers:
      struct mem_cgroup *page_memcg(struct page *page);
      struct mem_cgroup *page_memcg_rcu(struct page *page);
      struct mem_cgroup *page_memcg_check(struct page *page);
    
    page_memcg_check() is intended to be used in cases when the page can be a
    slab page and have a memcg pointer pointing at objcg vector.  It does
    check the lowest bit, and if set, returns NULL.  page_memcg() contains a
    VM_BUG_ON_PAGE() check for the page not being a slab page.
    
    To make sure nobody uses a direct access, struct page's
    mem_cgroup/obj_cgroups is converted to unsigned long memcg_data.
    Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    Reviewed-by: default avatarShakeel Butt <shakeelb@google.com>
    Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Acked-by: default avatarMichal Hocko <mhocko@suse.com>
    Link: https://lkml.kernel.org/r/20201027001657.3398190-1-guro@fb.com
    Link: https://lkml.kernel.org/r/20201027001657.3398190-2-guro@fb.com
    Link: https://lore.kernel.org/bpf/20201201215900.3569844-2-guro@fb.com
    bcfe06bf
page_io.c 11.6 KB