• Johannes Weiner's avatar
    mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting · db2ba40c
    Johannes Weiner authored
    During cgroup2 rollout into production, we started encountering css
    refcount underflows and css access crashes in the memory controller.
    Splitting the heavily shared css reference counter into logical users
    narrowed the imbalance down to the cgroup2 socket memory accounting.
    
    The problem turns out to be the per-cpu charge cache.  Cgroup1 had a
    separate socket counter, but the new cgroup2 socket accounting goes
    through the common charge path that uses a shared per-cpu cache for all
    memory that is being tracked.  Those caches are safe against scheduling
    preemption, but not against interrupts - such as the newly added packet
    receive path.  When cache draining is interrupted by network RX taking
    pages out of the cache, the resuming drain operation will put references
    of in-use pages, thus causing the imbalance.
    
    Disable IRQs during all per-cpu charge cache operations.
    
    Fixes: f7e1cb6e ("mm: memcontrol: account socket memory in unified hierarchy memory controller")
    Link: http://lkml.kernel.org/r/20160914194846.11153-1-hannes@cmpxchg.orgSigned-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Acked-by: default avatarTejun Heo <tj@kernel.org>
    Cc: "David S. Miller" <davem@davemloft.net>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Vladimir Davydov <vdavydov@virtuozzo.com>
    Cc: <stable@vger.kernel.org>	[4.5+]
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    db2ba40c
memcontrol.c 154 KB