• Vlastimil Babka's avatar
    mm/slub: introduce __kmem_cache_free_bulk() without free hooks · 520a688a
    Vlastimil Babka authored
    Currently, when __kmem_cache_alloc_bulk() fails, it frees back the
    objects that were allocated before the failure, using
    kmem_cache_free_bulk(). Because kmem_cache_free_bulk() calls the free
    hooks (KASAN etc.) and those expect objects that were processed by the
    post alloc hooks, slab_post_alloc_hook() is called before
    kmem_cache_free_bulk().
    
    This is wasteful, although not a big concern in practice for the rare
    error path. But in order to efficiently handle percpu array batch refill
    and free in the near future, we will also need a variant of
    kmem_cache_free_bulk() that avoids the free hooks. So introduce it now
    and use it for the failure path.
    
    In case of failure we however still need to perform memcg uncharge so
    handle that in a new memcg_slab_alloc_error_hook(). Thanks to Chengming
    Zhou for noticing the missing uncharge.
    
    As a consequence, __kmem_cache_alloc_bulk() no longer needs the objcg
    parameter, remove it.
    Reviewed-by: default avatarChengming Zhou <zhouchengming@bytedance.com>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    520a688a
slub.c 177 KB