• Vlastimil Babka's avatar
    mm, slab: call kvfree_rcu_barrier() from kmem_cache_destroy() · 6c6c47b0
    Vlastimil Babka authored
    We would like to replace call_rcu() users with kfree_rcu() where the
    existing callback is just a kmem_cache_free(). However this causes
    issues when the cache can be destroyed (such as due to module unload).
    
    Currently such modules should be issuing rcu_barrier() before
    kmem_cache_destroy() to have their call_rcu() callbacks processed first.
    This barrier is however not sufficient for kfree_rcu() in flight due
    to the batching introduced by a35d1690 ("rcu: Add basic support for
    kfree_rcu() batching").
    
    This is not a problem for kmalloc caches which are never destroyed, but
    since removing SLOB, kfree_rcu() is allowed also for any other cache,
    that might be destroyed.
    
    In order not to complicate the API, put the responsibility for handling
    outstanding kfree_rcu() in kmem_cache_destroy() itself. Use the newly
    introduced kvfree_rcu_barrier() to wait before destroying the cache.
    This is similar to how we issue rcu_barrier() for SLAB_TYPESAFE_BY_RCU
    caches, but has to be done earlier, as the latter only needs to wait for
    the empty slab pages to finish freeing, and not objects from the slab.
    
    Users of call_rcu() with arbitrary callbacks should still issue
    rcu_barrier() before destroying the cache and unloading the module, as
    kvfree_rcu_barrier() is not a superset of rcu_barrier() and the
    callbacks may be invoking module code or performing other actions that
    are necessary for a successful unload.
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    6c6c47b0
slab_common.c 35 KB