• Chengming Zhou's avatar
    mm/slub: directly load freelist from cpu partial slab in the likely case · 90b1e566
    Chengming Zhou authored
    The likely case is that we get a usable slab from the cpu partial list,
    we can directly load freelist from it and return back, instead of going
    the other way that need more work, like reenable interrupt and recheck.
    
    But we need to remove the "VM_BUG_ON(!new.frozen)" in get_freelist()
    for reusing it, since cpu partial slab is not frozen. It seems
    acceptable since it's only for debug purpose.
    
    And get_freelist() also assumes it can return NULL if the freelist is
    empty, which is not possible for the cpu partial slab case, so we
    add "VM_BUG_ON(!freelist)" after get_freelist() to make it explicit.
    
    There is some small performance improvement too, which shows by:
    perf bench sched messaging -g 5 -t -l 100000
    
                mm-stable   slub-optimize
    Total time      7.473    7.209
    Signed-off-by: default avatarChengming Zhou <zhouchengming@bytedance.com>
    Reviewed-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    90b1e566
slub.c 178 KB