• Kees Cook's avatar
    slub: relocate freelist pointer to middle of object · 3202fa62
    Kees Cook authored
    In a recent discussion[1] with Vitaly Nikolenko and Silvio Cesare, it
    became clear that moving the freelist pointer away from the edge of
    allocations would likely improve the overall defensive posture of the
    inline freelist pointer.  My benchmarks show no meaningful change to
    performance (they seem to show it being faster), so this looks like a
    reasonable change to make.
    
    Instead of having the freelist pointer at the very beginning of an
    allocation (offset 0) or at the very end of an allocation (effectively
    offset -sizeof(void *) from the next allocation), move it away from the
    edges of the allocation and into the middle.  This provides some
    protection against small-sized neighboring overflows (or underflows), for
    which the freelist pointer is commonly the target.  (Large or well
    controlled overwrites are much more likely to attack live object contents,
    instead of attempting freelist corruption.)
    
    The vaunted kernel build benchmark, across 5 runs. Before:
    
    	Mean: 250.05
    	Std Dev: 1.85
    
    and after, which appears mysteriously faster:
    
    	Mean: 247.13
    	Std Dev: 0.76
    
    Attempts at running "sysbench --test=memory" show the change to be well in
    the noise (sysbench seems to be pretty unstable here -- it's not really
    measuring allocation).
    
    Hackbench is more allocation-heavy, and while the std dev is above the
    difference, it looks like may manifest as an improvement as well:
    
    20 runs of "hackbench -g 20 -l 1000", before:
    
    	Mean: 36.322
    	Std Dev: 0.577
    
    and after:
    
    	Mean: 36.056
    	Std Dev: 0.598
    
    [1] https://twitter.com/vnik5287/status/1235113523098685440Signed-off-by: default avatarKees Cook <keescook@chromium.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Acked-by: default avatarChristoph Lameter <cl@linux.com>
    Cc: Vitaly Nikolenko <vnik@duasynt.com>
    Cc: Silvio Cesare <silvio.cesare@gmail.com>
    Cc: Christoph Lameter <cl@linux.com>Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Link: http://lkml.kernel.org/r/202003051624.AAAC9AECC@keescookSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    3202fa62
slub.c 144 KB