• Kees Cook's avatar
    slab: Introduce kmalloc_size_roundup() · 05a94065
    Kees Cook authored
    In the effort to help the compiler reason about buffer sizes, the
    __alloc_size attribute was added to allocators. This improves the scope
    of the compiler's ability to apply CONFIG_UBSAN_BOUNDS and (in the near
    future) CONFIG_FORTIFY_SOURCE. For most allocations, this works well,
    as the vast majority of callers are not expecting to use more memory
    than what they asked for.
    
    There is, however, one common exception to this: anticipatory resizing
    of kmalloc allocations. These cases all use ksize() to determine the
    actual bucket size of a given allocation (e.g. 128 when 126 was asked
    for). This comes in two styles in the kernel:
    
    1) An allocation has been determined to be too small, and needs to be
       resized. Instead of the caller choosing its own next best size, it
       wants to minimize the number of calls to krealloc(), so it just uses
       ksize() plus some additional bytes, forcing the realloc into the next
       bucket size, from which it can learn how large it is now. For example:
    
    	data = krealloc(data, ksize(data) + 1, gfp);
    	data_len = ksize(data);
    
    2) The minimum size of an allocation is calculated, but since it may
       grow in the future, just use all the space available in the chosen
       bucket immediately, to avoid needing to reallocate later. A good
       example of this is skbuff's allocators:
    
    	data = kmalloc_reserve(size, gfp_mask, node, &pfmemalloc);
    	...
    	/* kmalloc(size) might give us more room than requested.
    	 * Put skb_shared_info exactly at the end of allocated zone,
    	 * to allow max possible filling before reallocation.
    	 */
    	osize = ksize(data);
            size = SKB_WITH_OVERHEAD(osize);
    
    In both cases, the "how much was actually allocated?" question is answered
    _after_ the allocation, where the compiler hinting is not in an easy place
    to make the association any more. This mismatch between the compiler's
    view of the buffer length and the code's intention about how much it is
    going to actually use has already caused problems[1]. It is possible to
    fix this by reordering the use of the "actual size" information.
    
    We can serve the needs of users of ksize() and still have accurate buffer
    length hinting for the compiler by doing the bucket size calculation
    _before_ the allocation. Code can instead ask "how large an allocation
    would I get for a given size?".
    
    Introduce kmalloc_size_roundup(), to serve this function so we can start
    replacing the "anticipatory resizing" uses of ksize().
    
    [1] https://github.com/ClangBuiltLinux/linux/issues/1599
        https://github.com/KSPP/linux/issues/183
    
    [ vbabka@suse.cz: add SLOB version ]
    
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: linux-mm@kvack.org
    Signed-off-by: default avatarKees Cook <keescook@chromium.org>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    05a94065
slab.h 25.3 KB