• Uladzislau Rezki (Sony)'s avatar
    mm/vmalloc: remove preempt_disable/enable when doing preloading · 81f1ba58
    Uladzislau Rezki (Sony) authored
    Some background.  The preemption was disabled before to guarantee that a
    preloaded object is available for a CPU, it was stored for.  That was
    achieved by combining the disabling the preemption and taking the spin
    lock while the ne_fit_preload_node is checked.
    
    The aim was to not allocate in atomic context when spinlock is taken
    later, for regular vmap allocations.  But that approach conflicts with
    CONFIG_PREEMPT_RT philosophy.  It means that calling spin_lock() with
    disabled preemption is forbidden in the CONFIG_PREEMPT_RT kernel.
    
    Therefore, get rid of preempt_disable() and preempt_enable() when the
    preload is done for splitting purpose.  As a result we do not guarantee
    now that a CPU is preloaded, instead we minimize the case when it is
    not, with this change, by populating the per cpu preload pointer under
    the vmap_area_lock.
    
    This implies that at least each caller that has done the preallocation
    will not fallback to an atomic allocation later.  It is possible that
    the preallocation would be pointless or that no preallocation is done
    because of the race but the data shows that this is really rare.
    
    For example i run the special test case that follows the preload pattern
    and path.  20 "unbind" threads run it and each does 1000000 allocations.
    Only 3.5 times among 1000000 a CPU was not preloaded.  So it can happen
    but the number is negligible.
    
    [mhocko@suse.com: changelog additions]
    Link: http://lkml.kernel.org/r/20191016095438.12391-1-urezki@gmail.com
    Fixes: 82dd23e8 ("mm/vmalloc.c: preload a CPU with one object for split purpose")
    Signed-off-by: default avatarUladzislau Rezki (Sony) <urezki@gmail.com>
    Reviewed-by: default avatarSteven Rostedt (VMware) <rostedt@goodmis.org>
    Acked-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
    Acked-by: default avatarDaniel Wagner <dwagner@suse.de>
    Acked-by: default avatarMichal Hocko <mhocko@suse.com>
    Cc: Hillf Danton <hdanton@sina.com>
    Cc: Matthew Wilcox <willy@infradead.org>
    Cc: Oleksiy Avramchenko <oleksiy.avramchenko@sonymobile.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    81f1ba58
vmalloc.c 90.2 KB