• Davidlohr Bueso's avatar
    mm,compaction: serialize waitqueue_active() checks · 46acef04
    Davidlohr Bueso authored
    Without a memory barrier, the following race can occur with a high-order
    allocation:
    
    wakeup_kcompactd(order == 1)  		     kcompactd()
      [L] waitqueue_active(kcompactd_wait)
    						[S] prepare_to_wait_event(kcompactd_wait)
    						[L] (kcompactd_max_order == 0)
      [S] kcompactd_max_order = order;		      schedule()
    
    Where the waitqueue_active() check is speculatively re-ordered to before
    setting the actual condition (max_order), not seeing the threads that's
    going to block; making us miss a wakeup.  There are a couple of options
    to fix this, including calling wq_has_sleepers() which adds a full
    barrier, or unconditionally doing the wake_up_interruptible() and
    serialize on the q->lock.  However, to make use of the control
    dependency, we just need to add L->L guarantees.
    
    While this bug is theoretical, there have been other offenders of the
    lockless waitqueue_active() in the past -- this is also documented in
    the call itself.
    
    Link: http://lkml.kernel.org/r/1483975528-24342-1-git-send-email-dave@stgolabs.netSigned-off-by: default avatarDavidlohr Bueso <dbueso@suse.de>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    46acef04
compaction.c 57.2 KB