• Tejun Heo's avatar
    workqueue: disable irq while manipulating PENDING · 8930caba
    Tejun Heo authored
    Queueing operations use WORK_STRUCT_PENDING_BIT to synchronize access
    to the target work item.  They first try to claim the bit and proceed
    with queueing only after that succeeds and there's a window between
    PENDING being set and the actual queueing where the task can be
    interrupted or preempted.
    
    There's also a similar window in process_one_work() when clearing
    PENDING.  A work item is dequeued, gcwq->lock is released and then
    PENDING is cleared and the worker might get interrupted or preempted
    between releasing gcwq->lock and clearing PENDING.
    
    cancel[_delayed]_work_sync() tries to claim or steal PENDING.  The
    function assumes that a work item with PENDING is either queued or in
    the process of being [de]queued.  In the latter case, it busy-loops
    until either the work item loses PENDING or is queued.  If canceling
    coincides with the above described interrupts or preemptions, the
    canceling task will busy-loop while the queueing or executing task is
    preempted.
    
    This patch keeps irq disabled across claiming PENDING and actual
    queueing and moves PENDING clearing in process_one_work() inside
    gcwq->lock so that busy looping from PENDING && !queued doesn't wait
    for interrupted/preempted tasks.  Note that, in process_one_work(),
    setting last CPU and clearing PENDING got merged into single
    operation.
    
    This removes possible long busy-loops and will allow using
    try_to_grab_pending() from bh and irq contexts.
    
    v2: __queue_work() was testing preempt_count() to ensure that the
        caller has disabled preemption.  This triggers spuriously if
        !CONFIG_PREEMPT_COUNT.  Use preemptible() instead.  Reported by
        Fengguang Wu.
    
    v3: Disable irq instead of preemption.  IRQ will be disabled while
        grabbing gcwq->lock later anyway and this allows using
        try_to_grab_pending() from bh and irq contexts.
    Signed-off-by: default avatarTejun Heo <tj@kernel.org>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Fengguang Wu <fengguang.wu@intel.com>
    8930caba
workqueue.c 103 KB