• Mike Marciniszyn's avatar
    IB/hfi1: Add atomic triggered sleep/wakeup · 38fd98af
    Mike Marciniszyn authored
    When running iperf in a two host configuration the following trace can
    occur:
    
    [  319.728730] NETDEV WATCHDOG: ib0 (hfi1): transmit queue 0 timed out
    
    The issue happens because the current implementation relies on the netif
    txq being stopped to control the flushing of the tx list.
    
    There are two resources that the transmit logic can wait on and stop the
    txq:
    - SDMA descriptors
    - Ring space to hold completions
    
    The ring space is tested on the sending side and relieved when the ring is
    consumed in the napi tx reaping.
    
    Unfortunately, that reaping can run conncurrently with the workqueue
    flushing of the txlist.  If the txq is started just before the workitem
    executes, the txlist will never be flushed, leading to the txq being
    stuck.
    
    Fix by:
    - Adding sleep/wakeup wrappers
      * Use an atomic to control the call to the netif routines inside the
        wrappers
    
    - Use another atomic to record ring space exhaustion
      * Only wakeup when the a ring space exhaustion has happened and it
        relieved
    
    Add additional wrappers to clarify the ring space resource handling.
    
    Fixes: d99dc602 ("IB/hfi1: Add functions to transmit datagram ipoib packets")
    Link: https://lore.kernel.org/r/20200623204327.108092.4024.stgit@awfm-01.aw.intel.comReviewed-by: default avatarKaike Wan <kaike.wan@intel.com>
    Signed-off-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
    Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
    Signed-off-by: default avatarJason Gunthorpe <jgg@nvidia.com>
    38fd98af
ipoib.h 4.39 KB