• Chuck Lever's avatar
    xprtrdma: Prevent loss of completion signals · 4220a072
    Chuck Lever authored
    Commit 8301a2c0 ("xprtrdma: Limit work done by completion
    handler") was supposed to prevent xprtrdma's upcall handlers from
    starving other softIRQ work by letting them return to the provider
    before all CQEs have been polled.
    
    The logic assumes the provider will call the upcall handler again
    immediately if the CQ is re-armed while there are still queued CQEs.
    
    This assumption is invalid. The IBTA spec says that after a CQ is
    armed, the hardware must interrupt only when a new CQE is inserted.
    xprtrdma can't rely on the provider calling again, even though some
    providers do.
    
    Therefore, leaving CQEs on queue makes sense only when there is
    another mechanism that ensures all remaining CQEs are consumed in a
    timely fashion. xprtrdma does not have such a mechanism. If a CQE
    remains queued, the transport can wait forever to send the next RPC.
    
    Finally, move the wcs array back onto the stack to ensure that the
    poll array is always local to the CPU where the completion upcall is
    running.
    
    Fixes: 8301a2c0 ("xprtrdma: Limit work done by completion ...")
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    Reviewed-by: default avatarSagi Grimberg <sagig@mellanox.com>
    Reviewed-by: default avatarDevesh Sharma <devesh.sharma@avagotech.com>
    Tested-By: default avatarDevesh Sharma <devesh.sharma@avagotech.com>
    Signed-off-by: default avatarAnna Schumaker <Anna.Schumaker@Netapp.com>
    4220a072
verbs.c 33.5 KB