• Chuck Lever's avatar
    xprtrdma: Fix receive buffer accounting · 05c97466
    Chuck Lever authored
    An RPC can terminate before its reply arrives, if a credential
    problem or a soft timeout occurs. After this happens, xprtrdma
    reports it is out of Receive buffers.
    
    A Receive buffer is posted before each RPC is sent, and returned to
    the buffer pool when a reply is received. If no reply is received
    for an RPC, that Receive buffer remains posted. But xprtrdma tries
    to post another when the next RPC is sent.
    
    If this happens a few dozen times, there are no receive buffers left
    to be posted at send time. I don't see a way for a transport
    connection to recover at that point, and it will spit warnings and
    unnecessarily delay RPCs on occasion for its remaining lifetime.
    
    Commit 1e465fd4 ("xprtrdma: Replace send and receive arrays")
    removed a little bit of logic to detect this case and not provide
    a Receive buffer so no more buffers are posted, and then transport
    operation continues correctly. We didn't understand what that logic
    did, and it wasn't commented, so it was removed as part of the
    overhaul to support backchannel requests.
    
    Restore it, but be wary of the need to keep extra Receives posted
    to deal with backchannel requests.
    
    Fixes: 1e465fd4 ("xprtrdma: Replace send and receive arrays")
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    Reviewed-by: default avatarAnna Schumaker <Anna.Schumaker@Netapp.com>
    Signed-off-by: default avatarTrond Myklebust <trond.myklebust@primarydata.com>
    05c97466
verbs.c 33.7 KB