• Ido Schimmel's avatar
    mlxsw: pci: Ring CQ's doorbell before RDQ's · c9ebea04
    Ido Schimmel authored
    When a packet should be trapped to the CPU the device consumes a WQE
    (work queue element) from an RDQ (receive descriptor queue) and copies
    the packet to the address specified in the WQE. The device then tries to
    post a CQE (completion queue element) that contains various metadata
    (e.g., ingress port) about the packet to a CQ (completion queue).
    
    In case the device managed to consume a WQE, but did not manage to post
    the corresponding CQE, it will get stuck. This unlikely situation can be
    triggered due to the scheme the driver is currently using to process
    CQEs.
    
    The driver will consume up to 512 CQEs at a time and after processing
    each corresponding WQE it will ring the RDQ's doorbell, letting the
    device know that a new WQE was posted for it to consume. Only after
    processing all the CQEs (up to 512), the driver will ring the CQ's
    doorbell, letting the device know that new ones can be posted.
    
    Fix this by having the driver ring the CQ's doorbell for every processed
    CQE, but before ringing the RDQ's doorbell. This guarantees that
    whenever we post a new WQE, there is a corresponding CQE available. Copy
    the currently processed CQE to prevent the device from overwriting it
    with a new CQE after ringing the doorbell.
    
    Note that the driver still arms the CQ only after processing all the
    pending CQEs, so that interrupts for this CQ will only be delivered
    after the driver finished its processing.
    
    Before commit 8404f6f2 ("mlxsw: pci: Allow to use CQEs of version 1
    and version 2") the issue was virtually impossible to trigger since the
    number of CQEs was twice the number of WQEs and the number of CQEs
    processed at a time was equal to the number of available WQEs.
    
    Fixes: 8404f6f2 ("mlxsw: pci: Allow to use CQEs of version 1 and version 2")
    Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
    Reported-by: default avatarSemion Lisyansky <semionl@mellanox.com>
    Tested-by: default avatarSemion Lisyansky <semionl@mellanox.com>
    Acked-by: default avatarJiri Pirko <jiri@mellanox.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    c9ebea04
pci.c 49.4 KB