• John Fastabend's avatar
    net: sched, fix OOO packets with pfifo_fast · eb82a994
    John Fastabend authored
    After the qdisc lock was dropped in pfifo_fast we allow multiple
    enqueue threads and dequeue threads to run in parallel. On the
    enqueue side the skb bit ooo_okay is used to ensure all related
    skbs are enqueued in-order. On the dequeue side though there is
    no similar logic. What we observe is with fewer queues than CPUs
    it is possible to re-order packets when two instances of
    __qdisc_run() are running in parallel. Each thread will dequeue
    a skb and then whichever thread calls the ndo op first will
    be sent on the wire. This doesn't typically happen because
    qdisc_run() is usually triggered by the same core that did the
    enqueue. However, drivers will trigger __netif_schedule()
    when queues are transitioning from stopped to awake using the
    netif_tx_wake_* APIs. When this happens netif_schedule() calls
    qdisc_run() on the same CPU that did the netif_tx_wake_* which
    is usually done in the interrupt completion context. This CPU
    is selected with the irq affinity which is unrelated to the
    enqueue operations.
    
    To resolve this we add a RUNNING bit to the qdisc to ensure
    only a single dequeue per qdisc is running. Enqueue and dequeue
    operations can still run in parallel and also on multi queue
    NICs we can still have a dequeue in-flight per qdisc, which
    is typically per CPU.
    
    Fixes: c5ad119f ("net: sched: pfifo_fast use skb_array")
    Reported-by: default avatarJakob Unterwurzacher <jakob.unterwurzacher@theobroma-systems.com>
    Signed-off-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    eb82a994
sch_generic.c 32.1 KB