• Eric Dumazet's avatar
    net: release dst entry in dev_hard_start_xmit() · 93f154b5
    Eric Dumazet authored
    One point of contention in high network loads is the dst_release() performed
    when a transmited skb is freed. This is because NIC tx completion calls
    dev_kree_skb() long after original call to dev_queue_xmit(skb).
    
    CPU cache is cold and the atomic op in dst_release() stalls. On SMP, this is
    quite visible if one CPU is 100% handling softirqs for a network device,
    since dst_clone() is done by other cpus, involving cache line ping pongs.
    
    It seems right place to release dst is in dev_hard_start_xmit(), for most
    devices but ones that are virtual, and some exceptions.
    
    David Miller suggested to define a new device flag, set in alloc_netdev_mq()
    (so that most devices set it at init time), and carefuly unset in devices
    which dont want a NULL skb->dst in their ndo_start_xmit().
    
    List of devices that must clear this flag is :
    
    - loopback device, because it calls netif_rx() and quoting Patrick :
        "ip_route_input() doesn't accept loopback addresses, so loopback packets
         already need to have a dst_entry attached."
    - appletalk/ipddp.c : needs skb->dst in its xmit function
    
    - And all devices that call again dev_queue_xmit() from their xmit function
    (as some classifiers need skb->dst) : bonding, vlan, macvlan, eql, ifb, hdlc_fr
    Signed-off-by: default avatarEric Dumazet <dada1@cosmosbay.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    93f154b5
eql.c 14.6 KB