• Daniel Borkmann's avatar
    packet: introduce PACKET_QDISC_BYPASS socket option · d346a3fa
    Daniel Borkmann authored
    This patch introduces a PACKET_QDISC_BYPASS socket option, that
    allows for using a similar xmit() function as in pktgen instead
    of taking the dev_queue_xmit() path. This can be very useful when
    PF_PACKET applications are required to be used in a similar
    scenario as pktgen, but with full, flexible packet payload that
    needs to be provided, for example.
    
    On default, nothing changes in behaviour for normal PF_PACKET
    TX users, so everything stays as is for applications. New users,
    however, can now set PACKET_QDISC_BYPASS if needed to prevent
    own packets from i) reentering packet_rcv() and ii) to directly
    push the frame to the driver.
    
    In doing so we can increase pps (here 64 byte packets) for
    PF_PACKET a bit:
    
      # CPUs -- QDISC_BYPASS   -- qdisc path -- qdisc path[**]
      1 CPU  ==  1,509,628 pps --  1,208,708 --  1,247,436
      2 CPUs ==  3,198,659 pps --  2,536,012 --  1,605,779
      3 CPUs ==  4,787,992 pps --  3,788,740 --  1,735,610
      4 CPUs ==  6,173,956 pps --  4,907,799 --  1,909,114
      5 CPUs ==  7,495,676 pps --  5,956,499 --  2,014,422
      6 CPUs ==  9,001,496 pps --  7,145,064 --  2,155,261
      7 CPUs == 10,229,776 pps --  8,190,596 --  2,220,619
      8 CPUs == 11,040,732 pps --  9,188,544 --  2,241,879
      9 CPUs == 12,009,076 pps -- 10,275,936 --  2,068,447
     10 CPUs == 11,380,052 pps -- 11,265,337 --  1,578,689
     11 CPUs == 11,672,676 pps -- 11,845,344 --  1,297,412
     [...]
     20 CPUs == 11,363,192 pps -- 11,014,933 --  1,245,081
    
     [**]: qdisc path with packet_rcv(), how probably most people
           seem to use it (hopefully not anymore if not needed)
    
    The test was done using a modified trafgen, sending a simple
    static 64 bytes packet, on all CPUs.  The trick in the fast
    "qdisc path" case, is to avoid reentering packet_rcv() by
    setting the RAW socket protocol to zero, like:
    socket(PF_PACKET, SOCK_RAW, 0);
    
    Tradeoffs are documented as well in this patch, clearly, if
    queues are busy, we will drop more packets, tc disciplines are
    ignored, and these packets are not visible to taps anymore. For
    a pktgen like scenario, we argue that this is acceptable.
    
    The pointer to the xmit function has been placed in packet
    socket structure hole between cached_dev and prot_hook that
    is hot anyway as we're working on cached_dev in each send path.
    
    Done in joint work together with Jesper Dangaard Brouer.
    Signed-off-by: default avatarDaniel Borkmann <dborkman@redhat.com>
    Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    d346a3fa
af_packet.c 92.6 KB