• Eric Dumazet's avatar
    tcp: change tcp_skb_pcount() location · cd7d8498
    Eric Dumazet authored
    Our goal is to access no more than one cache line access per skb in
    a write or receive queue when doing the various walks.
    
    After recent TCP_SKB_CB() reorganizations, it is almost done.
    
    Last part is tcp_skb_pcount() which currently uses
    skb_shinfo(skb)->gso_segs, which is a terrible choice, because it needs
    3 cache lines in current kernel (skb->head, skb->end, and
    shinfo->gso_segs are all in 3 different cache lines, far from skb->cb)
    
    This very simple patch reuses space currently taken by tcp_tw_isn
    only in input path, as tcp_skb_pcount is only needed for skb stored in
    write queue.
    
    This considerably speeds up tcp_ack(), granted we avoid shinfo->tx_flags
    to get SKBTX_ACK_TSTAMP, which seems possible.
    
    This also speeds up all sack processing in general.
    
    This speeds up tcp_sendmsg() because it no longer has to access/dirty
    shinfo.
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    cd7d8498
tcp.c 84 KB