• Eric Dumazet's avatar
    tcp: fix tcp_ack() performance problem · ad971f61
    Eric Dumazet authored
    We worked hard to improve tcp_ack() performance, by not accessing
    skb_shinfo() in fast path (cd7d8498 tcp: change tcp_skb_pcount()
    location)
    
    We still have one spurious access because of ACK timestamping,
    added in commit e1c8a607 ("net-timestamp: ACK timestamp for
    bytestreams")
    
    By checking if sk_tsflags has SOF_TIMESTAMPING_TX_ACK set,
    we can avoid two cache line misses for the common case.
    
    While we are at it, add two prefetchw() :
    
    One in tcp_ack() to bring skb at the head of write queue.
    
    One in tcp_clean_rtx_queue() loop to bring following skb,
    as we will delete skb from the write queue and dirty skb->next->prev.
    
    Add a couple of [un]likely() clauses.
    
    After this patch, tcp_ack() is no longer the most consuming
    function in tcp stack.
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Cc: Willem de Bruijn <willemb@google.com>
    Cc: Neal Cardwell <ncardwell@google.com>
    Cc: Yuchung Cheng <ycheng@google.com>
    Cc: Van Jacobson <vanj@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    ad971f61
tcp_input.c 170 KB