• Willy Tarreau's avatar
    tcp: don't abort splice() after small transfers · 02275a2e
    Willy Tarreau authored
    TCP coalescing added a regression in splice(socket->pipe) performance,
    for some workloads because of the way tcp_read_sock() is implemented.
    
    The reason for this is the break when (offset + 1 != skb->len).
    
    As we released the socket lock, this condition is possible if TCP stack
    added a fragment to the skb, which can happen with TCP coalescing.
    
    So let's go back to the beginning of the loop when this happens,
    to give a chance to splice more frags per system call.
    
    Doing so fixes the issue and makes GRO 10% faster than LRO
    on CPU-bound splice() workloads instead of the opposite.
    Signed-off-by: default avatarWilly Tarreau <w@1wt.eu>
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    02275a2e
tcp.c 94.3 KB