• Jan Kara's avatar
    nfs: Block on write congestion · 2f1f3104
    Jan Kara authored
    Commit 6df25e58 ("nfs: remove reliance on bdi congestion")
    introduced NFS-private solution for limiting number of writes
    outstanding against a particular server. Unlike previous bdi congestion
    this algorithm actually works and limits number of outstanding writeback
    pages to nfs_congestion_kb which scales with amount of client's memory
    and is capped at 256 MB. As a result some workloads such as random
    buffered writes over NFS got slower (from ~170 MB/s to ~126 MB/s). The
    fio command to reproduce is:
    
    fio --direct=0 --ioengine=sync --thread --invalidate=1 --group_reporting=1
      --runtime=300 --fallocate=posix --ramp_time=10 --new_group --rw=randwrite
      --size=64256m --numjobs=4 --bs=4k --fsync_on_close=1 --end_fsync=1
    
    This happens because the client sends ~256 MB worth of dirty pages to
    the server and any further background writeback request is ignored until
    the number of writeback pages gets below the threshold of 192 MB. By the
    time this happens and clients decides to trigger another round of
    writeback, the server often has no pages to write and the disk is idle.
    
    To fix this problem and make the client react faster to eased congestion
    of the server by blocking waiting for congestion to resolve instead of
    aborting writeback. This improves the random 4k buffered write
    throughput to 184 MB/s.
    Reviewed-by: default avatarSagi Grimberg <sagi@grimberg.me>
    Reviewed-by: default avatarJeff Layton <jlayton@kernel.org>
    Signed-off-by: default avatarJan Kara <jack@suse.cz>
    Signed-off-by: default avatarAnna Schumaker <Anna.Schumaker@Netapp.com>
    2f1f3104
client.c 36.3 KB