• NeilBrown's avatar
    nfsd: Don't leave work of closing files to a work queue · ffb40259
    NeilBrown authored
    The work of closing a file can have non-trivial cost.  Doing it in a
    separate work queue thread means that cost isn't imposed on the nfsd
    threads and an imbalance can be created.  This can result in files being
    queued for the work queue more quickly that the work queue can process
    them, resulting in unbounded growth of the queue and memory exhaustion.
    
    To avoid this work imbalance that exhausts memory, this patch moves all
    closing of files into the nfsd threads.  This means that when the work
    imposes a cost, that cost appears where it would be expected - in the
    work of the nfsd thread.  A subsequent patch will ensure the final
    __fput() is called in the same (nfsd) thread which calls filp_close().
    
    Files opened for NFSv3 are never explicitly closed by the client and are
    kept open by the server in the "filecache", which responds to memory
    pressure, is garbage collected even when there is no pressure, and
    sometimes closes files when there is particular need such as for rename.
    These files currently have filp_close() called in a dedicated work
    queue, so their __fput() can have no effect on nfsd threads.
    
    This patch discards the work queue and instead has each nfsd thread call
    flip_close() on as many as 8 files from the filecache each time it acts
    on a client request (or finds there are no pending client requests).  If
    there are more to be closed, more threads are woken.  This spreads the
    work of __fput() over multiple threads and imposes any cost on those
    threads.
    
    The number 8 is somewhat arbitrary.  It needs to be greater than 1 to
    ensure that files are closed more quickly than they can be added to the
    cache.  It needs to be small enough to limit the per-request delays that
    will be imposed on clients when all threads are busy closing files.
    Signed-off-by: default avatarNeilBrown <neilb@suse.de>
    Reviewed-by: default avatarJeff Layton <jlayton@kernel.org>
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    ffb40259
nfssvc.c 26.1 KB