• Chuck Lever's avatar
    NFS: Cache state owners after files are closed · 0aaaf5c4
    Chuck Lever authored
    Servers have a finite amount of memory to store NFSv4 open and lock
    owners.  Moreover, servers may have a difficult time determining when
    they can reap their state owner table, thanks to gray areas in the
    NFSv4 protocol specification.  Thus clients should be careful to reuse
    state owners when possible.
    
    Currently Linux is not too careful.  When a user has closed all her
    files on one mount point, the state owner's reference count goes to
    zero, and it is released.  The next OPEN allocates a new one.  A
    workload that serially opens and closes files can run through a large
    number of open owners this way.
    
    When a state owner's reference count goes to zero, slap it onto a free
    list for that nfs_server, with an expiry time.  Garbage collect before
    looking for a state owner.  This makes state owners for active users
    available for re-use.
    
    Now that there can be unused state owners remaining at umount time,
    purge the state owner free list when a server is destroyed.  Also be
    sure not to reclaim unused state owners during state recovery.
    
    This change has benefits for the client as well.  For some workloads,
    this approach drops the number of OPEN_CONFIRM calls from the same as
    the number of OPEN calls, down to just one.  This reduces wire traffic
    and thus open(2) latency.  Before this patch, untarring a kernel
    source tarball shows the OPEN_CONFIRM call counter steadily increasing
    through the test.  With the patch, the OPEN_CONFIRM count remains at 1
    throughout the entire untar.
    
    As long as the expiry time is kept short, I don't think garbage
    collection should be terribly expensive, although it does bounce the
    clp->cl_lock around a bit.
    
    [ At some point we should rationalize the use of the nfs_server
    ->destroy method. ]
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    [Trond: Fixed a garbage collection race and a few efficiency issues]
    Signed-off-by: default avatarTrond Myklebust <Trond.Myklebust@netapp.com>
    0aaaf5c4
nfs4_fs.h 11.8 KB