- 12 Jul, 2024 1 commit
-
-
Christoph Hellwig authored
Look for the udev generated persistent device name for NVMe devices in addition to the SCSI ones and the Redhat-specific device mapper name. This is the client side implementation of RFC 9561 "Using the Parallel NFS (pNFS) SCSI Layout to Access Non-Volatile Memory Express (NVMe) Storage Devices". Note that the udev rules for nvme are a bit of a mess and udev will only create a link for the uuid if the NVMe namespace has one, and not the NGUID. As the current RFCs don't support UUID based identifications this means the layout can't be used on such namespaces out of the box. A small tweak to the udev rules can work around it, and as the real fix I will submit a draft to the IETF NFSv4 working group to support UUID-based identifiers for SCSI and NVMe. Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Sagi Grimberg <sagi@grimberg.me> Reviewed-by:
Benjamin Coddington <bcodding@redhat.com> Signed-off-by:
Anna Schumaker <Anna.Schumaker@Netapp.com>
-
- 10 Jul, 2024 2 commits
-
-
Christoph Hellwig authored
The nfs_page_length is not used anywhere, remove it. Signed-off-by:
-
Christoph Hellwig authored
max_deviceinfo_size is not set anywhere, remove it. Signed-off-by:
-
- 08 Jul, 2024 37 commits
-
-
Christoph Hellwig authored
When NFS requests are split into sub-requests, nfs_inode_remove_request calls nfs_page_group_sync_on_bit to set PG_REMOVE on this sub-request and only completes the head requests once PG_REMOVE is set on all requests. This means that when nfs_lock_and_join_requests sees a PG_REMOVE bit, I/O on the request is in progress and has partially completed. If such a request is returned to nfs_try_to_update_request, it could be extended with the newly dirtied region and I/O for the combined range will be re-scheduled, leading to extra I/O. Change the logic to instead restart the search for a request when any PG_REMOVE bit is set, as the completion handler will remove the request as soon as it can take the page group lock. This not only avoid extending the I/O but also does the right thing for the callers that want to cancel or flush the request. Signed-off-by:
-
Christoph Hellwig authored
nfs_wait_on_request is now only used in write.c. Move it there and mark it static. Signed-off-by:
-
Christoph Hellwig authored
Fold nfs_page_group_lock_subrequests into nfs_lock_and_join_requests to prepare for future changes to this code, and move the helpers to write.c as well. Signed-off-by:
-
Christoph Hellwig authored
Fold nfs_folio_find_and_lock_request into the only caller to prepare for changes to this code. Signed-off-by:
-
Christoph Hellwig authored
nfs_folio_find_and_lock_request and the nfs_page_group_lock_head helper called by it spend quite some effort to deal with head vs subrequests. But given that only the head request can be stashed in the folio private data, non of that is required. Fold the locking logic from nfs_page_group_lock_head into nfs_folio_find_and_lock_request and simplify the result based on the invariant that we always find the head request in the folio private data. Signed-off-by:
-
Christoph Hellwig authored
nfs_folio_private_request is a trivial wrapper around, which itself has fallen out of favor and has been replaced with plain ->private dereferences in recent folio conversions. Do the same for nfs. Signed-off-by:
-
Christoph Hellwig authored
Remove the code testing folio_test_swapcache either explicitly or implicitly in pagemap.h headers, as is now handled using the direct I/O path and not the buffered I/O path that these helpers are located in. Signed-off-by:
-
Olga Kornievskaia authored
Previously in order to mark the communication with the DS server, we tried to use NFS_CS_DS in cl_flags. However, this flag would only be saved for the DS server and in case where DS equals MDS, the client would not find a matching nfs_client in nfs_match_client that represents the MDS (but is also a DS). Instead, don't rely on the NFS_CS_DS but instead use NFS_CS_PNFS. Fixes: 379e4adf ("NFSv4.1: fixup use EXCHGID4_FLAG_USE_PNFS_DS for DS server") Signed-off-by:
Olga Kornievskaia <kolga@netapp.com> Signed-off-by:
-
Jan Kara authored
Commit 6df25e58 ("nfs: remove reliance on bdi congestion") introduced NFS-private solution for limiting number of writes outstanding against a particular server. Unlike previous bdi congestion this algorithm actually works and limits number of outstanding writeback pages to nfs_congestion_kb which scales with amount of client's memory and is capped at 256 MB. As a result some workloads such as random buffered writes over NFS got slower (from ~170 MB/s to ~126 MB/s). The fio command to reproduce is: fio --direct=0 --ioengine=sync --thread --invalidate=1 --group_reporting=1 --runtime=300 --fallocate=posix --ramp_time=10 --new_group --rw=randwrite --size=64256m --numjobs=4 --bs=4k --fsync_on_close=1 --end_fsync=1 This happens because the client sends ~256 MB worth of dirty pages to the server and any further background writeback request is ignored until the number of writeback pages gets below the threshold of 192 MB. By the time this happens and clients decides to trigger another round of writeback, the server often has no pages to write and the disk is idle. To fix this problem and make the client react faster to eased congestion of the server by blocking waiting for congestion to resolve instead of aborting writeback. This improves the random 4k buffered write throughput to 184 MB/s. Reviewed-by:
Jeff Layton <jlayton@kernel.org> Signed-off-by:
Jan Kara <jack@suse.cz> Signed-off-by:
-
Jan Kara authored
Atomic types should better be initialized with atomic_long_set() instead of relying on zeroing done by kzalloc(). Clean this up. Reviewed-by:
-
Jan Kara authored
nfss->writeback is updated only when we are ending page writeback and at that moment we also clear nfss->write_congested. So there's no point in rechecking congestion state in nfs_commit_release_pages(). Drop the pointless check. Reviewed-by:
-
Chuck Lever authored
An administrator cannot take action on these messages, but the reported errors might be helpful for troubleshooting. Transition them to trace points so these events appear in the trace log and can be easily lined up with other traced NFS client operations. Examples: append_writer-6147 [000] 80.247393: bl_pr_key_reg: dev=8,0 (sda) key=0x6675bfcf59112e98 append_writer-6147 [000] 80.247842: bl_pr_key_unreg: dev=8,0 (sda) key=0x6675bfcf59112e98 umount.nfs4-6172 [002] 84.950409: bl_pr_key_unreg_err: dev=8,0 (sda) key=0x6675bfcf59112e98 status=RESERVATION_CONFLICT Reviewed-by:Chuck Lever <chuck.lever@oracle.com> Signed-off-by:
-
Chuck Lever authored
Since commit f931d837 ("nfs/blocklayout: refactor block device opening"), an error is reported when no multi-path device is found. But this isn't a fatal error if the subsequent device open is successful. On systems without multi-path devices, this message always appears whether there is a problem or not. Instead, generate less system journal noise by reporting an error only when both open attempts fail. The new error message is more actionable since it indicates that there is a real configuration issue to be addressed. Reviewed-by:
-
Chuck Lever authored
During generic/069 runs with pNFS SCSI layouts, the NFS client emits the following in the system journal: kernel: pNFS: failed to open device /dev/disk/by-id/dm-uuid-mpath-0x6001405e3366f045b7949eb8e4540b51 (-2) kernel: pNFS: using block device sdb (reservation key 0x666b60901e7b26b3) kernel: pNFS: failed to open device /dev/disk/by-id/dm-uuid-mpath-0x6001405e3366f045b7949eb8e4540b51 (-2) kernel: pNFS: using block device sdb (reservation key 0x666b60901e7b26b3) kernel: sd 6:0:0:1: reservation conflict kernel: sd 6:0:0:1: [sdb] tag#16 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s kernel: sd 6:0:0:1: [sdb] tag#16 CDB: Write(10) 2a 00 00 00 00 50 00 00 08 00 kernel: reservation conflict error, dev sdb, sector 80 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2 kernel: sd 6:0:0:1: reservation conflict kernel: sd 6:0:0:1: reservation conflict kernel: sd 6:0:0:1: [sdb] tag#18 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s kernel: sd 6:0:0:1: [sdb] tag#17 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s kernel: sd 6:0:0:1: [sdb] tag#18 CDB: Write(10) 2a 00 00 00 00 60 00 00 08 00 kernel: sd 6:0:0:1: [sdb] tag#17 CDB: Write(10) 2a 00 00 00 00 58 00 00 08 00 kernel: reservation conflict error, dev sdb, sector 96 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0 kernel: reservation conflict error, dev sdb, sector 88 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0 systemd[1]: fstests-generic-069.scope: Deactivated successfully. systemd[1]: fstests-generic-069.scope: Consumed 5.092s CPU time. systemd[1]: media-test.mount: Deactivated successfully. systemd[1]: media-scratch.mount: Deactivated successfully. kernel: sd 6:0:0:1: reservation conflict kernel: failed to unregister PR key. This appears to be due to a race. bl_alloc_lseg() calls this: 561 static struct nfs4_deviceid_node * 562 bl_find_get_deviceid(struct nfs_server *server, 563 const struct nfs4_deviceid *id, const struct cred *cred, 564 gfp_t gfp_mask) 565 { 566 struct nfs4_deviceid_node *node; 567 unsigned long start, end; 568 569 retry: 570 node = nfs4_find_get_deviceid(server, id, cred, gfp_mask); 571 if (!node) 572 return ERR_PTR(-ENODEV); nfs4_find_get_deviceid() does a lookup without the spin lock first. If it can't find a matching deviceid, it creates a new device_info (which calls bl_alloc_deviceid_node, and that registers the device's PR key). Then it takes the nfs4_deviceid_lock and looks up the deviceid again. If it finds it this time, bl_find_get_deviceid() frees the spare (new) device_info, which unregisters the PR key for the same device. Any subsequent I/O from this client on that device gets EBADE. The umount later unregisters the device's PR key again. To prevent this problem, register the PR key after the deviceid_node lookup. Signed-off-by: -
Trond Myklebust authored
Some pNFS implementations, such as flexible files, want the client to send the layout stats and layout errors that may have incurred while the metadata server was booting. To do so, the client sends a layoutreturn with an all-zero stateid while the server is in grace during reboot recovery. Signed-off-by:
Trond Myklebust <trond.myklebust@hammerspace.com> Reviewed-by:
-
Trond Myklebust authored
The layout will be automatically unhashed on final release of the reference count. Signed-off-by:
-
Trond Myklebust authored
Replace the boolean in nfs4_proc_layoutreturn() with a set of flags that will allow us to craft a version that is appropriate for reboot recovery. Signed-off-by:
-
Trond Myklebust authored
If the layout return failed due to a timeout or reboot, then leave the layout segments on the list so that the layout return gets replayed later. The exception would be if we're freeing the inode. Signed-off-by:
-
Trond Myklebust authored
If the server reboots, then handle it by deferring the layout return. Signed-off-by:
-
Trond Myklebust authored
If the layoutreturn-on-close fails due to an RPC layer problem, such as a timeout, then we want to retry at a later time. Add a helper function to allow this. Signed-off-by:
-
Trond Myklebust authored
Add a flag PNFS_LAYOUT_FILE_BULK_RETURN, that will attempt to return all the layouts in a pnfs_layout_destroy_byfsid/pnfs_layout_destroy_byclid call, instead of just invalidating them. Signed-off-by:
-
Trond Myklebust authored
Change the bool argument to a flag so that we can add different modes for doing bulk destroy of a layout. In particular, we will want the ability to schedule return of all the layouts associated with a given NFS server when it reboots. Signed-off-by:
-
Trond Myklebust authored
Ensure that we encode the actual stateid, and not any metadata. Signed-off-by:
-
Trond Myklebust authored
Signed-off-by:
-
Trond Myklebust authored
pnfs_layout_free_bulk_destroy_list() already checks for whether the list is empty or not. Signed-off-by:
-
Trond Myklebust authored
When we set the new share access modes for CLOSE in nfs4_close_prepare(). we should only set a mode of NFS4_SHARE_ACCESS_READ, NFS4_SHARE_ACCESS_WRITE or NFS4_SHARE_ACCESS_BOTH. Currently, we may also be passing in the NFSv4.1 share modes for controlling delegation requests in OPEN, which is wrong. Signed-off-by:
-
Lance Shelton authored
Add a callback to return the delegation in order to allow generic NFS code to return the delegation when appropriate. Signed-off-by:
Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by:
Lance Shelton <lance.shelton@hammerspace.com> Reviewed-by:
-
Trond Myklebust authored
Turn on the optimisation to allow the client to request that the server not return the open stateid when it returns a delegation. Signed-off-by:
-
Trond Myklebust authored
If the server returns a delegation stateid only, then don't try to set an open stateid. Signed-off-by:
-
Trond Myklebust authored
If the server supports the NFSv4.2 protocol extension to optimise away returning a stateid when it returns a delegation, then we cache that information in another capability flag. Signed-off-by:
-
Trond Myklebust authored
Query the server for the OPEN arguments that it supports so that we can figure out which extensions we can use. Signed-off-by:
-
Trond Myklebust authored
If the timestamps and size are delegated to the client, then it is authoritative w.r.t. their values, so we should not be requesting those values from the server. Note that this allows us to optimise away most GETATTR calls if the only changes to the attributes are the result of read() or write(). Signed-off-by:
-
Trond Myklebust authored
nfs_setattr calls nfs_update_inode() directly, so we have to reset the m/ctime there. Signed-off-by:
-
Trond Myklebust authored
If the atime or mtime attributes were delegated, then we need to propagate their new values back to the server when returning the delegation. Signed-off-by:
-
Trond Myklebust authored
If we see that the server supports attribute delegations, then request them by setting the appropriate OPEN arguments. Signed-off-by:
-
Trond Myklebust authored
Cache whether or not the server may have support for delegated attributes in a capability flag. Signed-off-by:
-
Trond Myklebust authored
After a reboot of the NFSv4.2 server, the recovery code needs to specify whether the delegation to be recovered is an attribute delegation or not. Signed-off-by:
-
