- 12 Sep, 2014 10 commits
-
-
Trond Myklebust authored
Cc: Christoph Hellwig <hch@lst.de> Signed-off-by:Trond Myklebust <trond.myklebust@primarydata.com>
-
Christoph Hellwig authored
This code has been around for a while, but never was enabled, although it is in a working shape. Note that we implement NOTIFY_DEVICEID4_CHANGE identical to NOTIFY_DEVICEID4_DELETE. Given that in either case we can't do anything but preventing further lookups of a given device ID there isn't much difference in semantics for the two. For the delete case the server MUST ensure that there are no outstanding layouts, while for the change case it doesn't, but that has little relevance to the client. Signed-off-by:
Christoph Hellwig <hch@lst.de> Signed-off-by:
-
Christoph Hellwig authored
This patches moves parsing of the GETDEVICEINFO XDR to kernel space, as well as the management of complex devices. The reason for that is we might have multiple outstanding complex devices after a NOTIFY_DEVICEID4_CHANGE, which device mapper or md can't handle as they claim devices exclusively. But as is turns out simple striping / concatenation is fairly trivial to implement anyway, so we make our life simpler by reducing the reliance on blkmapd. For now we still use blkmapd by feeding it synthetic SIMPLE device XDR to translate device signatures to device numbers, but in the long runs I have plans to eliminate it entirely. Signed-off-by:
-
Christoph Hellwig authored
Create a file to house all the rpc_pipefs boilerplate code instead of sprinkling it over a few files. Signed-off-by:
-
Christoph Hellwig authored
Factor out a helper for all per-extent work, and merge the now trivial functions for lseg allocation and parsing. Signed-off-by:
-
Christoph Hellwig authored
This isn't device(id) related, so move it into the main file. Simple move for now, the next commit will clean it up a bit. Signed-off-by:
-
Christoph Hellwig authored
Instead of overflowing the XDR send buffer with our extent list allocate pages and pre-encode the layoutupdate payload into them. We optimistically allocate a single page use alloc_page and only switch to vmalloc when we have more extents outstanding. Currently there is only a single testcase (xfstests generic/113) which can reproduce large enough extent lists for this to occur. Signed-off-by:
-
Christoph Hellwig authored
The current GETDEVICELIST implementation is buggy in that it doesn't handle cursors correctly, and in that it returns an error if the server returns NFSERR_NOTSUPP. Given that there is no actual need for GETDEVICELIST, it has various issues and might get removed for NFSv4.2 stop using it in the blocklayout driver, and thus the Linux NFS client as whole. Signed-off-by:
-
Christoph Hellwig authored
The kbuild test robot complained about a new sparse warning in objio_alloc_deviceid_node, but it turns out that this was just a moved reference to an existing variable. Fix it to have the right big endian annotated type. Note that there are some other endianess issues in this file that I didn't bother to sort out as they involve global headers. Reported-by:
kbuild test robot <fengguang.wu@intel.com> Signed-off-by:
-
Christoph Hellwig authored
The kbuild test robot complained that we got the printk format wrong. Let's just kill these printks instead of fixing them as there is not point after the initial tree algorithm debugging. Reported-by:
-
- 10 Sep, 2014 30 commits
-
-
Jeff Layton authored
To make sparse happy... Signed-off-by:
Jeff Layton <jlayton@primarydata.com> Signed-off-by:
-
Jeff Layton authored
sparse says: fs/nfs/file.c:543:60: warning: incorrect type in argument 1 (different address spaces) fs/nfs/file.c:543:60: expected struct rpc_xprt *xprt fs/nfs/file.c:543:60: got struct rpc_xprt [noderef] <asn:4>*cl_xprt fs/nfs/file.c:548:53: warning: incorrect type in argument 1 (different address spaces) fs/nfs/file.c:548:53: expected struct rpc_xprt *xprt fs/nfs/file.c:548:53: got struct rpc_xprt [noderef] <asn:4>*cl_xprt cl_xprt is RCU-managed, so we need to take care to dereference and use it while holding the RCU read lock. Cc: Mel Gorman <mgorman@suse.de> Signed-off-by:
-
Christoph Hellwig authored
The VFS never calls setattr with ATTR_SIZE on anything but regular files. Remove the if check and turn it into an assert similar to what some other file systems do. Signed-off-by:
-
Christoph Hellwig authored
Signed-off-by:
-
Christoph Hellwig authored
This will be used by the block layout driver when splitting extents. Signed-off-by:
-
Christoph Hellwig authored
At a simple helper to issue a GETDEVICELIST operation and pre-load the device id cache based on the result. Signed-off-by:
-
Christoph Hellwig authored
Add support to the common pNFS core to issue GETDEVICEINFO calls on a device ID cache miss. The code is taken from the well debugged file layout implementation and calls out to the layoutdriver through a new alloc_deviceid_node method. The calling conventions for nfs4_find_get_deviceid are changed so that all information needed to send a GETDEVICEINFO request is passed to the common code. Signed-off-by:
-
Christoph Hellwig authored
This speads up truncate-heavy workloads like fsx by multiple orders of magnitude. Signed-off-by:
-
Christoph Hellwig authored
This allows removing extents from the extent tree especially on truncate operations, and thus fixing reads from truncated and re-extended that previously returned stale data. Signed-off-by:
-
Christoph Hellwig authored
Currently the block layout driver tracks extents in three separate data structures: - the two list of pnfs_block_extent structures returned by the server - the list of sectors that were in invalid state but have been written to - a list of pnfs_block_short_extent structures for LAYOUTCOMMIT All of these share the property that they are not only highly inefficient data structures, but also that operations on them are even more inefficient than nessecary. In addition there are various implementation defects like: - using an int to track sectors, causing corruption for large offsets - incorrect normalization of page or block granularity ranges - insufficient error handling - incorrect synchronization as extents can be modified while they are in use This patch replace all three data with a single unified rbtree structure tracking all extents, as well as their in-memory state, although we still need to instance for read-only and read-write extent due to the arcane client side COW feature in the block layouts spec. To fix the problem of extent possibly being modified while in use we make sure to return a copy of the extent for use in the write path - the extent can only be invalidated by a layout recall or return which has to wait until the I/O operations finished due to refcounts on the layout segment. The new extent tree work similar to the schemes used by block based filesystems like XFS or ext4. Signed-off-by:
-
Christoph Hellwig authored
The core nfs code handles setting pages uptodate on reads, no need to mess with the pageflags outselves. Also remove a debug function to dump page flags. Signed-off-by:
-
Christoph Hellwig authored
Use the new PNFS_READ_WHOLE_PAGE flag to offload read-modify-write handling to core nfs code, and remove a huge chunk of deadlock prone mess from the block layout writeback path. Signed-off-by:
-
Christoph Hellwig authored
If a layout driver keeps per-inode state outside of the layout segments it needs to be notified of any layout returns or recalls on an inode, and not just about the freeing of layout segments. Add a method to acomplish this, which will allow the block layout driver to handle the case of truncated and re-expanded files properly. Signed-off-by:
-
Christoph Hellwig authored
Like all block based filesystems, the pNFS block layout driver can't read or write at a byte granularity and thus has to perform read-modify-write cycles on writes smaller than this granularity. Add a flag so that the core NFS code always reads a whole page when starting a smaller write, so that we can do it in the place where the VFS expects it instead of doing in very deadlock prone way in the writeback handler. Note that in theory we could do less than page size reads here for disks that have a smaller sector size which are served by a server with a smaller pnfs block size. But so far that doesn't seem like a worthwhile optimization. Signed-off-by:
-
Christoph Hellwig authored
Expedite layout recall processing by forcing a layout commit when we see busy segments. Without it the layout recall might have to wait until the VM decided to start writeback for the file, which can introduce long delays. Signed-off-by:
-
Trond Myklebust authored
gcc reports: linux/fs/nfs/write.c: In function ‘nfs_page_find_head_request_locked.isra.17’: linux/fs/nfs/write.c:121:64: warning: ‘cinfo.mds’ may be used uninitialized in this function [-Wmaybe-uninitialized] list_for_each_entry_safe(freq, t, &cinfo.mds->list, wb_list) { ^ linux/fs/nfs/write.c:110:25: note: ‘cinfo.mds’ was declared here struct nfs_commit_info cinfo; Reported-by:Anna Schumaker <Anna.Schumaker@netapp.com> Cc: Weston Andros Adamson <dros@primarydata.com> Signed-off-by:
-
Christoph Hellwig authored
When we do non-page sized reads we can underflow the extent_length variable and read incorrect data. Fix the extent_length calculation and change to defensive <= checks for the extent length in the read and write path. Signed-off-by:
-
Christoph Hellwig authored
Make sure the block queue is plugged when performing pNFS blocklayout I/O. Signed-off-by:
-
Christoph Hellwig authored
Tell userspace what stage of GETDEVICEINFO failed so that there is a chance to debug it, especially with the userspace daemon clusterf***k in the block layout driver. Signed-off-by:
-
Christoph Hellwig authored
The Linux VM subsystem can't support block sizes larger than page size for block based filesystems very well. While this can be hacked around to some extent for simple filesystems the read-modify-write cycles required for pnfs block invalid extents are extremly deadlock prone when operating on multiple pages. Reject this case early on instead of pretending to support it (badly). Signed-off-by:
-
Christoph Hellwig authored
Currently there is no XDR buffer space allocated for the per-layout driver layoutcommit payload, which leads to server buffer overflows in the blocklayout driver even under simple workloads. As we can't do per-layout sizes for XDR operations we'll have to splice a previously encoded list of pages into the XDR stream, similar to how we handle ACL buffers. Signed-off-by:
-
Christoph Hellwig authored
After we issued a layoutreturn operations the may free the layout stateid and will thus cause bad stateid error when the client uses it again. We currently try to avoid this case by chosing the open stateid if not lsegs are present for this inode. But various places can hold refererence on lsegs and thus cause the list not to be empty shortly after a layout return. Add an explicit flag to mark the current layout stateid invalid and force usage of the openstateid after we did a full file layoutreturn. Signed-off-by:
-
Christoph Hellwig authored
Currently we fall through to nfs4_async_handle_error when we get a bad stateid error back from layoutget. nfs4_async_handle_error with a NULL state argument will never retry the operations but return the error to higher layer, causing an avoiable fallback to MDS I/O. Signed-off-by:
-
Christoph Hellwig authored
When layoutget returns an entirely new layout stateid it should not check the generation counter as the new stateid will start with a new counter entirely unrelated to old one. The current behavior causes constant layoutget failures against a block server which allocates a new stateid after an recall that removed all outstanding layouts. Signed-off-by:
-
Christoph Hellwig authored
Ensure the lsegs are initialized early so that we don't pass an unitialized one back to ->free_lseg during error processing. Signed-off-by:
-
Christoph Hellwig authored
pNFS servers may return arbitrarily large layouts. Trim back the I/O size to one that we can at least allocate the page array for. Signed-off-by:
-
Peng Tao authored
Following http://www.rfc-editor.org/errata_search.php?rfc=5661&eid=2751 Don't set layoutcommit for commit_through_mds case. For FILE_SYNC writes, don't set layoutcommit. For DATA_SYNC wirtes, set layout commit right after wirtes done. For UNSTABLE writes, set layout commit when commit done. Signed-off-by:
Peng Tao <tao.peng@primarydata.com> Signed-off-by:
-
Peng Tao authored
Track lwb in nfs_commit_data so that we can use it to setup layoutcommit in commit_done callback. Signed-off-by:
-
Anna Schumaker authored
can_open_cached() reads values out of the state structure, meaning that we need the so_lock to have a correct return value. As a bonus, this helps clear up some potentially confusing code. Signed-off-by:
-
Chris Perl authored
When attempting to establish a local ephemeral endpoint for a TCP or UDP socket, do not explicitly call bind, instead let it happen implicilty when the socket is first used. The main motivating factor for this change is when TCP runs out of unique ephemeral ports (i.e. cannot find any ephemeral ports which are not a part of *any* TCP connection). In this situation if you explicitly call bind, then the call will fail with EADDRINUSE. However, if you allow the allocation of an ephemeral port to happen implicitly as part of connect (or other functions), then ephemeral ports can be reused, so long as the combination of (local_ip, local_port, remote_ip, remote_port) is unique for TCP sockets on the system. This doesn't matter for UDP sockets, but it seemed easiest to treat TCP and UDP sockets the same. This can allow mount.nfs(8) to continue to function successfully, even in the face of misbehaving applications which are creating a large number of TCP connections. Signed-off-by:
Chris Perl <chris.perl@gmail.com> Signed-off-by:
-
