An error occurred fetching the project authors.
- 02 Aug, 2021 1 commit
-
-
Jakob Unterwurzacher authored
We enable FUSE_CAP_ASYNC_READ per default, which means that the kernel can (and does) submit multiple concurrent out-of-order read requests to service userspace reads and kernel readahead. For some backing storages, like Amazon Cloud Drive, out-of-order reads are expensive. gocryptfs has implemented a delay-based workaround with its `-serialize_reads` flag for this case (see https://github.com/rfjakob/gocryptfs/issues/92 for details). Not enabling FUSE_CAP_ASYNC_READ makes the kernel do this for us, as verified by adding debug output to gocryptfs, so expose it as a mount flag in MountOptions. Fixes: https://github.com/hanwen/go-fuse/issues/140 Graphs-at: https://github.com/hanwen/go-fuse/issues/395 Related: https://github.com/rfjakob/gocryptfs/issues/92 Change-Id: I10f947d71e1453989c4a9b66fbb0407f7163994f
-
- 08 May, 2021 1 commit
-
-
Jakob Unterwurzacher authored
Enables kernel ACL enforcement. Change-Id: I7fb5f6f8b350ad430da18e0173df942b98da7c02
-
- 04 Nov, 2020 1 commit
-
-
Jakob Unterwurzacher authored
nodeid and inode number used to be identical, which is why both could be marked as "iXXX". This is no longer the case, so make nodeids distinguishable. Example: 23:47:17.637736 rx 9156: MKDIR n28 {0777 (00)} ["d90X"] 5b 23:47:17.637801 tx 9156: OK, {n112 g1 tE=1s tA=1s {M040755 SZ=4096 L=2 1026:1026 B8*4096 i0:2769795 A 1601761637.637385 M 1601761637.637385 C 1601761637.637385}} v4: also update orphaned inode warning Change-Id: Ib29dabfd4cf6692cf4118191f8dc73529562426f
-
- 26 Nov, 2019 1 commit
-
-
Jakob Unterwurzacher authored
ListXAttr called with an empty buffer returns the current size of the list but does not touch the buffer (see man 2 listxattr). Fixes the xfstests generic/020 crash. Fixes https://github.com/hanwen/go-fuse/issues/337 . Add test Co-authored-by:
Han-Wen Nienhuys <hanwen@google.com> Change-Id: I94c94bc56fdf506d2eb924b246336b787fa2976a
-
- 07 Nov, 2019 1 commit
-
-
Han-Wen Nienhuys authored
open(2) has supports direct IO. When opening a file in the loopback FS, writes to the underlying would fail, because the FUSE server does not position the bytes to write (which come directly after the WriteIn header) on a 512-byte boundary. Fix this by allocating the input buffer with such that the first content byte is aligned. Add a test. Fixes #328. Change-Id: Ib7ad24aff673413ab8db4112d0b12143c2654617
-
- 25 Apr, 2019 2 commits
-
-
Kirill Smelkov authored
This complements commit "fuse: allow filesystems to disable CAP_AUTO_INVAL_DATA" and teaches go-fuse to request explicit data cache invalidation mode if fuse.MountOptions.ExplicitDataCacheControl is set. See https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/commit/?id=ad2ba64dd489 and https://lwn.net/ml/linux-fsdevel/20190315212556.9315-1-kirr%40nexedi.com/ for rationale and details.
-
Kirill Smelkov authored
CAP_AUTO_INVAL_DATA is capability of kernel, but from the point of view of a filesystem it is not a capability, but a behaviour request: if set, it requests to kernel - filesystem client - to perform data cache invalidations based on heuristics. Current heuristic is to drop data cache for a file if kernel sees file's mtime being changed: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/fuse/inode.c?id=v5.0-0-g1c163f4c7b3f#n238 https://git.kernel.org/linus/eed2179efe However this could be unwanted behaviour if filesystem is careful to explicitly invalidate local file cache: despite filesystem attempts to preciously keep data cache, the whole cache of the file is unnecessarily dropped. -> To fix add a new mount options for filesystems to indicate they are careful with respect to data cache invalidations and thus should be fully responsible for invalidating data cache. Teach go-fuse to not send CAP_AUTO_INVAL_DATA in this mode to kernel on FUSE handshake. Note, as of upcoming Linux 5.1 (estimated to be released mid 2019), FUSE client in kernel still automatically and unconditionally drops whole data cache of a file if its sees size change. Kernel and go-fuse patches to fix that are here: https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/commit/?id=ad2ba64dd489 https://github.com/hanwen/go-fuse/pull/273 The kernel patch is likely to enter mainline kernel when 5.2 merge window opens. We'll remove XXX in a follow-up go-fuse patch that adds support for CAP_EXPLICIT_INVAL_DATA.
-
- 23 Apr, 2019 1 commit
-
-
Kirill Smelkov authored
When kernel sends WRITE request it sends with it file handle, offset, size etc. We were not printing all this. Compare e.g. debug output sample for TestUtimesNano before and after the patch. before: rx 8: CREATE i1 {0100600 [CREAT,TRUNC,WRONLY,0x8000] (00)} ["hello.txt"] 10b tx 8: OK, {i3 g2 {M0100600 SZ=0 L=1 1000:1000 B0*4096 i0:733462 A 1556038355.426630 M 1556038355.426630 C 1556038355.426630} &{2 0 0}} rx 9: GETXATTR i3 {sz 0} ["security.capability"] 20b tx 9: 61=no data available rx 10: WRITE i3 3b <-- NOTE tx 10: OK after: rx 8: CREATE i1 {0100600 [WRONLY,CREAT,TRUNC,0x8000] (00)} ["hello.txt"] 10b tx 8: OK, {i3 g2 {M0100600 SZ=0 L=1 1000:1000 B0*4096 i0:736300 A 1556038379.359197 M 1556038379.359197 C 1556038379.359197} &{2 0 0}} rx 9: GETXATTR i3 {sz 0} ["security.capability"] 20b tx 9: 61=no data available rx 10: WRITE i3 {Fh 2 [0 +3) L 0 WRONLY,NONBLOCK,0x8000} 3b <-- NOTE tx 10: OK
-
- 06 Apr, 2019 1 commit
-
-
Han-Wen Nienhuys authored
-
- 27 Mar, 2019 7 commits
-
-
Han-Wen Nienhuys authored
Thanks to Ryan Lamore (ryanlamore) for spotting this
-
Han-Wen Nienhuys authored
This is unlikely to be useful, but makes the API more uniform.
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
add MaxPages to InitOut
-
Han-Wen Nienhuys authored
The protocol is: request: read (with buffer 1024) response: data (if buffer is big enough), or (data size, ERANGE) Before, the raw API provided * GetXAttrSize: called if the kernel sends a 0 sized buffer (which never happens * GetXAttrData: called in other cases. The return value is []byte forcing allocation of potentially large buffers * ListXAttr: return value []byte, so also problematic for allocation. Now, the raw API mirrors the kernel API more closely, and use recycled buffers for more efficiency.
-
Han-Wen Nienhuys authored
This should be tagged with a version bump because it changes the RawFileSystem API. Changes to nodefs and paths will be transparent: a Cancel channel was added to the fuse.Context struct. In addition fuse.Context now implements the Go context package. This also increases the minimum Go version to 1.7.
-
- 08 Mar, 2019 1 commit
-
-
Han-Wen Nienhuys authored
Use linear search to find requests to interrupt. This assumes that interrupts are rare, and the number of outstanding requests when it happens is low. Interrupted requests are not put back into the request pool to avoid racing with in-progress mutators.
-
- 24 Feb, 2019 2 commits
-
-
Han-Wen Nienhuys authored
INTERRUPT does not require a reply to the kernel. Trying to send it yields writer: Write/Writev failed, err: 2=no such file or directory. opcode: INTERRUPT
-
Han-Wen Nienhuys authored
-
- 23 Feb, 2019 2 commits
-
-
Han-Wen Nienhuys authored
In practice, this only triggers for RENAME_EXCHANGE, as the kernel issues the RENAME opcode in case of a normal rename or a RENAME_NOREPLACE. We have always advertised rename2 support (protocol v23), but it returned ENOSYS always. Instead, rename RenameIn => Rename1In, and add Flags to RenameIn. For raw filesystems that want to support rename2 should examine RenameIn.Flags and check for syscall.RENAME_XXX. For fuse/nodefs, we return ENOSYS in case of Rename2, since we lack API surface to propagate the flags.
-
Han-Wen Nienhuys authored
-
- 17 Feb, 2019 1 commit
-
-
Kirill Smelkov authored
After Linux VFS was enhanced to be able to do several lookups and readdirs in parallel, FUSE client in Linux, in fear to break existing filesystems, explicitly reverted it back to be serial in FUSE exchange and requested filesystems to indicate via CAP_PARALLEL_DIROPS flag that the filesystem server is ready to handle parallel lookup and readdir requests. https://git.kernel.org/linus/5c672ab3f0 We should be ready to handle and we provide MountOptions.SingleThreaded for those filesystems, which are not ready to handle requests in parallel.
-
- 30 Jan, 2019 1 commit
-
-
Jakob Unterwurzacher authored
Node numbers are now written as "i1234" everywhere except in the BATCH_FORGET output, fix that. Also show the transaction number so we can follow where each forget comes from. As a normal FORGET looks like this: 2019/01/27 14:38:07 rx 1488: FORGET i822 {Nlookup=1} with the new output, "grep FORGET i1234 " will catch both single forgets and batch forgets. Before: 2019/01/20 18:15:31 rx 1372: BATCH_FORGET i0 {Count=3} 48b 2019/01/20 18:15:31 doBatchForget: forgetting 1 of 3: NodeId: 40, Nlookup: 1 2019/01/20 18:15:31 doBatchForget: forgetting 2 of 3: NodeId: 41, Nlookup: 1 2019/01/20 18:15:31 doBatchForget: forgetting 3 of 3: NodeId: 42, Nlookup: 1 After: 2019/01/27 14:48:35 rx 2982: BATCH_FORGET i0 {Count=2} 32b 2019/01/27 14:48:35 doBatchForget: rx 2982 1/2: FORGET i804 {Nlookup=1} 2019/01/27 14:48:35 doBatchForget: rx 2982 2/2: FORGET i803 {Nlookup=1}
-
- 17 Dec, 2018 1 commit
-
-
Kirill Smelkov authored
This patch continues bdca0e6a (Add support for store notify) and adds corresponding support for counterpart to cache-store - to retrieve an inode data from kernel cache. As it was already noted in bdca0e6a, FUSE protocol provides primitives for pagecache control: to invalidate a data region for inode (notify_inval_inode), to store data into inode kernel's cache (notify_store), and to retrieve data from inode kernel's cache (notify_retrieve). For the latter 2 FUSE protocol messages and brief documentation about semantic can be seen here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n68 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n756 https://git.kernel.org/linus/a1d75f2582 https://git.kernel.org/linus/2d45ba381a In short, to retrieve data from kernel cache, filesystem server sends S > C NOTIFY_RETRIEVE_CACHE{notifyUnique, inode, offset, size} and if that message was sent correctly, the kernel sends back another write-style message with unique=notifyUnique S < C NOTIFY_REPLY{inode, offset, size, data} Since so far there were no cases when a server was querying the kernel, and the reply comes as separate kernel "request", we have to add infrastructure for tracking such in-flight queries. This is done by adding Server.retrieveTab and friends. Otherwise the implementation is straightforward. A particular note is that from a user-level API point of view we are not following e.g. libfuse to register a callback to be invoked upon reply, but instead provide {Inode,File}RetrieveCache that synchronously send notify query and wait for kernel's reply. This fits more naturally to Go and is easier to use.
-
- 11 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
- it is not NOTIFY_INVAL_DELETE, but just NOTIFY_DELETE as the kernel is notified of entry being deleted and there is no invalidation here. uapi/linux/fuse.h does not use "_INVAL_" for NOTIFY_DELETE neither. - similarly for "notify store/retrieve" _INVAL_ is not appropriate and neither is used in uapi/linux/fuse.h . However since we took more explicit approach for "notify store/retrieve" naming (see bdca0e6a "Add support for store notify") let's also add _CACHE suffix for "notify store/retrieve" operation names for consistency and for being less ambiguous. - for inode/entry invalidation operations, let's use _INVAL_ in internal _OP_NOTIFY_*, similarly to how it is used in corresponding NOTIFY_* constant. For example: _OP_NOTIFY_ENTRY -> _OP_NOTIFY_INVAL_ENTRY (corresponds to NOTIFY_INVAL_ENTRY)
-
- 08 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
I'm writing a networked filesystem which reads data in 2MB blocks from remote database. However read requests from the kernel come in much smaller sizes - for example 4K-128K. Since it would be very slow to refetch the 2MB block for e.g. every consecutive 4K reads, a cache for fetched data is needed. A custom cache would do, however since the kernel already implements pagecache to cache file data, it is logical to use it directly. FUSE protocol provides primitives for pagecache control. We already have support for e.g. invalidating a data region for inode (InodeNotify), but there is more there. In particular it is possible to store and retrieve data into/from the kernel cache with "store notify" and "retrieve notify" messages: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n68 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n756 https://git.kernel.org/linus/a1d75f2582 https://git.kernel.org/linus/2d45ba381a This patch adds support for "store notify". Adding support for "retrieve notify" might be added later since a) it is not needed right now, and b) supporting it is a bit more work since the kernel sends a separate reply with data and infrastructure has to be added to glue it back to original "retrieve notify" server-originated request. For user-visible API I decided not to duplicate FUSE-protocol naming 1-1 and to be more explicit in names - emphasizing it is about cache control - it is e.g. InodeNotifyStoreCache instead of InodeNotifyStore, and for retrieving it (hopefully) should be just InodeRetrieveCache instead of InodeNotifyRetrieveCache and a separate callback. Thanks beforehand, Kirill
-
- 27 Jul, 2018 1 commit
-
-
Han-Wen Nienhuys authored
This ensure VFS locking continues working, ensuring backward compatibility.
-
- 24 Jul, 2018 1 commit
-
-
Yufeng Cheng authored
Co-authored-by:
Paul Warren <paul.warren@emc.com> Co-authored-by:
Maria Shaldibina <mshaldibina@pivotal.io> Co-authored-by:
Han-Wen Nienhuys <hanwen@google.com>
-
- 02 May, 2018 1 commit
-
-
Jakob Unterwurzacher authored
Set the operationHandlers[op].DecodeIn function and add xattr name parsing. SETXATTR is special because it is the only opcode that takes a file name (the xattr name) and a binary blob (the xattr value). This was not supported by the file name parsing code: * setting FileNames = 1 would lump the xattr name and value together and truncates the last byte * setting FileNames = 2 truncated the last xattr value byte. This was solved by adding a special-case to parse(), which seemed less ugly than adding a special-case InputDebug(), or leaving the truncation as-is. Before: 2018/05/01 16:47:39 Dispatch 6: SETXATTR, NodeId: 3. 12 bytes After: 2018/05/01 16:48:36 Dispatch 6: SETXATTR, NodeId: 3. data: {sz 3 f0} names: [user.foo] 12 bytes The change only affects debug output as doSetXAttr() does its own bytes.SplitN(). The parsed filename *could* also be used in doSetXAttr(), but the code seems clearer as-is.
-
- 19 Oct, 2017 1 commit
-
-
Han-Wen Nienhuys authored
This saves a couple of bytes, and avoids a copy on serializing the response header. Add and document a hack for _OP_GETXATTR in serializeHeader.
-
- 18 Jul, 2017 1 commit
-
-
Han-Wen Nienhuys authored
Some versions of FUSE need this; fixes pathfs owner tests.
-
- 15 Jul, 2017 1 commit
-
-
Han-Wen Nienhuys authored
Go 1.9 uses epoll() for more efficient file I/O. File I/O causes a call to epoll, and the runtime makes this call take up a GOMAXPROCS slot. The FUSE kernel module also supports poll: polling on a file residing in a FUSE file system causes the kernel to sends a POLL request to the userspace process. If the process responds with ENOSYS, the kernel will stop forwarding poll requests to the FUSE process. In a test for Go FUSE file systems, it is normal to serve the filesystem out of the same process that opens files in the file system. If this happens in Go 1.9, the epoll call can take the only GOMAXPROCS slot left, leaving the process unable to respond to the FUSE POLL opcode, deadlocking the process. This change add support for a magic file "/ .go-fuse-epoll-hack" with node ID uint64(-1), and on starting up the file system, the library calls poll() on this file, triggering the POLL opcode before the Go runtime had a chance to do so. There are two problem scenarios left: * File system tests that start I/O before calling WaitMount() still risk deadlocking themselves. * The Linux kernel keeps track of feature support in fuse_conn, which notes * The following bitfields are only for optimization purposes * and hence races in setting them will not cause malfunction if our forced ENOSYS gets lost due to a race condition in the kernel, this can still trigger. Fixes golang/go#21014 and #165
-
- 21 Dec, 2016 1 commit
-
-
Nick Craig-Wood authored
-
- 09 Aug, 2016 1 commit
-
-
Adam H. Leventhal authored
On Mac OS X the Finder loses its mind if GetXAttr returns ENODATA rather than ENOATTR. On Linux, ENOATTR is just an alias for ENOATTR. The Linux man page references ENOATTR: http://man7.org/linux/man-pages/man2/fgetxattr.2.html
-
- 04 Aug, 2016 1 commit
-
-
Han-Wen Nienhuys authored
Add AUTHORS file.
-
- 03 Aug, 2016 1 commit
-
-
Han-Wen Nienhuys authored
-
- 11 Jul, 2016 1 commit
-
-
Han-Wen Nienhuys authored
SetDebug methods are maintained for compatibility. This is part of a patch series to process the INIT opcode on mount. This will enable filesystems to switch functionality depending on kernel settings. In order to get the debug output for the INIT opcode, the debug option must be passed to the mount method. Deprecate the SetDebug methods. There is often little practical use in toggling debug on the fly, since the error scenarios depend on timing details, and must usually be isolated into unittests. Change-Id: I221c995add5f7621b627f738d7e97af7daf81b2a
-
- 30 Jun, 2016 2 commits
-
-
Han-Wen Nienhuys authored
Any node that returns ENOSYS for Getxattr will cause the kernel to stop issuing Getxattr calls altogether. In file systems that must support GetXAttr and that has multiple node types, tracking down all node types and overriding GetXAttr for them individually is annoying. For filesystems that need the performance benefit of skipping GETXATTR calls, provide a DisableXAttrs option. Change-Id: I98327aa959b3b26192e15bc3be96b7a6c0ada5f6
-
Han-Wen Nienhuys authored
Change-Id: I6c17b6937483ad4a396cafe43d3b4605ce4a83cf
-
- 24 Apr, 2016 1 commit
-
-
Jakob Unterwurzacher authored
Change-Id: I5612016b2bccf9e28186a83f2f420cefe1c30120
-