gocryptfs user Felix Lechner reported a nil pointer dereference
in GetAttr: https://github.com/rfjakob/gocryptfs/issues/260

The crash is in line

	n.setClientInode(fi.Ino)

because fi is nil.

This can happen when file.GetAttr() returns an error code other than
ENOSYS and EBADF. For gocryptfs, this can only happen when an open
file descriptor breaks. In this case it was triggered by a failing
NFS volume.

Fix the crash by erroring out for error codes that are not handled
by the retry-by-path logic.

Current state is: fuse's godoc package-level description is completely
empty because there is a blank link between package-level description
and package clause.

Yes, a user still has to go through api.go and read it in full, but
having at least something for the description is a better start, and
removing one line is easy.

We already do it this way e.g. for nodefs and in other places.

I'm writing a networked filesystem which reads data in 2MB blocks from
remote database. However read requests from the kernel come in much
smaller sizes - for example 4K-128K.

Since it would be very slow to refetch the 2MB block for e.g. every
consecutive 4K reads, a cache for fetched data is needed. A custom cache
would do, however since the kernel already implements pagecache to cache
file data, it is logical to use it directly.

FUSE protocol provides primitives for pagecache control. We already have
support for e.g. invalidating a data region for inode (InodeNotify), but
there is more there. In particular it is possible to store and
retrieve data into/from the kernel cache with "store notify" and
"retrieve notify" messages:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n68
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n756

https://git.kernel.org/linus/a1d75f2582
https://git.kernel.org/linus/2d45ba381a

This patch adds support for "store notify". Adding support for "retrieve
notify" might be added later since a) it is not needed right now, and b)
supporting it is a bit more work since the kernel sends a separate reply
with data and infrastructure has to be added to glue it back to original
"retrieve notify" server-originated request.

For user-visible API I decided not to duplicate FUSE-protocol naming
1-1 and to be more explicit in names - emphasizing it is about cache
control - it is e.g. InodeNotifyStoreCache instead of
InodeNotifyStore, and for retrieving it (hopefully) should be just
InodeRetrieveCache instead of InodeNotifyRetrieveCache and a
separate callback.

Thanks beforehand,
Kirill

Current code was requiring that both

	in.Major >= maj		_and_
	in.Minor >= min

this way e.g. kernel's 7.27 would satisfy requirement of e.g. 7.25, but
8.1 won't do. Fix it.

out.Attr is already of type fuse.Attr .

Improve test to eliminate backing FS semantics

On a Linux system with go-fuse program running as root, the mount is
performed by calling /bin/fusermount, and the unmount is performed with
syscall.Unmount()

This creates a problem on systems (CentOS 6) with a static-but-edited-by-mount
/etc/mtab file.

 - fusermount adds a line to mtab when the go-fuse program starts
 - syscall.Unmount() doesn't edit the file on program exit
 - subsequent invocations of the program fail to mount with:

    "Mount fail: fusermount exited with code 256"

Deleting the now-inaccurate mtab entry clears things up.

There's probably a workaround by adding "-n" option so that mount doesn't
edit mtab in the first place, but it's not obvious where to insert that when
starting with the hello.go example.

This ensure VFS locking continues working, ensuring backward compatibility.

according to golint, comments of a type must start with the type name
- case sensitive.

This makes the package available on all platforms

Co-authored-by: Paul Warren <paul.warren@emc.com>
Co-authored-by: Maria Shaldibina <mshaldibina@pivotal.io>
Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>

When set, all Lookup calls will be forwarded from the kernel, even if
the current tree already has a child with the given name.

This allows the tree to be reconfigured driven by kernel events.

Fixes #211.

All but the newest MacOS versions (xnu-4570.1.46 / High Sierra)
lack utimensat() and UTIME_OMIT, see:
https://github.com/apple/darwin-xnu/blame/0a798f6738bc1db01281fc08ae024145e84df927/bsd/sys/stat.h#L537

The UTIME_OMIT value is intepreted literally by MacOS, resulting
in all files getting a 1970 timestamp
( https://github.com/rfjakob/gocryptfs/issues/229 ).

Emulate UTIME_OMIT by filling in the missing values
using an extra GetAttr call.

To not duplicate the logic in pathfs and nodefs, an
internal "utimens" helper package is created.

This patch fixes the failing utimens tests.

The File returned for NewReadOnlyFile only blocks writes not reads and
writes.

Tests loopbackFileSystem.Utimens() and loopbackfile.Utimens()
at 1-second precision.

The exising TestUtimesNano() test only works on Linux
because it relies on syscall.UtimesNano(), which is not
available on Darwin. The new tests call the Utimens()
functions directly, bypassing FUSE, and work on all
platforms.

Because Darwin does not have syscall.UtimesNano(),
getting the Utimens() implementation right is hard.

The tests currently fail on Darwin, underlining the
need for them ( https://github.com/rfjakob/gocryptfs/issues/229 ):

$ go test ./fuse/nodefs
[...]
--- FAIL: TestLoopbackFileUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384186 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

$ go test ./fuse/pathfs
--- FAIL: TestLoopbackFileSystemUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384379 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

Set the operationHandlers[op].DecodeIn function and
add xattr name parsing.

SETXATTR is special because it is the only opcode that
takes a file name (the xattr name) and a binary blob
(the xattr value). This was not supported by the
file name parsing code:
* setting FileNames = 1 would lump the xattr name
  and value together and truncates the last byte
* setting FileNames = 2 truncated the last xattr
  value byte.

This was solved by adding a special-case to parse(),
which seemed less ugly than adding a special-case
InputDebug(), or leaving the truncation as-is.

Before:
  2018/05/01 16:47:39 Dispatch 6: SETXATTR, NodeId: 3.  12 bytes
After:
  2018/05/01 16:48:36 Dispatch 6: SETXATTR, NodeId: 3. data: {sz 3 f0} names: [user.foo] 12 bytes

The change only affects debug output as doSetXAttr() does its
own bytes.SplitN(). The parsed filename *could* also be used in
doSetXAttr(), but the code seems clearer as-is.

We used to hand out a new generation number even
for already-known handles. This does not seem to
cause problems on Linux, but osxfuse throws errors
to userspace with

        osxfuse: vnode changed generation

showing up in the kernel log.

Introduce a per-handle generation number that stays
constant until the handle is forgotten.

Tested on Linux and MacOS via gocryptfs. Add test for handle map
behavior.

Fixes https://github.com/hanwen/go-fuse/issues/204
See also https://github.com/hanwen/go-fuse/pull/205

./cache_test.go:98: Skip call has possible formatting directive %d
FAIL	github.com/hanwen/go-fuse/fuse/test [build failed]

On mips64le, syscall.Getdents() and struct syscall.Dirent do
not fit together, causing our parseDirents() implementation to
return garbage ( see https://github.com/rfjakob/gocryptfs/issues/200
and https://github.com/golang/go/issues/23624 ).

Switch to unix.Getdents which does not have this problem -
the next Go release with the syscall package fixes is too
far away, and will take time to trickle into distros.

This issue has been reported by Debian maintainer
Felix Lechner.

Thanks to alexanderharm for suggesting this fix.

Markdown-ify README

The function was written with only a single return at
the end, which was elegant, but harder to follow.

Add early returns to handle common cases and add a few
comments.

A functional change is that we now catch when we get
OK with fi == nil and return EINVAL and log a message.
This should not happen and is a bug in the filesystem
implementation.

Passes ./all.bash and the gocryptfs test suite.

Consulting the output size is incorrect as it does not take into account the
cases where an xattr with an empty value exists.

Moreover, treat listxattr exactly the same as getxattr as the two both
can be used for two different operations.

Add a test for an xattr with an empty value
Signed-off-by: Shayan Pooya <shayan@arista.com>

This helps settle how much performance improvement splice()ing files
brings. The answer is that end-to-end read performance is improved by
about 10% for reading large files.

This saves a couple of bytes, and avoids a copy on serializing the
response header.

Add and document a hack for _OP_GETXATTR in serializeHeader.

Trivial patch that fixes a blank line in the debug output.

Before:

  2017/10/03 20:25:56 Dispatch 602: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 DIRECTORY,NONBLOCK,0x8000}
  2017/10/03 20:25:56 Serialize 602: READDIRPLUS code: OK value:  832 bytes data

  2017/10/03 20:25:56 Dispatch 603: GETXATTR, NodeId: 7. data: {sz 0} names: [system.posix_acl_access] 24 bytes
  2017/10/03 20:25:56 Serialize 603: GETXATTR code: 61=no data available value:

After:

  2017/10/03 21:02:46 Dispatch 22: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 22: READDIRPLUS code: OK value:  648 bytes data
  2017/10/03 21:02:46 Dispatch 23: READDIRPLUS, NodeId: 1. data: {Fh 2 off 4 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 23: READDIRPLUS code: OK value: