1. 03 Aug, 2020 2 commits
  2. 25 Jul, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: dump WRITE argument in debug logs · 9ca2d34b
      Han-Wen Nienhuys authored
      This provides feedback to file system authors about what we receive
      from the kernel. This can help figure out which system is to blame for
      data corruption.
      
      Change-Id: I7cd7f340bf705c9972c95cfc6d3d8612f1ce8b34
      9ca2d34b
  3. 18 Jul, 2020 3 commits
    • Jakob Unterwurzacher's avatar
      Make pollHack work on read-only mounts · fee50bf0
      Jakob Unterwurzacher authored
      Use LOOKUP & OPEN instead of CREATE so it also
      works on read-only mounts.
      
      go-fuse/fs$ go test -run TestRoMount -v
      === RUN   TestRoMount
      20:31:07.892736 rx 2: INIT i0 {7.31 Ra 0x20000 NO_OPEN_SUPPORT,ASYNC_READ,POSIX_LOCKS,EXPORT_SUPPORT,AUTO_INVAL_DATA,SPLICE_MOVE,SPLICE_READ,HANDLE_KILLPRIV,ABORT_ERROR,EXPLICIT_INVAL_DATA,BIG_WRITES,SPLICE_WRITE,READDIRPLUS,WRITEBACK_CACHE,IOCTL_DIR,READDIRPLUS_AUTO,ASYNC_DIO,PARALLEL_DIROPS,ATOMIC_O_TRUNC,DONT_MASK,FLOCK_LOCKS,POSIX_ACL,MAX_PAGES,CACHE_SYMLINKS,NO_OPENDIR_SUPPORT}
      20:31:07.892799 tx 2:     OK, {7.28 Ra 0x20000 BIG_WRITES,READDIRPLUS,PARALLEL_DIROPS,ASYNC_READ,AUTO_INVAL_DATA,NO_OPEN_SUPPORT 0/0 Wr 0x10000 Tg 0x0}
      20:31:07.892967 rx 4: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      20:31:07.893002 tx 4:     OK, {i18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      20:31:07.893030 rx 6: OPEN i18446744073709551615 {O_RDONLY,0x8000}
      20:31:07.893037 tx 6:     OK, {Fh 18446744073709551615 }
      20:31:07.893053 rx 8: POLL i18446744073709551615
      20:31:07.893059 tx 8:     38=function not implemented
      20:31:07.893074 rx 10: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      20:31:07.893079 tx 10:     34=numerical result out of range
      20:31:07.893131 rx 12: RELEASE i18446744073709551615 {Fh 18446744073709551615 0x8000  L0}
      20:31:07.893153 tx 12:     34=numerical result out of range
      20:31:07.896705 received ENODEV (unmount request), thread exiting
      20:31:07.896719 received ENODEV (unmount request), thread exiting
      20:31:07.896888 received ENODEV (unmount request), thread exiting
      
      Change-Id: I6de861bdccf03184a4ca3deeefb1cfdc930aabe3
      --- PASS: TestRoMount (0.01s)
      PASS
      ok  	github.com/hanwen/go-fuse/v2/fs	0.015s
      
      Change-Id: Ic66ca153f5b739e5e081988fdc48d51cde5470f0
      fee50bf0
    • Jakob Unterwurzacher's avatar
      example: loopback: implement -ro flag · 2beb8ed3
      Jakob Unterwurzacher authored
      Currently fails like this
      
      	18:13:57.008805 Mount fail: read-only file system
      
      because pollHack requires write access.
      
      This will be fixed in pollHack in a later commit.
      
      Change-Id: I37b459339eda2e8270ca8094455db3ddbf764911
      2beb8ed3
    • Jakob Unterwurzacher's avatar
      fs: add TestRoMount · f612a450
      Jakob Unterwurzacher authored
      Also add a test if "ro" works directly in the fs
      package.
      
      Currently fails like this (will be fixed later):
      
      	--- FAIL: TestRoMount (0.01s)
      	    simple_test.go:116: read-only file syste
      
      Change-Id: I8194360b1d092268784b9c2b17de274b86633aa2
      f612a450
  4. 06 Jun, 2020 2 commits
  5. 05 Apr, 2020 2 commits
  6. 03 Apr, 2020 3 commits
    • Han-Wen Nienhuys's avatar
      fs: cleanup after TestForget · 73071346
      Han-Wen Nienhuys authored
      Change-Id: Ie2aeb012346b2fbcc32de09ea05b39069291d803
      73071346
    • Han-Wen Nienhuys's avatar
      fs: add a test verifying that forget drops nodes · 18e94a3c
      Han-Wen Nienhuys authored
      Change-Id: Ife141f38f2b7598223935963e2916b0b9a609cca
      18e94a3c
    • Kirill Smelkov's avatar
      nodefs: Allow for several Lookup requests to be served simultaneously · 17c0c400
      Kirill Smelkov authored
      Commit d0fca860 introduced big lookup lock with the idea to "make sure
      we don't return forgotten nodes to the kernel". However this way it also
      started to prevent several Lookup handlers to be running simultaneously,
      which can deadlock if e.g. Lookup handler somehow synchronizes with
      other thread which caused the lookup:
      
      https://github.com/hanwen/go-fuse/commit/d0fca860#commitcomment-37772099
      
      On the surface the fix is easy: if we need to prevent lookups running in
      parallel to forget, we can turn lookupLock into shared one wrt Lookup,
      but exclusive wrt Forget.
      
      However a more correct fix would be to review nodefs locking completely:
      there is already per-filesystem treeLock RWMutex, which is _already_
      used to synchronize for forgets and lookups. From this point of view
      lookupLock is unneeded and the correct fix that d0fca860 should have been
      doing is to correct the scope of treeLock to be used more precisely.
      
      However that would be a more intrusive change, and given that nodefs is
      considered deprecated today, imho it is better to proceed with making
      lookupLock a shared one.
      
      The test that is added deadlocks without provided fix.
      
      I suggest not to reject this patch based on rationale that "nodefs is
      deprecated" as there still are real filesystems that use nodefs.
      
      I had not reviewed v2 go-fuse/fs locking yet.
      
      Change-Id: I18e01457f474dea31dc17186dfe6db582c2e6337
      17c0c400
  7. 23 Mar, 2020 2 commits
    • Kirill Smelkov's avatar
      Merge branch 'y/lookup-parallel' into t · e93dea2d
      Kirill Smelkov authored
      * y/lookup-parallel:
        nodefs: Allow for several Lookup requests to be served simultaneously
      e93dea2d
    • Kirill Smelkov's avatar
      nodefs: Allow for several Lookup requests to be served simultaneously · f9c9e8da
      Kirill Smelkov authored
      Commit d0fca860 introduced big lookup lock with the idea to "make sure
      we don't return forgotten nodes to the kernel". However this way it also
      started to prevent several Lookup handlers to be running simultaneously,
      which can deadlock if e.g. Lookup handler somehow synchronizes with
      other thread which caused the lookup:
      
      https://github.com/hanwen/go-fuse/commit/d0fca860#commitcomment-37772099
      
      On the surface the fix is easy: if we need to prevent lookups running in
      parallel to forget, we can turn lookupLock into shared one wrt Lookup,
      but exclusive wrt Forget.
      
      However a more correct fix would be to review nodefs locking completely:
      there is already per-filesystem treeLock RWMutex, which is _already_
      used to synchronize for forgets and lookups. From this point of view
      lookupLock is unneeded and the correct fix that d0fca860 should have been
      doing is to correct the scope of treeLock to be used more precisely.
      
      However that would be a more intrusive change, and given that nodefs is
      considered deprecated today, imho it is better to proceed with making
      lookupLock a shared one.
      
      The test that is added deadlocks without provided fix.
      
      I suggest not to reject this patch based on rationale that "nodefs is
      deprecated" as there still are real filesystems that use nodefs.
      
      I had not reviewed v2 go-fuse/fs locking yet.
      
      Change-Id: I18e01457f474dea31dc17186dfe6db582c2e6337
      f9c9e8da
  8. 09 Mar, 2020 2 commits
    • Kirill Smelkov's avatar
      Merge branch 'y/nodefs-cancel' into t · 5e88fbad
      Kirill Smelkov authored
      * y/nodefs-cancel:
        Y nodefs: Propagate context to File methods
      5e88fbad
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · ee19aa4c
      Kirill Smelkov authored
      * master: (60 commits)
        fuse: increment loops WaitGroup on Server creation
        loopback: enable diagnostics logging
        fs: detect Inode.Path() hitting an orphaned inode
        tests: make RenameOpenDir more sensitive
        fs: add Options.Logger, and use throughout
        fs: fix FileAllocater type assertion
        loopback: leave file permissions on "000" files as-is
        example/loopback: make -allow-other imply default_permissions
        fuse: implement GETATTR for pollHack
        loopback: preserve owner when running as root
        tests: posixtest: add FstatDeleted
        loopback: use Lgetxattr and friends
        fs: remove typeChangeDebug code
        fs: don't log error failing MvChild
        posixtest: fix some lint errors
        fs: fix lint errors
        fs: failure to unmount on cleanup is a fatal error
        fs: plug fd leak in bridge_test.go
        fs: don't enforce dirent ordering.
        internal: skip test with other group if user is only 1 group
        ...
      ee19aa4c
  9. 24 Feb, 2020 4 commits
    • Jakob Unterwurzacher's avatar
      tests: fs: TestOpenDirectIO: move to /var/tmp · c34eae0e
      Jakob Unterwurzacher authored
      This used to create a hidden folder in the user's home
      directory to avoid /tmp, which is on tmpfs on modern
      distros.
      
      Use /var/tmp instead, which is also a real filesystem,
      and avoid cluttering the home dir.
      
      Change-Id: Ifa0a1dbdf7a2c4b08e53d20e28f287207ff8ef47
      c34eae0e
    • Jakob Unterwurzacher's avatar
      Travis CI: disable output buffering using "go test -p 1" · b4058856
      Jakob Unterwurzacher authored
      Per default, "go test ./..." tests multiple packages in
      parallel.
      
      From "go help build":
      
      	-p n
      		the number of programs, such as build commands or
      		test binaries, that can be run in parallel.
      		The default is the number of CPUs available.
      
      This seems rather innocent, but a consequence of tests
      running in parallel is that the all the output of each test
      is buffered until it completes. This prevents multiple tests
      interleaving their output.
      
      This also means that when a test hangs, we get no output at
      all, which is pretty bad.
      
      Disable parallelization and trade test time for better
      debugability of hung tests.
      
      Change-Id: I4c670b65baa2df3bef70bce622b830530a316ee7
      b4058856
    • Jakob Unterwurzacher's avatar
      go.mod: upgrade to Go 1.13 · b36138d5
      Jakob Unterwurzacher authored
      This gets added on every build and every test run,
      dirtying the git tree again and again.
      
      Give in and let Go 1.13 have it's entry in go.mod.
      
      Change-Id: I13679e54fca09ab818bc0167dff3f1ad851e763e
      b36138d5
    • Jakob Unterwurzacher's avatar
      tests: fix data race in TestDeletedInodePath · 6b30e299
      Jakob Unterwurzacher authored
      Running `go test ./... -race` showed a data race
      at `rootNode.deleted`. This race cannot really happen,
      but the race detector does not know that as the calls
      go through FUSE an appear to not be synchronized.
      
      Use atomic loads and stores to make the race detector
      happy.
      
      Change-Id: I26bd3e7d8efdd5b967fdb360eb17d7f71a8005c8
      6b30e299
  10. 04 Feb, 2020 1 commit
  11. 02 Feb, 2020 1 commit
  12. 20 Jan, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: increment loops WaitGroup on Server creation · fe141f38
      Han-Wen Nienhuys authored
      The server.Serve routine can be called both inline and as
      goroutine. In the latter case, it is a synchronization error for Serve
      to call serve.loops.Add() by itself, leading to a detected race.
      
      Change-Id: I36f24bd36d1ae77d71e7d69a54ebdf5dbee9bd62
      fe141f38
  13. 03 Jan, 2020 11 commits
    • Jakob Unterwurzacher's avatar
      loopback: enable diagnostics logging · 0e3c45fc
      Jakob Unterwurzacher authored
      The loopback example is usually used to debug and develop.
      Enable the new diagnostics logging facility, unless -q (quiet)
      is passed.
      
      Change-Id: I9ae5214fc33616656832a219fc1470f421786b8c
      0e3c45fc
    • Jakob Unterwurzacher's avatar
      fs: detect Inode.Path() hitting an orphaned inode · 3f7bb2c2
      Jakob Unterwurzacher authored
      Some ".deleted" logic was in place, but did not work
      properly when the Inode was in the root directory.
      
      Fix that by introducing the `found` variable and
      add a test that verifies that it works.
      
      Also, return `.go-fuse.$RANDOM/deleted` to make it
      very unlikely that the placeholder name matches an
      actual file or directory. Introducing the `/deleted`
      subdir makes sure operations creating a file or
      directory fail reliably with ENOENT.
      
      Tests now look like this:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        === RUN   TestPosix
        === RUN   TestPosix/RenameOpenDir
        [...]
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:383: Fstat failed: no such file or directory. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.014s
      
      Change-Id: I2eb6fd48a11df543c9b7daf62647cb9d8a892568
      3f7bb2c2
    • Jakob Unterwurzacher's avatar
      tests: make RenameOpenDir more sensitive · ae87e918
      Jakob Unterwurzacher authored
      The test seemed to pass because the inode number is overridden
      in rawBridge.getattr, but looking at the permissions shows that
      the wrong directory is stat()ed:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        [...]
        17:49:46.454077 received ENODEV (unmount request), thread exiting
        17:49:46.454343 received ENODEV (unmount request), thread exiting
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:392: got permissions 0755, want 0700. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.016s
      
      Also, add a log message whenever the inode number is overridden,
      this should (probably) not happen during normal operation. And
      it actually only happens once in the test suite (in RenameOpenDir):
      
        $ go test ./... -count 1 -v 2>&1 | grep "overriding ino"
        14:48:44.143694 warning: rawBridge.getattr: overriding ino 188663 with 186314
      
      See https://github.com/hanwen/go-fuse/issues/55
      
      Change-Id: I8b2ddb84c35a3b28b4f5e032e7113f8d484a5981
      ae87e918
    • Han-Wen Nienhuys's avatar
      fs: add Options.Logger, and use throughout · ee9c8261
      Han-Wen Nienhuys authored
      Change-Id: I4fabf222a306e5d3abdfda28422b046cd75c9a8c
      ee9c8261
    • Jakob Unterwurzacher's avatar
      fs: fix FileAllocater type assertion · 46bf7a5b
      Jakob Unterwurzacher authored
      The type assertion was meant to operate on `f.file`,
      not on `n.ops` again.
      
      Fixes xfstests generic/228.
      
      Change-Id: I8f8ca0cead91a512a6d7826dd7e7e5d1887ae627
      46bf7a5b
    • Jakob Unterwurzacher's avatar
      loopback: leave file permissions on "000" files as-is · 32ac1d91
      Jakob Unterwurzacher authored
      Set `opts.NullPermissions = true` to stop us from falsifying
      file permissions on "000" files.
      
      Fixes xfstests generic/088.
      
      Change-Id: Ibabbdc97b1ae2531ca093bae6bb441ae15d3238e
      32ac1d91
    • Jakob Unterwurzacher's avatar
      example/loopback: make -allow-other imply default_permissions · c6370440
      Jakob Unterwurzacher authored
      With -allow-other, other users can access the mountpoint.
      However, without default_permissions, nobody checks file
      permissions.
      
      Make loopback behave more like a regular filesystem by
      enabling default_permissions if -allow-other is passed.
      
      Fixes xfstests generic/087:
      
      $ sudo ./check-loopback generic/087
      fuse-xfstests gocryptfs-2018-08-18/67408ac7
      Thu 26 Dec 2019 03:08:54 PM UTC
      
      loopback is /usr/local/bin/loopback
      FSTYP         -- fuse.loopback
      PLATFORM      -- Linux/x86_64 brikett 5.3.12-300.fc31.x86_64
      MKFS_OPTIONS  -- /var/tmp/fuse-xfstests/check-loopback/scratchdev
      MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /var/tmp/fuse-xfstests/check-loopback/scratchdev /var/tmp/fuse-xfstests/check-loopback/scratchdir
      
      generic/087	 1s
      Ran: generic/087
      Passed all 1 tests
      
      Runtime was 1 seconds, exit code 0
      
      Change-Id: Ifa8ac38a3775c3cacf20d37638a95887c9c24f93
      c6370440
    • Jakob Unterwurzacher's avatar
      fuse: implement GETATTR for pollHack · d13ebc4a
      Jakob Unterwurzacher authored
      When the default_permissions option is passed to the kernel,
      it issues a extra GETATTR on the .go-fuse-epoll-hack node.
      
      Rejecting that with EIO makes `syscall.Creat` in `pollHack` fail,
      ultimately erroring out during mount.
      
      $ loopback -allow-other -debug b a
      16:06:39.576856 rx 2: INIT i0 {7.31 Ra 0x20000 NO_OPEN_SUPPORT,PARALLEL_DIROPS,ABORT_ERROR,EXPLICIT_INVAL_DATA,SPLICE_WRITE,READDIRPLUS,ASYNC_DIO,DONT_MASK,NO_OPENDIR_SUPPORT,AUTO_INVAL_DATA,READDIRPLUS_AUTO,POSIX_ACL,HANDLE_KILLPRIV,MAX_PAGES,ATOMIC_O_TRUNC,EXPORT_SUPPORT,SPLICE_MOVE,BIG_WRITES,SPLICE_READ,FLOCK_LOCKS,IOCTL_DIR,WRITEBACK_CACHE,ASYNC_READ,POSIX_LOCKS,CACHE_SYMLINKS}
      16:06:39.576999 tx 2:     OK, {7.28 Ra 0x20000 AUTO_INVAL_DATA,BIG_WRITES,ASYNC_READ,NO_OPEN_SUPPORT,PARALLEL_DIROPS,READDIRPLUS 0/0 Wr 0x10000 Tg 0x0}
      16:06:39.578670 rx 4: GETATTR i1 {Fh 0}
      16:06:39.578717 rx 6: GETATTR i1 {Fh 0}
      16:06:39.578735 tx 4:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.578765 tx 6:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.579028 rx 8: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      16:06:39.579053 tx 8:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579087 rx 10: CREATE i1 {0100100 [WRONLY,TRUNC,CREAT,0x8000] (022)} [".go-fuse-epoll-hack"] 20b
      16:06:39.579113 tx 10:     OK, {i18446744073709551615 g0 {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000} &{18446744073709551615 0 0}}
      16:06:39.579199 rx 14: GETATTR i1 {Fh 0}
      16:06:39.579205 rx 12: GETATTR i18446744073709551615 {Fh 0}
      16:06:39.579216 tx 14:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.579237 rx 16: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      16:06:39.579242 tx 12:     5=input/output error, {tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579247 tx 16:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579270 Mount fail: input/output error
      16:06:39.579271 rx 20: LOOKUP i1 [".Trash"] 7b
      
      Change-Id: I20024baf2e8f386b637abbd236b188bfdfa8579f
      d13ebc4a
    • Jakob Unterwurzacher's avatar
      loopback: preserve owner when running as root · 493adfb7
      Jakob Unterwurzacher authored
      If we are running as root have caller information from
      the context, try to set the owner of newly created files
      and directories.
      
      Lays the foundation for fixing xfstests generic/087
      (xfstests always run as root).
      
      Change-Id: Ib8d768153f3ec82ce572021a433a6398560efd44
      493adfb7
    • Jakob Unterwurzacher's avatar
      tests: posixtest: add FstatDeleted · 037efc7d
      Jakob Unterwurzacher authored
      Excercises the fd-finding logic in rawBridge.GetAttr.
      No issues found, tests pass, logic works fine.
      
      Change-Id: I49731b8ec5b41344d58409c8ca615f466bd95f29
      037efc7d
    • Jakob Unterwurzacher's avatar
      loopback: use Lgetxattr and friends · f28fbbf1
      Jakob Unterwurzacher authored
      Stop following symlinks when working with extended
      attributes, and add a test for it.
      
      Problem found by xfstests generic/062.
      
      Change-Id: I67f94451322cdfebdcbcc3af21679ccd4e2800d7
      f28fbbf1
  14. 19 Dec, 2019 2 commits
  15. 11 Dec, 2019 2 commits
  16. 10 Dec, 2019 1 commit
    • Han-Wen Nienhuys's avatar
      fs: failure to unmount on cleanup is a fatal error · c8185a54
      Han-Wen Nienhuys authored
      Any subsequent attempt to remove the mountdir will fail if the unmount
      does not succeed.
      
      Use syscall.Rmdir rather than os.Remove. We know the mount point is a
      directory, and Go's cleverness can only complicate matters here.
      
      Change-Id: I62cfad63f34af17b78c3184884a2972673f631a3
      c8185a54