1. 03 Apr, 2020 1 commit
    • Kirill Smelkov's avatar
      nodefs: Allow for several Lookup requests to be served simultaneously · 17c0c400
      Kirill Smelkov authored
      Commit d0fca860 introduced big lookup lock with the idea to "make sure
      we don't return forgotten nodes to the kernel". However this way it also
      started to prevent several Lookup handlers to be running simultaneously,
      which can deadlock if e.g. Lookup handler somehow synchronizes with
      other thread which caused the lookup:
      
      https://github.com/hanwen/go-fuse/commit/d0fca860#commitcomment-37772099
      
      On the surface the fix is easy: if we need to prevent lookups running in
      parallel to forget, we can turn lookupLock into shared one wrt Lookup,
      but exclusive wrt Forget.
      
      However a more correct fix would be to review nodefs locking completely:
      there is already per-filesystem treeLock RWMutex, which is _already_
      used to synchronize for forgets and lookups. From this point of view
      lookupLock is unneeded and the correct fix that d0fca860 should have been
      doing is to correct the scope of treeLock to be used more precisely.
      
      However that would be a more intrusive change, and given that nodefs is
      considered deprecated today, imho it is better to proceed with making
      lookupLock a shared one.
      
      The test that is added deadlocks without provided fix.
      
      I suggest not to reject this patch based on rationale that "nodefs is
      deprecated" as there still are real filesystems that use nodefs.
      
      I had not reviewed v2 go-fuse/fs locking yet.
      
      Change-Id: I18e01457f474dea31dc17186dfe6db582c2e6337
      17c0c400
  2. 24 Feb, 2020 4 commits
    • Jakob Unterwurzacher's avatar
      tests: fs: TestOpenDirectIO: move to /var/tmp · c34eae0e
      Jakob Unterwurzacher authored
      This used to create a hidden folder in the user's home
      directory to avoid /tmp, which is on tmpfs on modern
      distros.
      
      Use /var/tmp instead, which is also a real filesystem,
      and avoid cluttering the home dir.
      
      Change-Id: Ifa0a1dbdf7a2c4b08e53d20e28f287207ff8ef47
      c34eae0e
    • Jakob Unterwurzacher's avatar
      Travis CI: disable output buffering using "go test -p 1" · b4058856
      Jakob Unterwurzacher authored
      Per default, "go test ./..." tests multiple packages in
      parallel.
      
      From "go help build":
      
      	-p n
      		the number of programs, such as build commands or
      		test binaries, that can be run in parallel.
      		The default is the number of CPUs available.
      
      This seems rather innocent, but a consequence of tests
      running in parallel is that the all the output of each test
      is buffered until it completes. This prevents multiple tests
      interleaving their output.
      
      This also means that when a test hangs, we get no output at
      all, which is pretty bad.
      
      Disable parallelization and trade test time for better
      debugability of hung tests.
      
      Change-Id: I4c670b65baa2df3bef70bce622b830530a316ee7
      b4058856
    • Jakob Unterwurzacher's avatar
      go.mod: upgrade to Go 1.13 · b36138d5
      Jakob Unterwurzacher authored
      This gets added on every build and every test run,
      dirtying the git tree again and again.
      
      Give in and let Go 1.13 have it's entry in go.mod.
      
      Change-Id: I13679e54fca09ab818bc0167dff3f1ad851e763e
      b36138d5
    • Jakob Unterwurzacher's avatar
      tests: fix data race in TestDeletedInodePath · 6b30e299
      Jakob Unterwurzacher authored
      Running `go test ./... -race` showed a data race
      at `rootNode.deleted`. This race cannot really happen,
      but the race detector does not know that as the calls
      go through FUSE an appear to not be synchronized.
      
      Use atomic loads and stores to make the race detector
      happy.
      
      Change-Id: I26bd3e7d8efdd5b967fdb360eb17d7f71a8005c8
      6b30e299
  3. 02 Feb, 2020 1 commit
  4. 20 Jan, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: increment loops WaitGroup on Server creation · fe141f38
      Han-Wen Nienhuys authored
      The server.Serve routine can be called both inline and as
      goroutine. In the latter case, it is a synchronization error for Serve
      to call serve.loops.Add() by itself, leading to a detected race.
      
      Change-Id: I36f24bd36d1ae77d71e7d69a54ebdf5dbee9bd62
      fe141f38
  5. 03 Jan, 2020 11 commits
    • Jakob Unterwurzacher's avatar
      loopback: enable diagnostics logging · 0e3c45fc
      Jakob Unterwurzacher authored
      The loopback example is usually used to debug and develop.
      Enable the new diagnostics logging facility, unless -q (quiet)
      is passed.
      
      Change-Id: I9ae5214fc33616656832a219fc1470f421786b8c
      0e3c45fc
    • Jakob Unterwurzacher's avatar
      fs: detect Inode.Path() hitting an orphaned inode · 3f7bb2c2
      Jakob Unterwurzacher authored
      Some ".deleted" logic was in place, but did not work
      properly when the Inode was in the root directory.
      
      Fix that by introducing the `found` variable and
      add a test that verifies that it works.
      
      Also, return `.go-fuse.$RANDOM/deleted` to make it
      very unlikely that the placeholder name matches an
      actual file or directory. Introducing the `/deleted`
      subdir makes sure operations creating a file or
      directory fail reliably with ENOENT.
      
      Tests now look like this:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        === RUN   TestPosix
        === RUN   TestPosix/RenameOpenDir
        [...]
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:383: Fstat failed: no such file or directory. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.014s
      
      Change-Id: I2eb6fd48a11df543c9b7daf62647cb9d8a892568
      3f7bb2c2
    • Jakob Unterwurzacher's avatar
      tests: make RenameOpenDir more sensitive · ae87e918
      Jakob Unterwurzacher authored
      The test seemed to pass because the inode number is overridden
      in rawBridge.getattr, but looking at the permissions shows that
      the wrong directory is stat()ed:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        [...]
        17:49:46.454077 received ENODEV (unmount request), thread exiting
        17:49:46.454343 received ENODEV (unmount request), thread exiting
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:392: got permissions 0755, want 0700. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.016s
      
      Also, add a log message whenever the inode number is overridden,
      this should (probably) not happen during normal operation. And
      it actually only happens once in the test suite (in RenameOpenDir):
      
        $ go test ./... -count 1 -v 2>&1 | grep "overriding ino"
        14:48:44.143694 warning: rawBridge.getattr: overriding ino 188663 with 186314
      
      See https://github.com/hanwen/go-fuse/issues/55
      
      Change-Id: I8b2ddb84c35a3b28b4f5e032e7113f8d484a5981
      ae87e918
    • Han-Wen Nienhuys's avatar
      fs: add Options.Logger, and use throughout · ee9c8261
      Han-Wen Nienhuys authored
      Change-Id: I4fabf222a306e5d3abdfda28422b046cd75c9a8c
      ee9c8261
    • Jakob Unterwurzacher's avatar
      fs: fix FileAllocater type assertion · 46bf7a5b
      Jakob Unterwurzacher authored
      The type assertion was meant to operate on `f.file`,
      not on `n.ops` again.
      
      Fixes xfstests generic/228.
      
      Change-Id: I8f8ca0cead91a512a6d7826dd7e7e5d1887ae627
      46bf7a5b
    • Jakob Unterwurzacher's avatar
      loopback: leave file permissions on "000" files as-is · 32ac1d91
      Jakob Unterwurzacher authored
      Set `opts.NullPermissions = true` to stop us from falsifying
      file permissions on "000" files.
      
      Fixes xfstests generic/088.
      
      Change-Id: Ibabbdc97b1ae2531ca093bae6bb441ae15d3238e
      32ac1d91
    • Jakob Unterwurzacher's avatar
      example/loopback: make -allow-other imply default_permissions · c6370440
      Jakob Unterwurzacher authored
      With -allow-other, other users can access the mountpoint.
      However, without default_permissions, nobody checks file
      permissions.
      
      Make loopback behave more like a regular filesystem by
      enabling default_permissions if -allow-other is passed.
      
      Fixes xfstests generic/087:
      
      $ sudo ./check-loopback generic/087
      fuse-xfstests gocryptfs-2018-08-18/67408ac7
      Thu 26 Dec 2019 03:08:54 PM UTC
      
      loopback is /usr/local/bin/loopback
      FSTYP         -- fuse.loopback
      PLATFORM      -- Linux/x86_64 brikett 5.3.12-300.fc31.x86_64
      MKFS_OPTIONS  -- /var/tmp/fuse-xfstests/check-loopback/scratchdev
      MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /var/tmp/fuse-xfstests/check-loopback/scratchdev /var/tmp/fuse-xfstests/check-loopback/scratchdir
      
      generic/087	 1s
      Ran: generic/087
      Passed all 1 tests
      
      Runtime was 1 seconds, exit code 0
      
      Change-Id: Ifa8ac38a3775c3cacf20d37638a95887c9c24f93
      c6370440
    • Jakob Unterwurzacher's avatar
      fuse: implement GETATTR for pollHack · d13ebc4a
      Jakob Unterwurzacher authored
      When the default_permissions option is passed to the kernel,
      it issues a extra GETATTR on the .go-fuse-epoll-hack node.
      
      Rejecting that with EIO makes `syscall.Creat` in `pollHack` fail,
      ultimately erroring out during mount.
      
      $ loopback -allow-other -debug b a
      16:06:39.576856 rx 2: INIT i0 {7.31 Ra 0x20000 NO_OPEN_SUPPORT,PARALLEL_DIROPS,ABORT_ERROR,EXPLICIT_INVAL_DATA,SPLICE_WRITE,READDIRPLUS,ASYNC_DIO,DONT_MASK,NO_OPENDIR_SUPPORT,AUTO_INVAL_DATA,READDIRPLUS_AUTO,POSIX_ACL,HANDLE_KILLPRIV,MAX_PAGES,ATOMIC_O_TRUNC,EXPORT_SUPPORT,SPLICE_MOVE,BIG_WRITES,SPLICE_READ,FLOCK_LOCKS,IOCTL_DIR,WRITEBACK_CACHE,ASYNC_READ,POSIX_LOCKS,CACHE_SYMLINKS}
      16:06:39.576999 tx 2:     OK, {7.28 Ra 0x20000 AUTO_INVAL_DATA,BIG_WRITES,ASYNC_READ,NO_OPEN_SUPPORT,PARALLEL_DIROPS,READDIRPLUS 0/0 Wr 0x10000 Tg 0x0}
      16:06:39.578670 rx 4: GETATTR i1 {Fh 0}
      16:06:39.578717 rx 6: GETATTR i1 {Fh 0}
      16:06:39.578735 tx 4:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.578765 tx 6:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.579028 rx 8: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      16:06:39.579053 tx 8:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579087 rx 10: CREATE i1 {0100100 [WRONLY,TRUNC,CREAT,0x8000] (022)} [".go-fuse-epoll-hack"] 20b
      16:06:39.579113 tx 10:     OK, {i18446744073709551615 g0 {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000} &{18446744073709551615 0 0}}
      16:06:39.579199 rx 14: GETATTR i1 {Fh 0}
      16:06:39.579205 rx 12: GETATTR i18446744073709551615 {Fh 0}
      16:06:39.579216 tx 14:     OK, {tA=1s {M040755 SZ=40 L=2 1026:1026 B0*4096 i0:1 A 1577370827.990394 M 1577370827.990394 C 1577370827.990394}}
      16:06:39.579237 rx 16: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      16:06:39.579242 tx 12:     5=input/output error, {tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579247 tx 16:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      16:06:39.579270 Mount fail: input/output error
      16:06:39.579271 rx 20: LOOKUP i1 [".Trash"] 7b
      
      Change-Id: I20024baf2e8f386b637abbd236b188bfdfa8579f
      d13ebc4a
    • Jakob Unterwurzacher's avatar
      loopback: preserve owner when running as root · 493adfb7
      Jakob Unterwurzacher authored
      If we are running as root have caller information from
      the context, try to set the owner of newly created files
      and directories.
      
      Lays the foundation for fixing xfstests generic/087
      (xfstests always run as root).
      
      Change-Id: Ib8d768153f3ec82ce572021a433a6398560efd44
      493adfb7
    • Jakob Unterwurzacher's avatar
      tests: posixtest: add FstatDeleted · 037efc7d
      Jakob Unterwurzacher authored
      Excercises the fd-finding logic in rawBridge.GetAttr.
      No issues found, tests pass, logic works fine.
      
      Change-Id: I49731b8ec5b41344d58409c8ca615f466bd95f29
      037efc7d
    • Jakob Unterwurzacher's avatar
      loopback: use Lgetxattr and friends · f28fbbf1
      Jakob Unterwurzacher authored
      Stop following symlinks when working with extended
      attributes, and add a test for it.
      
      Problem found by xfstests generic/062.
      
      Change-Id: I67f94451322cdfebdcbcc3af21679ccd4e2800d7
      f28fbbf1
  6. 19 Dec, 2019 2 commits
  7. 11 Dec, 2019 2 commits
  8. 10 Dec, 2019 2 commits
  9. 09 Dec, 2019 3 commits
    • Han-Wen Nienhuys's avatar
      fs: don't enforce dirent ordering. · be93dbf7
      Han-Wen Nienhuys authored
      In TestBridgeReaddirPlusVirtualEntries we test that "." comes first,
      and then "..". This ordering is not reliable. On
      4.19.67-2rodete2-amd64, I see ".." before "."
      
      Adapt the test to accept either ordering.
      
      Change-Id: Ifc003dca6c2b19d9df3045ac30569ee27f78fc52
      be93dbf7
    • Han-Wen Nienhuys's avatar
      internal: skip test with other group if user is only 1 group · d4b28605
      Han-Wen Nienhuys authored
      Change-Id: I59a1f187ff4c88bcefdb1e70c6461ca4b2a79c3b
      d4b28605
    • Jakob Unterwurzacher's avatar
      tests: fix TestReaddirTypeFixup leaking broken mounts · 672815b2
      Jakob Unterwurzacher authored
      After running the test suite, `df` used to report this:
      
        df: /tmp/TestReaddirTypeFixup090721515: Transport endpoint is not connected
      
      To catch problems like this earlier, testMount() now reports failures to umount
      cleanly, and catches the problem with TestReaddirTypeFixup:
      
        $ go test ./fs -run TestReaddirTypeFixup
        [...]
            mem_test.go:37: testMount: Unmount failed: /usr/bin/fusermount: failed to unmount /tmp/TestReaddirTypeFixup175482633: Device or resource busy
      
      TestReaddirTypeFixup was missing ds.Close(), which was also added,
      tests pass now and don't leave broken mounts behind.
      
      Change-Id: Ib5d96c8531b11080cf8da526b917446388537956
      672815b2
  10. 08 Dec, 2019 1 commit
    • Jakob Unterwurzacher's avatar
      tests: add OpenAt test · 6ae4ba6c
      Jakob Unterwurzacher authored
      Tests that openat(2) works as expected when the parent
      directory is renamed (it does).
      
      Originally written to get a FUSE log about what happens
      when userspace calls openat, but it might make sense to
      just keep it as a test.
      
      Change-Id: I3fac815a7c0f713b4f859f6876d60380df9bc953
      6ae4ba6c
  11. 04 Dec, 2019 1 commit
    • Jakob Unterwurzacher's avatar
      tests: make TestTypeChange robust against outside interference · da5ff9a7
      Jakob Unterwurzacher authored
      I have suddenly seen TestTypeChange hang forever, waiting for the
      FORGET that never arrives. The first concern was that the assumption
      that the FORGET always follows the delete was wrong. Checking the
      kernel source code, the FORGET should really, actually always follow
      the delete.
      
      Looking more closely on the debug trace showed this:
      
              LOOKUP i1 [".Trash-1026"] 12b
                   OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      
      This is the desktop environment looking for the trash folder.
      And because the test is stupid, it returned the same dummy
      inode i1234. This means the kernel had one extra reference
      to this inode and, of course, did not send a FORGET when
      "file" was deleted.
      
      I made testTypeChangeIno only react to expected file names
      now, and added a log output should we wait more than ~5
      seconds for the FORGET. The log message then repeats indefinitely
      every 5 seconds.
      
              ~/go/src/github.com/hanwen/go-fuse/fs$ go test -v -run TestTypeChange
              === RUN   TestTypeChange
              20:50:18.491584 rx 2: INIT i0 {7.31 Ra 0x20000 CACHE_SYMLINKS,POSIX_LOCKS,SPLICE_MOVE,SPLICE_READ,READDIRPLUS,READDIRPLUS_AUTO,PARALLEL_DIROPS,ABORT_ERROR,EXPLICIT_INVAL_DATA,ASYNC_READ,EXPORT_SUPPORT,DONT_MASK,NO_OPENDIR_SUPPORT,SPLICE_WRITE,FLOCK_LOCKS,AUTO_INVAL_DATA,HANDLE_KILLPRIV,MAX_PAGES,POSIX_ACL,ATOMIC_O_TRUNC,BIG_WRITES,IOCTL_DIR,ASYNC_DIO,WRITEBACK_CACHE,NO_OPEN_SUPPORT}
              20:50:18.491728 tx 2:     OK, {7.28 Ra 0x20000 NO_OPEN_SUPPORT,BIG_WRITES,PARALLEL_DIROPS,READDIRPLUS,ASYNC_READ,AUTO_INVAL_DATA 0/0 Wr 0x10000 Tg 0x0}
              20:50:18.491857 rx 4: ACCESS i1 {u=1026 g=1026 r}
              20:50:18.491873 tx 4:     OK
              20:50:18.492028 rx 6: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
              20:50:18.492110 tx 6:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492147 rx 8: CREATE i1 {0100100 [TRUNC,WRONLY,CREAT,0x8000] (00)} [".go-fuse-epoll-hack"] 20b
              20:50:18.492207 tx 8:     OK, {i18446744073709551615 g0 {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000} &{18446744073709551615 0 0}}
              20:50:18.492291 rx 10: LOOKUP i1 [".Trash"] 7b
              20:50:18.492340 tx 10:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492354 rx 12: POLL i18446744073709551615
              20:50:18.492359 tx 12:     38=function not implemented
              20:50:18.492370 rx 14: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
              20:50:18.492378 tx 14:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492389 rx 16: GETATTR i1234 {Fh 0}
              20:50:18.492398 tx 16:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492412 rx 18: FLUSH i18446744073709551615 {Fh 18446744073709551615}
              20:50:18.492417 tx 18:     5=input/output error
              20:50:18.492427 rx 20: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
              20:50:18.492435 tx 20:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492450 rx 22: RELEASE i18446744073709551615 {Fh 18446744073709551615 WRONLY,0x8000  L0}
              20:50:18.492456 tx 22:     5=input/output error
              20:50:18.492469 rx 24: LOOKUP i1 ["file"] 5b
              20:50:18.492477 tx 24:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492489 rx 26: LOOKUP i1 [".Trash-1026"] 12b
              20:50:18.492497 tx 26:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492506 rx 28: GETATTR i1234 {Fh 0}
              20:50:18.492513 tx 28:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492521 rx 30: GETATTR i1234 {Fh 0}
              20:50:18.492528 tx 30:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492537 rx 32: LOOKUP i1 ["file"] 5b
              20:50:18.492545 tx 32:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
              20:50:18.492555 rx 34: FORGET i18446744073709551615 {Nlookup=1}
              20:50:18.492575 rx 36: UNLINK i1 ["file"] 5b
              20:50:18.492588 tx 36:     OK
              20:50:18.492621 rx 38: LOOKUP i1 ["dir"] 4b
              [hangs here]
      
      Change-Id: Ibee48faf09c5a8993dfc7bb75251551f699cd63e
      da5ff9a7
  12. 03 Dec, 2019 2 commits
    • Han-Wen Nienhuys's avatar
      fs: introduce ServerCallbacks · 0ff7c8ac
      Han-Wen Nienhuys authored
      Options.ServerCallbacks can set the callbacks used when not mounting
      the filesystem.
      
      Change-Id: I51bf875a816c23e587ec8b575110221bdfcdaffe
      0ff7c8ac
    • Han-Wen Nienhuys's avatar
      fs: use syscall.Rename rather than os.Rename · 1e5729e0
      Han-Wen Nienhuys authored
      This provides more fidelity for reproducing file system behavior. Case
      in point, it lets us rename a directory on top of an existing
      directory
      
      Fixes #55.
      
      Change-Id: I82572aae32ee225b1629303c448a09f1066eb4ed
      1e5729e0
  13. 26 Nov, 2019 2 commits
  14. 25 Nov, 2019 6 commits
    • Jakob Unterwurzacher's avatar
      fs: wait out apparent inode type change · 7090b027
      Jakob Unterwurzacher authored
      When a inode is deleted and recreated, the inode number may
      be reused, and it may appear that it has changed type.
      
      A simple solution is to wait for the FORGET that *always*
      follows the deletion. This is what this patch does.
      
      Makes fsstress happy (tested with fsstress-loopback.bash from
      gocryptfs, which is similar to xfstests generic/013).
      
      Also fixes the test that was introduced in the last commit
      (TestTypeChange).
      
      fsstress-loopback.bash before:
      
          ~/go/src/github.com/rfjakob/gocryptfs/tests/stress_tests$ ./fsstress-loopback.bash
          Recompile go-fuse loopback
          Waiting for mount:  ok
          Starting fsstress loop
          1
              fsstress.1 seed = 1572931176
              fsstress.2 seed = 1572919573
              fsstress.3 seed = 1573005097
              rm
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.2/p4/d1XXXX': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.2/p6/d0/d44XXXXXXXXXXXX/d4cXXXX': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p1/d11/d124/d12f/d154/d15a/d143/df1/d100/d135/d123/d3a': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p1/d11/f22': Not a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p3/d0/d7e/d4b/d102/d157/d134/d12f/d14b/d17a/d130': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p3/d0/d7e/d4b/d102/d157/d158/df2/ff5': Not a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p3/d0/d1c': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p0/d0/d4': Is a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p2/d6/dfd/da6/df2/d124/dea/d100/d120/d131/d13e/d13b/d149/d13a/f9c': Not a directory
          rm: cannot remove '/var/tmp/fsstress-loopback.bash.qgK.mnt/fsstress.3/p2/d6/dfd/da6/df2/d124/dea/dd5/dfa/d127/d129/d14a/d117/d14c/d82': Is a directory
          ./fsstress-loopback.bash: line 1: kill: %1: no such job
      
      After (with typechangeDebug = true):
      
          $ ./fsstress-loopback.bash
          Recompile go-fuse loopback
          Waiting for mount:  ok
          Starting fsstress loop
          1
              fsstress.1 seed = 1574198874
              fsstress.2 seed = 1574016238
          17:51:25.770396 Ino:13634604 typechange 0x8000 -> 0xa000, sleep=0s
          17:51:25.890382 Ino:13899580 typechange 0xa000 -> 0x8000, sleep=0s
          17:51:25.890400 Ino:13899580 typechange 0xa000 -> 0x8000, sleep=1µs
          17:51:25.890412 Ino:13899580 typechange 0xa000 -> 0x8000, sleep=1.41µs
              fsstress.3 seed = 1574847533
          17:51:27.472057 Ino:14158985 typechange 0x4000 -> 0x8000, sleep=0s
          17:51:27.472070 Ino:14158985 typechange 0x4000 -> 0x8000, sleep=1µs
          17:51:27.472304 Ino:14158985 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:27.478628 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:27.478640 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=1µs
          17:51:27.478700 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=1.051µs
          17:51:27.478747 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=1.817µs
          17:51:27.478796 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=3.18µs
          17:51:27.478840 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=5.718µs
          17:51:27.478884 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=8.396µs
          17:51:27.478976 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=10.784µs
          17:51:27.479024 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=13.017µs
          17:51:27.479067 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=16.065µs
          17:51:27.479108 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=19.767µs
          17:51:27.479145 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=27.427µs
          17:51:27.479240 Ino:14159000 typechange 0x8000 -> 0x4000, sleep=36.272µs
          17:51:27.500756 Ino:14291431 typechange 0x4000 -> 0x8000, sleep=0s
          17:51:27.500770 Ino:14291431 typechange 0x4000 -> 0x8000, sleep=1µs
          17:51:27.500779 Ino:14291431 typechange 0x4000 -> 0x8000, sleep=1.873µs
          17:51:27.521194 Ino:14291506 typechange 0x4000 -> 0x8000, sleep=0s
          17:51:27.521207 Ino:14291506 typechange 0x4000 -> 0x8000, sleep=1µs
              rm
          2
              fsstress.1 seed = 1574600636
              fsstress.2 seed = 1574203490
          17:51:29.009666 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=0s
          17:51:29.010068 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=1µs
          17:51:29.010078 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=1.79µs
          17:51:29.010179 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=3.331µs
          17:51:29.010255 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=4.537µs
          17:51:29.010364 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=8.354µs
          17:51:29.010416 Ino:12209562 typechange 0x4000 -> 0xa000, sleep=8.897µs
              fsstress.3 seed = 1573917850
          17:51:33.623359 Ino:14291439 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:33.623378 Ino:14291439 typechange 0x8000 -> 0x4000, sleep=1µs
          17:51:33.623392 Ino:14291439 typechange 0x8000 -> 0x4000, sleep=1.378µs
          17:51:33.644130 Ino:14291494 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:33.644151 Ino:14291494 typechange 0x8000 -> 0x4000, sleep=1µs
          17:51:33.644159 Ino:14291494 typechange 0x8000 -> 0x4000, sleep=1.072µs
          17:51:33.654370 Ino:14291501 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:33.654384 Ino:14291501 typechange 0x8000 -> 0x4000, sleep=1µs
          17:51:33.654393 Ino:14291501 typechange 0x8000 -> 0x4000, sleep=1.588µs
          17:51:33.654399 Ino:14291501 typechange 0x8000 -> 0x4000, sleep=2.639µs
          17:51:33.655098 Ino:14291476 typechange 0x4000 -> 0x8000, sleep=0s
          17:51:33.667216 Ino:14291504 typechange 0x4000 -> 0x8000, sleep=0s
          17:51:33.667228 Ino:14291504 typechange 0x4000 -> 0x8000, sleep=1µs
          17:51:33.667236 Ino:14291504 typechange 0x4000 -> 0x8000, sleep=1.443µs
          17:51:33.667242 Ino:14291504 typechange 0x4000 -> 0x8000, sleep=2.286µs
              rm
          3
              fsstress.1 seed = 1574749722
              fsstress.2 seed = 1574328972
          17:51:35.220180 Ino:13771167 typechange 0x8000 -> 0xa000, sleep=0s
          17:51:35.220205 Ino:13771167 typechange 0x8000 -> 0xa000, sleep=1µs
          17:51:35.514144 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=0s
          17:51:35.514165 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=1µs
          17:51:35.514172 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=1.739µs
          17:51:35.515875 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=1.969µs
          17:51:35.516021 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=3.744µs
          17:51:35.516094 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=4.308µs
          17:51:35.516514 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=5.112µs
          17:51:35.516765 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=5.299µs
          17:51:35.516870 Ino:13771355 typechange 0x4000 -> 0xa000, sleep=8.169µs
          17:51:36.429950 Ino:13771381 typechange 0x8000 -> 0xa000, sleep=0s
          17:51:36.430068 Ino:13771381 typechange 0x8000 -> 0xa000, sleep=1µs
          17:51:36.430082 Ino:13771381 typechange 0x8000 -> 0xa000, sleep=1.071µs
          17:51:36.547108 Ino:13899567 typechange 0x8000 -> 0x4000, sleep=0s
          17:51:36.547121 Ino:13899567 typechange 0x8000 -> 0x4000, sleep=1µs
          17:51:36.547158 Ino:13899567 typechange 0x8000 -> 0x4000, sleep=1.907µs
          17:51:36.547181 Ino:13899567 typechange 0x8000 -> 0x4000, sleep=2.358µs
          17:51:38.112588 Ino:13771503 typechange 0x8000 -> 0xa000, sleep=0s
          17:51:38.112602 Ino:13771503 typechange 0x8000 -> 0xa000, sleep=1µs
          17:51:38.112611 Ino:13771503 typechange 0x8000 -> 0xa000, sleep=1.11µs
          17:51:38.524020 Ino:13899675 typechange 0xa000 -> 0x8000, sleep=0s
          17:51:38.524042 Ino:13899675 typechange 0xa000 -> 0x8000, sleep=1µs
      
      Change-Id: I6120ce76730018d22898127290f492fa9f85a957
      7090b027
    • Jakob Unterwurzacher's avatar
      fs: add TestTypeChange (simulate inode number reuse) · 53f16b0a
      Jakob Unterwurzacher authored
      TestTypeChange simulates inode number reuse that happens on real
      filesystems. For go-fuse, inode number reuse can look like a file changing
      to a directory or vice versa. Acutally, the old inode does not exist anymore,
      we just have not received the FORGET yet.
      
      The test fails reliably with iteration count 100. Will be fixed in the
      next commit.
      
      Test failure looks like this:
      
      	~/go/src/github.com/hanwen/go-fuse/fs$ go test -run TestTypeChange -v
      	=== RUN   TestTypeChange
      	21:43:36.065494 rx 2: INIT i0 {7.31 Ra 0x20000 MAX_PAGES,SPLICE_READ,FLOCK_LOCKS,READDIRPLUS,READDIRPLUS_AUTO,ASYNC_DIO,WRITEBACK_CACHE,HANDLE_KILLPRIV,SPLICE_WRITE,PARALLEL_DIROPS,ABORT_ERROR,EXPLICIT_INVAL_DATA,NO_OPENDIR_SUPPORT,ASYNC_READ,BIG_WRITES,DONT_MASK,IOCTL_DIR,AUTO_INVAL_DATA,POSIX_ACL,CACHE_SYMLINKS,POSIX_LOCKS,ATOMIC_O_TRUNC,EXPORT_SUPPORT,SPLICE_MOVE,NO_OPEN_SUPPORT}
      	21:43:36.065573 tx 2:     OK, {7.28 Ra 0x20000 PARALLEL_DIROPS,AUTO_INVAL_DATA,ASYNC_READ,BIG_WRITES,NO_OPEN_SUPPORT,READDIRPLUS 0/0 Wr 0x10000 Tg 0x0}
      	21:43:36.065742 rx 4: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      	21:43:36.065798 tx 4:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.065837 rx 6: CREATE i1 {0100100 [WRONLY,CREAT,TRUNC,0x8000] (00)} [".go-fuse-epoll-hack"] 20b
      	21:43:36.065861 tx 6:     OK, {i18446744073709551615 g0 {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000} &{18446744073709551615 0 0}}
      	21:43:36.065945 rx 8: POLL i18446744073709551615
      	21:43:36.065954 tx 8:     38=function not implemented
      	21:43:36.066009 rx 10: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      	21:43:36.066046 tx 10:     5=input/output error
      	21:43:36.066065 rx 12: RELEASE i18446744073709551615 {Fh 18446744073709551615 WRONLY,0x8000  L0}
      	21:43:36.066071 tx 12:     5=input/output error
      	21:43:36.066107 rx 14: LOOKUP i1 ["file"] 5b
      	21:43:36.066181 tx 14:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066217 rx 16: GETATTR i1234 {Fh 0}
      	21:43:36.066240 tx 16:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066264 rx 18: LOOKUP i1 ["file"] 5b
      	21:43:36.066285 tx 18:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066301 rx 20: UNLINK i1 ["file"] 5b
      	21:43:36.066321 tx 20:     OK
      	21:43:36.066361 rx 24: LOOKUP i1 ["dir"] 4b
      	21:43:36.066371 tx 24:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066383 rx 22: FORGET i1234 {Nlookup=2}
      	21:43:36.066390 rx 26: GETATTR i1234 {Fh 0}
      	21:43:36.068259 tx 26:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.069664 received ENODEV (unmount request), thread exiting
      	21:43:36.073309 received ENODEV (unmount request), thread exiting
      	21:43:36.073396 received ENODEV (unmount request), thread exiting
      	--- FAIL: TestTypeChange (0.01s)
      	    bridge_test.go:96: should be a dir now
      	FAIL
      	exit status 1
      	FAIL	github.com/hanwen/go-fuse/v2/fs	0.017s
      
      Change-Id: I7a1a10303ecc0a8c008d29d5d81079868fc2cc43
      53f16b0a
    • Jakob Unterwurzacher's avatar
      fs: addNewChild(): handle concurrent FORGETs · 68f70527
      Jakob Unterwurzacher authored
      As seen in debug logs like shown below, concurrent
      FORGETs and inode number reuse could make use give
      an Inode to the kernel that has already been deleted
      from the nodes[] map.
      
      With this change, I can no langer trigger a panic
      in xfstests generic/007.
      
      Fixes https://github.com/hanwen/go-fuse/issues/324
      
      Nodes that appear to change type are not handled yet
      (actually caused by inode number reuse), but this is
      a separate problem.
      
      Debug log:
      
        17:09:07.651735 rx 84444: CREATE i13632136 {0100644 [WRONLY,TRUNC,CREAT,0x8000] (022)} ["XXXXXXXXXXXX.38"] 16b
        17:09:07.651790 tx 84444:     OK, {i13632483 g1 {M0100644 SZ=0 L=1 0:0 B0*4096 i0:13632483 A 1572192547.650470 M 1572192547.650470 C 1572192547.650470} &{7 0 0}}
        [...]
        17:09:07.684439 rx 87988: CREATE i13632119 {0100644 [CREAT,TRUNC,WRONLY,0x8000] (022)} ["XXXXXXXXXXXX.37"] 16b
        [...]
        17:09:07.684472 rx 87992: FORGET i13632483 {Nlookup=1}
        17:09:07.684483 tx 87988:     OK, {i13632483 g1 {M0100644 SZ=0 L=1 0:0 B0*4096 i0:13632483 A 1572192547.683470 M 1572192547.683470 C 1572192547.683470} &{8 0 0}}
        [...]
        17:09:07.684557 rx 88006: FLUSH i13632483 {Fh 8}
        17:09:07.684560 unknown node 13632483
        17:09:07.684564 tx 88002:     OK,  "XXXXXXXXXXXX.830"
        panic: unknown node 13632483
      
      Change-Id: Ibb1a7a6a459d2e4133a4e512e0816e8ed950a8d7
      68f70527
    • Jakob Unterwurzacher's avatar
      example: loopback: support -q (quiet) · 713e99eb
      Jakob Unterwurzacher authored
      Getting rid of the "Mounted!" message makes it easier
      to integrate into xfstests, which barks at all and any
      unexpected output.
      
      Used by https://github.com/rfjakob/fuse-xfstests/commit/ab1d77ccdb8cf16f79cb312bcb3aaadb23660807
      
      Change-Id: Idff314474b4980ef6f6fc2460b338a840bb0e409
      713e99eb
    • Jakob Unterwurzacher's avatar
      fuse: replace DirEntryList.FixMode() pointer arithmetic · be5a8417
      Jakob Unterwurzacher authored
      As pointed out by @slackner at
      https://github.com/hanwen/go-fuse/commit/8581b7e950b2d520370eba8997ddd9493192e9c7#r35845728
      FixMode() failed to take into account the length
      of the file name and the padding.
      
      To fix this without adding a lot of additional
      pointer arithmetic, add the "lastDirent" field
      to DirEntryList. This stores the last serialized
      _Dirent, so _Dirent.Typ can be accessed easily
      by FixMode().
      
      Change-Id: I6ca3fe551e1a072779b1d5ed5293add057e55287
      be5a8417
    • Han-Wen Nienhuys's avatar
      fs: check outcome of mode fixup result · c9fd0958
      Han-Wen Nienhuys authored
      Add TestReaddirTypeFixup(). This test currently fails
      because FixMode is broken. Will be fixed in a later commit.
      
      Change-Id: Ie4eab9c5817ddb772b73aa22e8f75f5d4f2ad49c
      c9fd0958
  15. 22 Nov, 2019 1 commit