1. 01 Feb, 2021 1 commit
    • Tommy Lindgren's avatar
      fuse: increase max readers for better throughput · 0f728ba1
      Tommy Lindgren authored
      Previously maximum number of requests readers was hard-coded to 2.
      Setting it to max(2, min(16, GOMAXPROCS)) improves the throughput on
      systems with more cores available. Going beyond 16 readers seems to hurt
      performance even if there are more cores available.
      
      Benchmark GoFuseRead can be used for demonstrating the effects of this
      variable. The benchmark reads 32 streams in parallel from a dummy
      filesystem (read operations immediately return zeros). Example results
      from an i7-8550U (8 cores):
      
      | Max readers | Total throughput |
      | ----------: | ---------------: |
      |           2 |       13217 MB/s |
      |           4 |       19202 MB/s |
      |           8 |       19973 MB/s |
      |          16 |       18994 MB/s |
      
      On a 96 core system:
      
      | Max readers | Total throughput |
      | ----------: | ---------------: |
      |           2 |       11490 MB/s |
      |           4 |       16129 MB/s |
      |           8 |       24263 MB/s |
      |          16 |       29568 MB/s |
      |          32 |       28262 MB/s |
      
      Note that improvements won't be as dramatic for real filesytem
      implementations. In benchmarks for a filesystem doing real work I see a
      30-40% improvement (8.3 -> 11.4 GB/s) on the 96 core system.
      
      Also tweaked some of the other benchmarks so they don't leave behind
      mountpoints.
      
      Fixes #388.
      
      Change-Id: Ibff17d7fc92195f078a9ccff818a31f3a58873f2
      0f728ba1
  2. 25 Jan, 2021 1 commit
  3. 21 Jan, 2021 1 commit
    • Han-Wen Nienhuys's avatar
      fs: allow extending LoopbackNode · e29d3106
      Han-Wen Nienhuys authored
      Make LoopbackNode and LoopbackRoot public, and add an example of how
      to extend them.
      
      Fixes #364.
      
      Change-Id: Idcce1bf3765e93ece3bcf9d14de8a957b8c6b63a
      e29d3106
  4. 04 Jan, 2021 1 commit
  5. 31 Dec, 2020 1 commit
  6. 27 Dec, 2020 4 commits
  7. 09 Dec, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: fix pollHack response to SETATTR · 9473722e
      Han-Wen Nienhuys authored
      On OSX, syscall.O_TRUNC on the Create call apparently triggers a
      SETATTR call with size 0, which must be responded to correctly.
      
      Change-Id: Icb99891eae99ced6ffbd5fb1dadf42ffad687534
      9473722e
  8. 08 Dec, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: try to fix mount errors on OSX · 4a458845
      Han-Wen Nienhuys authored
      Some of the error codes due to the poll hack get returned back to the
      OSX mount process, which leads to failing mounts.
      
      Change-Id: I5b10509c3b8173343caf3f67c0e12a25ddfae32b
      4a458845
  9. 01 Dec, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fs: redeclare syscall.Dirent locally · 588c83d1
      Han-Wen Nienhuys authored
      syscall.Dirent declares a 256-byte "name" field. In reality, this
      field is only as long as necessary to transmit the name, so more
      entries can be packed in a page.
      
      When casting the result to syscall.Dirent, the result might run over
      the end of the byte buffer, which the race detector doesn't like.
      
      Address this by using our own flavor of syscall.Dirent. All linux
      flavors use the same structure layout, except for the amount of
      padding at the end.
      
      Change-Id: I618f7ef9e167894e1ab25b2113dcd20d7b76ceb4
      588c83d1
  10. 04 Nov, 2020 5 commits
    • Jakob Unterwurzacher's avatar
      fuse: debug logging: mark nodeids as "nXXX" · be8e5f4a
      Jakob Unterwurzacher authored
      nodeid and inode number used to be identical, which is
      why both could be marked as "iXXX". This is no longer the
      case, so make nodeids distinguishable.
      
      Example:
      
      23:47:17.637736 rx 9156: MKDIR n28 {0777 (00)} ["d90X"] 5b
      23:47:17.637801 tx 9156:     OK, {n112 g1 tE=1s tA=1s {M040755 SZ=4096 L=2 1026:1026 B8*4096 i0:2769795 A 1601761637.637385 M 1601761637.637385 C 1601761637.637385}}
      
      v4: also update orphaned inode warning
      
      Change-Id: Ib29dabfd4cf6692cf4118191f8dc73529562426f
      be8e5f4a
    • Jakob Unterwurzacher's avatar
      fs: split inode number tracking and kernel nodeid tracking · 1d0096e5
      Jakob Unterwurzacher authored
      Using the inode numbers as the nodeid causes problems
      when the fs reuses inode numbers. This is the case with
      any overlay filesystem that is backed by ext4 like the
      loopback example or gocryptfs.
      
      We already had the expSleep() and re-add hack,
      
      	7090b027 fs: wait out apparent inode type change
      	68f70527 fs: addNewChild(): handle concurrent FORGETs
      
      to mitigate some of the problems (at the risk of deadlocking
      forever), but I could not find a way to work around the following
      case uncovered by fsstress:
      
      The kernel expects a fresh nodeid from MKDIR (see #372 for a
      debug log).
      
      This is now guaranteed by passing the O_EXCL to addNewChild().
      However, this also means that the hard link detection must
      happen in addNewChild() as opposed to newInodeUnlocked()
      before. The expSleep and re-add hacks are no longer needed
      and have been dropped.
      
      This survived 24 hours (42587 iterations) of fsstress testing.
      Tested was the loopback example running on top of ext4 on
      Linux 5.8.10.
      
      Fixes https://github.com/hanwen/go-fuse/issues/372 .
      
      v2: Rename inoMap -> stableAttrs, nodeidMap -> kernelNodeIds
          acc. to feedback from Han-Wen.
      v3: initialize Inode.nodeId early
      v4: drop leftover file stash.txt
      
      Change-Id: Ibb36a886f15d48727daa10b9717ea88e45a6b8af
      1d0096e5
    • Jakob Unterwurzacher's avatar
      fs: TestFsstress: add rename, link ; reduce runtime to 1 sec · 6a785da2
      Jakob Unterwurzacher authored
      With rename and link, it hangs within 20ms, so we can reduce the
      runtime.
      
      Also add deadlock detection & statistics printout.
      
      A good run looks like this:
      
      	$ TMPDIR=/var/tmp go test -run TestFsstress -v
      	=== RUN   TestFsstress
      	    simple_test.go:601: Operation statistics:
      	    simple_test.go:604:      creat:    33 ok, 132978 fail,  0 hung
      	    simple_test.go:604:       link:    85 ok,  37196 fail,  0 hung
      	    simple_test.go:604:      mkdir:    60 ok, 155273 fail,  0 hung
      	    simple_test.go:604:     mkfifo:    45 ok, 161937 fail,  0 hung
      	    simple_test.go:604: mknod_fifo:    55 ok, 154459 fail,  0 hung
      	    simple_test.go:604:  mknod_reg:    77 ok, 180614 fail,  0 hung
      	    simple_test.go:604: mknod_sock:    83 ok, 139479 fail,  0 hung
      	    simple_test.go:604:    readdir:    13 ok,      0 fail,  0 hung
      	    simple_test.go:604:     remove:   180 ok,     40 fail,  0 hung
      	    simple_test.go:604:     rename:   646 ok,    137 fail,  0 hung
      	    simple_test.go:604:      rmdir:    48 ok,  48342 fail,  0 hung
      	    simple_test.go:604:    symlink:    30 ok, 119954 fail,  0 hung
      	    simple_test.go:604:     unlink:   150 ok,   6121 fail,  0 hung
      
      	--- PASS: TestFsstress (1.02s)
      	PASS
      	ok  	github.com/hanwen/go-fuse/v2/fs	1.022s
      
      A bad run like this:
      
      	go-fuse/fs$ TMPDIR=/var/tmp go test -run TestFsstress -v
      	=== RUN   TestFsstress
      	    simple_test.go:592: timeout waiting for goroutines to exit (deadlocked?)
      	    simple_test.go:601: Operation statistics:
      	    simple_test.go:604:      creat:     0 ok,     44 fail, 10 hung
      	    simple_test.go:604:       link:     0 ok,      6 fail, 10 hung
      	    simple_test.go:604:      mkdir:     1 ok,    283 fail, 10 hung
      	    simple_test.go:604:     mkfifo:     1 ok,    140 fail, 10 hung
      	    simple_test.go:604: mknod_fifo:     1 ok,   1246 fail, 10 hung
      	    simple_test.go:604:  mknod_reg:     0 ok,     19 fail, 10 hung
      	    simple_test.go:604: mknod_sock:     0 ok,    110 fail, 10 hung
      	    simple_test.go:604:    readdir:     1 ok,      0 fail,  1 hung
      	    simple_test.go:604:     remove:     1 ok,      0 fail, 10 hung
      	    simple_test.go:604:     rename:     0 ok,     12 fail, 10 hung
      	    simple_test.go:604:      rmdir:     1 ok,    346 fail, 10 hung
      	    simple_test.go:604:    symlink:     1 ok,   1456 fail, 10 hung
      	    simple_test.go:604:     unlink:     2 ok,     15 fail, 10 hung
      	    simple_test.go:51: /usr/bin/fusermount: failed to unmount /var/tmp/TestFsstress302646914/mnt: Device or resource busy
      	         (code exit status 1)
      
      	--- FAIL: TestFsstress (2.30s)
      	FAIL
      
      	[hangs]
      
      Change-Id: I9eb0ce02eb1df766be257d6a87de53c6ffa6d21e
      6a785da2
    • Jakob Unterwurzacher's avatar
      fs: add TestFsstress · e5a9167d
      Jakob Unterwurzacher authored
      TestFsstress is loosely modeled after xfstest's fsstress. It performs rapid
      parallel removes / creates / readdirs. Coupled with inode reuse, this test
      used to deadlock go-fuse quite quickly.
      
      Change-Id: Ia3af099e08c74d3c160062c1ed414430b6c1e7c9
      e5a9167d
    • Jakob Unterwurzacher's avatar
      fs: detect lookupCount underflow · e9983956
      Jakob Unterwurzacher authored
      If this happens, it means our node accounting is messed up.
      Abort the process by panicing instead of wrapping around
      to UINT_MAX.
      
      Change-Id: Id662ab03877d44a16785bfd164a96173a3b7519f
      e9983956
  11. 07 Oct, 2020 3 commits
    • Kirill Smelkov's avatar
      nodefs: Skip TestNodeParallelLookup on kernels that don't have CAP_PARALLEL_DIROPS · 3a8e4783
      Kirill Smelkov authored
      If kernel internally serializes lookup requests for a directory, the
      test will deadlock, becaue it tries to issue two such request and pause
      lookup handlers, untill both requests come in.
      
      For example here is how this test hangs on Ubuntu Xenial 16.04:
      
      kirr@kubu:~/t/go-fuse$ go test -count=1 -v -run TestNodeParallelLookup ./fuse/test/
      === RUN   TestNodeParallelLookup
      18:24:38.494651 rx 1: INIT i0 {7.23 Ra 0x20000 POSIX_LOCKS,ATOMIC_O_TRUNC,EXPORT_SUPPORT,DONT_MASK,FLOCK_LOCKS,READDIRPLUS_AUTO,ASYNC_DIO,WRITEBACK_CACHE,SPLICE_WRITE,SPLICE_READ,IOCTL_DIR,AUTO_INVAL_DATA,READDIRPLUS,NO_OPEN_SUPPORT,ASYNC_READ,BIG_WRITES,SPLICE_MOVE}
      18:24:38.494713 tx 1:     OK, {7.23 Ra 0x20000 ASYNC_READ,BIG_WRITES,AUTO_INVAL_DATA,READDIRPLUS,NO_OPEN_SUPPORT 0/0 Wr 0x10000 Tg 0x0}
      18:24:38.495351 rx 2: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      18:24:38.495467 tx 2:     OK, {i18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      18:24:38.495530 rx 3: OPEN i18446744073709551615 {O_RDONLY,0x8000}
      18:24:38.495575 tx 3:     OK, {Fh 18446744073709551615 }
      18:24:38.495624 rx 4: POLL i18446744073709551615
      18:24:38.495659 tx 4:     38=function not implemented
      18:24:38.495706 rx 5: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      18:24:38.495741 tx 5:     34=numerical result out of range
      18:24:38.495812 rx 6: RELEASE i18446744073709551615 {Fh 18446744073709551615 0x8000  L0}
      18:24:38.495849 tx 6:     34=numerical result out of range
      18:24:38.495904 rx 7: LOOKUP i1 ["world"] 6b
      18:24:38.495957 I: <- lookup "world"
      < hang >
      
      Amends: 17c0c400 (nodefs: Allow for several Lookup requests to be served simultaneously)
      Updates: https://github.com/hanwen/go-fuse/issues/261
      
      Change-Id: I5d0a76d1190af805351326c378f69cf8ff781a37
      3a8e4783
    • Kirill Smelkov's avatar
      Travis CI: Exercise tests both GOMAXPROCS=`nproc` and GOMAXPROCS=1 · ffb96eff
      Kirill Smelkov authored
      GOMAXPROCS=1 currently triggers tests hang:
      https://github.com/hanwen/go-fuse/issues/261
      
      Change-Id: I3dc98c8bbdaac7a4ce86639e405ba1a1cef4c295
      ffb96eff
    • Kirill Smelkov's avatar
      Travis CI += Go 1.14, Go1.15 · bf11ce7c
      Kirill Smelkov authored
      Change-Id: I8291dceac66bd02bc8fbb845d1f125c4cf22f48e
      bf11ce7c
  12. 08 Sep, 2020 1 commit
    • Jakob Unterwurzacher's avatar
      pollHack: handle GETATTR · 0b6cbc51
      Jakob Unterwurzacher authored
      When a filesystem is mounted with the default_permissions mount
      option (which you want to do when other users can access the
      mount), the kernel issues extra GETATTR calls to determine access
      rights. The GETATTR was not handled and pollHack failed with
      "numerical result out of range", preventing the mount from succeeding.
      
      Handle GETATTR so mounting with default_permissions works again.
      
      Before:
      
      	$ gocryptfs -fusedebug -extpass "echo test" -ko default_permissions a b
      	Reading password from extpass program "echo", arguments: ["test"]
      	Decrypting master key
      	2020/09/08 19:09:21 rx 2: INIT i0 {7.31 Ra 0x20000 ABORT_ERROR,MAX_PAGES,ATOMIC_O_TRUNC,EXPORT_SUPPORT,READDIRPLUS_AUTO,ASYNC_DIO,NO_OPEN_SUPPORT,PARALLEL_DIROPS,ASYNC_READ,BIG_WRITES,SPLICE_MOVE,WRITEBACK_CACHE,HANDLE_KILLPRIV,NO_OPENDIR_SUPPORT,CACHE_SYMLINKS,EXPLICIT_INVAL_DATA,POSIX_LOCKS,SPLICE_WRITE,SPLICE_READ,IOCTL_DIR,AUTO_INVAL_DATA,READDIRPLUS,DONT_MASK,FLOCK_LOCKS,POSIX_ACL}
      	2020/09/08 19:09:21 tx 2:     OK, {7.28 Ra 0x20000 AUTO_INVAL_DATA,READDIRPLUS,NO_OPEN_SUPPORT,PARALLEL_DIROPS,ASYNC_READ,BIG_WRITES 0/0 Wr 0x20000 Tg 0x0}
      	2020/09/08 19:09:21 rx 4: GETATTR i1 {Fh 0}
      	2020/09/08 19:09:21 tx 4:     OK, {tA=1s {M040755 SZ=80 L=2 1026:1026 B0*4096 i0:1 A 1599584934.254242 M 1599584467.201817 C 1599584467.201817}}
      	2020/09/08 19:09:21 rx 6: GETATTR i1 {Fh 0}
      	2020/09/08 19:09:21 tx 6:     OK, {tA=1s {M040755 SZ=80 L=2 1026:1026 B0*4096 i0:1 A 1599584934.254242 M 1599584467.201817 C 1599584467.201817}}
      	2020/09/08 19:09:21 rx 8: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      	2020/09/08 19:09:21 tx 8:     OK, {i18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      	2020/09/08 19:09:21 rx 10: LOOKUP i1 [".Trash"] 7b
      	2020/09/08 19:09:21 tx 10:     2=no such file or directory, {i0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      	2020/09/08 19:09:21 rx 12: GETATTR i18446744073709551615 {Fh 0}
      	2020/09/08 19:09:21 tx 12:     34=numerical result out of range, {tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      	fs.Mount failed: numerical result out of range
      
      After:
      
      	$ gocryptfs -fusedebug -extpass "echo test" -ko default_permissions a b
      	Reading password from extpass program "echo", arguments: ["test"]
      	Decrypting master key
      	2020/09/08 19:21:33 rx 2: INIT i0 {7.31 Ra 0x20000 READDIRPLUS,POSIX_ACL,CACHE_SYMLINKS,SPLICE_WRITE,IOCTL_DIR,AUTO_INVAL_DATA,FLOCK_LOCKS,ASYNC_DIO,HANDLE_KILLPRIV,ABORT_ERROR,ATOMIC_O_TRUNC,BIG_WRITES,DONT_MASK,SPLICE_MOVE,PARALLEL_DIROPS,EXPORT_SUPPORT,SPLICE_READ,READDIRPLUS_AUTO,WRITEBACK_CACHE,NO_OPEN_SUPPORT,ASYNC_READ,POSIX_LOCKS,MAX_PAGES,NO_OPENDIR_SUPPORT,EXPLICIT_INVAL_DATA}
      	2020/09/08 19:21:33 tx 2:     OK, {7.28 Ra 0x20000 NO_OPEN_SUPPORT,ASYNC_READ,AUTO_INVAL_DATA,READDIRPLUS,BIG_WRITES,PARALLEL_DIROPS 0/0 Wr 0x20000 Tg 0x0}
      	2020/09/08 19:21:33 rx 4: GETATTR i1 {Fh 0}
      	2020/09/08 19:21:33 tx 4:     OK, {tA=1s {M040755 SZ=80 L=2 1026:1026 B0*4096 i0:1 A 1599585687.491337 M 1599585688.791343 C 1599585688.791343}}
      	2020/09/08 19:21:33 rx 6: GETATTR i1 {Fh 0}
      	2020/09/08 19:21:33 tx 6:     OK, {tA=1s {M040755 SZ=80 L=2 1026:1026 B0*4096 i0:1 A 1599585687.491337 M 1599585688.791343 C 1599585688.791343}}
      	2020/09/08 19:21:33 rx 8: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      	2020/09/08 19:21:33 tx 8:     OK, {i18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      	2020/09/08 19:21:33 rx 10: GETATTR i18446744073709551615 {Fh 0}
      	2020/09/08 19:21:33 tx 10:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      	2020/09/08 19:21:33 rx 12: OPEN i18446744073709551615 {O_RDONLY,0x8000}
      	2020/09/08 19:21:33 tx 12:     OK, {Fh 18446744073709551615 }
      	2020/09/08 19:21:33 rx 14: POLL i18446744073709551615
      	2020/09/08 19:21:33 tx 14:     38=function not implemented
      	2020/09/08 19:21:33 rx 16: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      	2020/09/08 19:21:33 tx 16:     34=numerical result out of range
      	2020/09/08 19:21:33 rx 18: RELEASE i18446744073709551615 {Fh 18446744073709551615 0x8000  L0}
      	2020/09/08 19:21:33 tx 18:     34=numerical result out of range
      	Filesystem mounted and ready.
      
      Change-Id: Idfaea994ee6a4b8f28acbe8343063f6250cbef35
      0b6cbc51
  13. 25 Jul, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: dump WRITE argument in debug logs · 9ca2d34b
      Han-Wen Nienhuys authored
      This provides feedback to file system authors about what we receive
      from the kernel. This can help figure out which system is to blame for
      data corruption.
      
      Change-Id: I7cd7f340bf705c9972c95cfc6d3d8612f1ce8b34
      9ca2d34b
  14. 18 Jul, 2020 3 commits
    • Jakob Unterwurzacher's avatar
      Make pollHack work on read-only mounts · fee50bf0
      Jakob Unterwurzacher authored
      Use LOOKUP & OPEN instead of CREATE so it also
      works on read-only mounts.
      
      go-fuse/fs$ go test -run TestRoMount -v
      === RUN   TestRoMount
      20:31:07.892736 rx 2: INIT i0 {7.31 Ra 0x20000 NO_OPEN_SUPPORT,ASYNC_READ,POSIX_LOCKS,EXPORT_SUPPORT,AUTO_INVAL_DATA,SPLICE_MOVE,SPLICE_READ,HANDLE_KILLPRIV,ABORT_ERROR,EXPLICIT_INVAL_DATA,BIG_WRITES,SPLICE_WRITE,READDIRPLUS,WRITEBACK_CACHE,IOCTL_DIR,READDIRPLUS_AUTO,ASYNC_DIO,PARALLEL_DIROPS,ATOMIC_O_TRUNC,DONT_MASK,FLOCK_LOCKS,POSIX_ACL,MAX_PAGES,CACHE_SYMLINKS,NO_OPENDIR_SUPPORT}
      20:31:07.892799 tx 2:     OK, {7.28 Ra 0x20000 BIG_WRITES,READDIRPLUS,PARALLEL_DIROPS,ASYNC_READ,AUTO_INVAL_DATA,NO_OPEN_SUPPORT 0/0 Wr 0x10000 Tg 0x0}
      20:31:07.892967 rx 4: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      20:31:07.893002 tx 4:     OK, {i18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
      20:31:07.893030 rx 6: OPEN i18446744073709551615 {O_RDONLY,0x8000}
      20:31:07.893037 tx 6:     OK, {Fh 18446744073709551615 }
      20:31:07.893053 rx 8: POLL i18446744073709551615
      20:31:07.893059 tx 8:     38=function not implemented
      20:31:07.893074 rx 10: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      20:31:07.893079 tx 10:     34=numerical result out of range
      20:31:07.893131 rx 12: RELEASE i18446744073709551615 {Fh 18446744073709551615 0x8000  L0}
      20:31:07.893153 tx 12:     34=numerical result out of range
      20:31:07.896705 received ENODEV (unmount request), thread exiting
      20:31:07.896719 received ENODEV (unmount request), thread exiting
      20:31:07.896888 received ENODEV (unmount request), thread exiting
      
      Change-Id: I6de861bdccf03184a4ca3deeefb1cfdc930aabe3
      --- PASS: TestRoMount (0.01s)
      PASS
      ok  	github.com/hanwen/go-fuse/v2/fs	0.015s
      
      Change-Id: Ic66ca153f5b739e5e081988fdc48d51cde5470f0
      fee50bf0
    • Jakob Unterwurzacher's avatar
      example: loopback: implement -ro flag · 2beb8ed3
      Jakob Unterwurzacher authored
      Currently fails like this
      
      	18:13:57.008805 Mount fail: read-only file system
      
      because pollHack requires write access.
      
      This will be fixed in pollHack in a later commit.
      
      Change-Id: I37b459339eda2e8270ca8094455db3ddbf764911
      2beb8ed3
    • Jakob Unterwurzacher's avatar
      fs: add TestRoMount · f612a450
      Jakob Unterwurzacher authored
      Also add a test if "ro" works directly in the fs
      package.
      
      Currently fails like this (will be fixed later):
      
      	--- FAIL: TestRoMount (0.01s)
      	    simple_test.go:116: read-only file syste
      
      Change-Id: I8194360b1d092268784b9c2b17de274b86633aa2
      f612a450
  15. 06 Jun, 2020 2 commits
  16. 03 Apr, 2020 3 commits
    • Han-Wen Nienhuys's avatar
      fs: cleanup after TestForget · 73071346
      Han-Wen Nienhuys authored
      Change-Id: Ie2aeb012346b2fbcc32de09ea05b39069291d803
      73071346
    • Han-Wen Nienhuys's avatar
      fs: add a test verifying that forget drops nodes · 18e94a3c
      Han-Wen Nienhuys authored
      Change-Id: Ife141f38f2b7598223935963e2916b0b9a609cca
      18e94a3c
    • Kirill Smelkov's avatar
      nodefs: Allow for several Lookup requests to be served simultaneously · 17c0c400
      Kirill Smelkov authored
      Commit d0fca860 introduced big lookup lock with the idea to "make sure
      we don't return forgotten nodes to the kernel". However this way it also
      started to prevent several Lookup handlers to be running simultaneously,
      which can deadlock if e.g. Lookup handler somehow synchronizes with
      other thread which caused the lookup:
      
      https://github.com/hanwen/go-fuse/commit/d0fca860#commitcomment-37772099
      
      On the surface the fix is easy: if we need to prevent lookups running in
      parallel to forget, we can turn lookupLock into shared one wrt Lookup,
      but exclusive wrt Forget.
      
      However a more correct fix would be to review nodefs locking completely:
      there is already per-filesystem treeLock RWMutex, which is _already_
      used to synchronize for forgets and lookups. From this point of view
      lookupLock is unneeded and the correct fix that d0fca860 should have been
      doing is to correct the scope of treeLock to be used more precisely.
      
      However that would be a more intrusive change, and given that nodefs is
      considered deprecated today, imho it is better to proceed with making
      lookupLock a shared one.
      
      The test that is added deadlocks without provided fix.
      
      I suggest not to reject this patch based on rationale that "nodefs is
      deprecated" as there still are real filesystems that use nodefs.
      
      I had not reviewed v2 go-fuse/fs locking yet.
      
      Change-Id: I18e01457f474dea31dc17186dfe6db582c2e6337
      17c0c400
  17. 24 Feb, 2020 4 commits
    • Jakob Unterwurzacher's avatar
      tests: fs: TestOpenDirectIO: move to /var/tmp · c34eae0e
      Jakob Unterwurzacher authored
      This used to create a hidden folder in the user's home
      directory to avoid /tmp, which is on tmpfs on modern
      distros.
      
      Use /var/tmp instead, which is also a real filesystem,
      and avoid cluttering the home dir.
      
      Change-Id: Ifa0a1dbdf7a2c4b08e53d20e28f287207ff8ef47
      c34eae0e
    • Jakob Unterwurzacher's avatar
      Travis CI: disable output buffering using "go test -p 1" · b4058856
      Jakob Unterwurzacher authored
      Per default, "go test ./..." tests multiple packages in
      parallel.
      
      From "go help build":
      
      	-p n
      		the number of programs, such as build commands or
      		test binaries, that can be run in parallel.
      		The default is the number of CPUs available.
      
      This seems rather innocent, but a consequence of tests
      running in parallel is that the all the output of each test
      is buffered until it completes. This prevents multiple tests
      interleaving their output.
      
      This also means that when a test hangs, we get no output at
      all, which is pretty bad.
      
      Disable parallelization and trade test time for better
      debugability of hung tests.
      
      Change-Id: I4c670b65baa2df3bef70bce622b830530a316ee7
      b4058856
    • Jakob Unterwurzacher's avatar
      go.mod: upgrade to Go 1.13 · b36138d5
      Jakob Unterwurzacher authored
      This gets added on every build and every test run,
      dirtying the git tree again and again.
      
      Give in and let Go 1.13 have it's entry in go.mod.
      
      Change-Id: I13679e54fca09ab818bc0167dff3f1ad851e763e
      b36138d5
    • Jakob Unterwurzacher's avatar
      tests: fix data race in TestDeletedInodePath · 6b30e299
      Jakob Unterwurzacher authored
      Running `go test ./... -race` showed a data race
      at `rootNode.deleted`. This race cannot really happen,
      but the race detector does not know that as the calls
      go through FUSE an appear to not be synchronized.
      
      Use atomic loads and stores to make the race detector
      happy.
      
      Change-Id: I26bd3e7d8efdd5b967fdb360eb17d7f71a8005c8
      6b30e299
  18. 02 Feb, 2020 1 commit
  19. 20 Jan, 2020 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: increment loops WaitGroup on Server creation · fe141f38
      Han-Wen Nienhuys authored
      The server.Serve routine can be called both inline and as
      goroutine. In the latter case, it is a synchronization error for Serve
      to call serve.loops.Add() by itself, leading to a detected race.
      
      Change-Id: I36f24bd36d1ae77d71e7d69a54ebdf5dbee9bd62
      fe141f38
  20. 03 Jan, 2020 4 commits
    • Jakob Unterwurzacher's avatar
      loopback: enable diagnostics logging · 0e3c45fc
      Jakob Unterwurzacher authored
      The loopback example is usually used to debug and develop.
      Enable the new diagnostics logging facility, unless -q (quiet)
      is passed.
      
      Change-Id: I9ae5214fc33616656832a219fc1470f421786b8c
      0e3c45fc
    • Jakob Unterwurzacher's avatar
      fs: detect Inode.Path() hitting an orphaned inode · 3f7bb2c2
      Jakob Unterwurzacher authored
      Some ".deleted" logic was in place, but did not work
      properly when the Inode was in the root directory.
      
      Fix that by introducing the `found` variable and
      add a test that verifies that it works.
      
      Also, return `.go-fuse.$RANDOM/deleted` to make it
      very unlikely that the placeholder name matches an
      actual file or directory. Introducing the `/deleted`
      subdir makes sure operations creating a file or
      directory fail reliably with ENOENT.
      
      Tests now look like this:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        === RUN   TestPosix
        === RUN   TestPosix/RenameOpenDir
        [...]
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:383: Fstat failed: no such file or directory. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.014s
      
      Change-Id: I2eb6fd48a11df543c9b7daf62647cb9d8a892568
      3f7bb2c2
    • Jakob Unterwurzacher's avatar
      tests: make RenameOpenDir more sensitive · ae87e918
      Jakob Unterwurzacher authored
      The test seemed to pass because the inode number is overridden
      in rawBridge.getattr, but looking at the permissions shows that
      the wrong directory is stat()ed:
      
        $ go test ./fs -run TestPosix/RenameOpenDir -count 1  -v
        [...]
        17:49:46.454077 received ENODEV (unmount request), thread exiting
        17:49:46.454343 received ENODEV (unmount request), thread exiting
        --- PASS: TestPosix (0.01s)
            --- SKIP: TestPosix/RenameOpenDir (0.01s)
                test.go:392: got permissions 0755, want 0700. Known limitation - see https://github.com/hanwen/go-fuse/issues/55
        PASS
        ok    github.com/hanwen/go-fuse/v2/fs 0.016s
      
      Also, add a log message whenever the inode number is overridden,
      this should (probably) not happen during normal operation. And
      it actually only happens once in the test suite (in RenameOpenDir):
      
        $ go test ./... -count 1 -v 2>&1 | grep "overriding ino"
        14:48:44.143694 warning: rawBridge.getattr: overriding ino 188663 with 186314
      
      See https://github.com/hanwen/go-fuse/issues/55
      
      Change-Id: I8b2ddb84c35a3b28b4f5e032e7113f8d484a5981
      ae87e918
    • Han-Wen Nienhuys's avatar
      fs: add Options.Logger, and use throughout · ee9c8261
      Han-Wen Nienhuys authored
      Change-Id: I4fabf222a306e5d3abdfda28422b046cd75c9a8c
      ee9c8261