1. 25 Nov, 2019 5 commits
    • Jakob Unterwurzacher's avatar
      fs: add TestTypeChange (simulate inode number reuse) · 53f16b0a
      Jakob Unterwurzacher authored
      TestTypeChange simulates inode number reuse that happens on real
      filesystems. For go-fuse, inode number reuse can look like a file changing
      to a directory or vice versa. Acutally, the old inode does not exist anymore,
      we just have not received the FORGET yet.
      
      The test fails reliably with iteration count 100. Will be fixed in the
      next commit.
      
      Test failure looks like this:
      
      	~/go/src/github.com/hanwen/go-fuse/fs$ go test -run TestTypeChange -v
      	=== RUN   TestTypeChange
      	21:43:36.065494 rx 2: INIT i0 {7.31 Ra 0x20000 MAX_PAGES,SPLICE_READ,FLOCK_LOCKS,READDIRPLUS,READDIRPLUS_AUTO,ASYNC_DIO,WRITEBACK_CACHE,HANDLE_KILLPRIV,SPLICE_WRITE,PARALLEL_DIROPS,ABORT_ERROR,EXPLICIT_INVAL_DATA,NO_OPENDIR_SUPPORT,ASYNC_READ,BIG_WRITES,DONT_MASK,IOCTL_DIR,AUTO_INVAL_DATA,POSIX_ACL,CACHE_SYMLINKS,POSIX_LOCKS,ATOMIC_O_TRUNC,EXPORT_SUPPORT,SPLICE_MOVE,NO_OPEN_SUPPORT}
      	21:43:36.065573 tx 2:     OK, {7.28 Ra 0x20000 PARALLEL_DIROPS,AUTO_INVAL_DATA,ASYNC_READ,BIG_WRITES,NO_OPEN_SUPPORT,READDIRPLUS 0/0 Wr 0x10000 Tg 0x0}
      	21:43:36.065742 rx 4: LOOKUP i1 [".go-fuse-epoll-hack"] 20b
      	21:43:36.065798 tx 4:     2=no such file or directory, {i0 g0 tE=0s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.065837 rx 6: CREATE i1 {0100100 [WRONLY,CREAT,TRUNC,0x8000] (00)} [".go-fuse-epoll-hack"] 20b
      	21:43:36.065861 tx 6:     OK, {i18446744073709551615 g0 {M0100644 SZ=0 L=1 0:0 B0*0 i0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000} &{18446744073709551615 0 0}}
      	21:43:36.065945 rx 8: POLL i18446744073709551615
      	21:43:36.065954 tx 8:     38=function not implemented
      	21:43:36.066009 rx 10: FLUSH i18446744073709551615 {Fh 18446744073709551615}
      	21:43:36.066046 tx 10:     5=input/output error
      	21:43:36.066065 rx 12: RELEASE i18446744073709551615 {Fh 18446744073709551615 WRONLY,0x8000  L0}
      	21:43:36.066071 tx 12:     5=input/output error
      	21:43:36.066107 rx 14: LOOKUP i1 ["file"] 5b
      	21:43:36.066181 tx 14:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066217 rx 16: GETATTR i1234 {Fh 0}
      	21:43:36.066240 tx 16:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066264 rx 18: LOOKUP i1 ["file"] 5b
      	21:43:36.066285 tx 18:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066301 rx 20: UNLINK i1 ["file"] 5b
      	21:43:36.066321 tx 20:     OK
      	21:43:36.066361 rx 24: LOOKUP i1 ["dir"] 4b
      	21:43:36.066371 tx 24:     OK, {i1234 g0 tE=0s tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.066383 rx 22: FORGET i1234 {Nlookup=2}
      	21:43:36.066390 rx 26: GETATTR i1234 {Fh 0}
      	21:43:36.068259 tx 26:     OK, {tA=0s {M0100644 SZ=0 L=0 0:0 B0*4096 i0:1234 A 0.000000 M 0.000000 C 0.000000}}
      	21:43:36.069664 received ENODEV (unmount request), thread exiting
      	21:43:36.073309 received ENODEV (unmount request), thread exiting
      	21:43:36.073396 received ENODEV (unmount request), thread exiting
      	--- FAIL: TestTypeChange (0.01s)
      	    bridge_test.go:96: should be a dir now
      	FAIL
      	exit status 1
      	FAIL	github.com/hanwen/go-fuse/v2/fs	0.017s
      
      Change-Id: I7a1a10303ecc0a8c008d29d5d81079868fc2cc43
      53f16b0a
    • Jakob Unterwurzacher's avatar
      fs: addNewChild(): handle concurrent FORGETs · 68f70527
      Jakob Unterwurzacher authored
      As seen in debug logs like shown below, concurrent
      FORGETs and inode number reuse could make use give
      an Inode to the kernel that has already been deleted
      from the nodes[] map.
      
      With this change, I can no langer trigger a panic
      in xfstests generic/007.
      
      Fixes https://github.com/hanwen/go-fuse/issues/324
      
      Nodes that appear to change type are not handled yet
      (actually caused by inode number reuse), but this is
      a separate problem.
      
      Debug log:
      
        17:09:07.651735 rx 84444: CREATE i13632136 {0100644 [WRONLY,TRUNC,CREAT,0x8000] (022)} ["XXXXXXXXXXXX.38"] 16b
        17:09:07.651790 tx 84444:     OK, {i13632483 g1 {M0100644 SZ=0 L=1 0:0 B0*4096 i0:13632483 A 1572192547.650470 M 1572192547.650470 C 1572192547.650470} &{7 0 0}}
        [...]
        17:09:07.684439 rx 87988: CREATE i13632119 {0100644 [CREAT,TRUNC,WRONLY,0x8000] (022)} ["XXXXXXXXXXXX.37"] 16b
        [...]
        17:09:07.684472 rx 87992: FORGET i13632483 {Nlookup=1}
        17:09:07.684483 tx 87988:     OK, {i13632483 g1 {M0100644 SZ=0 L=1 0:0 B0*4096 i0:13632483 A 1572192547.683470 M 1572192547.683470 C 1572192547.683470} &{8 0 0}}
        [...]
        17:09:07.684557 rx 88006: FLUSH i13632483 {Fh 8}
        17:09:07.684560 unknown node 13632483
        17:09:07.684564 tx 88002:     OK,  "XXXXXXXXXXXX.830"
        panic: unknown node 13632483
      
      Change-Id: Ibb1a7a6a459d2e4133a4e512e0816e8ed950a8d7
      68f70527
    • Jakob Unterwurzacher's avatar
      example: loopback: support -q (quiet) · 713e99eb
      Jakob Unterwurzacher authored
      Getting rid of the "Mounted!" message makes it easier
      to integrate into xfstests, which barks at all and any
      unexpected output.
      
      Used by https://github.com/rfjakob/fuse-xfstests/commit/ab1d77ccdb8cf16f79cb312bcb3aaadb23660807
      
      Change-Id: Idff314474b4980ef6f6fc2460b338a840bb0e409
      713e99eb
    • Jakob Unterwurzacher's avatar
      fuse: replace DirEntryList.FixMode() pointer arithmetic · be5a8417
      Jakob Unterwurzacher authored
      As pointed out by @slackner at
      https://github.com/hanwen/go-fuse/commit/8581b7e950b2d520370eba8997ddd9493192e9c7#r35845728
      FixMode() failed to take into account the length
      of the file name and the padding.
      
      To fix this without adding a lot of additional
      pointer arithmetic, add the "lastDirent" field
      to DirEntryList. This stores the last serialized
      _Dirent, so _Dirent.Typ can be accessed easily
      by FixMode().
      
      Change-Id: I6ca3fe551e1a072779b1d5ed5293add057e55287
      be5a8417
    • Han-Wen Nienhuys's avatar
      fs: check outcome of mode fixup result · c9fd0958
      Han-Wen Nienhuys authored
      Add TestReaddirTypeFixup(). This test currently fails
      because FixMode is broken. Will be fixed in a later commit.
      
      Change-Id: Ie4eab9c5817ddb772b73aa22e8f75f5d4f2ad49c
      c9fd0958
  2. 22 Nov, 2019 1 commit
  3. 08 Nov, 2019 2 commits
  4. 07 Nov, 2019 4 commits
    • Jakob Unterwurzacher's avatar
      Travis CI: drop Go 1.9, add Go 1.13 · 692daa62
      Jakob Unterwurzacher authored
      Go 1.9 fails to build the fs tests:
      
        # github.com/kylelemons/godebug/diff
        ../../kylelemons/godebug/diff/diff.go:50:13: undefined: strings.Builder
        ?   	github.com/hanwen/go-fuse/example/zipfs	[no test files]
        FAIL	github.com/hanwen/go-fuse/fs [build failed]
      
      So drop Go 1.9.
      
      Add Go 1.13, which is already at 1.13.3.
      
      Change-Id: Id2286cddc3973a1388f94e1b294db36d7663c37b
      692daa62
    • Jakob Unterwurzacher's avatar
      DirEntryList: implement changing of mode after entry was serialized · 8581b7e9
      Jakob Unterwurzacher authored
      When a directory is modified while READDIRPLUS is running,
      the file type may change between readdir and lookup.
      
      This patch adds the function FixMode() which allows to
      overwrite the file type of the last serialized dirent.
      
      Add test
      
      Fixes https://github.com/hanwen/go-fuse/issues/323
      
      Change-Id: Id45a9c4633e9abcb075588dc610ce8341d104825
      8581b7e9
    • Han-Wen Nienhuys's avatar
      fuse: support direct IO in loopback · 3a40eb5c
      Han-Wen Nienhuys authored
      open(2) has supports direct IO. When opening a file in the loopback
      FS, writes to the underlying would fail, because the FUSE server does
      not position the bytes to write (which come directly after the WriteIn
      header) on a 512-byte boundary.
      
      Fix this by allocating the input buffer with such that the first
      content byte is aligned.
      
      Add a test.
      
      Fixes #328.
      
      Change-Id: Ib7ad24aff673413ab8db4112d0b12143c2654617
      3a40eb5c
    • Han-Wen Nienhuys's avatar
      Filter O_APPEND from open flags. · d1b56cc4
      Han-Wen Nienhuys authored
      As of Go 1.13, os.File doesn't allow WriteAt to O_APPEND. Since the
      kernel translates writes to correct offsets, loopback does not have to
      open the file as O_APPEND.
      
      Fixes #325.
      
      Change-Id: Ib1a7026b94b5604d415ac9fbb5850842dbeffc2a
      d1b56cc4
  5. 06 Nov, 2019 2 commits
  6. 03 Nov, 2019 3 commits
    • Jakob Unterwurzacher's avatar
      Travis CI: set test timeout to 5 minutes · d6dc0f2a
      Jakob Unterwurzacher authored
      Travis CI has a no-output-timeout of 10 minutes.
      Set a lower test timeout so we get proper backtraces
      on a hung test.
      
      Actual test runtime should be about 1 minute.
      
      Change-Id: I3fb40d13fb5bf9efc5e96f3551733205a7502481
      d6dc0f2a
    • Jakob Unterwurzacher's avatar
      posixtest: print more infos on fd leak · d258ee8d
      Jakob Unterwurzacher authored
      gocryptfs has a similar test with somewhat more detailed
      parsing of the open fds. Import the ListFds function from
      gocryptfs and use it to make the error more informative.
      
      Change-Id: I2442364e2e5d49749683072b91bda5691ad8902e
      d258ee8d
    • Jakob Unterwurzacher's avatar
      fs: ReadDirPlus: don't add ".." and "." to the tree · c4765ac7
      Jakob Unterwurzacher authored
      These are not actual directories, and adding them to
      our tree causes directory cycles, so don't.
      
      Before this change, running
      
        $ ./extractloop.bash -loopback
      
      from https://github.com/rfjakob/gocryptfs/tree/master/tests/stress_tests
      quickly caused loopback to get stuck and consume all
      available memory. Things look stable now.
      
      Add a test to verify that rawBridge does not call LOOKUP
      on "." and "..".
      
      Also, add a comment to the "for pd = range p.parents" loop,
      that has caused some head-scratching during debugging.
      
      Change-Id: I15eedeee30043dc57e5fa580fefe59a48674b193
      c4765ac7
  7. 19 Oct, 2019 4 commits
    • Jakob Unterwurzacher's avatar
      fs: ReadDirPlus: don't forget overflow entry · 1cb24af2
      Jakob Unterwurzacher authored
      Even when the directory stream does not have additional
      entries, we may still have an entry in the overflow slot.
      
      Don't forget about it. Test now passes:
      
        go-fuse/fs$ go test
        PASS
        ok  	github.com/hanwen/go-fuse/v2/fs	0.582s
      
      Change-Id: I98b39d4e7df235ed57ae4bf2702532f3b17d1bd0
      1cb24af2
    • Jakob Unterwurzacher's avatar
      posixtest: ReadDir: check directory listing after each file · 0fb2d39e
      Jakob Unterwurzacher authored
      There is an unhandled corner case in the directory stream
      handling. The stricter test catches it:
      
        go-fuse/fs$ go test
        --- FAIL: TestPosix (0.20s)
            --- FAIL: TestPosix/ReadDir (0.09s)
                test.go:342: got 19 entries, want 20
                test.go:342: got 40 entries, want 41
                test.go:342: got 61 entries, want 62
                test.go:342: got 82 entries, want 83
                test.go:342: got 103 entries, want 104
        FAIL
        exit status 1
        FAIL	github.com/hanwen/go-fuse/v2/fs	0.610s
      
      Fix comes next.
      
      Change-Id: I3e22904fce96bf2264ede47656ebd2936a7d13cd
      0fb2d39e
    • Jakob Unterwurzacher's avatar
      loopback: show original directory in "df" · 17c7e2cd
      Jakob Unterwurzacher authored
      Filesystems usually show the underlying device in the first
      column of df. Follow suite and show the original dir, making
      "df" and "mount" output more useful.
      
      The actual reason for this change, however, is that it facilitates
      running xfstests against go-fuse loopback. The xfstests machinery
      checks that the expected device is mounted at the right place.
      
      Commit https://github.com/rfjakob/fuse-xfstests/commit/182841fea487419f493d9b547bf703bee9e39322
      adds support for go-fuse loopback to xfstests and depends on this
      change.
      
      Before:
      
        $ df -T
        Filesystem                 Type           1K-blocks       Used Available Use% Mounted on
        loopback                   fuse.loopback  237371192  149885928  75357828  67% /home/jakob/go/src/github.com/hanwen/go-fuse/example/loopback/b
      
      After:
      
        $ df -T
        Filesystem                 Type           1K-blocks       Used Available Use% Mounted on
        a                          fuse.loopback  237371192  150332024  74911732  67% /home/jakob/go/src/github.com/hanwen/go-fuse/example/loopback/b
      
      Change-Id: I3d5a2e7ab021af4eed657e66654b636e08292219
      17c7e2cd
    • Jakob Unterwurzacher's avatar
      loopback: set filesystem type as shown in "df -T" · 8c10cb4e
      Jakob Unterwurzacher authored
      Per default, "rawBridge" is shown here, which does not tell the user
      much about the filesystem. Set to "loopback" as shown below.
      
      Before:
      
        $ df -T
        Filesystem                 Type            1K-blocks       Used Available Use% Mounted on
        rawBridge                  fuse.rawBridge  237371192  149873220  75370536  67% /home/jakob/go/src/github.com/hanwen/go-fuse/example/loopback/b
      
      After:
      
        $ df -T
        Filesystem                 Type           1K-blocks       Used Available Use% Mounted on
        loopback                   fuse.loopback  237371192  149885928  75357828  67% /home/jakob/go/src/github.com/hanwen/go-fuse/example/loopback/b
      
      Change-Id: Iaaa18b4193b31567f6589669e8c443715afbd362
      8c10cb4e
  8. 04 Oct, 2019 3 commits
  9. 16 Sep, 2019 2 commits
  10. 05 Aug, 2019 1 commit
  11. 02 Aug, 2019 2 commits
  12. 26 Jul, 2019 2 commits
    • Han-Wen Nienhuys's avatar
      travis: disable -race · 2f298055
      Han-Wen Nienhuys authored
      Travis CI appears to be too slow for the race detector. 
      
      Change-Id: I5f4e7dc379938cf8000ab2d8ba95e09fab8ed6c9
      2f298055
    • Han-Wen Nienhuys's avatar
      fuse: don't get Pool elements under lock · 33b2c2f1
      Han-Wen Nienhuys authored
      Pool.Get() can trigger a GC, and we see this GC being hung in stack
      traces for the TestDeleteNotify test case.
      
      See issue #261.
      
      Change-Id: I278a05fc3c6a59c701686aebd410511c5502d9d5
      33b2c2f1
  13. 25 Jul, 2019 1 commit
    • Han-Wen Nienhuys's avatar
      Fix typo. · 48d90837
      Han-Wen Nienhuys authored
      Reported-by: Philipp Gillé
      Change-Id: I3db9f54016c17744bafa57a813531a99c01e47fa
      48d90837
  14. 20 Jul, 2019 8 commits