• Levin Zimmermann's avatar
    wcfs: v↑ go dependencies · 764d4da8
    Levin Zimmermann authored
    This patch updates:
    
    - github.com/golang/glog: we already wanted to do so in
        !23,
        but we deferred it to keep go 1.18 support. However in recent patches
        we already dropped go 1.18 support and we can therefore update glog now.
    - lab.nexedi.com/kirr/neo/go: add fix in handshake, see here for more information:
        kirr/neo@d75f4ac2 and
        kirr/neo@03db1d8a
    
    This patch doesn't update:
    
    - github.com/hanwen/go-fuse: This was updated upstream and Kirill already
        reviewed and integrated patches in custom branch. However when updating
        go-fuse to v2.4.3-0.20240904154523-9546fc238dc6 (this is
        kirr/go-fuse@9546fc23),
        WCFS tests fail on my machine [1] => let's defer update
    - github.com/kisielk/og-rek: there are new patches that will be needed
        in the future, but we didn't update NEO/go og-rek dependency yet,
        so let's defer the update in wendelin.core until we updated og-rek
        in NEO/go
    - github.com/johncgriffin/overflow: no update on upstream
    - github.com/pkg/errors: no update on upstream
    - github.com/stretchr/testify: This was already updated with
        c559ec1a
        'testify' seems to have a major release in the future which may break
        some of our test code, but for now major version 1 is still the
        stable release.
    
    ----
    kirr: I confirm that
    kirr/go-fuse@9546fc23 brings in
    regression to WCFS tests. It seems I missed some error in that go-fuse
    update and it will need to be bisected and debugged.
    
    ---
    
    [1] Test failure log:
    
    ========================================== FAILURES ==========================================
    ______________________________________ test_wcfs_basic _______________________________________
    
        @func
        def test_wcfs_basic():
            t = tDB(); zf = t.zfile
            defer(t.close)
    
            # >>> lookup non-BigFile -> must be rejected
            with raises(OSError) as exc:
                t.wc._stat("head/bigfile/%s" % h(t.nonzfile._p_oid))
            assert exc.value.errno == EINVAL
    
            # >>> file initially empty
            f = t.open(zf)
            f.assertCache([])
            f.assertData ([], mtime=t.at0)
    
            # >>> (@at1) commit data -> we can see it on wcfs
            at1 = t.commit(zf, {2:'c1'})
    
            f.assertCache([0,0,0])  # initially not cached
            f.assertData (['','','c1'], mtime=t.head)
    
            # >>> (@at2) commit again -> we can see both latest and snapshotted states
            # NOTE blocks e(4) and f(5) will be accessed only in the end
            at2 = t.commit(zf, {2:'c2', 3:'d2', 5:'f2'})
    
            # f @head
    >       f.assertCache([1,1,0,0,0,0])
    
    wcfs/wcfs_test.py:1341:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    t = <wcfs.wcfs_test.tFile instance at 0x7ff61457b960>, incorev = [1, 1, 0, 0, 0, 0]
    
        def assertCache(t, incorev):
    >       assert t.cached() == incorev
    E       assert [0, 0, 0, 0, 0, 0] == [1, 1, 0, 0, 0, 0]
    E         At index 0 diff: 0 != 1
    E         Use -v to get the full diff
    
    wcfs/wcfs_test.py:791: AssertionError
    ------------------------------------ Captured stdout call ------------------------------------
    
    M: commit -> @at0 (03fb5dfbe3c1cd55)
    
    M: commit -> @at1 (03fb5dfbe4936a66)
    M:      f<0000000000000002>     [2]
    
    M: commit -> @at2 (03fb5dfbe4d01166)
    M:      f<0000000000000002>     [2, 3, 5]
    >>> Change history by file:
    
    f<0000000000000002>:
                                    0 1 2 3 4 5 6 7
                                    a b c d e f g h
            @at0 (03fb5dfbe3c1cd55)
            @at1 (03fb5dfbe4936a66)     2
            @at2 (03fb5dfbe4d01166)     2 3   5
    
    ------------------------------------ Captured stderr call ------------------------------------
    I0917 12:43:53.392222  124283 wcfs.go:2752] start "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    I0917 12:43:53.392282  124283 wcfs.go:2758] (built with go1.21.13)
    W0917 12:43:53.392404  124283 storage.go:232] zodb: FIXME: open file:///tmp/testdb_fs.z5ZoMH/1.fs: raw cache is not ready for invalidations -> NoCache forced
    W0917 12:43:53.567807  124283 wcfs.go:2331] /head/bigfile: lookup "0000000000000001": bigfopen 0000000000000001 @03fb5dfbe3c1cd55: invalid argument: ZODB.Broken("persistent.Persistent") is not a ZBigFile
    I0917 12:43:53.710208  124283 wcfs.go:2933] stop "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    ------------------------------------- Captured log call --------------------------------------
    WARNING  ZODB.FileStorage:FileStorage.py:412 Ignoring index for /tmp/testdb_fs.z5ZoMH/1.fs
    _________________________________ test_wcfs_watch_vs_access __________________________________
    
        @func
        def test_wcfs_watch_vs_access():
            t = tDB(); zf = t.zfile; at0=t.at0
            defer(t.close)
    
            f = t.open(zf)
            at1 = t.commit(zf, {2:'c1'})
            at2 = t.commit(zf, {2:'c2', 3:'d2', 5:'f2'})
            at3 = t.commit(zf, {0:'a3', 2:'c3', 5:'f3'})
    
            f.assertData(['a3','','c3','d2','x','x'])
            f.assertCache([1,1,1,1,0,0])
    
            # watched + commit -> read -> receive pin messages.
            # read vs pin ordering is checked by assertBlk.
            #
            # f(5) is kept not accessed to check later how wcfs.go handles δFtail
            # rebuild after it sees not yet accessed ZBlk that has change history.
            wl3  = t.openwatch();  w3 = wl3.watch(zf, at3);  assert at3 == t.head
            assert w3.at     == at3
            assert w3.pinned == {}
    
            wl3_ = t.openwatch();  w3_ = wl3_.watch(zf, at3)
            assert w3_.at     == at3
            assert w3_.pinned == {}
    
            wl2  = t.openwatch();  w2 = wl2.watch(zf, at2)
            assert w2.at     == at2
            assert w2.pinned == {0:at0, 2:at2}
    
            # w_assertPin asserts on state of .pinned for {w3,w3_,w2}
            def w_assertPin(pinw3, pinw3_, pinw2):
                assert w3.pinned   == pinw3
                assert w3_.pinned  == pinw3_
                assert w2.pinned   == pinw2
    
            f.assertCache([1,1,1,1,0,0])
            at4 = t.commit(zf, {1:'b4', 2:'c4', 5:'f4', 6:'g4'})
    >       f.assertCache([1,0,0,1,0,0,0])
    
    wcfs/wcfs_test.py:1702:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    t = <wcfs.wcfs_test.tFile instance at 0x7ff614512050>, incorev = [1, 0, 0, 1, 0, 0, ...]
    
        def assertCache(t, incorev):
    >       assert t.cached() == incorev
    E       assert [0, 0, 0, 0, 0, 0, ...] == [1, 0, 0, 1, 0, 0, ...]
    E         At index 0 diff: 0 != 1
    E         Use -v to get the full diff
    
    wcfs/wcfs_test.py:791: AssertionError
    ------------------------------------ Captured stdout call ------------------------------------
    
    M: commit -> @at0 (03fb5dfc0fd82300)
    
    M: commit -> @at1 (03fb5dfc10b92ecc)
    M:      f<0000000000000049>     [2]
    
    M: commit -> @at2 (03fb5dfc10cee9dd)
    M:      f<0000000000000049>     [2, 3, 5]
    
    M: commit -> @at3 (03fb5dfc1100c999)
    M:      f<0000000000000049>     [0, 2, 5]
    
    C: setup watch f<0000000000000049> @at3 (03fb5dfc1100c999)
    
    C: setup watch f<0000000000000049> @at3 (03fb5dfc1100c999)
    
    C: setup watch f<0000000000000049> @at2 (03fb5dfc10cee9dd)
    
    M: commit -> @at4 (03fb5dfc120ed611)
    M:      f<0000000000000049>     [1, 2, 5, 6]
    >>> Change history by file:
    
    f<0000000000000049>:
                                    0 1 2 3 4 5 6 7
                                    a b c d e f g h
            @at0 (03fb5dfc0fd82300)
            @at1 (03fb5dfc10b92ecc)     2
            @at2 (03fb5dfc10cee9dd)     2 3   5
            @at3 (03fb5dfc1100c999) 0   2     5
            @at4 (03fb5dfc120ed611)   1 2     5 6
    
    ------------------------------------ Captured stderr call ------------------------------------
    I0917 12:44:03.733037  125217 wcfs.go:2752] start "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    I0917 12:44:03.733126  125217 wcfs.go:2758] (built with go1.21.13)
    W0917 12:44:03.733418  125217 storage.go:232] zodb: FIXME: open file:///tmp/testdb_fs.z5ZoMH/1.fs: raw cache is not ready for invalidations -> NoCache forced
    I0917 12:44:04.475273  125217 wcfs.go:2933] stop "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    ============================ 2 failed, 42 passed in 55.81 seconds ============================
    I0917 12:44:17.882140  125540 wcfs.go:2933] stop "/dev/shm/wcfs/c4d833a0bdea4c51decf5425b8ad2cc4d017280f" "file:///tmp/testdb_fs.bvHBy9/1.fs"
    make: *** [Makefile:174: test.wcfs] Error 1
    
    /reviewed-by @kirr
    /reviewed-on !30
    764d4da8
go.sum 21.5 KB