1. 18 Jan, 2021 4 commits
  2. 15 Jan, 2021 20 commits
  3. 14 Jan, 2021 4 commits
    • Kirill Smelkov's avatar
      go/neo/neonet: Kill pktBuf.Dump · dfec9278
      Kirill Smelkov authored
      It is not used anywhere and would have to be reworked with upcoming
      introduction of msgpack encoding. -> Simply remove that.
      dfec9278
    • Kirill Smelkov's avatar
      go/neo/neonet: msgPack -> pktEncode · 47e3cdd9
      Kirill Smelkov authored
      Just renaming.
      47e3cdd9
    • Kirill Smelkov's avatar
      go/neo/neonet: tests: Introduce T · 41a120c1
      Kirill Smelkov authored
      This represents a neonet testing environment and in the future will be
      used to run tests under different protocol versions, protocol encodings, etc.
      
      For now it is noop wrapper around testing.T + t.bin that wraps []byte
      and will be amended to return something different depending on t's
      encoding.
      41a120c1
    • Kirill Smelkov's avatar
      go/neo/neonet: Rework handshake to differentiate client and server parts · e407f725
      Kirill Smelkov authored
      Previously we were doing handshake symmetrically: both client and server
      were transmitting hello and receiving peer's hello simultaneously.
      However this does not allow server to adjust its behaviour depending on
      which client (protocol version, protocol encoding, ...) is connecting to it.
      
      -> Rework handshake so that client always sends its hello first, and
      only then the server side replies. This matches actual NEO/py behaviour:
      
      https://lab.nexedi.com/nexedi/neoppod/blob/v1.12-67-g261dd4b4/neo/lib/connector.py#L293-294
      
      even though the "NEO protocol" states that
      
      	Handshake transmissions are not ordered with respect to each other and can go in parallel.
      
      	( https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/2 )
      
      If I recall correctly that sentence was authored by me in 2018 based on
      previous understanding of should-be full symmetry in-between client and
      server.
      
      However soon we are going to teach server sides to autodetect client
      encoding and adjust server to talk to client via its preferred way.
      This needs handshake for client and server to be differentiated.
      
      The protocol needs to be adjusted as well. However I'm not sure it is
      going to happen...
      e407f725
  4. 13 Jan, 2021 1 commit
  5. 12 Jan, 2021 1 commit
    • Kirill Smelkov's avatar
      go/neo/neonet: Fix thinko in recvPkt · 4f6adb26
      Kirill Smelkov authored
      We were returning packet with tail read for next packet data.
      Previously decoding was forgiving, but upcoming rework for msgpack
      support will add check to catch packets with overlong payload, e.g.
      
          === RUN   TestEmptyDB/py/!ssl
          I: runneo.py: /tmp/neo214694750/1 !ssl: started master(s): 127.0.0.1:30621
          127.0.0.1:42266 > 127.0.0.1:30621: .1 RequestIdentification &{CLIENT ?(0)0  1 ø [] []}
          127.0.0.1:42266 < 127.0.0.1:30621: (N: decode header: len(payload) != msgLen) 00 00 00 01 80 01 00 00 00 09 00 f0 00 00 01 e0 00 00 01 00 00 00 00 00 06 00 00 00 58 41 d7 ff 69 82 0a 91 25 00 00 00 03 02 00 00 00 00 e0 00 00 01 02 41 d7 ff 69 82 0a 78 ff 01 00 00 00 09 31 32 37 2e 30 2e 30 2e 31 96 f7 00 00 00 01 02 41 d7 ff 69 81 c6 65 f5 00 00 00 00 09 31 32 37 2e 30 2e 30 2e 31 77 9d f0 00 00 01 02 ff ff ff ff ff ff ff ff 00 00 00 02 00 0a 00 00 00 19 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00 01 01
      
      -> Fix is: pkt.data should be data[:pktLen], not data[:n] since n is how
      much we have read at all.
      4f6adb26
  6. 29 Dec, 2020 3 commits
  7. 18 Dec, 2020 1 commit
  8. 17 Dec, 2020 1 commit
  9. 30 Nov, 2020 4 commits
    • Kirill Smelkov's avatar
      go/internal/xtesting: DrvTestWatch: Verify that watching works with empty transactions · 463ef9ad
      Kirill Smelkov authored
      It was not the case with ZEO4 server because we did not had
      ZEO@bf80d23d yet.
      463ef9ad
    • Kirill Smelkov's avatar
      go/zeo: tests: Terminate spawned processses with SIGTERM instead of SIGKILL · b516190c
      Kirill Smelkov authored
      For ZEO it is not strictly required, but for upcoming NEO for example
      NEOCluster needs to shutdown gracefully, else there are processes left for e.g.
      storage nodes and they dump somthing as below on the terminal after tests
      completion:
      
          === RUN   TestLoad
          2020/10/21 14:33:00 zodb: FIXME: open ../zodb/storage/fs1/testdata/1.fs: raw cache is not ready for invalidations -> NoCache forced
          === RUN   TestLoad/py
          I: runneo.py: /tmp/neo445013868/1: Started master(s): 127.0.0.1:24661
          WARNING: This is not the recommended way to import data to NEO: you should use the Importer backend instead.
          NEO also does not implement IStorageRestoreable interface, which means that undo information is not preserved when using this tool: conflict resolution could happen when undoing an old transaction.
          Migrating from ../zodb/storage/fs1/testdata/1.fs to 127.0.0.1:24661
          Migration done in 0.19877
          --- PASS: TestLoad (0.75s)
              --- PASS: TestLoad/py (0.74s)
          PASS
          ok      lab.nexedi.com/kirr/neo/go/neo  0.749s
          (neo) (z-dev) (g.env) kirr@deco:~/src/neo/src/lab.nexedi.com/kirr/neo/go/neo$ Traceback (most recent call last):
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/tests/functional/__init__.py", line 182, in start
              getattr(neo.scripts,  command).main()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/scripts/neostorage.py", line 66, in main
              app.run()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 147, in run
              self._run()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 178, in _run
              self.doOperation()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 266, in doOperation
              poll()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 87, in _poll
              self.em.poll(1)
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 155, in poll
              self._poll(blocking)
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 253, in _poll
              timeout_object.onTimeout()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 259, in onTimeout
              on_timeout()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 207, in _deferredCommit
              self.commit()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 193, in commit
              self._commit()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 90, in _commit
              retry_if_locked(self.conn.commit)
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 45, in retry_if_locked
              return f(*args)
          OperationalError: disk I/O error
      b516190c
    • Kirill Smelkov's avatar
      go/zodb/zeo: Factor-out verifying empty database into xtesting.DrvTestEmptyDB · 8fafde00
      Kirill Smelkov authored
      And use it to verify other storages.
      8fafde00
    • Kirill Smelkov's avatar
  10. 05 Nov, 2020 1 commit