1. 06 May, 2024 2 commits
  2. 30 Apr, 2024 2 commits
  3. 24 Apr, 2024 6 commits
    • Levin Zimmermann's avatar
      Revert "proto/msgpack: Allow nil=false simplification" · 1791b053
      Levin Zimmermann authored
      This reverts commit d779695c.
      1791b053
    • Levin Zimmermann's avatar
      NEO/py/storage: Ensure compression is always BOOL on the wire · 0e4bb572
      Levin Zimmermann authored
      Currently it can be:
      
      - nil
      - bool
      - integer
      
      In python this isn't really a problem due to '__bool__', but in Golang
      we don't have this. Yes - it would be possible to make a parser in
      Golang to decode all of these values as either True/False, but this is
      much more complex than just adjusting python to send consistent data
      types. It's also easier to describe a protocol where one parameter is
      always 'bool' instead of having a parameter that is 'bool | int | NoneType'.
      0e4bb572
    • Levin Zimmermann's avatar
      fixup! proto/msgpack: Fix encoding INVALID_{TID,OID} · ccdfd363
      Levin Zimmermann authored
      This is the decoding part of 37dd83e9
      ccdfd363
    • Levin Zimmermann's avatar
      proto/msgpack: Allow nil=false simplification · d779695c
      Levin Zimmermann authored
      NEO/py sometimes sends 'None' for boolean values (being equal to 'False'
      due to pythons 'bool(None) == False' logic). Let's accept python based
      False values (= Nil) as equally acceptable.
      
      Without this patch we can see errors as:
      
      xtesting.go:330: load 0285cbad6e147b66:0000000000000005: returned err unexpected:
              have: neo://127.0.0.1:30002/1: load 0285cbad6e147b66:0000000000000005: 127.0.0.1:48928 - 127.0.0.1:33051 .251: decode: decode: M: AnswerObject.Compression/op: got opcode 00;  expected any from c3c2
      d779695c
    • Levin Zimmermann's avatar
      9e4dd249
    • Levin Zimmermann's avatar
      proto/msgpack: Fix encoding INVALID_{TID,OID} · 37dd83e9
      Levin Zimmermann authored
      In pre-msgpack protocol an 'INVALID_{TID,OID}' was always
      decoded as 'None' in NEO/py [1]. But in msgpack protocol
      this isn't true anymore. An `INVALID_TID` is now decoded
      as an `INVALID_TID`. And this then leads to errors later [2].
      Instead of restoring NEO/py to a pre-msgpack behaviour, we
      fix this on NEO/go side by encoding 'INVALID_{TID,OID}' to NIL
      on the wire. This is good as it makes the messages we send
      smaller.
      
      [1] https://lab.nexedi.com/nexedi/neoppod/-/blob/6332112cba979dfd29b40fe9f98d097911fde696/neo/lib/protocol.py#L579-583
      [2] With SQLite backend we can see the following exception:
      
      Traceback (most recent call last):
        File "/home/levin/neo/neo/tests/functional/__init__.py", line 192, in start
          self.run()
        File "/home/levin/neo/neo/tests/functional/__init__.py", line 288, in run
          getattr(neo.scripts,  self.command).main()
        File "/home/levin/neo/neo/scripts/neostorage.py", line 32, in main
          app.run()
        File "/home/levin/neo/neo/storage/app.py", line 196, in run
          self._run()
        File "/home/levin/neo/neo/storage/app.py", line 227, in _run
          self.doOperation()
        File "/home/levin/neo/neo/storage/app.py", line 301, in doOperation
          poll()
        File "/home/levin/neo/neo/storage/app.py", line 145, in _poll
          self.em.poll(1)
        File "/home/levin/neo/neo/lib/event.py", line 160, in poll
          to_process.process()
        File "/home/levin/neo/neo/lib/connection.py", line 508, in process
          self._handlers.handle(self, self._queue.pop(0))
        File "/home/levin/neo/neo/lib/connection.py", line 93, in handle
          self._handle(connection, packet)
        File "/home/levin/neo/neo/lib/connection.py", line 108, in _handle
          pending[0][1].packetReceived(connection, packet)
        File "/home/levin/neo/neo/lib/handler.py", line 125, in packetReceived
          self.dispatch(*args)
        File "/home/levin/neo/neo/lib/handler.py", line 75, in dispatch
          method(conn, *args, **kw)
        File "/home/levin/neo/neo/storage/handlers/client.py", line 67, in askObject
          o = app.dm.getObject(oid, at, before)
        File "/home/levin/neo/neo/storage/database/manager.py", line 484, in getObject
          before_tid and u64(before_tid))
        File "/home/levin/neo/neo/storage/database/sqlite.py", line 336, in _getObject
          r = q(sql + ' AND tid=?', (partition, oid, tid))
      OverflowError: Python int too large to convert to SQLite INTEGER
      37dd83e9
  4. 10 Apr, 2024 1 commit
  5. 09 Apr, 2024 1 commit
  6. 08 Apr, 2024 2 commits
  7. 07 Apr, 2024 4 commits
  8. 06 Apr, 2024 1 commit
  9. 19 Mar, 2024 1 commit
    • Levin Zimmermann's avatar
      fixup! go/neo/neonet: MessagePack support for link layer (draft) · d64f1baf
      Levin Zimmermann authored
      In pre-msgpack protocol, the msgcode increment logic is like this:
      
      Notify 	 add +1 to next msgcode
      Request  add +0 to next msgcode   (next msg is answer & should therefore be the same)
      Answer 	 add +1 to next msgcode
      
      ('Answer' msgcode is adjusted by 'AnswerBit')
      
      In post-msgpack protocol, the logic is a bit different:
      
      Notify 	 add +1 to next msgcode
      Request  add +0 to next msgcode   (next msg is answer & should therefore be the same)
      Answer 	 add +2 to next msgcode
      
      So here we produce gaps after Request/Answer pairs.
      
      In the current version, this patch breaks the functionality of
      the pre-msgpack protocol, because for both protocols the same
      msgcodes are used. To still support 'N' we need to use different
      msgcodes depending on if 'N' or 'M' is used.
      d64f1baf
  10. 18 Mar, 2024 1 commit
  11. 15 Mar, 2024 2 commits
  12. 14 Mar, 2024 5 commits
    • Levin Zimmermann's avatar
      . · c19abad1
      Levin Zimmermann authored
      c19abad1
    • Levin Zimmermann's avatar
      fixup! X on msgpack support · 1282b808
      Levin Zimmermann authored
      This is a fix of 1cf2712f:
      'AppendString' is the equivalent to NEO/py 'NEO' encoding, not
      'AppendBytes'.
      1282b808
    • Levin Zimmermann's avatar
      go/neo/neonet: Fix segmentation violation in case handshake fails · 9501173e
      Levin Zimmermann authored
      Because 'err' was locally assigned inside the loop of 'DialLink' [1],
      it was always 'nil' as a function return value, even when
      'handshakeClient' actually returned an error. This lead to the
      unfortunate situation that the function sometimes returned 'link=nil'
      and 'err=nil', so that the function caller tried to access 'link'
      attributes which lead to 'runtime error: invalid memory address or nil
      pointer dereference'. This patch fixes this and now the function
      correctly returns an error if the dialing fails.
      
      [1] `peerConn, err := networker.Dial(ctx, addr)`
      9501173e
    • Levin Zimmermann's avatar
      . · 5e90fc24
      Levin Zimmermann authored
      5e90fc24
    • Julien Muchembled's avatar
      protocol: switch to msgpack for packet serialization · ccd47c48
      Julien Muchembled authored
      Not only for performance reasons (at least 3% faster) but also because of
      several ugly things in the way packets were defined:
      - packet field names, which are only documentary; for roots fields,
        they even just duplicate the packet names
      - a lot of repetitions for packet names, and even confusion between the name
        of the packet definition and the name of the actual notify/request packet
      - the need to implement field types for anything, like PByte to support new
        compression formats, since PBoolean is not enough
      
      neo/lib/protocol.py is now much smaller.
      ccd47c48
  13. 02 Feb, 2024 3 commits
  14. 29 Jan, 2024 8 commits
  15. 22 Aug, 2023 1 commit