go/zodb/zeo: Fix ZRPC/pickle binary and string encoding/decoding wrt py3
ZEO/go client was asserting "method" field to be bytestring. As the result interoperation with ZEO/py3 server was failing as e.g. 2024-07-18T20:41:14 INFO ZEO.asyncio.server received handshake 'Z5' 2024/07/18 20:41:14 zlink @ - /tmp/zeo2281880331/1.fs.zeosock: serveRecv1: decode: .0: method: got string; expected str zeo_test.go:221: openzeo /tmp/zeo2281880331/1.fs.zeosock: zeo: open /tmp/zeo2281880331/1.fs.zeosock: zlink @ - /tmp/zeo2281880331/1.fs.zeosock: call register: zlink is closed because ZEO/py3 sends "method" as unicode. -> Fix it by using pickle.AsString when decoding "method" field. Similarly ZEO/go client was sending strings as bytestr, which on py3 decode to unicode ok only if they are ASCII. Adjust string encoding to be emitted as unicode always which works wrt both ZEO/py3 and ZEO/py2. -------- ZEO/go client was sending binary ZRPC arguments as bytestr as well, which was resulting in the following exception on ZEO/py3 side: 2024-07-18T20:42:31 ERROR ZEO.asyncio.server Can't deserialize message Traceback (most recent call last): File "/home/kirr/src/wendelin/z/ZEO/src/ZEO/asyncio/server.py", line 102, in message_received message_id, async_, name, args = self.decode(message) File "/home/kirr/src/wendelin/z/ZEO/src/ZEO/asyncio/marshal.py", line 122, in pickle_server_decode return unpickler.load() # msgid, flags, name, args UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 1: ordinal not in range(128) 2024/07/18 20:42:31 zlink @ - /tmp/zeo627526011/1.fs.zeosock: recvPkt: EOF xtesting.go:361: load 0285cbac70a3d733:0000000000000001: returned err unexpected: have: /tmp/zeo627526011/1.fs.zeosock: load 0285cbac70a3d733:0000000000000001: zlink @ - /tmp/zeo627526011/1.fs.zeosock: call loadBefore: zlink is closed want: nil -> Fix it by using pickle.Bytes which should be encoded as bytes in the pickle stream. This fix is ok but depends on the next patch to upgrade ZODB serialization protocol to 3, because on pickle protocol 2 bytes are serialized as `_codecs.encode(byt.decode('latin1'), 'latin1')` and ZEO/py forbids import of _codecs: 2024-07-18T20:44:39 ERROR ZEO.asyncio.server Can't deserialize message Traceback (most recent call last): File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/server.py", line 100, in message_received message_id, async_, name, args = self.decode(message) File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 119, in pickle_server_decode return unpickler.load() # msgid, flags, name, args File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 176, in server_find_global raise ImportError("Module not allowed: %s" % (module,)) ImportError: Module not allowed: _codecs We will address this in the next patch.
Showing
Please register or sign in to comment