• Kirill Smelkov's avatar
    go/zodb/fs1: Fix index save wrt py3 · 1e2de699
    Kirill Smelkov authored
    We were saving index data with bytestrings inside index records. This
    works for py2 but on py3 it causes failure on unpickling because, by default,
    *STRING opcodes are unpickled into unicode on py3:
    
        === RUN   TestIndexSaveToPy/py2_pickle1/py3
        UnicodeDecodeError: 'ascii' codec can't decode byte 0x8d in position 25: ordinal not in range(128)
    
        The above exception was the direct cause of the following exception:
    
        Traceback (most recent call last):
          File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/fs1/./py/indexcmp", line 43, in <module>
            main()
          File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/fs1/./py/indexcmp", line 30, in main
            d1 = fsIndex.load(path1)
          File "/home/kirr/src/wendelin/z/ZODB/src/ZODB/fsIndex.py", line 138, in load
            v = unpickler.load()
        SystemError: <built-in method read of _io.BufferedReader object at 0x7fb631a86670> returned a result with an error set
            index_test.go:229: zodb/py read/compare index: exit status 1
    
    -> Fix it by explicitly emitting index entries in binary form.
    
    To be able to compare "index from py2 -> loaded into go -> saved by go ->
    compared by py3" implement a bit of backward compatibility for  loading py2
    index files on py3. Do this compatibility only for known-good py2 files, not
    index produced by go code which is loaded by py3 without any compatibility
    shims.
    1e2de699
index.go 18.4 KB