-
Jérome Perrin authored
With upcoming ZODB 5, oids (used as persistent references in pickles) are no longer str as it use to be with ZODB 4, but instances of zodbpickle.binary, which with zodbpickle 1 are a subclass of str on python2. OrderedPickler was a subclass of pickle.Pickler, the pickler from standard library, but this pickler was not able to use a str subclass for persistent references, when pickles are loaded with noload method, persistent_load is called with `None` instead of the actual string subclass instance. This was problematic in the XMLExportImport handling of business templates, because ZODB.serialize.referencesf was unable to find persistent references. The error was: ZODB-5.6.0-py2.7.egg/ZODB/serialize.py", line 664, in referencesf assert isinstance(reference, list) AssertionError because the reference was None. zodbpickle 2 changed to make zodbpickle.binary implemented in C, which was failing earlier, because pickle.Pickle can not pickle these objects, failing in an error like this: lib/python2.7/copy_reg.py", line 70, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle binary objects This change also simplify our own implementation, by dropping jython support and calling save_dict on the super class instead of copying the implementation. Further references: - minimal script to reproduce the issues: ```python from __future__ import print_function import io import pickle import zodbpickle import zodbpickle.pickle import zodbpickle.fastpickle class ExternalObject(object): def __init__(self, oid): self.oid = oid def persistent_id(obj): if isinstance(obj, ExternalObject): return obj.oid def persistent_load(persid): print('persistent_load called with persid', repr(persid)) o = ExternalObject(oid=zodbpickle.binary("binary persid")) for pickler_class in pickle.Pickler, zodbpickle.pickle.Pickler: f = io.BytesIO() p = pickler_class(f, 1) p.persistent_id = persistent_id p.dump(o) print('dump with pickler %s:\n %r' % (pickler_class, f.getvalue())) # ZODB uses this unpickler up = zodbpickle.fastpickle.Unpickler(io.BytesIO(f.getvalue())) up.persistent_load = persistent_load up.noload() ``` ```console $ python2 repro.py # with zodbpickle 1 dump with pickler pickle.Pickler: 'ccopy_reg\n_reconstructor\nq\x00(czodbpickle\nbinary\nq\x01c__builtin__\nstr\nq\x02U\rbinary persidq\x03tq\x04Rq\x05Q.' persistent_load called with persid None dump with pickler zodbpickle.pickle_2.Pickler: 'U\rbinary persidq\x00Q.' persistent_load called with persid 'binary persid' ``` ```console $ python2 repro.py # with zodbpickle 2 Traceback (most recent call last): File "repro.py", line 45, in <module> p.dump(o) File ".../lib/python2.7/pickle.py", line 224, in dump self.save(obj) File ".../lib/python2.7/pickle.py", line 273, in save self.save_pers(pid) File ".../lib/python2.7/pickle.py", line 340, in save_pers self.save(pid) File ".../lib/python2.7/pickle.py", line 306, in save rv = reduce(self.proto) File ".../lib/python2.7/copy_reg.py", line 70, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle binary objects ``` * ZODB change starting to use zodbpickle.binary instead of str: 12ee41c4 (-ZODB now uses pickle protocol 3 for both Python 2 and Python 3., 2018-03-26) Since of 5.4.0 release * zodbpickle change starting to use C objects for zodbpickle.binary: bbef98c (Implement zodbpickle.binary in C for Py27., 2019-11-12) Since of 2.0.0 release
8310bf30