• Kirill Smelkov's avatar
    Allow to hook-in application-level logic to handle persistent references (#38) · d51cc146
    Kirill Smelkov authored
    Continuing 84598fe1 (Add support for persistent references) theme let's
    develop support for persistent references more: the usual situation (at
    least in ZODB context, which is actually the original reason
    persistent references exist) is that decode converts a reference of
    (type, oid) tuple form into corresponding "ghost" object - an instance
    of type, but not yet loaded with database object's data. This way
    resulted object tree is created right away with types application
    expects (and ZODB/py further cares to automatically load object data
    when that ghost objects are accessed).
    
    To mimic this on Go side, with current state, after decoding, one would
    need to make a full reflect pass on the resulted decoded objects, find
    Refs, and convert them to instances of other types, also caring to
    patch pointers in other objects consistently that were pointing to such
    Refs. In other words it is some work and it coincides almost 100% with
    what Decoder already does by itself.
    
    Thus, not to duplicate that work and conducting parallel with Python
    world, let's add ability to configure Decoder and Encoder so that
    persistent references could be handled with user-specified application
    logic right in the process, and e.g. Decoder would return resulted
    object with references converted to corresponding Go types.
    
    To do so we introduce DecoderConfig and EncoderConfig - configurations
    to tune decode/encode process. To maintain backward compatibility
    NewDecoder and NewEncoder signatures are not adjusted and instead
    NewDecoderWithConfig and NewEncoderWithConfig are introduces. We should
    be sure we won't need e.g. NewDecoderWithConfigAndXXX in the future,
    since from now on we could be adding fields to configuration structs
    without breaking backward compatibility.
    
    For decoding there is DecoderConfig.PersistentLoad which mimics
    Unpickler.persistent_load in Python:
    
    https://docs.python.org/3/library/pickle.html#pickle.Unpickler.persistent_load
    
    For encoding there is EncoderConfig.PersistentRef which mimics
    Pickler.persistent_id in Python:
    
    https://docs.python.org/3/library/pickle.html#pickle.Pickler.persistent_id
    
    ( for Persistent{Load,Ref}, following suggestion from @kisielk, the
      choice was made to explicitly pass in/out Ref, instead of raw
      interface{} pid, because that makes the API cleaner. )
    
    Then both Decoder and Encoder are adjusted correspondingly with tests
    added.
    
    About tests: I was contemplating to patch-in support for decoder and
    encoder configurations, and handling errors, into our main test
    TestDecode, but for now decided not to go that way and to put the test
    as separate TestPersistentRefs.
    
    By the way, with having configurations in place, we could start to add
    other things there - e.g. the protocol version to use for Encoder.
    Currently we have several places where we either don't use opcodes from
    higher protocol versions (fearing to break compatibility), or instead
    always use higher-version opcodes without checking we should be able to
    do so by allowed protocol version.
    d51cc146
encode.go 8.48 KB