• Barry Warsaw's avatar
    A new algorithm for packing which seems much more straightforward. · 27f80e8e
    Barry Warsaw authored
    Here's how it works:
    
    - On every store(), we write an entry to a objrev table containing the
      tuple of information (newserial, oid, oldserial).  We don't write
      this entry if the store is the first revision of an object on a new
      version.
    
      We do basically the same thing on restore() and transactionalUndo().
    
    - On an abortVersion(), we write two entries to the objrev table, one
      that has (newserial, oid, oldserial) -- which points to the old
      serial in the version, and (newserial, oid, nvserial) -- which
      points to the non-version revision of the version revision.
    
    - On commitVersion(), we do the same as abortVersion() except that we
      don't write the non-version data if we're committing to a different
      version.
    
    - Now, when we pack, all we need to do is cruise from the beginning of
      the objrev table until we find an entry with a newserial > packtime.
      If the oldserial is ZERO, it's an object creation event which we
      don't need to worry about because there's no previous revision.  But
      otherwise, we can delete the oid+oldserial revision because we know
      it's not current.  We do this, updating pickle refcounts and then
      collecting any objects that are left unreferenced.
    
    The cute thing is that autopacking will use the same algorithm.  The
    main difference between autopack and classic pack, is that the latter
    does a mark and sweep garbage collection phase after the normal objrev
    collection phase.  Also, this algorithm means autopack needs only
    three pieces of information:
    
    - How often the thread should run (e.g. once per hour)
    
    - How far in the past it should pack (e.g. pack to 4 hours ago).  We
      don't need a start time for the autopack window, because we'll
      always just start at the beginning of the objrev table.
    
    - How often should autopack also do a classic pack (e.g. do a classic
      pack once per day).
    
    Autopack isn't implemented in this checkin, but I believe it will be
    nearly trivial to add.  That comes next.
    27f80e8e
BDBFullStorage.py 77.3 KB