----

kirr: This is likely to be ZODB3-backport of https://github.com/zopefoundation/ZODB/pull/69
Taken from https://lab.nexedi.com/nexedi/neoppod/blob/v1.12-52-gfb746e6b/ZODB3.patch

People say that ZODB3 is kind of closed:

	https://github.com/zopefoundation/ZODB/pull/306#issuecomment-606735225

Givent that even for ZEO4 we were suggested that 4 is considered "dead" by upstream:

	https://github.com/zopefoundation/ZEO/pull/161#pullrequestreview-447245642

Let's keep on patching ZODB3 in our local fork until we keep on using it.

Changes are backported from commits 227953b9
and 03a326be.

( This is backport of https://github.com/zopefoundation/persistent/pull/44
  to ZODB-3.10 )

On ._p_deactivate() and ._p_invalidate(), when an object goes to ghost
state, objects referenced by all its attributes, except related to
persistence machinery, are released, this way freeing memory (if they
were referenced only from going-to-ghost object).

That's the idea - an object in ghost state is simply a stub, which loads
its content on first access (via hooking into get/set attr) while
occupying minimal memory in not-yet-loaded state.

However the above is not completely true right now, as currently on
ghostification only object's .__dict__ is released, while in-slots objects
are retained attached to ghost object staying in RAM:

    ---- 8< ----
    from ZODB import DB
    from persistent import Persistent
    import gc

    db = DB(None)
    jar = db.open()

    class C:
        def __init__(self, v):
            self.v = v
        def __del__(self):
            print 'released (%s)' % self.v

    class P1(Persistent):
        pass

    class P2(Persistent):
        __slots__ = ('aaa')

    p1 = P1()
    jar.add(p1)
    p1.aaa = C(1)

    p2 = P2()
    jar.add(p2)
    p2.aaa = C(2)

    p1._p_invalidate()
    # "released (1)" is printed

    p2._p_invalidate()
    gc.collect()
    # "released (2)" is NOT printed     <--
    ---- 8< ----

So teach ghostify() & friends to release objects in slots to free-up
memory when an object goes to ghost state.

NOTE PyErr_Occurred() added after ghostify() calls because
pickle_slotnames() can raise an error, but we do not want to change
ghostify() prototype for backward compatibility reason - as it is used
in cPersistenceCAPIstruct.

( I hit this bug with wendelin.core which uses proxies to load
  data from DB to virtual memory manager and then deactivate proxy right
  after load has been completed:

  https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L239
  https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L295 )

This reverts to the behaviour of 3.10.3 and older.

(cherry picked from commit b74eef76)

Conflicts:
	src/ZODB/tests/testconflictresolution.py

Multi-threaded IO support, which is new to ZODB 3.10, allows clients to read
data (load & loadBefore) even after tpc_vote has started to write a new
transaction to disk. This is done by using different 'file' objects.

Issues start when a transaction is rolled back after data has been appended
(using the writing file object). Truncating is not enough because the FilePool
may have been used concurrently to read the end of the last transaction:
file objects have their own read buffers which, in this case, may also contain
the beginning of the aborted transaction.

So a solution is to invalidate read buffers whenever they may contain wrong
data. This patch does it on truncation, which happens rarely enough to not
affect performance.

We discovered this bug in the following conditions:
- ZODB splitted in several FileStorage
- many conflicts in the first committed DB, but always resolved
- unresolved conflict in another DB
If the transaction is replayed with success (no more conflict in the other DB),
a subsequent load of the object that could be resolved in the first DB may, for
example, return a wrong serial (tid of the aborted transaction) if the layout
of the committed transaction matches that of the aborted one.

The bug usually manifests with POSKeyError & CorruptedDataError exceptions in
ZEO logs, for example while trying to resolve a conflict (and restarting the
transaction does not help, causing Site Errors in Zope). But theorically,
this could also cause silent corruption or unpickling errors at client side.

(cherry picked from commit 028b1922)

Conflicts:
	src/ZODB/FileStorage/FileStorage.py

Fixing uncaught exception problem on shutdown

Nested connections try to clean up themselves multiple
times. This does not work

see https://github.com/zopefoundation/ZODB/issues/16

see https://github.com/zopefoundation/ZODB/issues/16

when objects had been added in readCurrent

still need to come up with a test