.

zrace.py

 #!/usr/bin/env python
-"""Program zopenrace.py demonstrates race-condition bug in ZODB
-Connection.open() that leads to stale live cache and wrong data provided by
-database to users.
+"""Program zopenrace.py demonstrates concurrency bug in ZODB Connection.open()
+that leads to stale live cache and wrong data provided by database to users.
 
+The bug is that when a connection is opened, it syncs to storage and processes
+invalidations received from the storage in two _separate_ steps, potentially
+leading to situation where invalidations for transactions _past_ opened
+connection's view of the database are included into opened connection's cache
+invalidation. This leads to stale connection cache and old data provided by
+ZODB.Connection when it is reopened next time.
 
-newTransaction
+That in turn can lead to loose of Consistency of the database if mix of current
+and old data is used to process a transaction. A classic example would be bank
+accounts A, B and C with A<-B and A<-C transfer transactions. If transaction
+that handles A<-C sees stale data for A when starting its processing, it
+results in A loosing what it should have received from B.
+
+Below is timing diagram on how the bug happens on ZODB5:
+
+    Client1 or Thread1                                  Client2 or Thread2
+
+    # T1 begins transaction and opens zodb connection
+    newTransaction():
+        # implementation in Connection.py[1]
         ._storage.sync()
         invalidated = ._storage.poll_invalidations():
+            # implementation in MVCCAdapterInstance [2]
 
+            # T1 settles on as of which particular database state it will be
+            # viewing the database.
             ._storage._start = ._storage._storage.lastTrasaction() + 1:
                 s = ._storage._storage
                 s._lock.acquire()
                 head = s._ltid
                 s._lock.release()
                 return head
-                                               XXX T2 commits here: objX
-
+                                                        # T2 commits here.
+                                                        # Time goes by and storage server sends
+                                                        # corresponding invalidation message to T1,
+                                                        # which T1 queues in its _storage._invalidations
+
+            # T1 retrieves queued invalidations which _includes_
+            # invalidation for transaction that T2 just has committed,
+            # that goes past @head.
             ._storage._lock.acquire()
-           r = _storage._invalidations        ; T1 receives invalidations for some transactions after head
+                r = _storage._invalidations
             ._storage._lock.release()
             return r
 
-   # T1 processes invalidates for [... head] _and_ some next transactions.
-   # T1 thus will _not_ process invalidations for those next transactions when
+        # T1 processes invalidations for [... head] _and_ invalidations for past-@head transaction.
+        # T1 thus will _not_ process invalidations for that next transaction when
         # opening zconn _next_ time. The next opened zconn will thus see _stale_ data.
         ._cache.invalidate(invalidated)
+
+
+[1] https://github.com/zopefoundation/ZODB/blob/5.5.1-29-g0b3db5aee/src/ZODB/Connection.py#L734-L742
+[2] https://github.com/zopefoundation/ZODB/blob/5.5.1-29-g0b3db5aee/src/ZODB/mvccadapter.py#L130-L139
 """
 
 from ZODB import DB