- 09 Jun, 2016 2 commits
-
-
Jim Fulton authored
-
Jim Fulton authored
By resetting self.__onCloseCallbacks before calling the close callbacks to forestall infinite recursion (as can be the case when a close callback is the database close methos).
-
- 07 Jun, 2016 3 commits
-
-
Jim Fulton authored
Changed it to the value corresponding to the maximim *signed* big-endian 64-bit integer, because of LxBTrees. Note that this value isn't used anywhere in ZODB yet. Maybe it should be.
-
Jim Fulton authored
-
Jim Fulton authored
To mitigate errors during tearDown. Lots of tests are sloppy about closing connection (because they could be), but storages tend to get closed and transactions aborted. This has the effedt of calling afterCompletion on connections which tries to call storage lastTransaction, which causes errors.
-
- 06 Jun, 2016 3 commits
-
-
Jim Fulton authored
-
Jim Fulton authored
Fixed: FileStorage loadBefore didn't handle deleted/undone data correctly
-
Jim Fulton authored
Enforce minimum required Python versions: 2.7, 3.3. +1
-
- 05 Jun, 2016 1 commit
-
-
Jim Fulton authored
-
- 31 May, 2016 1 commit
-
-
Jim Fulton authored
Merge pull request #62 from zopefoundation/dont-depend-on-implementation-detail-in-tpc_finish-test-assertions Fixed a test that depended on some off behavior of tpc_finish in some…
-
- 30 May, 2016 1 commit
-
-
Jim Fulton authored
-
- 17 May, 2016 2 commits
-
-
Jim Fulton authored
This and https://github.com/zodb/relstorage/pull/31 make RelStorage pass with this branch (as much as they pass with master).
-
Tres Seaver authored
Closes #59.
-
- 12 May, 2016 1 commit
-
-
Julien Muchembled authored
This answers #13
-
- 09 May, 2016 3 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
- It failed with oids that already exist in the base storage. - MappingStorage returned timestamps in wrong format.
-
Julien Muchembled authored
-
- 05 May, 2016 5 commits
-
-
Jim Fulton authored
-
Jim Fulton authored
-
Jim Fulton authored
-
Jim Fulton authored
On Python 2 and Mac OS X, checkFlushAfterTruncate was failing.
-
Jim Fulton authored
-
- 04 May, 2016 7 commits
-
-
Jim Fulton authored
Even though the previous commit otherwise made MVCCMappingStorage and IMVCCStorage handling better.
-
Jim Fulton authored
The previous commit, made in anger, made a test pass, but wasn't really the right fix. This commit fixes MVCCMappingStorage's loadBefore implementation by fixing handling of the internal _ltid variable so that it's updated during poll_invalidations. This allowed the base class version of loadBefore to be used and, I'm 97% sure has the right semantics. Fixing this revealed a problem with the Connection changes. Fixed Connection.py to poll for invalidations before computing _txn_time by calling lastTransaction, so as to get a current value. We still apply invalidations after computing _txn_time, so that persistent classes can be loaded correctly when they are invalidated. This was accomplished by weaving _flush_invalidations into newTransaction.
-
Jim Fulton authored
Changed to always empty the file pool. I doubt that just flushing on Python 2 was much of an optimization, and ... Python 2. :)
-
Jim Fulton authored
This is needed because pack temporarily deletes data from the data structure that loadBefore reads. Note that in the new scheme of things, we avoid using load, but instead use loadBefore. This we bypass the _data_snapshot shenanigans, which makes me think we aren't testing what we should be, but I have no confidence that the machinery in MVCCMappingStorage actually reflects anything meaningful.
-
Jim Fulton authored
-
Jim Fulton authored
-
Jim Fulton authored
This implements: https://github.com/zopefoundation/ZODB/issues/50 Rather than watching invalidations, simply use 1 + the storages lastTransaction, which is equivalent to but much simpler than waiting for the first invalidation after a transaction starts. More importantly, it means we can always use loadBefore and get away from load. We no longer have to worry about ordering of invalidations and load() results. Much thanks to NEO for pointing the way toward this simplification! Implementing this initially caused a deadlock, because DB.open() called Connection.open() while holding a database lock and Connection.open() now calls IStotage.lastTransaction(), which acquires a storage lock. (It's not clear that lastTransaction() really needs a storage lock.) Meanwhile, IStotage.tpc_finish() calls a DB function that requires the DB lock while holding the storage lock. Fixing this required moving the call to Connection.open() outside the region where the DB lock was held. To debug the problem above, I greatly improved lock-debugging support. Now all of the ZODB code imports Lock, RLock and Condition from ZODB.utils. If the DEBUG_LOCKING is set to a non-empty value, then these are wrapped in such a way that debugging information is printed as they are used. This made spotting the nature of the deadlock easier. Of course, a change this basic broke lots of tests. Most of the breakage arises from the fact that connections now call lastTransaction on storages at transaction boundaries. Lots of tests didn't clean up databases and connections properly. I fixed many tests, but ultimately gave up and added some extra cleanup code that violated transaction-manager underware (and the underware's privates) to clear transaction synchonizers in test setup and tear-down. I plan to add a transaction manager API for this and to use it in a subsequent PR. This tests makes database and connection hygiene a bit more important, especially for tests, because a connection will continue to interact with storages if it isn't properly closed, which can lead to errors if the storage is closed. I chose not to swallow these errors in Connection, choosing rather to clean up tests. The thread debugging and test changes make this PR larger than I would have liked. Apologies in advance to the reviewers.
-
- 29 Apr, 2016 2 commits
-
-
Tres Seaver authored
Incididentally, should make the Py26 winbot happy too, until we can get it disabled.
-
Tres Seaver authored
Fixed test failure introduced in pull-request 51
-
- 28 Apr, 2016 2 commits
-
-
Jim Fulton authored
Fix possible data corruption after FileStorage is truncated to roll back a transaction
-
https://github.com/zopefoundation/ZODB/pull/51Jim Fulton authored
The PR depended on system time being UTC.
-
- 27 Apr, 2016 4 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
Multi-threaded IO support, which is new to ZODB 3.10, allows clients to read data (load & loadBefore) even after tpc_vote has started to write a new transaction to disk. This is done by using different 'file' objects. Issues start when a transaction is rolled back after data has been appended (using the writing file object). Truncating is not enough because the FilePool may have been used concurrently to read the end of the last transaction: file objects have their own read buffers which, in this case, may also contain the beginning of the aborted transaction. So a solution is to invalidate read buffers whenever they may contain wrong data. This patch does it on truncation, which happens rarely enough to not affect performance. We discovered this bug in the following conditions: - ZODB splitted in several FileStorage - many conflicts in the first committed DB, but always resolved - unresolved conflict in another DB If the transaction is replayed with success (no more conflict in the other DB), a subsequent load of the object that could be resolved in the first DB may, for example, return a wrong serial (tid of the aborted transaction) if the layout of the committed transaction matches that of the aborted one. The bug usually manifests with POSKeyError & CorruptedDataError exceptions in ZEO logs, for example while trying to resolve a conflict (and restarting the transaction does not help, causing Site Errors in Zope). But theorically, this could also cause silent corruption or unpickling errors at client side.
-
Julien Muchembled authored
-
Julien Muchembled authored
Cleanup dependencies, remove ZEO leftovers
-
- 25 Apr, 2016 2 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
- remove now useless `zdaemon` dependency - remove ZEO documentation - update dependency list in README
-
- 04 Apr, 2016 1 commit
-
-
Jason Madden authored
patch time.time in historical_connections to avoid windows failures; fix race condition in pool.availableGC * patch time.time in historical_connections to avoid windows failures. * Also use a mock version of datetime.datetime.utcnow. We can't patch the method in-place because it's an extension type. * synchronize sleep with time * increment by less than the pool timeout to avoid a race condition. * Avoid mutating the dict as we iter over it. Seen in Python 3.4 tests: File "/home/travis/build/zopefoundation/ZODB/src/ZODB/historical_connections.txt", line 263, in historical_connections.txt Failed example: conn = db.open() Exception raised: Traceback (most recent call last): File "/opt/python/3.4.2/lib/python3.4/doctest.py", line 1324, in __run compileflags, 1), test.globs) File "<doctest historical_connections.txt[0]>", line 1, in <module> conn = db.open() File "/home/travis/build/zopefoundation/ZODB/src/ZODB/DB.py", line 761, in open self.historical_pool.availableGC() File "/home/travis/build/zopefoundation/ZODB/src/ZODB/DB.py", line 277, in availableGC for key, pool in self.pools.items(): RuntimeError: dictionary changed size during iteration
-