It was broken since review of logging
(commit 1fce5cc4).

_getPartition optimization can't be done at class level and properties hide
instance attribute so implement laziness in __getattr__

This is not a valid place to call app.newTask

The following 3 methods are renamed:
 unlockData -> releaseData
 storeData -> holdData
 _storeData -> storeData

and StorageOperationHandler use the new storeData instead of the old one.

For the moment, this is only useful for debugging.

Transaction manager uses a new getLastObjectTID method instead of
getObjectHistory, because the latter fetches the data and this is
a waste of time here.

Similarly, minimize SQL work in getObject if the object can't be found.

Although it's a warning for developers, it's only informational for users
so lower severity in order not to frighten them.

- Fix case of empty values: there's no reason to do anything special for them.
- Do not warn about multiple level of indirection to get data serial.
  With the current structure of tables, this does not cause significant
  performance issue as it did before.

Export:
- Remove leftover warning about a bug that was fixed in
  commit e76af297
- In neomigrate script, open NEO storage read-only.
- IStorageIteration is already implemented.

Import:
- Review comments.
- In neomigrate script, warn that IStorageRestoreable is not implemented.
- Do not call 'close' method on source iterator. BaseStorage does not do it and
  this is not part of ZODB API. In the case of FileStorage, resource are freed
  automatically during garbage collection.

This is more realistic than testing with a single partition, in particular when
there are more storage nodes that copies.

There is simply no way to guess data serials and instead of producing random
values, the only solution is to retrieve the values from storages.

There are still differences for data serials between FileStorage and NEO:
- NEO always resolves to original serial, to avoid any indirection
  (which slightly speeds up undo at the expense of a more complex pack code)
- NEO does not make any difference between object deletion and creation undone
  (data serial always null in storage)
It has to be decided whether NEO implementation should be changed about this.

Apart from that, conversion database back from NEO should be fixed.
testExportFileStorageBug passes and there was in fact no FileStorage bug.

Another change is that iterator does not trash the client cache anymore.

- _[gs]etPackTID accessors implementation is not backend-specific
  so move them to superclass
- _getObjectLength method is useless since data_tid always contains the wanted
  information, regardless the contents of value_tid column

Sometimes, the tested cluster reacts so quickly that a new primary master arised
before we test that at some point, there is no primary master.

This fixes:

Traceback (most recent call last):
  File "neo/tests/functional/testMaster.py", line 50, in testStoppingSecondaryMaster
    self.neo.expectDead(master)
  File "neo/tests/functional/__init__.py", line 615, in expectDead
    self.expectCondition(callback, *args, **kw)
  File "neo/tests/functional/__init__.py", line 509, in expectCondition
    'History: %s' % opaque_history)
AssertionError: Timeout while expecting condition. History: [False, False, False, False, False, False, False, False, False, False, False]

See commit d9ab77b8

One entry should have been removed before v1.1

If anything wrong happens after a transaction is locked and before the end of
onTransactionCommitted, recovery phase should be run again, so that the master
gets correct last tid.

Following patch by Vincent is an attempt to fix this:

--- a/neo/master/app.py
+++ b/neo/master/app.py
@@ -329,8 +329,8 @@ def playPrimaryRole(self):

         # recover the cluster status at startup
         try:
-            self.runManager(RecoveryManager)
             while True:
+                self.runManager(RecoveryManager)
                 self.runManager(VerificationManager)
                 try:
                     if self.backup_tid:
@@ -338,10 +338,6 @@ def playPrimaryRole(self):
                             raise RuntimeError("No upstream cluster to backup"
                                                " defined in configuration")
                         self.backup_app.provideService()
-                        # Reset connection with storages (and go through a
-                        # recovery phase) when leaving backup mode in order
-                        # to get correct last oid/tid.
-                        self.runManager(RecoveryManager)
                         continue
                     self.provideService()
                 except OperationFailure:

This should following random errors:

>   File "neo/lib/event.py", line 77, in unregister
>     self.epoll.unregister(fd)
> IOError: [Errno 2] No such file or directory

>   File "neo/tests/threaded/test.py", line 670, in testClientReconnection
>     c, = cluster.storage.nm.getClientList()
> ValueError: need more than 0 values to unpack

This implementation proper cache invalidation.

Connection to master is also made optional to load from storage nodes, as long
as partition table contains up-to-date data (which is anyway unlikely to change
when there is no master).

This is enough because on disconnection, the master already aborts all
transactions on its side.