- catch OperationFailure
- reset transaction manager when leaving backup mode
- send appropriate target tid to a storage that updates a outdated cell
- clean up partition table when leaving BACKINGUP state unexpectedly
- make sure all readable cells of a partition have the same 'backup_tid'
  if they have the same data, so that we know when internal replication is
  finished when leaving backup mode
- fix storage not finished internal replication when leaving backup mode

This includes an API change of Node.isIdentified, which now tells whether
identification packets have been exchanged or not.
All handlers must be updated to implement '_acceptIdentification' instead of
overriding EventHandler.acceptIdentification: this patch only does it for
StorageOperationHandler

Replication is also fully reimplemented:
- It is not done anymore on whole partitions.
- It runs at lowest priority not to degrades performance for client nodes.

Schema of MySQL table is changed to optimize storage layout: rows are now
grouped by age, for good partial replication performance.
This certainly also speeds up simple loads/stores.

- Storage nodes start to replicate a partition when all transactions that were
pending when the oudated partition was added are committed.
- Transactions are registered by the master from the tpc_begin step.
Signed-off-by: Grégory <gregory@nexedi.com>

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2649 71dcc9de-d417-0410-9af5-da40c76e7ee4

In order to prepare the eggification of the different neo parts, we created a
new neo/lib module, containing all of the main neo's functions. It allows to
make neo a virtual namespace, and so not containing module code anymore.




git-svn-id: https://svn.erp5.org/repos/neo/trunk@2615 71dcc9de-d417-0410-9af5-da40c76e7ee4

- AnswerInformationLocked give ttid instead of tid
- Master transaction manager always use ttid in data structures
- It's no more makes sense to check if the tid is greater than the last
generated as it never comes back from a storage, just check if the ttid is
well known by the transaction manager.
- Rename all tid variable that now hold a ttid
- Transaction manager's queue contains ttids but the corresponding tids are
increasing to keep commit order.
- Adjust tests

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2613 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2563 71dcc9de-d417-0410-9af5-da40c76e7ee4

This allows parallel execution of tpc_begin, stores & related conflict
resolution and tpc_vote for different transactions.
This requires an extension to ZODB allowing to keep TID secret until
tpc_finish (ie, so that it doesn't require tpc_vote to return tid for each
stored object).

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2534 71dcc9de-d417-0410-9af5-da40c76e7ee4

This ensures invalidations are sent in strict ascending TID values order.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2530 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2478 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2427 71dcc9de-d417-0410-9af5-da40c76e7ee4

A storage might notify the master after the cluster as fall down to the
verification state.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2426 71dcc9de-d417-0410-9af5-da40c76e7ee4

Some requests can be safely ignored when received over a closed connection.
This was previously done explicitly in handlers, but it turns out it would
cause a lot of code duplication. Instead, define the policy on a packet
type basis, and apply it to all packets upon reception, before passing it
to handler.
Also, protect request handlers when they respond, as connection might be
closed.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2419 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2374 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2371 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2366 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2365 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2287 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2239 71dcc9de-d417-0410-9af5-da40c76e7ee4

Master transactions objects are instanciated during the finish phase only,
which means that any transaction known by the master is being committed.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2237 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2184 71dcc9de-d417-0410-9af5-da40c76e7ee4

- The storages no more check the last OID during a store
- The storages inconditionnaly store the last OID notified by the master
- The master check during the if a greater oid was used by a client
- The master always notify the last OID when a pool is generated or if the
check above is True
- The master's transaction manager manager the last oid and oid generator

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2180 71dcc9de-d417-0410-9af5-da40c76e7ee4

If the last storage node gets disconnected, an operation failure will
occur, so there is no risk of sending AnswerTransactionFinished to client
if some object was not stored.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2150 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2101 71dcc9de-d417-0410-9af5-da40c76e7ee4

Between closure request and effect, a client node can be registered to the
node manager without a connection. As there is no needs to use an
unconnected client node, always set only_identified parameter to True.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2098 71dcc9de-d417-0410-9af5-da40c76e7ee4

This fix a race condition when switching from recovery to verification
steps where NotifyLastOID packet could be handled by verification handler
instead of initialisation (storage side).

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1983 71dcc9de-d417-0410-9af5-da40c76e7ee4

This should improve scalability when the number of non-client nodes grows.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1868 71dcc9de-d417-0410-9af5-da40c76e7ee4

Singeltons can not be used in NEO because client side process may deals with
multiple NEO cluster. Add a callback on connection when linked with a node to
update node's connection property at connection closure.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1826 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1819 71dcc9de-d417-0410-9af5-da40c76e7ee4

Remove the binding at connection closure.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1817 71dcc9de-d417-0410-9af5-da40c76e7ee4