Safe activities are:
- those which are not visible outside current transaction (IOW, queued
  in activity buffer)
- those which are already marked as failed, as no activity node will try
  to process these
Other activities may already be running.
Note: deleting even failed activities may violate activity dependencies
(other activities may be waiting on the ones being deleted, even those
not spawned yet), but if this is an issue then some other code should
take care of these as well.
Also, rework a CMFActivity test checking that activties already-spawned on
a later-deleted context get discarded instead of being executed: previous
implementation was working around the activity deletion removed in this commit.
Also, test indexation/unindexation parallelism.

uids are a way one can signal that different objects (from a ZODB point of
view, and hence an oid point of view) are actually to be considered as the
same objet (from a higher abstration point of view). For example, another
variant of the same object, as imported over an older variant.
In turn, this allows extending the protection to activities spawned from
brains, and not just from live objects.

Vincent Pelletier:
- Disable timeout by default.

Simplifies activity spawning when dependencies are conditionally set:
  activate(
    after_tag=some_value if some_condition else None,
  )
instead of having to do a ** dance.
All columns involved in dependency checking are declared NOT NULL, so providing
"None" and expecting a dependency to happen was already not working.
This change pushes this one step further by allowing activity
auto-validation also happen on these activities.
Also, simplify getOrderValidationText: avoid iterating on keys and then
retrieving values, use a list-comprehension, simplify condition.
Add a test for after_tag, also covering activity auto-validation.

Add 10 seconds timeouts on all blocking calls in this test method, which
should be a thousand times more than required on a slow machine but avoids
completely blocking tests.

The intent is to be able to tell that an independently-defined group of
activity nodes may execute given activity, and no other node.
This allows more flexible parallelism control than serialization_tag.

With recent changes in CMFActivity, getServerAddress is now used
when setting up the site, before the ZServer is started, which means:
- getServerAddress memoized a wrong return value;
- the test didn't fail with --load.

The goal is to make better use of the ZODB Storage cache. It is common to do
processing on a data set in several sequential transactions: in such case, by
continuing execution of these messages on the same node, data is loaded from
ZODB only once. Without this, and if there are many other messages to process,
processing always continue on a random node, causing much more load from ZODB.

To prevent nodes from having too much work to do, or too little compared to
other nodes, this new parameter is only a hint for CMFActivity. It remains
possible for a node to execute a message that was intended for another node.

Before this commit, a processing node selects the first message(s) according to
the following ordering:

  priority, date

and now:

  priority, node_preference, date

where node_preference is:

  -1 -> same node
   0 -> no preferred node
   1 -> another node

The implementation is tricky for 2 reasons:
- MariaDB can't order this way in a single simple query, so we have 1
  subquery for each case, potentially getting 3 times the wanted maximum of
  messages, then order/filter on the resulting union.
- MariaDB also can't filter efficiently messages for other nodes, so the 3rd
  subquery returns messages for any node, potentially duplicating results from
  the first 2 subqueries. This works because they'll be ordered last.
  Unfortunately, this requires extra indices.

In any case, message reservation must be very efficient, or MariaDB deadlocks
quickly happen, and locking an activity table during reservation reduces
parallelism too much.

In addition to better cache efficiency, this new feature can be used as a
workaround for a bug affecting serialiation_tag, causing IntegrityError when
reindexing many new objects. If you have 2 recursive reindexations for both a
document and one of its lines, and if you have so many messages than grouping
is split between these 2 messages, then you end up with 2 nodes indexing the
same line in parallel: for some tables, the pattern DELETE+INSERT conflicts
since InnoDB does not take any lock when deleting a non-existent row.

If you have many activities creating such documents, you can combine with
grouping and appropriate priority to make sure that such pair of messages won't
be executed on different nodes, except maybe at the end (when there's no
document to create anymore; then activity reexecution may be enough).
For example:

  from Products.CMFActivity.ActivityTool import getCurrentNode
  portal.setPlacelessDefaultReindexParameters(
    activate_kw={'node': 'same', 'priority': priority},
    group_id=getCurrentNode())

where `priority` is the same as the activity containing the above code, which
can also use grouping without increasing the probability of IntegrityError.

This fixes the issue that a transaction with many big messages failed to
commit. By dynamically find the maximum allowed size of a query, it also
speeds up insertion by minimizing the number of queries.

The original goal was to improve performance by removing the
`processing_node_processing` index and the queries that modified
these 2 useless columns.

- getCurrentNode method can be directly imported from CMFActivity
- corresponding unittests

/reviewed-on nexedi/erp5!647

Duplicate message detection is not good enough: different messages with
the same unicity value may bear different serialization_tags. This code
does not takes this into account, which can lead to deleting such tagged
message and validate an untagged one, which breaks serialization_tag
contract of preventing any further activity validation until execution
of all such-tagged validated activities is successful.
Also, it is not validation's node job to deduplicate: it can happen during
message execution without slowing down this crucial (performance-wise)
activity node.
As a result, distribute methods of SQLDict and SQLQueue can be factorised.

This column is not a significant condition for this test. It is an
unreliable transient piece of information intended for a human observer
monitoring activity execution. processing_node is the reliable piece of
information this test should care about.

The main change here is the reduction of try..except scope, so failing the
first assertion does not cause an AttributeError about original_query.
Also, avoid code duplication by just checking final number of messages
outside the try..except block.
Also, improve coding style (spaces around operators and commas).
Also, avoid single-use variable.
Also, fix typo in comment.

Instead of using obsolete processing_node parameter, use invalid_onlym
with True, in order to check if there are activities with error.

As INVOKE_ERROR_STATE is needed only in SQLBase, move it there and so drop it
from ActiveObject and ActiveProcess.

hasErrorActivity is tested on ActiveObject and ActiveProcess classes.

The recent API change was not enough.
A grouping method may need more information: in particular, the dummy grouping
method must be fixed to change user.

When grouped messages fail, ActivityTool must distinguish 3 groups,
in order to reexecute them separately, as follows:
- first, those that succeeded
- then, those that were skipped
- at last, failed ones

Grouping methods are updated to handle partial failures, and stop doing
anything when something goes wrong.

Without this, we would have the following pathological cases.

1. Let's suppose first that skipped messages are marked as succeeded.

The problem is that each skipped message that will fail causes the reexecution
of those that didn't fail.

Exemple: A:ok B:ok C:err D:err E:err F:err
  1: A:ok, B:ok, C:err, D:skipped, E:skipped, F:skipped
  2: A:ok, B:ok, D:err, E:skipped, F:skipped
  3: A:ok, B:ok, E:err, F:skipped
  4: A:ok, B:ok, F:err
  5: A:ok, B:ok -> commit

And worst, the first failed (C) may be processable again before 5, entering
a failing loop if it is executed again in the same group as A & B.

2. Another implementation is to mark all skipped as failed.

Example:
  1: A:ok, B:ok, C:err, D:skipped, E:skipped, F:skipped
  2: A:ok, B:ok -> commit
  3: C:err, D:skipped, E:skipped, F:skipped
 >3: same as 3

=> D, E or F are never tried.

For even more refactoring between SQLDict & SQLQueue,
which now uses SQL tables with the same schema.

The action to recreate activity tables while preserving existing messages
was unsafe for 2 reasons:
- if any error happened, messages could be lost
- it relied on Message.reactivate

Which this patch, any instance created after commit d881edd1 (Aug 2010) will
upgrade successfully. For older instances, make sure you have no activity left.

For cases where 'ALTER TABLE' would not work, a better way to implement repair
functionality would be:
- one action to backup all messages in ZODB
- and another to restore them
And maybe a security so that during the backup-clear-restore sequence,
activities can't be created nor processed.

If any column is added in the future, it would still be possible to write code
that fills them by inspecting messages.

see http://docs.python.org/2/library/unittest.html#deprecated-aliases

CMFActivity: Drop messages when document is removed but another document can be acquired with the same path.
Signed-off-by: Julien Muchembled <jm@nexedi.com>