Python scripts can be used as catalog methods, including as "clear catalog"
methods. This was causing error when upgradeSchema was called:

    product/ERP5Catalog/CatalogTool.py", line 1372, in upgradeSchema
      method_list_by_connection_id[method.connection_id].append(method)
    AttributeError: connection_id

Safe activities are:
- those which are not visible outside current transaction (IOW, queued
  in activity buffer)
- those which are already marked as failed, as no activity node will try
  to process these
Other activities may already be running.
Note: deleting even failed activities may violate activity dependencies
(other activities may be waiting on the ones being deleted, even those
not spawned yet), but if this is an issue then some other code should
take care of these as well.
Also, rework a CMFActivity test checking that activties already-spawned on
a later-deleted context get discarded instead of being executed: previous
implementation was working around the activity deletion removed in this commit.
Also, test indexation/unindexation parallelism.

During indexation we don't apply security checks, so this should be a
little bit faster.

With this change _getProperty becomes a more "official" API, so some
small changes had to be made to classes which do not inherits from
Products.ERP5Type.Base.Base, so that they also implements _getProperty:
 - for ERP5Site we simply use getProperty
 - for the test class from testERP5Catalog, the change is a bit more
important, because this class never defined getProperty, so during that
test we were just acquiring a getProperty from portal.

This is needed to support datawarehouse on another database.

This is called when checking access permission on objects, which happens
very often. CachingMethod has a hit cost which is too high for this use.
Instead, generate this method as part of the portal type class, removing
all call-time logic.

This fixes:

  Traceback (innermost last):
    Module Products.CMFActivity.ActivityTool, line 1373, in invokeGroup
      traverse(method_id)(expanded_object_list)
    Module Products.ERP5Catalog.CatalogTool, line 946, in catalogObjectList
      super(CatalogTool, self).catalogObjectList(tmp_object_list, **m.kw)
    Module Products.ZSQLCatalog.ZSQLCatalog, line 813, in catalogObjectList
      **kw
  TypeError: catalogObjectList() got an unexpected keyword argument 'group_id'

Before this change, `index_html` for Catalog Tool was None that means trying to
access view for CatalogTool from UI always make a function call to the
` __call__` which function was overridden with `searchResults`, which
means everytime someone tried to access `portal_catalog/`, a useless query was
being made and an undesirable query result was displyed on the UI.

In this case, `index_html` is prefered whenever we try to access portal_catalog
from UI.

Also, add test for proper display of view when CatalogTool is accessed from XHTML UI

/reviewed-on nexedi/erp5!793

This allows breaking a circular dependency of catalog on itself: to
index a local role, catalog must tell whether it's granted to a group or
to a user, which used to happen using a catalog lookup (via PAS API),
preventing catalog from being rebuilt from scratch.

With this change, ERP5Site_filterUserIdSet must be implemented when
custom role columns are used (viewable_owner is not concerned by this
change).
If it is implemented in a way which does not rely on catalog,
catalog become rebuildable from a lone ZODB, without double-indexation
of users nor security_uid explosion.
Nothing prevents relying on catalog (which mayu be desired for
backward-compatibility purposes), but such script will not be provided
by generic ERP5, as the impossibility of recreating catalog from ZODB
is considered a bug, and introducing such script amounts to introducing
this bug.

Create ERP5Site_filterUserIdList in tests which add custom viewable_*
columns.

Use Owner role to help filling user_set: owners must be users, never
groups.
Only look for users among group_ids when they are candidate for indexation
into a catalog_role column. This should very significantly reduce the
number of user lookups, even bringing it to zero on instance with default
catalog setting of catalog_role_set = set(["viewable_owner"]) .
Also, only look for users if there are candidates for lookup.
Also, rename local variable to better describe its content.
Also, use set.__iadd__ instead of set.update (shorter).

To not receive catalog already tainted by clearing & reindexing.

In turn, this allows testing catalog reference content for equality:
catalog must be exactly the same before/after an ERP5Site_reindexAll, and
before/after a hot-reindex. So issubset is not an acceptable method.
Instead, use assertItemsEqual, and raise max diff size.

Also, tidy code a bit:
- Avoid clearing catalog only to restrict result set.
- Improve coding style.
- Simplify code (single-use variables...).
- Call assert methods producing more helpful error messages on failure.

Before this change, expression_instance would not be set when expression is
modified, leading to either non-applied filter expression (when no
expression_instance existed) or use of an outdated expression.
Use a volatile attribute to store the expression: it does not change the
amount of work needed to load the Expression instance (pickle is the raw
python expression as text) and it remove data duplication in exported
documents.

Catalog used in this test is now close/above 1k rows, so go one API level
down to get control over that pesky "limit 1000".

These tests do not test what they advertise anymore. Advertised features
are in widespread use in other tests, so there should be no need to test
specifically for these.
Also, avoids polluting other tests by clearing catalog for futile reasons.

ZSQLCatalog/ERP5Catalog is only supposed to work on catalog table, by
definition.

Tic'ing mid-hot-reindex is not possible, so immediate reindexation is
not avoidable here. Also, immediate indexation of just-created document
is acceptable, so this is not exactly a bad example.

Also, add an XXX about manage_test annoying behaviour.

These methods must not be called synchronously:
- they can break catalog by not being careful enough about other
  reindexations which may happen in parallel. See the serialization_tag
  mechanism for more.
- indexation gets executed in the security context of the user causing the
  call, which may lead to an indexation result different from what happens
  when indexation happens with an all-accesses user.
These lines of code (some even commented-out) give a bad example. Replace
them with safe equivalents.

Move from SQLCatalog to ERP5Catalog as the default Catalog inside ERP5.
The major difference is use of Products.ERP5Type.Core.Folder as Catalog
base class.

Significant addition/changes in
-------------------------------

  ERP5Catalog class:
    Inherit from Catalog class from Products.ZSQLCatalog.SQLCatalog instead of copy-pasting the whole code again.
    Monkey patch some property setters and getters to maintain consistency
    Override getCatalogMethodIds cause it uses global variable in SQLCatalog.Catalog
    Add FilterDict and Filter class to have consistency with `filter_dict` attribute of SQLCatalog

  BusinessTemplate:
    Update BusinessTemplate installation with updated filter_dict
    Also, use dynamic migration while installing the catalog method objects for
    bt5. This way we have SQL Methods migrated just after installation.

  Tests:
    Update tests according to changes in portal_catalog

  SQLCatalog, testZSQLCatalog:
    Cleanup for unusable functions

These are the easy uses, there are more to come.

* to quickly setup catalog_full_text table, you can use the following SQL.

  REPLACE INTO catalog_full_text SELECT uid, title, description FROM catalog;

* non fulltext queries like '=abc', '>abc', '%abc%' are supported.

* now erp5_full_text_mroonga_catalog is used for unit tests thus I recommend using it instead of erp5_full_text_myisam_catalog.

* to migrate existing MyISAM full_text table into Mroonga, you can use the following SQL.

  ALTER TABLE full_text DROP KEY SearchableText,
    ENGINE = mroonga,
    ADD FULLTEXT KEY SearchableText (`SearchableText`) COMMENT 'parser "TokenBigramSplitSymbolAlpha"';

* fulltext search score is no longer provided as (column_name) but now provided as (column_name)__score__.

* (category)_title, like source_title, related keys are automatically generated. (category)_description keys as well.