Also, make sure that Listbox always knows how many lines are to be displayed.

Commit adds also testPagerWithCustomListMethod in listbox_zuite of erp5_ui_test,
which checks that listbox jumping to previous/next/first/last works
correctly in displaying the elements

predecessor, successor and trade_phase ListBox fields didn't have their
own fields and thus only the title was displayed (eg 'Ordered' for
'Manufacturing/Ordered').

This allows breaking a circular dependency of catalog on itself: to
index a local role, catalog must tell whether it's granted to a group or
to a user, which used to happen using a catalog lookup (via PAS API),
preventing catalog from being rebuilt from scratch.

With this change, ERP5Site_filterUserIdSet must be implemented when
custom role columns are used (viewable_owner is not concerned by this
change).
If it is implemented in a way which does not rely on catalog,
catalog become rebuildable from a lone ZODB, without double-indexation
of users nor security_uid explosion.
Nothing prevents relying on catalog (which mayu be desired for
backward-compatibility purposes), but such script will not be provided
by generic ERP5, as the impossibility of recreating catalog from ZODB
is considered a bug, and introducing such script amounts to introducing
this bug.

Create ERP5Site_filterUserIdList in tests which add custom viewable_*
columns.

Use Owner role to help filling user_set: owners must be users, never
groups.
Only look for users among group_ids when they are candidate for indexation
into a catalog_role column. This should very significantly reduce the
number of user lookups, even bringing it to zero on instance with default
catalog setting of catalog_role_set = set(["viewable_owner"]) .
Also, only look for users if there are candidates for lookup.
Also, rename local variable to better describe its content.
Also, use set.__iadd__ instead of set.update (shorter).

Test loop invariant outside of loop.
Use clearer and more consistent variable names.
Avoid iterating twice when once is enough, allowing removal of
"local_role_dict" local and simplification of corresponding dict
structure.
Avoid creating new instances when the existing one can be mutated.
Evaluate dict as boolean instead of constructing an empty dict to compare
against.
Avoid using %-operator just to concatenate strings (allows en-passant
typechecking).
Drop unused locals.
Do not use a dict when a set would be sufficient.

So that it can be reused outside of group manager without having to
duplicate the backward-compatibility order.
Also, simplify code.

Move variable initialisations closer to where they are actually used.
Do not initialise a variable which will always be initilised again before
its first use.
Reduce argument indentation.
Drop unhelpful comments.
User document may not be a Person instance, use a more neutral variable
name.
Avoid single-use locals.
Take advantage of zLOG.LOG's "error=True" idiom to not have to pull
exception information ourselves.

Speed up JSON calculation by dropping not needed info for rendering.

Render a listfield to select the relation field listboxes on the 'relation_form' page template.

This remove the requirement to add an action for every form.

No need to keep many references on the gadget itself.

Javascript can calculate the length of an array...

Reduce not needed DOM modification.

Stop writing in the DOM when changing the template value.
This keeps the focus on the select.

Fixup nexedi/erp5@bf57228a

This reduces the number of logs.

When editing components, this warning is displayed on the console:
```
Ambiguous name for method of Products.ERP5.Document.SQLMethod.SQLMethod: 'manage_main' != 'manage'
```
fix by adding the same `_setName` that the [parent class does](https://github.com/zopefoundation/Products.ZSQLMethods/blob/2.13/src/Products/ZSQLMethods/SQL.py#L143).


While looking a this, I realized that clicking on the breadcrumb in the ZMI, any part of

![image](/uploads/5e42659d8124b7566fb09cc76958a345/image.png)

would redirect to the ERP5 xhtml style view.

/reviewed-on nexedi/erp5!782

It is extremely slow for some reason on large BTreeFolders inside
BTreeFolder2's has_key (even on warm caches). Using get instead is
significantly faster.
Found by chance, calling getAcquiredCategoryList on a random Person when
person_module contains of the order of magnitude of a million documents.

Do the calculations while ERP5 renders the form in JSON

Web Illustration was duplicated in erp5_officejs and erp5_dms before nexedi/erp5@c838767b.

After being removed, update erp5_officejs will uninstall this portal type

/reviewed-on nexedi/erp5!783

we used to add this value because ```select``` without ```option```
(that happens in MultiListField only) is invalid on XHTML.

But it was not at all required for other cases like MultiCheckboxField or RadioField.

Also,  ```select``` without ```option``` is no longer invalid on HTML5.

Considering that showing dummy unselectable <option> is anyway meaningless for users,
we drop it for all cases.

/reviewed-on nexedi/erp5!777

The warning is:
```
Ambiguous name for method of Products.ERP5.Document.SQLMethod.SQLMethod: 'manage_main' != 'manage'
```

It happens when components are reloaded

erp5_mrp: Manufacturing Order Portal Type must have its own init script so that it can be customized independently of Production Order.

This is needed when running livetests in a runUnitTest instance, in the
following scenario:
 - `runUnitTest --save test_bt:testXXX`
 - `runUnitTest --save --load`
 - run the live test from the browser

because livetest way of processing activities is to unsubscribe at the
beginning of the test and subscribe at the end to have control of
activities when running .tic, if ProcessingNodeTestCase also process the
activities automatically, they interfere and test often fail with errors like:

```
File "ERP5Type/tests/ProcessingNodeTestCase.py", line 249, in tic
  raise RuntimeError(error_message)
RuntimeError: tic is looping forever. These messages are pending: [('/erp5/portal_components/test.erp5.testSupportRequest', 'immediateReindexObject', 1, 0), ('/erp5/support_request_module/2617', 'immediateReindexObject', -1, 0)]
```

/reviewed-on nexedi/erp5!779

This step just checks that multi relation field listbox shows a list of
related document and that clicking on any item of the list leads to the
related document.
The order of related documents in the list has always been unspecified,
but the first one was always 1 until recently.
Because this test does not really care which one of the two related
document was clicked, just assert that it's a document from foo_module.

/reviewed-on nexedi/erp5!774

89115b88 is great, but with "testFunctionalAnonymousSelection" the
result HTML is more than 40Mo.

/reviewed-on nexedi/erp5!772

This will insure that the URL created for diff by `Base_redirect` will have the paramters
`your_first_path` and `your_second_path` in it, thus making it unique and hence
sharable. Before, we only used to have the name of selection in the URL which
returned different result for different users.

also use '>' instead of image

Assorted fixes and new features for support request app:

Features
 * post are now HTML and use the preferred editor ( CKEditor by default )
 * posts are ingested in Web Message and the app uses same data model as erp5_crm ( so it is able to display support request created with "standard" ERP5 interfaces)
 * date of post uses momentjs relative time (New message by Bob 1 hour ago...)

Bug fixes:
 * post API no longer use proxy roles / immediate reindex
 * RSS was re-implemented to list events. The previous approach of listing support requests had an issue that the date of new posts was still the date of the original support request.
 * attached files to the "submit new support request" dialog where not uploaded
 * using a handlebars template we prevent html injection / XSS 
 * increased test coverage

/reviewed-on nexedi/erp5!769

This was inefficient for two reasons:
- any message we could validate during current iteration means a message we
  did not consider is now in the range we just scanned. And it will not be
  considered until validation node starts over and scan this same range
  again.
- "LIMIT x,1000" pattern on >1000 messages causes a quick-growing number of
  extra rows scanned by the SQL database just to skip the "x" first rows:
  at 2000 rows present it must scan 1000 + 2000 = 3000 rows for a complete
  loop over all pending activities. At 3k rows it must scan 6k rows.
  At 4k, 10k.
  While this is an overestimation (some rows should be possible to
  validate, so these would be scanned once only), this overhead grows so
  large that this overestimation can become negligible.

Instead, use a range condition consistent with query's "SORT ON", which is
already efficiently materialised by an index: SQL database just has to
dive into the existing index to start just above the last message from
previous iteration, and resume scanning from there, solving both issues
listed above.

This method is called on every __getattr__ (guarded_getattr, actually) of
every restricted python scripts. Which means each "." in the code triggers:
- traversal to relevant acl_user folder
- a user lookup
just to get the same value throughout the execution of a script (as its
owner does not change during execution).
Also, in practice we have extremely few possible owners: very few users are
allowed to edit code in an ERP5 instance, and if such instance is managed
using the upgrader alarms, they will even further reduce this owner set to
System Processes only.
On a real-world web page rendering, this reduces the total number of
traversal calls from 1500 to 1100, getting rid of the two hottest spots:
/acl_users and /$site_id/acl_users .

Publishing was too open, sharing is a better state.

Setting the document as followup of the ticket and the support request
project and the support request organisation seems a good default to set
security so that project members or users working on the ticket can see
documents.

Also ingest document in two steps:

* first step is to create the document and activate DMS metadata discovery
* second step is to share the document, once the necessary roles have been
  given to the user

What's still not good with this approach is that DMS ingestion might
merge this new document with an already existing document. This case
is not supported and needs an extension to ingestion API