urlmd5 is useless: better store url. And if metadata contains url,
file also becomes useless. There remain so little to check that we can
do better with custom validation instead of using a JSON validator.

Also, if we wanted to set a file name on a shadir document, which is
json data, it should be different from the file name of the referenced
cached data. At least, the extension would end with '.json'

This was still working with Zope 2.13 because the default output_encoding for
HTML was iso-8859-15 but this is not the case anymore with Zope 4. Moreover this
should be explicit rather than implicit anyway...

Revert "testBusinessTemplateTwoFileExport: Make output_encoding explicit as it should have always been."

This reverts commit 968991e7 which does not
actually work properly with Zope4.

SQLCatalog_trimInventoryCacheOnCatalog needs resource category on movement.

This fixes a failure due to Zope4 now defaulting to utf-8.

Use `portal_alarms/check_consistency` after configuration to make sure that every document created by configurator or part of the dependency business template is valid.

For now we have two exceptions (that are some TODO for the future):
 - Web Sites, because upgrader constraints are not ran by configurator at this point
 - Business Configuration, because most of the configurator item classes don't fully follow the protocol of constraints, their `fixConsistency` method always return "something needs to be fixed" without actually checking.

See merge request !1562

Creation date is defined as the oldest date of any workflow history on the
considered document.
Modification date is defined as the newest date of any workflow history
on the considered document.
As soon as there is more than one workflow on a given document, these
values can differ: they are set with the wall-clock time at the moment
the workflow history of each workflow is initially filled, so one can land
just before a second change and the other right after.
So comparing them is meaningless, catalog or not, and just causes this test
to be unstable.

Because we want to check consistency on all documents created by
configurator, we also need documents created during the test to be
consistent (or deleted)

We had trade phase duplication:
 - `trade_phase/default/*` were provided by erp5_simulation_test
 - `trade_phase/trade/*` are provided by erp5_configurator_standard_categories

so the former was only for testing and the later were actually used in
instance configured by configurator.

But in some paths from configurator_standard_*template, which are also
used on configurated instances, were using trade_phase/default (the one
from test). Some rules were referencing these categories, but apparenlty
not using them.

To fix this confusion, use trade_phase/trade/* everywhere, and install
erp5_configurator_standard_categories where needed in tests.

BusinessConfiguration have a resource category, but it's the workflow,
it's not really a resource, so some APIs from the base Amount class are
not available. Because they have no meaning on BusinessConfiguration
anyway, define the broken APIs to return nothing.

This fixes errors trying to use check_consistency alarm on Business
Configurations.

Tools also contain documents, so it makes sense to check them as well

These documents were probably edited before they had correct property
sheets, because they had local properties as incorrect type, as reported
by PropertyTypeValidity:

    <ERP5Type.ConsistencyMessage for PropertyTypeValidity type_check on portal_gadgets/erp5_persons (message: Attribute gadget_type should be of type lines but is of type <type 'str'>)>
    <ERP5Type.ConsistencyMessage for PropertyTypeValidity type_check on portal_gadgets/erp5_gadget_new_sale_opportunity (message: Attribute gadget_type should be of type lines but is of type <type 'str'>)>
    <ERP5Type.ConsistencyMessage for PropertyTypeValidity type_check on portal_gadgets/erp5_gadget_contact_person (message: Attribute gadget_type should be of type lines but is of type <type 'str'>)>

For historical reasons, EncryptedPasswordMixin.setPassword was public
and did its own security checks, this was the case since 7d0882ef (
setPassword have to do explicit security checks…, 2007-11-12), this was
because we wanted to support cases where user can edit the login ("Edit
portal content" permission), but not changed the password ("Set own
password" permission).

Also, we wanted to support the case where login is edited through a view
form, in that case we have a my_password field that is empty and we
don't want to set the password to None in that case.

For these two reasons the API to set password was very complex and
behaving differently from other accessors: usually setSomething(None)
just set something to None, ie. "unset" something, but for passwords it
was not the case. Also we had to introduce _forceSetPassword method,
which sets the password without security checks, so that it can be
called from unrestricted code for cases where user does not have the
permission to reset password (like in the reset password scenario).

Since d1312cdb ( make edit check the security remove all useless
security declaration on private method, 2008-05-23), edit supports
restricted properties, so we can simplify all this and make setPassword
a more standard accessor, ie:
 - setPassword has a security declaration, so if it is called from
  restricted python the security will apply at `__getattr__` time.
  `edit` method will also check security
 - setPassword(None) reset the password.
 - The logic to not change the password when editing in view mode is now
  `edit` responsability. ie. `login.setPassword(None)` resets, but
  `login.edit(password=None)` does not reset.

This also correct some usage of the lower level API (`pw_encrypt` and
`pw_validate`) which were never supposed to use `None`:
 - `pw_validate` was called with None when a user without password was
  trying to login, causing a TypeError that was cached by PAS and logged
  with level debug (and refusing login). Now the error is no longer raised.
 - `pw_encrypt` was called with None (but apparently only in the tests,
  when doing `user.newContent(portal_type='ERP5 Login', password=None)`)
  and this was creating a login with password `'None'` with AccessControl 2.
  With AccessControl 4 this was an Error.

reorder methods, make some docstrings a bit more informative and fix
several typos

This disable price lookup on domains, because it errors and because the
meaning is not clear.

It errors because Domains are MetaResources, so they are subclass of
Resource and they inherit the price lookup mechanism, which does not
work on domains because of their different `getRelativeUrl` implementation.

check_consistency alarm checks that every document created satisfy
contraints. This is done in order to verify that configurator it self
create consistent documents and also that all default documents from a
realistic configuration are consistent.

the API is not to return a list of strings, but a list of ConsistencyMessage

Replace it with a method on ERP5Type.Base so these activities get found
by CopySupport.unindexObject and flushed, rather than remaining on the
interaction workflow's context and failing when run.
It seems a lot more likely for a document to be deleted while interactions
are being spawned than an interaction workflow itself. So this should be
a net benefit in activity stability.

This currently only works because CookieCrumbler reacts on *all* urls which
contain the magic login & password fields. But the POST request body lacks
any value from submit fields, which bypasses the "logged-in" scripts, which
is harmless with current code but lack realism.

See merge request !1516

The initial idea was that publishing documents is something really exceptional, the "attach document" normal use case is typically use cases like attaching a PDF invoice to an invoice document in accounting, ie. most of the time it's with sensitive information that we don't want users to be mistakenly publish on the internet just because they selected a wrong value in the field.

Now we have a project with use cases where the attached documents needs to be published, we did not change our mind that publishing an attached document is an exceptional case, but we want to make it possible to configure so that in certain contexts, publishing documents is possible.

So we reuse the existing configuration by type based method idea and when the getPreferredAttachedDocumentPublicationState returns "published" we make it possible to publish by default.

Theses methods have always been using user_id, but where written at
a time where there was not such disctinction

This test case tries to provide message helpful for debugging in case
of assertion failure for assertUserCanPassWorkflowTransition, but
this was not correctly using new workflow API and in case of failure
there was an error like this:

  File ".../custom/test.py"
    self.assertUserCanPassWorkflowTransition(user, 'stop_action', packing_list)
  File "product/ERP5Type/tests/SecurityTestCase.py", line 237, in failUnlessUserCanPassWorkflowTransition
    if wf_transition.trigger_type == TRIGGER_USER_ACTION:
AttributeError: 'NoneType' object has no attribute 'trigger_type'

The previous implementation was using getGuardSummary, which no
longer exist in new workflow, so we implement similar logic here.

The new message changes a bit, it now look like this:

    AssertionError: User X can NOT pass stop_action transition on Internal Packing List at /erp5/internal_packing_list_module/20220218-22A38 (draft on delivery_causality_workflow, draft on internal_packing_list_notification_workflow, started on packing_list_workflow).
     Roles: [Owner, Member, Authenticated, Associate]
     Available transitions:
              deliver_action[packing_list_workflow]
                    Expression:
                    Permissions:
                    Groups:
            * stop_action[packing_list_workflow]
                    Expression: python: not(state_change['object'].getPortalType() == "Sale Packing List" and state_change['object'].getSimulationState() == "confirmed")
                    Permissions:
                    Groups:

1b1dbf60 (tests: also consider python unittest failures in
functional tests, 2021-06-16) was not counting properly the cases
where we have selenium failures. In that case we only want to count
selenium failures, if we add with the python failures from
status_dict, we report one extra failures.

The correct approach is to count selenium failures if any and otherwise
count python failures
Co-authored-by: Vincent Pelletier <vincent@nexedi.com>

Avoids duplicating items from that list.

Both so that changes to the list of core business templates are applied on
upgrade, and to avoid uninstalling core business templates, if no other
maintained-up-to-date business templates depend on them.
Also, improve the documentation of the erp5_upgrader version of this
script.

But make it non-publishable.

The main reason is to use portal type setters, and not set the properties
directly: setting the properties directly bypasses interactions which
trigger type refresh, which hence prevents such changes from being applied
to the types until something else would cause a reload.
While at it:
- modify the property sheet list once only instead of once per added
  property sheet
- only modify the property sheet list when fixit is true
- improve constraint message to actually tell what is being detected
- do not report a constraint error when no change is necessary
- follow naming conventions: avoid abbreviations, variables holding
  documents must end in "_value"
- avoid single-use local variables

This reverts commit 77b3f202.
For some obscure reason, this affects unit tests related to inventory
unit conversion. There seems to be too much code to cleanup to keep this
patch for now, so drop it.