key()-based sorting is now used instead, available since Python 2.4.

Also, remove `if True/1/False/0` statement raising pylint warnings.
Co-authored-by: Arnaud Fontaine <arnaud.fontaine@nexedi.com>

* Zope API changes:
  + publish() stdin is now BytesIO rather than StringIO.
  + HTTPResponse `body` property is now bytes().
  + OFS.Image.File file parameter is bytes().
* Python3 API changes:
  + builtin reduce() was removed.
  + Use BytesIO rather than StringIO to follow py3 API.
  + hmac.new() requires digestmod argument from Python 3.8.
  + Use six.moves library to handle moved objects from py2 to py3.
  + `modernize -f xrange_six` then slightly adjusted manually to just use range
     where it does not make a significant difference (for example in test).
  + base64.b64encode() now expects bytes().
  + UserDict() interface changed:
    - New parameter in update() and pop().
    - `failobj` setdefault parameter renamed to `default`.
  + ensure_list() on dict.{values,items}() and list(dict) for dict.keys() when
    we really need a list and not an iterable (Python3).
    - Add ensure_list() to RestrictedPython safe_builtins as it's very common usage.
  + Make dict iteration works on both version of Python.
    - Use six.iter{items,values,keys}().
    - has_key() has been removed.
    - Make sure that dict.{items,values,keys}() returns a real list when
    modified (ensure_list()).
  + Comparisons between int and NoneType raises TypeError.
  + No more unbound methods in python3 so use six.get_unbound_function().
  + Exceptions:
    - No longer behave as sequences. Using `args` attribute instead.
    - When an exception has been assigned using `as target`, it is cleared at
      the end of the except clause.
  + file: py2 was returning `str` upon reading, now it returns text strings.
    Also, opening mode is text strings by default.
  + Data strings are bytes().
    - Replace str() by bytes().
  + iterators no longer have next() method, instead there is next() builtin.
  + New ConnectionError exception so rename existing one to not clash.
  + Integer division is now with //.
Co-Authored-by: Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>
  Arnaud Fontaine <arnaud.fontaine@nexedi.com>
  Carlos Ramos Carreño <carlos.ramos@nexedi.com>
  Emmeline Vouriot <emmeline.vouriot@nexedi.com>

some conversion (TextDocument) depend on the content type, so it's
necessary to first update the content type before trying to convert to
base format

This executes even if setup() encounters an error and the cleanup hook is added
when opening the file, not later on.

Also, unify makeFileUpload() and makeFilePath() to remove duplicated code.
Co-authored-by: Arnaud Fontaine <arnaud.fontaine@nexedi.com>

See merge request nexedi/erp5!1979

- {'destination_list': ['person_module/he', 'person_module/me'],
+ {'destination_list': ['person_module/me', 'person_module/he'],

See merge request nexedi/erp5!1963

   The script is supposed to be called on portal context (or any context) and not on MailevaSOAPConnector, and it should return a MailevaSOAPConnector for a given reference.

    In the implementation where the user cannot directly insert holidays (by an specialised role/implementation) it allow the accountant insert the values more easily. Using a simple action.

    Add simple jump for navigate to the holidays/leave

  This helps to review value on the print out

See merge request nexedi/erp5!1959

Fix UI tests error introduced by f159d4f2. As it checks for timeout,
simulation finished state cannot be reached anymore.

Fix projection by not making a square map from non square limits (width != depth).
This way rotation does not result anymore in deformed shapes (it was the case until now because proportionality was not respected between width and depth).

Fix typo (cbe50b0f): erp5_accounting_l10n_fr: the French Fiscal Report (FEC) can be generated for different ledgers.

WebSection.getDocumentValueList() returns absolutely all the documents (as catalog brains) reachable by the web section. Then, calling .getUid() on them load all the objects in memory. The number of documents is of the order of hundreds, even on a small ERP5 (web pages, web scripts, etc.).

Limit the documents to retrieve in WebSection_getLatestDiscussionPostList by filtering on the portal_type, and do not load them needlessly in memory.

  This aims to add compatibility with BTreeFolder2 API, even it is not
  required. Since some checkConsistency may call self._cleanup() regardless
  expecting that the folder is a [H]BTreeFolder2 always.

  This was detected when a post upgrade constrant was included to
  portal_categories

The methods used in indexing did not make a difference between the case
where the price is None (ie. price is not set) or where the price is
set to 0 - in both cases this was saved as NULL in stock.total_price
column. This is incorrect, we need to keep the distinction between
these two cases also for inventory calculation.
We had some places where we select IFNULL(stock.total_price, 0) to work
around this, we don't plan to change the existing ones for now, but
while discussing on nexedi/erp5!1974
we concluded that a newly idenfified case of a problem consequence of
these NULL should be handled by fixing the indexation. To benefit from
the fix, impacted instances will have to reindex documents present in
the stock table with stock.total_price is null.

The ad-hoc handling of boolean in protocol 1 was not implemented
correctly and they were serialized as integers (0 for False and 1 for
True), this fixes the export code and re-export everything

With these changes, we are able to install py2 business templates on py3,
but export is slightly different, because we already export using pickle
protocol 3 on py3.

To be able to install py2 business templates, we included heuristics to
guess the str or bytes from business template XML: oids are bytes and
also some strings that do not decode to UTF-8, so that we can install
python2 business templates on py3.

When exporting business templates, we need to build a list of referenced
persistent objects to export them separately in the XML, this is is done
using Unpickler.noload, in a way which does not support pickle protocol
1 on py3 (the persistent ids are None and the assertion in
https://github.com/zopefoundation/ZODB/blob/d698507bb89eeb38c6e655199bc9f54c909dbf4d/src/ZODB/serialize.py#L669
fails), so we need to use pickle protocol 3 on py3.

In the future, we might switch to exporting on protocol 3 on py2 as well
so that we have stable output on both py2 and py3, or maybe we'll do
this only when we stop supporting py2.

__import__ `fromlist` argument was wrong. It was working anyway with Python2 but
not anymore with Python3, raising a `ModuleNotFoundError` exception. According
to Python `__import__(name, globals, locals, fromlist)` documentation:

  When the `name` variable is of the form `package.module`, normally, the
  top-level package (the `name` up till the first dot) is returned, *not* the
  module named by `name`. However, when a non-empty `fromlist` argument is
  given, the module named by `name` is returned.

Thus, the following patterns were wrong:
  * __import__(MODULE_NAME, globals(), locals(), MODULE_NAME)
    => Iterate through each character of MODULE_NAME as fromlist is expected to
       be a list/tuple.
  * __import__(MODULE_NAME, globals(), locals(), [MODULE_NAME])
    => This works but actually tries to import MODULE_NAME object from
       MODULE_NAME module (no error if it cannot).

The goal of such __import__ calls were for __import__ to return the right-end
module instead of the top-level package. In such case, `fromlist=['']` is the
way to go as it __import__ does not check if the object exists in the module if
it's an empty string. However, it is even better and easier to read to use
importlib.import_module() for that...

Also, add `from __future__ import absolute_import` because python2 tries both
relative and absolute import (level=-1 __import__ parameter) whereas python3
does absolute import by default (level=0).
Co-authored-by: Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>

When under a pinDateTime context, d.isPast() was comparing d with the
actual current date, not the fake date from the pinned context.

Also, make the methods class methods, so that they can be used as
external methods in Zelenium tests.

 - Products.PythonScripts now has a __loader__
 - do not use < > as filename for component on py3, this makes linecache
 work out of the box. On py2 I think it was causing errors trying to
 actually open the file, but this does not seem needed on py3 and
 simplifies everything

new test with ERP5-style workflow to cover python3 migration

with ROUND_DOWN, we have a different behaviour between py2/py3, that caused failures in
erp5_simplified_invoicing:testTradeModelLine.

Here is what happened in _round(1.9999999999999998) in
bt5/erp5_simulation/DocumentTemplateItem/portal_components/document.erp5.FloatEquivalenceTester.py

* py2
decimal.Decimal(str(1.9999999999999998)).quantize(decimal.Decimal('0.000001'), 'ROUND_DOWN')
=> Decimal('2.000000')
(because str(1.9999999999999998) is '2.0')

* py3
decimal.Decimal(str(1.9999999999999998)).quantize(decimal.Decimal('0.000001'), 'ROUND_DOWN')
=> Decimal('1.999999')
(because str(1.9999999999999998) is '1.9999999999999998')

But ROUND_DOWN result of 1.9999999999999998 with 0.000001 precision should be 1.999999 thus py2 behaviour is wrong.