See merge request nexedi/wendelin!101

This commit is a rebase of the following original commits:

Analysis execution methods supporting transient in- and output

---

Conflicting files:
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/DataAnalysis_executeDataOperation.py
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/DataAnalysis_executeDataOperation.xml
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/ERP5Site_executeDataAnalysisList.py

processing of analysis with transient items

---

Conflicts:
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/DataAnalysis_executeDataOperation.py
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/ERP5Site_createDataAnalysisList.py

See merge request nexedi/wendelin!100

The new tests ensures the correct creation of Data Analysis documents.

pass causality_relative_url to operation script to support multiple intervals with same sensor at same location

/reviewed-on klaus/wendelin!10

---

Hint: this cherry-pick created a conflict in "ERP5Site_createDataAnalysisList.py"
with the following commit:
529d5430

The conflict has been resolved by accepting the solution by the cherry-picked commit,
and then afterwards moving the changed code from the above commit to the relevant position.

See merge request !98

Shows children (Data Array View Lines) which were hidden before.

For creating aggregated views on one or multiple other Data Arrays

We have to set the 'email_from_address' property after changing the
MailHost settings. This change was not yet described in the tutorial.

See merge request !97

After ERP5's 992c0ff13e8b5ac78cca4e7195de383418291892 we need to explicitly add this bt5 dependancy.

This patch is similar to
nexedi/erp5@530e8b4e
(nexedi/erp5!1445) and amends Wendelin test
driver to run tests on a real storage and spawn WCFS as needed.

Running tests on a real storage instead of in-RAM MappingStorage is
required in wendelin.core 2 case, because WCFS and Zope need to access the
same ZODB, and if Zope keeps data inside in-RAM MappingStorage, there
is no easy way for WCFS to access it. In such a case tests are failing as e.g.

    ERROR: test_03_DataArray (erp5.component.test.erp5_version.testWendelin.Test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "<portal_components/test.erp5.testWendelin>", line 185, in test_03_DataArray
        self.tic()
      File "/srv/slapgrid/slappart15/srv/runner/software/f0bddbb3930203c84c98ece4787c4df6/parts/erp5/product/ERP5Type/tests/ProcessingNodeTestCase.py", line 309, in tic
        raise RuntimeError(error_message)
      ...
    Last error message:
    ValueError
    ZODB.MappingStorage.MappingStorage is in-RAM storage
            in-RAM storages are not supported:
            a zurl pointing to in-RAM storage in one process would lead to
            another in-RAM storage in WCFS process.
    Traceback (innermost last):
      Module Products.CMFActivity.ActivityTool, line 361, in __call__
        result = method(*self.args, **self.kw)
      Module Products.ERP5Type.patches.PythonScript, line 179, in __call__
        return self._orig_bindAndExec(args, kw, None)
      Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
        return self._exec(bound_data, args, kw)
      Module Products.ERP5Type.tests.ERP5TypeTestCase, line 1576, in _exec
        return PythonScript_exec(self, *args)
      Module Products.PythonScripts.PythonScript, line 344, in _exec
        result = f(*args, **kw)
      Module script, line 21, in DataStream_readChunkListAndTransform
       - <PythonScript at /erp5_portal_30347777537315e139b1afecf8479bcc/DataStream_readChunkListAndTransform used for /erp5_portal_30347777537315e139b1afecf8479bcc/data_stream_module/1>
       - Line 21
        **kw)
      Module AccessControl.ZopeGuards, line 369, in guarded_apply
        return builtin_guarded_apply(func, args, kws)
      Module AccessControl.ZopeGuards, line 391, in builtin_guarded_apply
        return func(*arglist, **argdict)
      Module Products.ERP5Type.patches.ExternalMethod, line 114, in __call__
        return _f[0](*args, **kw)
      Module erp5.component.extension.erp5_version.Wendelin, line 58, in DataStream_copyCSVToDataArray
        zarray[-ndarray_shape[0]:] = ndarray
      Module wendelin.bigarray, line 444, in __setitem__
        a = self.__getitem__(idx)
      Module wendelin.bigarray, line 411, in __getitem__
        vmaM = self._fileh.mmap(pageM_min, pageM_max-pageM_min+1)
      Module wendelin.bigarray.array_zodb, line 94, in _fileh
        self._v_fileh = self.zfile.fileh_open()
      Module wendelin.bigfile.file_zodb, line 628, in fileh_open
        fileh = _ZBigFileH(self, _use_wcfs)
      Module wendelin.bigfile.file_zodb, line 689, in __init__
        self.zfileh = zfile._v_file.fileh_open(use_wcfs)
      Module wendelin.bigfile._file_zodb, line 112, in wendelin.bigfile._file_zodb._ZBigFile.fileh_open
        pywconn   = wczsync.pywconnOf(zconn)
      Module wendelin.wcfs.client._wczsync, line 54, in wendelin.wcfs.client._wczsync.pywconnOf
        zurl  = pyzodb.zstor_2zurl(zstor)
      Module wendelin.lib.zodb, line 338, in zstor_2zurl
        return "demo:(%s)/(%s)" % (zstor_2zurl(zstor.base), zstor_2zurl(zstor.changes))
      Module wendelin.lib.zodb, line 403, in zstor_2zurl
        "\tanother in-RAM storage in WCFS process.") % ztype)
    ValueError: ZODB.MappingStorage.MappingStorage is in-RAM storage
            in-RAM storages are not supported:
            a zurl pointing to in-RAM storage in one process would lead to
            another in-RAM storage in WCFS process.

-> Fix it by using a real storage for tests via --load/--save hack, and via
--with_wendelin_core requesting to spawn WCFS when built with wendelin.core 2.

Please see more details in added comment and in
nexedi/erp5@530e8b4e.

Test results:

- Wendelin/WC1: https://nexedijs.erp5.net/#/test_result_module/20210728-55E6A5B6
- Wendelin/WC2: https://nexedijs.erp5.net/#/test_result_module/20210729-A15AF293

/cc @tomo, @rafael, @Tyagov, @klaus, @romain, @jerome, @seb, @jp (reminder to review all tests to run on a real storage)
/reviewed-on nexedi/wendelin!96

See merge request nexedi/wendelin!95

- fix ebulk install command for ios
- update embulk manual installation url

See merge request nexedi/wendelin!94

See merge request nexedi/wendelin!93

See merge request nexedi/wendelin!92

See merge request !91