Commit cdbeda8d authored by Kirill Smelkov's avatar Kirill Smelkov

tests: Teach test driver to pass tests when run with wendelin.core 2

This patch is similar to
nexedi/erp5@530e8b4e
(nexedi/erp5!1445) and amends Wendelin test
driver to run tests on a real storage and spawn WCFS as needed.

Running tests on a real storage instead of in-RAM MappingStorage is
required in wendelin.core 2 case, because WCFS and Zope need to access the
same ZODB, and if Zope keeps data inside in-RAM MappingStorage, there
is no easy way for WCFS to access it. In such a case tests are failing as e.g.

    ERROR: test_03_DataArray (erp5.component.test.erp5_version.testWendelin.Test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "<portal_components/test.erp5.testWendelin>", line 185, in test_03_DataArray
        self.tic()
      File "/srv/slapgrid/slappart15/srv/runner/software/f0bddbb3930203c84c98ece4787c4df6/parts/erp5/product/ERP5Type/tests/ProcessingNodeTestCase.py", line 309, in tic
        raise RuntimeError(error_message)
      ...
    Last error message:
    ValueError
    ZODB.MappingStorage.MappingStorage is in-RAM storage
            in-RAM storages are not supported:
            a zurl pointing to in-RAM storage in one process would lead to
            another in-RAM storage in WCFS process.
    Traceback (innermost last):
      Module Products.CMFActivity.ActivityTool, line 361, in __call__
        result = method(*self.args, **self.kw)
      Module Products.ERP5Type.patches.PythonScript, line 179, in __call__
        return self._orig_bindAndExec(args, kw, None)
      Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
        return self._exec(bound_data, args, kw)
      Module Products.ERP5Type.tests.ERP5TypeTestCase, line 1576, in _exec
        return PythonScript_exec(self, *args)
      Module Products.PythonScripts.PythonScript, line 344, in _exec
        result = f(*args, **kw)
      Module script, line 21, in DataStream_readChunkListAndTransform
       - <PythonScript at /erp5_portal_30347777537315e139b1afecf8479bcc/DataStream_readChunkListAndTransform used for /erp5_portal_30347777537315e139b1afecf8479bcc/data_stream_module/1>
       - Line 21
        **kw)
      Module AccessControl.ZopeGuards, line 369, in guarded_apply
        return builtin_guarded_apply(func, args, kws)
      Module AccessControl.ZopeGuards, line 391, in builtin_guarded_apply
        return func(*arglist, **argdict)
      Module Products.ERP5Type.patches.ExternalMethod, line 114, in __call__
        return _f[0](*args, **kw)
      Module erp5.component.extension.erp5_version.Wendelin, line 58, in DataStream_copyCSVToDataArray
        zarray[-ndarray_shape[0]:] = ndarray
      Module wendelin.bigarray, line 444, in __setitem__
        a = self.__getitem__(idx)
      Module wendelin.bigarray, line 411, in __getitem__
        vmaM = self._fileh.mmap(pageM_min, pageM_max-pageM_min+1)
      Module wendelin.bigarray.array_zodb, line 94, in _fileh
        self._v_fileh = self.zfile.fileh_open()
      Module wendelin.bigfile.file_zodb, line 628, in fileh_open
        fileh = _ZBigFileH(self, _use_wcfs)
      Module wendelin.bigfile.file_zodb, line 689, in __init__
        self.zfileh = zfile._v_file.fileh_open(use_wcfs)
      Module wendelin.bigfile._file_zodb, line 112, in wendelin.bigfile._file_zodb._ZBigFile.fileh_open
        pywconn   = wczsync.pywconnOf(zconn)
      Module wendelin.wcfs.client._wczsync, line 54, in wendelin.wcfs.client._wczsync.pywconnOf
        zurl  = pyzodb.zstor_2zurl(zstor)
      Module wendelin.lib.zodb, line 338, in zstor_2zurl
        return "demo:(%s)/(%s)" % (zstor_2zurl(zstor.base), zstor_2zurl(zstor.changes))
      Module wendelin.lib.zodb, line 403, in zstor_2zurl
        "\tanother in-RAM storage in WCFS process.") % ztype)
    ValueError: ZODB.MappingStorage.MappingStorage is in-RAM storage
            in-RAM storages are not supported:
            a zurl pointing to in-RAM storage in one process would lead to
            another in-RAM storage in WCFS process.

-> Fix it by using a real storage for tests via --load/--save hack, and via
--with_wendelin_core requesting to spawn WCFS when built with wendelin.core 2.

Please see more details in added comment and in
nexedi/erp5@530e8b4e.

Test results:

- Wendelin/WC1: https://nexedijs.erp5.net/#/test_result_module/20210728-55E6A5B6
- Wendelin/WC2: https://nexedijs.erp5.net/#/test_result_module/20210729-A15AF293

/cc @tomo, @rafael, @Tyagov, @klaus, @romain, @jerome, @seb, @jp (reminder to review all tests to run on a real storage)
/reviewed-on nexedi/wendelin!96
parent 44f9e393
# -*- coding: utf-8 -*-
from test_suite import ERP5TypeTestSuite from test_suite import ERP5TypeTestSuite
import glob import glob
import os.path import os.path
...@@ -19,9 +20,24 @@ class WendelinERP5(ERP5TypeTestSuite): ...@@ -19,9 +20,24 @@ class WendelinERP5(ERP5TypeTestSuite):
def run(self, full_test): def run(self, full_test):
test = ':' in full_test and full_test.split(':')[1] or full_test test = ':' in full_test and full_test.split(':')[1] or full_test
# from https://lab.nexedi.com/nexedi/erp5/commit/530e8b4e:
# ---- 8< ----
# Combining Zope and WCFS working together requires data to be on a real
# storage, not on in-RAM MappingStorage inside Zope's Python process.
# Force this via --load --save for now.
#
# Also manually indicate via --with_wendelin_core, that this test needs
# WCFS server - corresponding to ZODB test storage - to be launched.
#
# In the future we might want to rework custom_zodb.py to always use
# FileStorage on tmpfs instead of δ=MappingStorage in DemoStorage(..., δ),
# and to always spawn WCFS for all tests, so that this hack becomes
# unnecessary.
# ---- 8< ----
status_dict = self.runUnitTest('--load', '--save', '--with_wendelin_core', full_test)
if test.startswith('testFunctional'): if test.startswith('testFunctional'):
return self._updateFunctionalTestResponse(self.runUnitTest(full_test)) status_dict = self._updateFunctionalTestResponse(status_dict)
return super(WendelinERP5, self).run(full_test) return status_dict
### this is dublicate code from erp5, needed to display functional tests ontestnodes nicely ### this is dublicate code from erp5, needed to display functional tests ontestnodes nicely
def _updateFunctionalTestResponse(self, status_dict): def _updateFunctionalTestResponse(self, status_dict):
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment