WIP: run tests wit zope.testrunner

testing with to list tests

      cd software/erp5/test

      env PYTHONNOUSERSITE=1 \
          MAKEFLAGS=-j128 \
          SLAPOS_SR_TEST_NAME=erp5-py3 \
          SLAPOS_TEST_SKIP_SOFTWARE_REBUILD=1 \
          SLAPOS_TEST_VERBOSE=0 \
          SLAPOS_TEST_SKIP_SOFTWARE_CHECK=1 \
          SLAPOS_TEST_DEBUG=0 \
          PYTHONWARNINGS=ignore \
          zope-testrunner  --color -vD --path . --tests-pattern test --list-tests

and to run :

      env PYTHONNOUSERSITE=1 \
          MAKEFLAGS=-j128 \
          SLAPOS_SR_TEST_NAME=erp5-py3 \
          SLAPOS_TEST_SKIP_SOFTWARE_REBUILD=1 \
          SLAPOS_TEST_VERBOSE=0 \
          SLAPOS_TEST_SKIP_SOFTWARE_CHECK=1 \
          SLAPOS_TEST_DEBUG=0 \
          PYTHONWARNINGS=ignore \
          zope-testrunner  --color -vD --path . --tests-pattern test

problems:
 - zope.testrunner could support setupModule / setupClass directly (or
    at least the workaround could be put directly in slapos.core testing
    on the default SlapOSInstanceTestCase)
 - zope.testrunner -D trigger debuggers on expected failures (probably similar to
   https://github.com/zopefoundation/zope.testrunner/issues/141 )
 - hitting Ctrl+C seems "even worse" than default unit test -> we can probably
   register a signal handler in setupClass to stop slapos standalone subsystem
   ourselves.
 - nxdtest probably needs support for good reporting on test_result_module

zope.testrunner

software/erp5/test/test/__init__.py

@@ -52,6 +52,26 @@ def setUpModule():
   setup_module_executed = True
 
 
+
+## zope.testrunner support
+class SetupModule:
+    @classmethod
+    def setUp(cls):
+        setUpModule()
+
+
+class SetupClass:
+    def __init__(self, _cls):
+        self._cls = _cls
+        self.__name__ = f'SetupClass for {_cls.__name__}'
+        self.__bases__ = (SetupModule, )  # TODO: different setup module per module ?
+
+    def setUp(self):
+        self._cls.setUpClass()
+    def tearDown(self):
+        self._cls.tearDownClass()
+
+
 # Metaclass to parameterize our tests.
 # This is a rough adaption of the parameterized package:
 #   https://github.com/wolever/parameterized
@@ -76,6 +96,7 @@ class ERP5InstanceTestMeta(type):
     base_class = super().__new__(cls, name, bases, attrs)
     if base_class._isParameterized():
       cls._parameterize(base_class)
+    base_class.layer = SetupClass(base_class)
     return base_class
 
   # _isParameterized tells whether class is parameterized.

software/slapos-sr-testing/software.cfg

@@ -27,6 +27,7 @@ parts =
   system-python-disabled
   slapos-cookbook
   template
+  zope-testrunner
 
 shared-part-list =
 
@@ -445,6 +446,18 @@ scripts =
   supervisord
   caucase
 
+[zope-testrunner]
+recipe = zc.recipe.egg
+#recipe = zc.recipe.testrunner
+eggs =
+  zope.testrunner
+  ${python-interpreter:eggs}
+initialization =
+  # XXX is this really necessary ??
+  sys.path.insert(0, '.')
+scripts =
+  zope-testrunner
+
 [git-clone-repository]
 recipe = slapos.recipe.build:gitclone
 git-executable = ${git:location}/bin/git