See merge request nexedi/slapos!1499

/reviewed-by @kirr and @jerome
/reviewed-on nexedi/slapos!1494

wendelin.core 2.0.alpha3.post9 supports golang 1.21: with this new
version we can set golang 1.21 as the new default.

/reviewed-by @kirr and @jerome
/reviewed-on nexedi/slapos!1494

Go1.21 already has the fifth minor revision (released 2023-12-05) and
should therefore already be sufficiently stable. Furthermore we need it
to fix a bug in a NEO/go dependency [1].

Please find all details in the official release note:

    https://go.dev/doc/go1.21

It was released on 2023-08-08 [2]. Due to the go promise of compatibility
most software should still compile without any problems.

In golang < 1.21 we needed to patch golang to fix
https://github.com/golang/go/issues/42525. In golang 1.21 we still need
to apply a fix, but can't apply the old patch because the code changed.
In golang > 1.21 this problem is already fixed with https://go-review.googlesource.com/c/go/+/520055.

Because of https://github.com/golang/go/commit/092671423cd95eaa6df93eb29442fef41504d097
'TestUnshareMountNameSpace' fails on golang 1.21 [3]. In golang 1.20 this test
was skipped [4]. To fix this failure the additional patch
'skip-unshare-mount-test.patch' has been added.

---

[1] nexedi/wendelin.core!22 (comment 195769)]
[2] https://go.dev/doc/devel/release
[3] --- FAIL: TestUnshareMountNameSpace (0.18s)
    exec_linux_test.go:243: unshare failed: exit status 2
        unshare: mount /tmp/TestUnshareMountNameSpace2210137852/001 failed: 0x1
[4] === RUN TestUnshareMountNameSpace exec_linux_test.go:333: kernel prohibits unshare in unprivileged process, unless using user namespace — SKIP: TestUnshareMountNameSpace (0.00s)

/reviewed-by @kirr and @jerome
/reviewed-on nexedi/slapos!1494

This version is compatible with gcc 12

using https://github.com/ilevkivskyi/com2ann

drop versions not running on debian 12 and add new versions

This was disabled to "keep compatibility with current font selection
problems", but I don't think we need to care about compatibility for
this. This was anyway causing a problem that system fontconfig will be
used if it's available, so it's better to define things explictly to
prevent falling back to system configuration.

This caused cloudooo to select different fonts, because LibreOffice-bin
comes with some Noto fonts and fontconfig now prefers Noto font

Testing conversion of HTML with font-family:"FontFamily FontVariant"
CSS rule does not really make sense. Remove a few cases for which the
behavior is not stable.

Using RegExp to validate hostnames is a bad practice, and has a lot of reasons to be wrong.
On top of that, the JSON Schema specification allows, since draft 7, to validate hostnames
against an IDN hostname, by using the `idn-hostname` format.

With these changes, IDN are now supported (.рф and .中國 for instance), and long TLD
should not be a problem anymore.

With backported fixes so that old software supports debian 12

In 9636d79c (Nextcloud Upgrade fixes, 2023-12-13) parameter changed:
 - instance.trusted-domain-* parameters are replaced by instance.trusted-domain-list
 - the frontend is part of trusted domain, so with "bypassed" frontends
from slapos proxy the backend appears twice (because the backend URL is
returned as frontend URL)

This actually updates firefox from version 68 to version 115

So that we can see when something changes.

See merge request !1486

Use runTestSuite interface to launch tests that request instances
on actual SlapOS cloud.

As described in https://github.com/msgpack/msgpack-python#major-breaking-changes-in-msgpack-10
raw is False by default for unpacker so we receive strings, not bytes

See merge request nexedi/slapos!1490

ZODB4 does not support python3

These must be installed through dependencies, but we explicitly need
scripts from supervisor because this is used by slapos-core tests

on python3 we use lzma directly

This software only use nodejs for a simple build step, it should be OK
to use any version.

Hello up there.

I've noticed that wendelin.core tests became failing to build and
automatically disabled some time ago. Not good. I've tried to fix build
failures. Please see individual patches for details.

/cc @levin.zimmermann
/reviewed-by @jerome
/reviewed-on nexedi/slapos!1492

After probably 02fad6a8 (component/{numpy,scipy}: Version up for Python3.)
pygolang test build started to fail in numpy compilation as shown in the
appendix. The failure is correct as pygolang never explicitly depended on
${numpy:egg}. We did not noticed before probably because in earlier versions
numpy did not required Cython to be present at install time.

-> Fix it by explicitly adding ${numpy:egg} to pygolang[all_test] dependencies.

Apendix. Log of test build failure

    networkcache: Trying to download pypi:numpy=1.22.0 from network cache...
    Getting distribution for 'numpy==1.22.0'.
    WARNING: The easy_install command is deprecated and will be removed in a future version.
    Running from numpy source directory.
    Processing numpy/random/_bounded_integers.pxd.in
    Processing numpy/random/_pcg64.pyx
    Traceback (most recent call last):
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 53, in process_pyx
        import Cython
    ModuleNotFoundError: No module named 'Cython'

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 234, in <module>
        main()
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 230, in main
        find_process_files(root_dir)
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 221, in find_process_files
        process(root_dir, fromfile, tofile, function, hash_db)
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 187, in process
        processor_function(fromfile, tofile)
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 60, in process_pyx
        raise OSError(msg) from e
    OSError: Cython needs to be installed in Python as a module
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 154, in save_modules
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
      File "<string>", line 7, in setup_context
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 250, in run_setup
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 45, in _execfile
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 450, in <module>
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 432, in setup_package
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 237, in generate_cython
    RuntimeError: Running cythonize failed!

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 10, in <module>
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 2317, in main
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/__init__.py", line 162, in setup
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 424, in run
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 666, in easy_install
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 711, in install_item
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 896, in install_eggs
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 1164, in build_and_install
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 1150, in run_setup
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 253, in run_setup
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
        self.gen.throw(typ, value, traceback)
      File "<string>", line 7, in setup_context
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
        self.gen.throw(typ, value, traceback)
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
      File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
        self.gen.throw(typ, value, traceback)
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 166, in save_modules
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 141, in resume
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/_vendor/six.py", line 685, in reraise
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 154, in save_modules
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
      File "<string>", line 7, in setup_context
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 250, in run_setup
      File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 45, in _execfile
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 450, in <module>
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 432, in setup_package
      File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 237, in generate_cython
    RuntimeError: Running cythonize failed!
    Cythonizing sources
    An error occurred when trying to install numpy 1.22.0. Look above this message for any errors that were output by easy_install.
    While:
      Installing gpython.
      Base installation request: 'pygolang[all_test]'
        Requirement of pygolang[all_test]: wheel
        Requirement of pygolang[all_test]: setuptools_dso>=2.8
        Requirement of pygolang[all_test]: setuptools
        Requirement of pygolang[all_test]: pytest
        Requirement of pygolang[all_test]: numpy
        Requirement of pygolang[all_test]: ipython
        Requirement of pygolang[all_test]: cython<3
        Requirement of pygolang[all_test]: geventmp
        Requirement of pygolang[all_test]: setuptools_dso>=2.8
        Requirement of pygolang[all_test]: decorator
        Requirement of pygolang[all_test]: six
        Requirement of pygolang[all_test]: gevent
        Requirement of setuptools_dso>=2.8: setuptools
        Requirement of pytest: wcwidth
        Requirement of pytest: six>=1.10.0
        Requirement of pytest: py>=1.5.0
        Requirement of pytest: pluggy<1.0,>=0.12
        Requirement of pytest: packaging
        Requirement of pytest: more-itertools>=4.0.0
        Requirement of pytest: attrs>=17.4.0
        Requirement of pytest: atomicwrites>=1.0
      Getting distribution for 'numpy==1.22.0'.
    Error: Couldn't install: numpy 1.22.0