These must be installed through dependencies, but we explicitly need
scripts from supervisor because this is used by slapos-core tests

on python3 we use lzma directly

now we have a test extra

The official version of manuel has now the fix for which we needed a dev
version.

Pick the latest python2 compatible version.

use unshare to create new user-root namesapce to have the perrmission to
create network device, use mount to update files information of network
device in /sys(nemu)
test use iptables to manage route condition, miniupnpd to manage upnp,
brctl(nemu) to create bridge, so add them in component.

add openvpn, add babel, modify ovpn-client/server

due to outdated pypi version, nemu don't include need moudle in
setup.py, so add them in buildout

mostly packages with known vulnerabilities or causing failures/warnings
in the tests.

This also cleanup duplicates version definitions, sometimes the test
profiles overloaded some versions from stack/slapos.cfg

software.cfg becomes python 3
software-py3.cfg no longer exists
software-py2.cfg is introduced

Test suites will have to be updated accordingly

Change python2 softwares to use python2 explicitly and python3
softwares to use the default, but leave a few softwares (html5as,
html5as-base, fluent-bit, packer, seleniumrunner) using the default
python, as they were using python2 not because they only support python2
but because they don't have strong dependency to the python version
being used.

Use slapos.recipe.template instead of slapos.recipe.template if possible

Harmonize template keys :
rendered -> output
template -> url
template = inline: -> inline =

Delete "mode" key

See merge request nexedi/slapos!1151

slapos.core now depends on PyYAML and jsonschema

exposing environment variable SLAPOS_TEST_POSTGRESQL_PREFIX containing
the location where postgresql is installed

This allow caucase test use the random partition IP
address instead of fixed localhost:8000. Which can avoid
the conflict if one test node running multiple test suite.

This is used by caucase.sh test

Instead of duplicating the versions and having an old pin to a version
which is now older than caucase minimum required version.

217f7f05 (slapos-testing: don't pin tested eggs, 2021-01-12) was wrong,
some versions needs to be pinned for initial installation of the software.
The problem was not visible when the software was already installed and
updated in place, but when installing software from scratch, it was failing
very early with an error like:

    While:
      Bootstrapping.
      Getting distribution for 'slapos.libnetworkcache'.
    Error: Picked: slapos.libnetworkcache = 0.20

We want the tests to use the code from the repository, but when we have a
pinned version, buildout will use the code from repository only when the
version from the repository match the version spec from the pin.

The way slapos.recipe.template was pinned to 4.4 caused issues when we made
a change in slapos.recipe.template to make it 4.5, because tests were no
longer using code from slapos.recipe.template repository, but from 4.4 egg
from pypi.

To prevent this kind of issues, unset the slapos.recipe.template = 4.4
version pin and also unset the version for all tested egg, to prevent
similar issues if extended profiles also have version pins.

We also try to switch the tests to use python -m unittest as a way of
invoking tests, instead of the deprecated python setup.py test

When invoking tests with setup.py test, missing eggs are installed before
running tests, using their latest available version, but we want to always
install eggs using buildout and using pinned versions.

Adds the newly added to nexedi's repositories rubygemsrecipe (https://lab.nexedi.com/nexedi/rubygemsrecipe) to the list of tested eggs.

section to install dependencies of slapos.toolbox using slapos libraries

section to install dependencies of slapos.cookbook using slapos libraries

Since this version introduces a dependency on backports.lzma we had to
explicitly install backports.lzma in a slapos compliant way before installing
slapos.toolbox, because installing the egg needs xz-utils.

For slapos-sr-testing it's also a version up of backports.lzma to 0.14.0

For some SR, it implies the following upgrades:
- decorator: 4.3.0
- gitdb2: 2.0.5
- smmap2: 2.0.5
- subprocess32: 3.5.3

extra_requires was added in slapos.core, which simplifies this part.

/reviewed-on nexedi/slapos!693

Because we run egg tests with setup.py test, which installs missing
eggs, if an egg was not installed by buildout, then it installed before
running test. This was the case for mock, which is now python3 only (
https://pypi.org/project/mock/4.0.0b1/ ) and we started to see test
failures.

To solve this issue, refactor the setup definition to use
extra_requires, which seems to work fine in buildout now. Keep
test_requires because it's the what `python setup.py test` uses.

Clean buildout profiles to install slapos.cookbook[test] for test
instead of duplicating the content of test_requires.

/reviewed-on nexedi/slapos!690

and make sure we really use the develop version, by clearing version pin
that might be for another version.

Also clean up comments in version section.

This will be needed for standalone tests.

Needed for nexedi/kedifa!2

/reviewed-on nexedi/slapos!537

They are dependencies of slapos.toolbox and we already agreed to put slapos.toolbox version in a shared file.

/cc @jerome @jm

/reviewed-on nexedi/slapos!502

slapos.core test suite needs `sensors` command in $PATH
nexedi/slapos.core@7467ef4c (comment 71574)

/reviewed-on nexedi/slapos!479

Depend on caucase, as it shall be installed before as it is dependency.