Default behavior of runUnitTest was to listen on `127.0.0.1`/`localhost`, which is not
allowed for SlapOS instance and caused issues when the used DNS server does not resolve localhost.

Instead, we decided to run the zserver on the partition IP. Because this partition's IP is not seen as a secure origin (as defined in https://goo.gl/Y0ZkNV ) from firefox when running zelenium tests, we also add some entries in apache from balancer partition to act as an https proxy on this zserver and change the testrunners to access the https URL.

See also erp5!729 for the erp5 part.

/reviewed-on !374

We don't want runUnitTest is to listen on 127.0.0.1, as it is not
allowed for SlapOS instances, runUnitTest wrapper is extended to include
`--zserver` with a pre-assigned ip:port.

runTestSuite starts several test in parrallel (controlled by
`--node_quantity` argument, which is passed by erp5testnode), so we need
to make sure that we provide it with enough ip:port. For this, we
extended runTestSuite with a `--zserver_address_list` argument and we
generate a wrapper with a list of `testrunner.node-count` pre-assigned
ip:ports.

Because zelenium tests needs to access this zserver over a secure origin
(otherwise modern browser features such as service worker are not
available), use an https proxy in the apache from the balancer
partition.

runUnitTest and runTestSuite have been extended with resp.
`--zserver_frontend_url` and `--zserver_frontend_url_list` arguments and
the URLs published by the balancer paritions are set in the wrappers.
For compatibility reasons, runTestSuite pass parameters as environment
variables.

Implementation notes:

This introduces a circular depencency, balancer partition needs to know
the address of the testrunners and zope partitions needs to know the
URLs of the corresponding http proxies on the apache. This is is handled
by `slapos.recipe:publish-early`:

 1. request zope family with an empty `test-runner-apache-url-list`.
    zope is instanciated a first time.
    zope returns `test-runner-address-list` ( a list of (host, port)
    tuples )
 2. request balancer with `test-runner-address-list`
    balancer is instanciated.
    balancer returns `{{ family_name }}-test-runner-url-list` ( a list
    of apache URLs ), which are published in the root partition.
 3. zope family is re-requested with
    updated`test-runner-apache-url-list` information

instance-erp5.cfg.in template was also reorganised to move
`[publish-early]` next to `[publish]` at the bottom of the file because
these sections are semantically related.
Also test runner generation is moved after zope generation, because we
want to allocate test runners ports after zopes, otherwise existing
zopes would get new ports when existing instances are upgraded.

`enable_authentication` parameter of `backend-list`
was not documented.

We have new parameters "how many parallel tests this instance can run"
( test-runner.node-count ) and "how many supplementary databases each
test needs" ( test-runner.extra-database-count ).

Instead of configuring "I need a total of 15 databases", we configure "I
need to be able to run 3 tests in parallel, each test requiring 4 extra
databases" which generates 3 * (4 +1) databases.

`mariadb.test-database-amount` becomes deprecated in favour of these new
arguments, but compatibility is preserved.

Set cache.direct=on when using native aio (needed by qemu)

/reviewed-on nexedi/slapos!399

Instead of relying on slapos.cookbook:certificate_authority recipe, which
stops buildout processing, extract the minimal implementation to runtime
key/certificate validator and reject slaves, which does not pass this test.

This commits results in TODO item being done.

As slave requester is able to enter any string in server-alias validate it
against being correct domain name and in case if validation fails reject that
slave.

Also use a trick to have access to global slave state, see
https://fabianlee.org/2016/10/18/saltstack-setting-a-jinja2-variable-from-an-inner-block-scope/

Install validators dependency, which is a way to easily check if email is an
email or domain is correct.

As slave requester is able to enter any string in custom domain validate it
against being correct domain name and in case if validation fails reject that
slave.

Create caddyprofiledeps egg with dummy noop recipe.

Thanks to setting dependencies of this egg and enabling it on the instance
profile, buildout will install eggs during software run and activate them
during instance run.

No existing egg (like slapos.cookbook) is used, as this technique is to allow
profile/software release developer to choose required eggs used during
instantiation.

Another apporach would be to add dependency for validators in
slapos.recipe.template (in install_requires).

Thank to escaping whole command-line it is possible to process buildout
dangerous strings, like ${section:option}, pass them to the wrapper,
instead of killing the whole profile processing.

If the value does not contain "," drop its contents instead of stopping
processing of the whole profile.

Instead of needlessly storing information in configuration section, pass it
via jinja2 parameter. This is safe, in case if extra_slave_instance_list
would contain value like ${section:option}.

Mainly to fix parallel build.

https://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options#TestConfigurationOptions-MaximumTestDuration

There is a way to pass monitor port to each replicated frontend by using
frontend-config-N-monitor-httpd-port, thus there is no need to pass through
the default value, as it comes default on each replicated slave.

I noticed by accident that tests for `software/wendelin/software-dev.cfg` where failing because buildout was picking eggs:

```
Installing eggs.
While:
  Installing eggs.
  Getting distribution for 'scikit-learn'.
Error: Picked: scikit-learn = 0.19.2
Failed to run buildout profile in directory '/srv/slapgrid/slappart7/srv/testnode/acv/soft/78ae8a6d8b0639addb22de2c7c96af63'
```

Turned out it was a problem with usage of `zc.recipe.egg:develop`

There was also a problem which `${gcc-fortran:}` that does not exist anymore.

/reviewed-on nexedi/slapos!395

Turned out we are not ready to apply the part about storing wrappers in
${bin:directory} from "Firefox: install wrappers in
${buildout:bin-directory} and version up"

This commit is to be reverted once we update test nodes to 1.0.73 or
higher

Slapos compatible wrapper is no longer installed in `parts/firefox/firefox-slapos` , but directly as `firefox` in the `${buildout:bin-directory}` of the software. `geckodriver` is also in the same directory.

Firefox version up to 52.9.0esr

Geckodriver version up to 0.17.0


/reviewed-on nexedi/slapos!387

:develop is a bit special in that it does not install the egg, but just
tell buildout that this directory contain the egg, so that recipes that
will install this egg later will pick the develop-eggs.