Real frontend is always requested.

Additional frontend is requested only if frontend-additional-instance-guid is
not empty. This frontend is published as url-additional and can be used for
access to users from some restricted networks.

publish-early does not work in this case, and monitor password is stored
in buildout elsewhere anyway.

Update almost all tests to use nexedi/slapos.core!64 

Notable changes:
 * the testcase class is generated dynamically with `makeModuleSetUpAndTestCaseClass`
 * `self.slap` allow low level control of slapos
 * IP addresses are available as `_ipv6_address` and `_ipv4_address` class attributes
 * `getSupervisorRPCServer` no longer exist, instead use `self.slap.instance_supervisor_rpc` (as context manager)
 * now the framework takes care of running promises and instance step fail when instanciation fail or when promises are not satisfied (as a result, some tests checking promises are dropped)
 * test needs `slapos` and `supervisord` commands in PATH. It's important to have a recent enough `slapos` and not the `~/bin/slapos` from slaprunner which sets `$SLAPOS_CONFIGURATION` 

Following tests were not updated:
 * `caddy-frontend` as this will require more work, because this software uses shared partitions and maybe other things will need to be adapted in the tests. 
 * `slapos-master` as it is same as ERP5 and I thought maybe we can do better than duplicating code, so I left it as is for now.
 * `nextcloud` as this fail the `slapos node instance` step.

There are also a few style changes or small fixes to make pylint happy and some other fixes for problems with softwares and also a fix for seleniumrunner flaky test.

/reviewed-on nexedi/slapos!624

also drop test_turnserver_promises as promises are now checked by
framework.

slapos.cookbook:zero-knowledge.read has this limitation that because it
reads the secret file on __init__, ie. before any other parts are
installed, the value cannot be read the first time.
On first slapos node instance run, the generated turnserver-config has
an empty secret, it's only after the second execution that the secret is
generated. What happens is:
 - first run:
   - read-secret __init__ cannot find the file and does not override
     secret
   - turnserver-config uses the default ${read-secret:secret} (empty
     string)
   - gen-secret install creates the secret file
 - second run
   - read-secret __init__ finds the file and override secret with the
     value read from the file.
   - turnserver-config uses the overriden ${read-secret:secret} (good
     secret)

This was not a problem before, because some frontend promise were
always failing, so we processed this partition several time, but now it
is succesfully processed the first time.

To workadound this, we prevent turnserver from starting if secret is
empty, then promise will fail, instance will be processed a second time
and then turnserver will run with a secret.

This was causing "No JSON object could be decoded"

now that promises are satisfied, we don't retry slapos node instance
again and again, so this test starts earlier and we need to wait longer
for ERP5 to be ready.

Also tolerate that this 503 can be 404 (I think this happened a few time
on test nodes in the past)

This test was just checking the data from the first recv call, but the
data might arrive in multiple times. Call recv in a loop until we
received the end of the expected message (or until timeout when
something goes wrong).

needed for slapos.testing

and make sure we really use the develop version, by clearing version pin
that might be for another version.

Also clean up comments in version section.

This will be needed for standalone tests.

This will support shared builds from multiple directories

Prevent creating 2 wrapper for the same service if hash changed. Here, one service is exited because port is used by the firt to service to start:

    slappart6:runner-sshd-4248650e36a9a26a6481df1baffd9f58-on-watch                RUNNING   pid 27835, uptime 0:03:45
    slappart6:runner-sshd-b3b68f4278ceb84691ec27521ea229eb-on-watch                EXITED    Mar 06 04:52 PM

To achieve that, update slapos.cookbook and use hash-existing-files option of wrapper recipe

hash-existing-files list all the files used for hash that are not
handled by buildout. For those files, the hash is calculated as soon as
the __init__ function so that if there is a change in those files,
buildout will remove the existing wrapper (it will uninstall the
section) and replace it with the new wrapper.

/reviewed-on nexedi/slapos!525

Fixed slapos.core stabilises connection_dict during publish, so it is able
to calculate connection-parameter-hash the same way on client and on server, this
having same results when it is expected to have them.

Tasks:

 - [x] release `slapos.core` with nexedi/slapos.core!131 and use the version here

/reviewed-on nexedi/slapos!625

These are changes to profiles to satisfy the requirements from "checkSoftware" function from `slapos.testing`, ie:
 1. rpath is properly set on all executables.
 2. shared parts are not referencing non shared parts.

Eventhough current shared was working, we had several component with the problem 2. This was a problem in the following conditions:
 - because their hash was computed with a signature including the software installation path ( eg `/opt/slapgrid/e96b88e28dac4de1ba005bb48c6daac9/parts/something/`), shared parts were in practice never shared with other softwares. It was effective when re-installing the same software from scratch a second time, but not when using other softwares.
 - when the software was uninstalled, the shared parts stopped working because of missing paths.

As a result of these changes, some parts which used to be shared are no longer shared, but this MR also tries to share a few more parts, the goal was that all parts needed until rebootstrap were shared and also gcc and golang that are significantly large.

/reviewed-on !628

ERP5Catalog assumes that dates in catalog are UTC, this works
fine with DATETIME columns that does not carry any timezone
information, but TIMESTAMP columns are converted to mariadb timezone,
which is by default system timezone.

If we want ERP5Catalog to query on TIMESTAMP columns with the existing
logic of converting dates to UTC (because "dates are in UTC in
catalog"), we need to have TIMESTAMP columns in UTC.

This also generally seem a better default than falling back to system
timezone.

/reviewed-on nexedi/slapos!623

backports.lzma was missing from dependencies.