slapos.cookbook:softwaretype tends to lowercase keys in each section, which
has been undetected due to using lowercase references of slaves in the tests.

By restructuring information in the sections, and putting slave references
inside of dumped part of information, now the slave reference case is kept.

Also real care was taken to stabilise published lists by sorting them, as it
also slipped, that they could be unstable.

Tests has been updated to catch this issue, also other tests were fixed, as
they had wrong assertions.

validators.url is enough, even for Caddy, to check that URL is correct, and
caddy_backend_url_validator was introduced before validators.

Also calling an external command for each slave takes a lot of time.

The password is anyway present in the section itself, so it's eventual change
will result with reinstalling the section.

stop-on-error during instantiation can lead to endless instantiation in some
cases, which disallows to create services required for given part to pass, and
in the same time in many cases the called scripts are smart enough to continue
and restart on error.

It's a dict, and in SlapOS usage of Jinja2 it's good to see the type of
a variable immediately.

"parameter_dict" says nothing, whereas "software_parameter_dict" explains
source and purpose of the information.

There is needless duplication of information.

That's true, that those are templates, but the important information which
shall be in the name of the parameter is its purpose - a profile.

Each node allows for global statistic access for full backend-haproxy, which
is exposed using special frontend, and then transferred back to the master
partition, so that the administrator can access it.

By default do not offer authentication certificate, the switch
authenticate-to-backend can be used on cluster or slave level to control
this feature.

ssl_proxy_ca_crt can be just empty value, and that's not acceptable.

Customized configuration support is not used since introduction of
Caddy software, so there is no need to support it anymore.

Instead of forcing to set monitor port in some cases, just generate them,
so it's possible to correctly instantiate caddy-frontend on one partition
scenario like in webrunner or tests.

/reviewed-on nexedi/slapos!633

Prevent creating 2 wrapper for the same service if hash changed. Here, one service is exited because port is used by the firt to service to start:

    slappart6:runner-sshd-4248650e36a9a26a6481df1baffd9f58-on-watch                RUNNING   pid 27835, uptime 0:03:45
    slappart6:runner-sshd-b3b68f4278ceb84691ec27521ea229eb-on-watch                EXITED    Mar 06 04:52 PM

To achieve that, update slapos.cookbook and use hash-existing-files option of wrapper recipe

hash-existing-files list all the files used for hash that are not
handled by buildout. For those files, the hash is calculated as soon as
the __init__ function so that if there is a change in those files,
buildout will remove the existing wrapper (it will uninstall the
section) and replace it with the new wrapper.

/reviewed-on nexedi/slapos!525

Sorting will make dumped data "canonical", so it will limit amount of order
based changes.

Destroyed nodes shall be just destroyed, and there is no need to send whole
configuration to them.

This will stabilise list of published slaves.

Server returns slave list with request and publish keys, but only request keys
are important.

In order to avoid needless updates and nonsense data remove those polluted
keys before publishing information about each slave.

/reviewed-on nexedi/slapos!597

Due to missing monitor frontend in tests this change is not covered by tests.

/reviewed-on nexedi/slapos!592

Frontend operator shall have easy access to information about rejected
slaves, possibly the best in the JSON file.

Also the keys for the human readable information are slave's titles, not
references.

The information is published via hand crafted HTTPS endpoint.

Note: The SSL certificate is generated manually. Existing caucase is special
      for KeDiFa, this is another step to move all generated certificates (or
      otherwise self-signed) to internal, full automatic caucase.

By default whole slave makes websocket connection to the backend.
With websocket-path, only the path has websocket style connections,
the rest is standard HTTP.

There is no need anymore to have two processes for normal and nginx slaves,
as nginx ones are served by caddy anyway.

Also inform the requester that type:eventsource is not implemented.

This reverts commit 7993ff81.

Custom configuration checks are hard to be trusted, as they can impact too
many aspects of running frontend.

Frontend administrator knows the risks of custom configuration, and shall take
proper care.

/reviewed-on nexedi/slapos!543

This mostly useful during tests to have stable results, especially when
some slaves are rejected.

This change is expected to be no-op during normal run.

Note: The slave rejection system does not guarantee any ordering, as the sort
      order can change, because of parameters can reorder slaves. Thus, even
      if slave A was requested before slave B, and they conflict each other,
      slave A can be rejected instead of "expected" slave B.

It is better to have automation similar to previous implementation by
default.