/reviewed-on nexedi/slapos!622

https://github.com/grafana/loki/blob/master/docs/README.md

> Grafana Loki is a set of components that can be composed into a fully featured logging stack.

The plan is not to really use this, just to see what interesting ideas are in there.

/reviewed-on nexedi/slapos!658

/reviewed-on nexedi/slapos!695

Many parts were still built for nothing now that cloudooo is removed
from this SR. This saves 14MB.

Many extends and many parts from ${buildout:parts}
also don't need to be listed explicitly.

Monitoring promise break instanciation:
  UserError: No section named 'monitor-base-promise'

and telegraf startup script is using influxdb promise, because it has a
bug that it needs to start only after influxdb is available, so it does
not look so easy to switch to monitoring stack.

Also, this software is experimental (and also is conceptually a
monitoring software on its own), so it does not need monitoring for now.

This reverts commit 48b6c27c.

Because we run egg tests with setup.py test, which installs missing
eggs, if an egg was not installed by buildout, then it installed before
running test. This was the case for mock, which is now python3 only (
https://pypi.org/project/mock/4.0.0b1/ ) and we started to see test
failures.

To solve this issue, refactor the setup definition to use
extra_requires, which seems to work fine in buildout now. Keep
test_requires because it's the what `python setup.py test` uses.

Clean buildout profiles to install slapos.cookbook[test] for test
instead of duplicating the content of test_requires.

/reviewed-on nexedi/slapos!690

This enable vieiwing applications logs directly in grafana, as long as
loki agent can access the log file and the regular expressions to parse
the logs are defined

Other changes:
 - provision data sources automatically so that user do not have to add
   them manually
 - update TODO in README
 - introduce a SR test
 - switch to dep to install dependencies, because that's how these
  packages manage their dependencies.

Versions up:
  grafana: v6.3.0
  telegraf: v1.11.1-0
  influxdb: v1.7.8rc1
New:
  loki: v0.3.0

It seems some programs can only be built by the same gcc than the one used to
build golang itself.

For example, when system gcc is gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516,
grafana-server fail to build with:

    /data/slappart11_testnode/cqg/inst/test0-0/tmp/shared/golang1.12/fbee59cfb3c995382cf70d409615aa54/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /tmp/go-link-305363633/000000.o: file not recognized: File format not recognized
    collect2: error: ld returned 1 exit status

Also generally, if we don't trust system gcc to build golang, we cannot trust
it to build golang programs.

The downside is that if components or software want to use a specifig
golang version they have to set both golang= and the corresponding
gcc-bin-directory= in their [gowork]

/reviewed-on nexedi/slapos!627

For now we just use a directory inside the partition as shared parts,
but we could also maybe one day also inject the shared_part_list from
this slapos.

This recipe already use all options defined in buildout for substitution
in the erp5testnode.cfg.in

slap_connection is a deprecated alias, we use kebab case in buildout
profiles.

/reviewed-on nexedi/slapos!688

This requires the "prerm" plugin be enabled to be used in addition to devperm. 

I tested on production and it works fine.

/cc @tomo

/reviewed-on nexedi/slapos!687

If there is IPv6 on both interface, Linux will put a default route on
ens3 preventing IPv6 to work correctly on ens4.
Even disabling totally IPv6 from inside the host on ens3 may not work.
It is safer to disable it from qemu process directly. Also, it will ease
the configuration of the host.

NBD server is not running until an image is uploaded. Make sure that the
promise doesn't fail until the image is there. Also display a WARNING in
the connection parameter to say that the NBD server is not running.

Recent fixes for proftpd were still not correct, we still have issues with rpath for mod_auth_web and proftpd itself which uses libcap if available, error was:
    
    ======================================================================
    ERROR: setUpModule (test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slappart11/cqg/soft/1ae6b62cded47f49a2e77af2f372cf2a/parts/slapos.core-repository/slapos/testing/testcase.py", line 154, in setUpModule
        installSoftwareUrlList(cls, [software_url], debug=debug)
      File "/srv/slappart11/cqg/soft/1ae6b62cded47f49a2e77af2f372cf2a/parts/slapos.core-repository/slapos/testing/testcase.py", line 348, in installSoftwareUrlList
        raise e
    RuntimeError: /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/bin/ftpdctl uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2 /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/libexec/mod_auth_web.so has some not found libraries:
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/libexec/mod_auth_web.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /srv/slappart11/cqg/inst/test0-0/tmp/shared/curl/da798e87b4f10345c33fbae0a4593114/lib/libcurl.so.4)
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/sbin/proftpd uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/sbin/in.proftpd uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2
    Ran 0 tests in 530.067s
    ----------------------------------------------------------------------

/reviewed-on nexedi/slapos!685

Because we were simply using "setupClass", all classes where storing
snapshots in the same folder, so if more than one class failed we also
had an error in storing snapshot.

/reviewed-on nexedi/slapos!683

libcap is a system library that some system may have and some may not,
but we don't want slapos software to depend on the base system.

See also d0a88346

This build command never used the LDFLAGS that we set, so instead
rewrite this to inject path as a -R option that will be passed to
libtool.