When PYTHONNOUSERSITE environemnt variable is set, python does not load user
site.
User site packages may contain a different version of buildout, which would
be selected when slapos runs buildout, causing unexpected behaviours.

Some buildout profiles are expecting this.

 - Use dict.items() to iterate on dict, instead of dict.keys() then dict[key]
 - don't use six.iterkeys(), this dict is small, so we don't need iterkeys()
   on python2 and can already write python3 version
 - don't % string when invoking logger, logging framework do it for us and do
   it only if message will be logged.

    For historical reason some aggregated packing lists could be merged (due a bug). This changes introduced backward compatibiltiy with those cases, rather them require a data migration.

    This is frequently updated on production and development instances.

Show exactly what type is used for a request, it makes debugging easier.

By default proxy checks if software type is None, then it's setting it to
RootSoftwareInstance, but if it comes as "" (empty string) it's kept as is.

So for consistency with proxy set default software type, so that proxy can
manage it correctly.

    This helps sync ZODB and mariadb (considering mariadb is newer). This is useful when restore a clone or synchronize after restore the mariadb catalog.

   The most appropriated way is reindex the site, however in case of slapos master (with large amount of data) it is too long (make this solution more appealing.

   If the Destroyed Hosting subscription is only present on Invalidated Open Orders, we just skip and converge the Hosting Subscription.

  This means the user deleted everything he had at once, too fast, before everything get processed.

See merge request nexedi/slapos.core!263

See merge request nexedi/slapos.core!262

See merge request nexedi/slapos.core!261

    ... else it leads to nowhere.

      He can access it or open a ticket.

Introduce new "Managed Resources" and assorted fixes accompanying nexedi/slapos!840 

Also respect meaning of verbose (more output in the console) and debug (drop in debugger on errors and keep things around for inspection)

See merge request nexedi/slapos.core!259

Really remove security_uid from roles_and_users table

Don't pollute console output, store this in log file

When we execute crontab commands, these commands might be python scripts, so
we don't want the PYTHONPATH set by `python setup.py test` to apply to these
scripts.

Respect semantics of "verbose" vs "debug", to make it possible to run tests in
debug mode but without verbose messages.

Also adjust "Starting", this message was not clear

No need to query the catalog multiple times for every partitions

In SlapOS software release tests we often have to start a process or store
temporary files in a directory for the lifetime of the test. Cleaning up the
resources is a bit error prone and source of code duplication.

This introduce a registry of resources that are automatically created and freed
by the test framework, to simplify tests.

See merge request nexedi/slapos.core!250

scan installed eggs and check if they have known vulnerabilities listed on
https://github.com/pyupio/safety-db

See merge request nexedi/slapos.core!255

cliff complete command only supports bash, but fish offers much more user friendly
completions.

cliff uses a pluggable system, but this where new shell completions can be registered,
but the plugin (CompleteFish) can not access the command manager to get more
details about options, since fish can also show help of suggestion, we want to
access the help of each option to show this during suggestions.
Also we want to make some more clever suggestions (for example slapos node start complete
with services identifiers).
For these reasons we need to register a command to replace the default complete command
and can not achieve this with simply a cliff.formatter.completion entry point.

This can be installed with:

    slapos complete > ~/.bash_completion.d/slapos
    slapos complete --shell fish > ~/.config/fish/completions/slapos.fish

github.com might be not available in many places.

In a scenario of an instance with child instances which does not properly
support destruction, a scenario like this happens:

 1. first report detect that it's destroying a root partition but child
partitions are still started and request destruction of these child instances
 2. child partitions are destroyed
 3. root partition is destroyed

For this case we need to run 3 times

So far implementation of slapos proxy was incomplete regarding partition destructions:
A partition requested as destroyed was properly destroyed "physically", files were
removed from filesystem, services were removed from supervisor, but the partition was
still marked as busy in proxy database, so it was never re-used in subsequent requests.

To solve this, implement the destroyedComputerPartition endpoint to mark the partition
as free.

This is straightworfard for root partitions, but the cases of child partitions was a bit
more tricky, especially because software releases usually does not implement properly
the chain of deletion (it seems this is not properly implemted with softwaretype, only
with switch_softwaretype) and generally because we don't want to leave orpheans partitions.

In the case of ERP5 implementation of SlapOS master we have an alarm which garbage
collect in these cases, so we implement something to reach similar goals. We refuse freeing
a partition while their child paritions are not freed - and we request deletion of these
partitions so that they are deleted on next slapos node report run. Then after a few runs,
the whole partition tree will be completely destroyed and freed, even for software which
does not implement deletion properly.