Add scripts in bin directory that can be called manually to apply a
new primary-url (after having edited it in the instance parameters)
by turning a primary into a replica or by making a replica follow a
new primary, all without deleting existing data - so this is useful
when the existing data does not diverge from the new primary.

Being able to start replicating from a new primary without deleting
existing data is convenient, but maybe it's better or sufficient or
both to only be able to tell a mariadb to just destroy its existing
data and re-bootstrap from scratch.

Being able to delete all existing data and start from scratch seems
necessary in any case, but it's not clear that a special feature is
needed for this: maybe it's completely enough to be able to destroy
and re-request the instance through standard slapos operations.

Questions remain for automated usage:

1. One option is to provide a url the user may simply interact with
   (clickable button ...) to call the appropriate script. This kind
   of interaction could also be used to trigger the deletion of all
   existing data for a fresh bootstrap.

2. Another option is that editing the primary-url in the parameters
   results in the appropriate script being called. This makes sense
   as a way to remain in sync with given parameters, although there
   are some drawbacks: the uncertain propagation time, and the fact
   there is no guarantee that the existing data can converge to the
   state of the new primary. This might be grounds to keep all this
   strictly manual.

In any case, this relies already on the propagation of the instance
parameters to input the new primary-url, so it may not be useful at
all for any use-case, manual or automated.

Add a script in bin directory that can be called manually to turn a
replica into a primary. Being able to turn a replica into a primary
is an absolutely necessary feature of replication. It's a necessary
step of a takoever procedure.

Questions remain for automatisation. One option is to provide a url
the user may simply interact with (clickable button ...) to trigger
the script, that may also be used automatically as part of a larger
takoever procedure.

Triggering the change to primary by editing the instance parameters
- such as by removing the replication-related parameters - does not
seem best: one immediate drawback is the uncertain propagation time
of instance parameters; another is the reliance on the availability
of the SlapOS master to trigger a usually time-sensitive operation.

Allow requesting a mariadb set-up to replicate another mariadb:
- bootstrap-url: bootstrap from a statically served backup file
- primary-url: replicate from a primary mariadb

This happens in mariadb first initialization, when no data exists yet.
That way existing data in a non-replicating mariadb cannot be deleted
by setting the replication parameters after the fact.

A promise checks that the state (replica / primary, replication source)
of the running mariadb matches the requested state; but if it doesn't,
the mariadb will not automatically converge without human intervention
if ~/srv/mariadb data directory already exists, to avoid deleting data.

This service does on-the-fly modifications on the running mariadb that
can conflict with and break replication and are anyway unneeded in the
replica.

Maybe this service should be dropped entirely and its functionality be
implemented another way.

Use a generated password and publish its url.

Remove

```
[gcc:python2]
part = gcc-8.5
```

from neoppod/software-common.cfg and dream/software.cfg as it causes
gcc being pinned to a different version before and after rebootstrap
when the initial python version is not Python2.

This reverts dc91e5d4.

When use-ipv6 is enabled, make mariadb publish an url of the form:
  `mysql://user:password@[ipv6]:port/database`

instead of:
  `mysql://user:password@ipv6:port/database`

which results in the zope instances crashing during processing
due to urlparse.urlsplit failing to parse the url.

See merge request nexedi/slapos!1668

See merge request nexedi/slapos!1668

See merge request nexedi/slapos!1668

See merge request nexedi/slapos!1668

GDAL is a library, and thus it should not mess with global settings.
This was causing the logs to be flooded with deprecation messages.

See merge request nexedi/slapos!1674

The syntax to compare strings is

      STRING1 = STRING2
                     True if the strings are equal.
      STRING1 != STRING2
                     True if the strings are not equal.

STRING1=STRING2 is not a valid syntax for strings comparisons.

This is same fix as 6cf1769d (ERP5: fix handling of repozo restoration
failure, 2024-10-24) and also a fix to a wrong error message, because
this is not restoration script, it's backup script.

Fluentbit Tail input documentation[1] says that by default maximum buffer
size is 32K which turned out to be too small in practice because we hit
a situation where enb.xlog started to have lines with ~ 34K and so
fluentbit ingestion stopped to work with the following error in
fluentbit log:

    [2024/10/23 20:30:23] [error] [input:tail:tail.0] file=/srv/slapgrid/slappart19/srv/monitor/public/enb.xlog requires a larger buffer size, lines are too long. Skipping file.

-> Fix that by increasing max buffer size to 1M which seems to be high
   enough at least for now.

Maybe it will make sense to configure this as unlimited, but I'm not
sure if going as unlimited is universally a good idea.

[1] https://docs.fluentbit.io/manual/pipeline/inputs/tail

/cc @lu.xu, @jhuge, @tomo
/reviewed-by @paul.graydon
/reviewed-on nexedi/slapos!1672

As discussed on nexedi/slapos@bb841a7b (comment 219278)
when using storage-path and passwd option, the storage file could not
be updated to the new format because of AttributeError _needs_migration.

This changes to no longer try to detect if the storage needs migration,
but just compare the expected content of the storage file during install
and overwrite the file if it is different.

This new approach also fix a behavior that re-running buildout with
storage-path option and a different passwd option did not update the
storage file. Now it is also updated.

( this also fixes a potential encoding problem on py2 )

fix a SyntaxWarning on py3.9

See merge request nexedi/slapos!1664

This test is using two connection one with a client to subscribe to a
topic and wait for message and another one with publish.single to
publish to the topic.
The test was failing from time to time because the publish might have
happened after the client was subscribed.

Refactor the test to use `loop` on the client to have more control
and be able to wait for the client to be subscribed using the
`on_subscribe` callback.

The test is also factorized, instead of having the same test twice for
IPv4 and IPv6, we pass the host as parameter.

See merge request !1665

from repozo doc:

> If a full backup is created, remove any prior full or incremental
> backup files (and associated metadata files) from the repository
> directory.

This solves a problem that after a pack some old repozo files were left
around, with this option they are automatically removed.

Products.TIDStorage was not ported to python3 and is not installed on
software-py3.cfg but the backup crontab expects tidstorage to be
present - as a result, it was silently failing to produce backups.

This brings minimal support to repozo backups on python3, without
Products.TIDStorage interraction and also extends software release test
to have a simple test checking that backups are produced and can be
restored.

Split the instances in two:
 - "default" instance is grafana, loki (for logs) and influxdb (for
metrics)
 - "agent" instance is telegraf collecting metrics and logs and sending
 it the the "default" instance.

Next steps will be that the agent becomes not used, instead the
slapos instances will be able to push metrics or logs directly, probably
using fluentbit and sending to either loki/influxdb or wendelin.

boot-image-url-select is used instead of default image being downloaded by the
software release.

If nothing is selected, the default boot-image-url-select is used, but not if
other way to obatin boot image is enabled.

The vm-img uses ISOs internally and let it handle them, thus qemu-kvm can be
just a component providing qemu machine.