We can not rely on system python here, it's a bit better to rely on
buildout's python ( the main reason is that it complies with
slapos-sr-testing not having system python ).

The implementation is a bit complicated, we can not just reference
${buildout:executable} in the profile, because doing so will cause an
infinite loop with rebootstrap and software installation never finish
because options of ca-certificates change at every run if they include
the python path. Instead, we use a pre-make-hook to rewrite the Makefile
without saving the interpreter in option. This python is only used
during the build process (to generate the certificate files) and not
used at run time, so it does not really depend on python strictly
speaking. Anyway, because it's a component used very early in the
bootstrap, we can not reference python here.

This software autodetects python2 and python3, so provide both

this component needs both python2 and python3

replace deprecated rainloop by the recommended replacement snappymail

fix promise race with check-nc-install, CAN_INSTALL is removed before
slapos specific configuration is finished. Create a file at the end and
check its presence as well.

now that slapos uses php 8

not used and seems not working (using php5)

stdout/stderr is sent to supervisor and we don't want buffering the
output in that case, because it introduces a delay in the output and
break the usage of `slapos node supervisorctl fg` to re-attach the
process and use pdb

This test fail since 450a6999 (slapos-master: Add missing
RevocationCheck on apache, 2023-06-01) and we are reimplementing all
this differently. Mark the test as expected failure for now, so that
it does not impact the general state of test suite.

ORS software-release emits many *.json.log files with information about current
state of eNB and radio units. Those files are used by code in slapos.toolbox to
implement corresponding promises. The files, it seems, try to match JSON Lines
format(*) with every line containing a JSON object describing timestamp, log
level and log payload of corresponding log entry.

So far all is good. One peculiarity, however, is that, contrary to everything
else, the content of the `data` field in the logs is not emitted via
JSON-encoding. Initially, starting from 4968d55c (ors-amarisoft: get eNB stats
and add promises), that `data` payload was being emitted via %s py formatting,
which is similar to what JSON-encoding would produce, but is not it exactly.
For example if log payload is {'abc': 123} then %s and JSON would quote `abc`
differently:

    {'abc': 123}  # %s
    {"abc": 123}  # JSON

That would not be a big deal if what %s produces would be still valid JSON
accepted by JSON-decoder, but unfortunately it is not so: JSONDecodeError is
raised when {'abc': 123} is tried to be decoded.

The code in slapos.toolbox, so far, is handling this situation by feeding to
JSON-decoder original input with ' characters replaced by "

    json.loads(line.decode().replace("'", '"'))     (+)

which more or less works in basic cases, but breaks when log payload contains '
character itself. For example for {"a'bc": 123} replacing gives {"a"bc": 123},
which, being invalid JSON, raises

    JSONDecodeError: Expecting ':' delimiter: line 1 column 5 (char 4)

when fed to json.loads() .

I've actually hit this problem for real with amarisoft-rf-info.json.log where
the data part contains output of `rf_info` looking e.g. as

    TX underflows=0 RX overflows=0
    TRX SDR driver 2023-09-07, API v15/18
    PCIe CPRI /dev/sdr0@0:
      Hardware ID: 0x4b12
      DNA: [0x0048248a334a7054]
      Serial: ''
      FPGA revision: 2023-06-23  10:05:24
      FPGA vccint: 0.98 V
      FPGA vccaux: 1.76 V
      FPGA vccbram: 0.98 V
      FPGA temperature: 71.9 °C
      Clock tune: 0.0 ppm
      NUMA: 0
      CPRI_option: '5' (x8) lock=no
      DMA0: TX fifo: 66.67us  Usage=16/32768 (0%)
      DMA0: RX fifo: 66.67us  Usage=16/32768 (0%)
      DMA0 Underflows: 0
      DMA0 Overflows: 0
    PCIe CPRI /dev/sdr0@1:
      Hardware ID: 0x4b12
      DNA: [0x0048248a334a7054]
      Serial: ''
      FPGA revision: 2023-06-23  10:05:24
      FPGA vccint: 0.98 V
      FPGA vccaux: 1.77 V
      FPGA vccbram: 0.98 V
      FPGA temperature: 71.7 °C
      Clock tune: 0.0 ppm
      NUMA: 0
      CPRI_option: '5' (x8) lock=HW+SW rx/tx=46.606us
        Port #0: T14=46.606us
      DMA0: TX fifo: 66.67us  Usage=16/32768 (0%)
      DMA0: RX fifo: 66.67us  Usage=16/32768 (0%)
      DMA0 Underflows: 0
      DMA0 Overflows: 0

which lead to breakage when trying to load that log due to ' symbols.

The fix is simple: change the code, that emits *.json.log to emit data payload
via json.dumps instead of %s. We can do that without breaking anything because,
contrary to enb.xlog, those *.json.log are not uploaded to any separate system
and currently are only used to implement promises without preserving log files
in any storage for any non-small amount of time. In other words, currently
those *.json.log are temporary files whose format can be adjusted without
strongly caring about backward compatibility.

b32b4a8e (software/ors-amarisoft: general improvement for RU (logs/promises/input parameters))
already changed %s to json.dumps for amarisoft-stats.json.log , but left all other logs at %s.

-> Fix this for everything by replacing %s to json.dumps in all generated logs.
-> Corresponding slapos.toolbox adjustments are in nexedi/slapos.toolbox!120.

/reviewed-on nexedi/slapos!1447
/reviewed-by @jhuge, @jerome, @tomo
/cc @lu.xu, @xavier_thompson, @Daetalus

(*) https://jsonlines.org/
(+) see e.g. https://lab.nexedi.com/nexedi/slapos.toolbox/blob/453dce5f/slapos/promise/plugin/util.py#L50

after 6e8b97ee (Publish IPv6 URL to SlapOs master., 2023-10-02) the
parameters are different

after recent updates, probably 3c514224 (component/freetype: version
up 2.13.2, including security fixes., 2023-10-10), the rendering is
different.

See merge request nexedi/slapos!1443

See merge request !1437