See merge request nexedi/slapos!1243

See merge request nexedi/slapos!1245

See merge request nexedi/slapos!1229

our shacache uploading code for pypi cannot handle the prefix 0 in the version number right now.
it will upload the version 2020.4.5.1 instead of 2020.04.05.1

pango needs glib/glib-mkenums, and glib-mkenums needs python2.7:
-mkenums: /usr/bin/env: ‘python2.7’: No such file or directory

Direct logs to instance's var/log and call postfix's logrotate command,
instead of the slapos-hostile syslog.
Unfortunately, rotated logs will stay in that directory, unlike the
de-facto standard for slapos.

Resolve warnings emitted by postfix commands themselves.

Fixes build on recent libc headers:
set -e; for i in src/util src/global src/dns src/tls src/xsasl src/master src/milter src/postfix src/fsstone src/smtpstone src/sendmail src/error src/pickup src/cleanup src/smtpd src/local src/trivial-rewrite src/qmgr src/oqmgr src/smtp src/bounce src/pipe src/showq src/postalias src/postcat src/postconf src/postdrop src/postkick src/postlock src/postlog src/postmap src/postqueue src/postsuper src/qmqpd src/spawn src/flush src/verify src/virtual src/proxymap src/anvil src/scache src/discard src/tlsmgr src/postmulti src/postscreen src/dnsblog src/tlsproxy src/posttls-finger src/postlogd; do \
 (set -e; echo "[$i]"; cd $i; make 'WARN=-Wmissing-prototypes -Wformat -Wno-comment -fcommon' update MAKELEVEL=) || exit 1; \
done
[src/util]
gcc -I. -I../../include -DUSE_SASL_AUTH -DUSE_CYRUS_SASL -DUSE_TLS -DHAS_PCRE -DHAS_DB -I/home/vifib/software/b3708eca6720657934c5dee0b8247042/parts/libdb/include -I/home/vifib/software/b3708eca6720657934c5dee0b8247042/parts/pcre/include -I/home/vifib/software/b3708eca6720657934c5dee0b8247042/parts/openssl/include -I/home/vifib/software/b3708eca6720657934c5dee0b8247042/parts/cyrus-sasl/include/sasl -I/home/vifib/software/b3708eca6720657934c5dee0b8247042/parts/libnsl/include -DNO_EAI -DHAS_DEV_URANDOM -UUSE_DYNAMIC_LIBS -DDEF_SHLIB_DIR=\"no\" -UUSE_DYNAMIC_MAPS -Wmissing-prototypes -Wformat -Wno-comment -fcommon  -g -O -I. -DLINUX5 -c attr_clnt.c
In file included from attr_clnt.c:87:
./sys_defs.h:763: warning: "HAS_DB" redefined
  763 | #define HAS_DB
      |
<command-line>: note: this is the location of the previous definition
In file included from attr_clnt.c:88:
/usr/include/unistd.h:363:13: error: conflicting types for 'closefrom'; have 'void(int)'
  363 | extern void closefrom (int __lowfd) __THROW;
      |             ^~~~~~~~~
./sys_defs.h:1512:12: note: previous declaration of 'closefrom' with type 'int(int)'
 1512 | extern int closefrom(int);
      |            ^~~~~~~~~
make: *** [Makefile:192: attr_clnt.o] Error 1
make: *** [Makefile:94: update] Error 1

Also, switch from Wietse's own source ftp to a mirror.

The default "skin" changed, which results in diagram being grey instead
of looking like rational rose. The previous behavior can be kept by
using instruction `skin rose` in diagram code (see the changes in tests
for example)

python3.8 can run only buildout versions greater than 2.7.1+slapos019
which restrict a lot the number of SR buildable by slapos.

See merge request nexedi/slapos!1240

this prevents ResourceWarning because this way the sockets are closed

see commit e6deadaca7f4469ab32e028cdbb029acc9c7d3c3 from slapos.core:

We fork a subprocess to serve requests, but the listening socket is bound
in server constructor and inherited by subprocess. With this pattern, we
have to explicitly close the socket from parent process to prevent the
leak.

Now that this test is python 3, test output has lots of warnings like this:

    software/caddy-frontend/test/test.py:305: ResourceWarning: unclosed file <_io.TextIOWrapper name='software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlave.test00cluster_request_instance_parameter_dict.txt' mode='r' encoding='UTF-8'>
      test_data = open(test_data_file).read().strip()
    ResourceWarning: Enable tracemalloc to get the object allocation traceback

revert nexedi/slapos@fac8ceb5  now that mroonga install.sql is fixed

See merge request nexedi/slapos!1242

See merge request !1238

html5as-base, grafana and metabase are python3 software releases
and we don't need to run these tests twice for python2 and python3.

This continues the changes from a576000f (software/html5as: serialize
as json-in-xml, 2022-08-19) to fix a test failure and serialize also
in json-in-xml for the replicate software type.

Minor changes to tests were made, to use assertIn for better messages
in case of assertion failures

See merge request nexedi/slapos!1239

after slapos commit cc7f36a0 (component/defaults: use python3 by
default, 2022-08-27) this test accidentally switched to using python3.

using python3 showed an incompatibility:

   File ".nxdtest", line 23, in <module>
     TestCase('transaction',['python', '-m', 'unittest', 'discover', '-s', P(transaction)])
   File ".nxdtest", line 22, in P
     def P(path): return os.path.join(path, '..')
   File "lib/python3.8/posixpath.py", line 90, in join
     genericpath._check_arg_types('join', a, *p)
   File "lib/python3.8/genericpath.py", line 155, in _check_arg_types
     raise TypeError("Can't mix strings and bytes in path components") from None
 TypeError: Can't mix strings and bytes in path components

that we also address here by using universal_newlines to the subprocess

test.cfg becomes the python3 version
a new test-py2.cfg is introduced
test-py3.cfg is removed

cf slapos commit cc7f36a0 (component/defaults: use python3 by default, 2022-08-27)

Change python2 softwares to use python2 explicitly and python3
softwares to use the default, but leave a few softwares (html5as,
html5as-base, fluent-bit, packer, seleniumrunner) using the default
python, as they were using python2 not because they only support python2
but because they don't have strong dependency to the python version
being used.

See merge request nexedi/slapos!1235