See merge request nexedi/slapos!1314

Updated eggs for software/jupyter:
  pathlib2 2.2.1 -> 2.3.5
  pytz 2020.4 -> 2022.2.1
  scandir 1.5 -> 1.10.0
  wcwidth 0.1.7 -> 0.2.5

Updated eggs for stack/slapos.cfg:
  prompt-toolkit 2.0.10 -> 3.0.19
  Pygments 2.7.4 -> 2.9.0
  traitlets 4.3.3 -> 5.0.5

Update ipython to fix https://github.com/ipython/ipython/issues/11590

See merge request nexedi/slapos!1201

In the same time extract described variables to be available in the comment
context.

Changes:

 * clean up tests from simply removing headers before asserting
 * check out unstable headers and correct them
 * improve assertion name

Adapt tests to cleaner responses from Haproxy.

It's just type:websocket without any path and with transparent parameter,
thus resulting with providing X-Real-Ip to the backend.

haproxy supports websocket by default everywhere, so reuse this and make
type=websocket only needed if some specific configuration like
websocket-path-list or websocket-transparent have to be set.

As a result of implementing this feature X-Forwarded-Proto and
X-Forwarded-Port header support is now enabled per slave and not just
globally.

X-Real-Ip is only available for websocket paths, previous implementation
was simply wrong, and this has been fixed and asserted in tests.

Tests have been slightly updated to make it easier to follow the real
websocket logic.

Cipher translation is implented on the node, so that old style and new style
nodes can co-exists in the same cluster, thus making partial upgrade possible.

Caddy was automat(g)ically adding Content-Type header to the backend
response, but Haproxy does not do it, and change of the servers shall
follow generic approach that not additional information is added to the
response by the CDN.

In case if Date header is present do nothing otherwise add one.
Test backend is adapted to have full control over headers sent.

Backward incompatible change happens, as the Date header is generated
before passing the request to the cache, thus resulting with caching it.
Nevertheless this is good change, as previous behavior was simply wrong.

mpm-graceful-shutdown-timeout is dropped, as it's historical leftover and never
really useful in the caddy-frontend CDN usage context - stopping the server is
the most rare situation, and any grace period is solved eventually outside of
the running process (like redirecting traffic elsewhere before stopping).

haproxy since 2.4 requires correctly ending files, thus do it by adding
in-jinja comment which will result with LF in the end.

It's based on phased out caddy-frontend, especially as next step is to drop
Caddy software from the software release.

2.6 implementes url normalization, needed on front facing part, see
https://cbonte.github.io/haproxy-dconv/2.6/configuration.html#4.2-http-request%20normalize-uri

We don't care about exact exception message, but we only want to test
that the client can't connect anymore once banned. Relaxing this
assertion should stabilize previously flaky
'TestBan.test_client_are_banned_after_5_wrong_passwords'.

See nexedi/slapos@97f4a5fb (comment 150049)
and nexedi/slapos!1305 (comment 175255)
for additional context.

The workaround seems to be required for several OlinuXino boards (https://github.com/OLIMEX/OLINUXINO).
See !915

Disable java

Do not update vscode-uri yet, because theia does not support vscode-uri 3 currently, so we have to keep our patch.

See merge request nexedi/slapos!1315

Sometimes this script is slow, add some logging to display progress
also use a Session to pool connections and apply a timeout to fail more
faster in case of network/server problems

This is big, fails on raspberry pi and we don't really use it

See merge request nexedi/slapos!1303

The NEO/go commit kirr/neo@8c974485
changed the URL with which a NEO/go client can be loaded from

    // neo://name@master1,master2,...,masterN?options

to

    neo(s)://[credentials@]master1,master2,...,masterN/name?options

We need to apply this change in the NEO URL structure to instance-wcfs.cfg.in
to use WCFS with NEO storage.

Why?
====

The URL in instance-wcfs.cfg.in is finally send to wcfs binary, which
will open a NEO/go client with this URL via the "openClientByURL" function [1].
If the URL formatting instance-wcfs.cfg.in follows the old pattern,
NEO/go is unable to create a NEO/go client and may even raise an
exception [2], because information regarding NEO server is at the wrong
position of the URL.

---

[1] See https://lab.nexedi.com/kirr/neo/blob/87199da2b163c09ed3946b7ab9bb00a5b987d377/go/neo/client.go#L416-534
for openClientByURL.

[2] For instance when using the old pattern of putting the cluster name
in the "user" part, NEO/go will raise "credentials can be specified only
with neos:// scheme", because NEO/go reserves the "user" part for
encryption information,
see https://lab.nexedi.com/kirr/neo/blob/87199da2b163c09ed3946b7ab9bb00a5b987d377/go/neo/client.go#L449

---

/reviewed-by @kirr
/reviewed-on nexedi/slapos!1307

Go1.18 is the next stable go release. There is already Go1.19
(released 2022-08-02), but we incrementally increase versions.
For Go1.18 there are already many bug fix releases (last one
2022-12-06) which gives it a more stable/mature impression.

/reviewed-on nexedi/slapos!1305

Go 1.18 is the incremental improvement for earlier go 1.17 release.
Is "is a significant release, including changes to the language,
implementation of the toolchain, runtime, and libraries". Please find
all details in the official release note:

    https://go.dev/doc/go1.18

It was released on 2022-03-15. Due to the go promise of compatibility
most software should still compile without any problems.

/reviewed-on nexedi/slapos!1305