If a test fails we may want to invest further why this test fails and
reading the NEO server log files can be an efficient way for these
investigations. Therefore we should keep log files if a test fails.

If tests fail we may want to check NEO logs. The new function LogContent
of the NEOSrv interface provides an API to fetch the relevant parts of
the log file of a NEO server. This API can be used by test code in order
to automatically print relevant server log information if a test fails.

For example option `compress` controls kind of compression that _client_
performs when saving data to server. Similarly cache-size, logfile and
read-only adjust on-client behaviour, not server.

From nexedi/neoppod!18 (comment 124725) :

    In general the most correct thing to do is:

    - use host part for where to connect (host:port, list of host ports, UNIX socket, etc)
    - use path part to identify a database or other on-server resource
    - use query part for parameters that are passed to remote server (e.g. `storage` in case of ZEO)
    - use fragment part for local parameters that are not passed to remote server (e.g. local `logfile`)
    - use credentials part for things required to authenticate/encrypt.

    To normalize an URL wcfs client would drop credentials and fragment, but keep host, path and query.

    Fragments are documented not to be sent to remote side and to be evaluated by local side only.

-> Move options that control client behaviour to fragment.

Because list of masters and cluster name must be already present in
netloc and path. Previously e.g.

	neo://α,β,γ/db?master_nodes=a,b,c"

would mean to use master nodes {a,b,c} not {α,β,γ}. Now it is treated as
invalid URI to remove ambiguity. Same for cluster name.

* master:
  go/neo: Expand user prefix in TLS key/cert paths
  go/neo/interal += xpath, xfilepath

This patch fixes a discrepancy between NEO/py and NEO/go: NEO/py expands
the '~' and the '~username' prefix in the file path of the TLS
certificate/key files [1]. This syntax is used in NEO/py SlapOS SR [2].
We need to fix this discrepancy in NEO/go in order to use TLS encryption
with NEO + WCFS.

[1] https://lab.nexedi.com/nexedi/neoppod/blob/7c539f0f/neo/lib/config.py#L149
and https://lab.nexedi.com/nexedi/neoppod/blob/fa63d856/neo/lib/app.py#L25-31

[2] https://lab.nexedi.com/nexedi/slapos/blob/397726e1/stack/erp5/instance-zodb-base.cfg.in#L18-20
and https://lab.nexedi.com/nexedi/slapos/blob/a8150a1a/software/neoppod/instance-neo-input-schema.json#L62

/reviewed-by @kirr
/reviewed-on !1

The xfilepath package supports resolving filepaths with a user prefix
to absolute paths: it converts '~' and '~username' to $HOME of user
(as it's done by for  instance bash). No builtin golang module supports this
functionality [1]. We need this functionality in order to imitate the
behaviour of NEO/py in NEO/go [2].

---

[1] https://stackoverflow.com/questions/47261719/how-can-i-resolve-a-relative-path-to-absolute-path-in-golang)
[2] nexedi/slapos!1307 (comment 17574)

/reviewed-by @kirr
/reviewed-on !1

* master:
  go/neo/t/neotest: Use python -c 'print ...' in a way that works on both py2 and py3
  go/neo/t/tzodb.py: Fix zhash for Python3

Without parenthesis it was failing on py3:

	(neo) (py3.venv) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t$ ./neotest info-local
	date:   Wed, 18 May 2022 11:05:50 +0300
	xnode:  kirr@deca.navytux.spb.ru (2401:5180:0:af::1 192.168.0.3 (+ 1·ipv4))
	uname:  Linux deca 5.10.0-13-amd64 #1 SMP Debian 5.10.106-1 (2022-03-17) x86_64 GNU/Linux
	cpu:    Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
	  File "<string>", line 1
	    print '%.2fGHz' % (400000 / 1E6)
	          ^
	SyntaxError: invalid syntax

On py3 dict.keys() returns iterator instead of list:

    $ ./tzodb.py zhash
    Traceback (most recent call last):
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 141, in <module>
        main()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 138, in main
        zhash()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 59, in zhash
        optv, argv = getopt(sys.argv[2:], "h", ["help", "check=", "bench="] + hashRegistry.keys())
    TypeError: can only concatenate list (not "dict_keys") to list

* master:
  go/internal/xurl: New package (draft)

With xurl.ParseQuery being simpler analog of url.ParseQuery.
Simpler: It returns regular map instead of url.Values by not allowing duplicates.

Instead of requiring users to explicitly mark their tests as finished in
particular subject-to-deadlock places, tracetest was fixed to
unconditionally detect "extra sends" deadlocks automatically:

kirr/go123@3b19f68c

go get: upgraded github.com/fsnotify/fsnotify v1.4.10-0.20200417215612-7f4cf4dd2b52 => v1.5.1
go get: upgraded github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b => v1.0.0
go get: upgraded github.com/gwenn/gosqlite v0.0.0-20201008200117-82e079acf5b6 => v0.0.0-20211101095637-b18efb2e44c8
go get: upgraded github.com/gwenn/yacr v0.0.0-20200112083327-bbe82c1f4d60 => v0.0.0-20211101095056-492fb0c571bc
go get: upgraded github.com/kisielk/og-rek v1.1.0 => v1.2.0
go get: upgraded github.com/soheilhy/cmux v0.1.5-0.20210205191134-5ec6847320e5 => v0.1.5
go get: upgraded github.com/tinylib/msgp v1.1.5 => v1.1.6
go get: upgraded golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 => v0.0.0-20211111160137-58aab5ef257a
go get: upgraded golang.org/x/sys v0.0.0-20210301091718-77cc2087c03b => v0.0.0-20211111213525-f221eed1c01e
go get: upgraded golang.org/x/text v0.3.5 => v0.3.7
go get: upgraded lab.nexedi.com/kirr/go123 v0.0.0-20210128150852-c20e95f0f789 => v0.0.0-20210906140734-c9eb28d9e408

Highlights:

- og-rek brings support for pickle protocol 5
- go123  brings in support for Go1.17

* master:
  go/zodb/btree: Change V<op> family to also provide visited node key coverage on visit callback
  go/zodb/btree: Add KeyRange type
  go/zodb/btree: Introduce constants for min/max key value
  go/zodb/btree: tests: Don't forget to close storage
  go/zodb/btree: Cosmetics

WCFS needs to know key coverage for every visited node.

WARNING: this is API change.

KeyRange represents [lo,hi) key range. It simplifies working with ranges
of keys. We will use it in the next commit.

KeyRange originated in WCFS and was copied from there:

https://lab.nexedi.com/kirr/wendelin.core/blob/57be0126/wcfs/internal/xbtree/blib/keyrange.go

We were already using math.Min<Key> in one place, but the number of such
places is going to increase. -> Keep min/max definition in only one place.

NOTE: db was already being closed in the test's code.

* t+btree:
  .
  .
  .
  .
  .
  .
  .
  X Start reworking BTree to provide keycov on visit callback

Due to security concerns, go1.17, even though very reluctantly,
similarly to Python broke backward compatibility and stopped to treat ';'
as separator when parsing URL queries.

We still want to accept ';' as separator in credential part.
-> Work it around by replacing ';' -> to '&'.

See https://golang.org/doc/go1.17#semicolons and https://github.com/golang/go/issues/25192 for context.

See also: cf685fb5 (fixup! Y client: Fix URI scheme to move credentials out of query)

parse_qsl no longer treats ';' as valid query separator for security
reason because most proxies did not do so and it was possible to poison
proxy cache due to difference in query separator handling (see bugs.python.org/issue42967).

To handle credentials we don't have any proxy here, and it is still
perfectly valid to use ';' as credentials separator.

-> Fix it with ';' -> '&' replace workaround, before feeding credentials
string to parse_qsl.

Amends: b9a42957.

* master:
  go/internal/xtesting: Add missing X
  go/zodb/{fs1,zeo}: ~staticcheck
  go/zodb/btree: Fix missing return on data-consistency error
  go/zodb, go/zodb/btree: Fix go generate after rename on zodbtools side

staticcheck says:

	xtesting.go:386:2: this value of err is never used (SA4006)

fs1/format.go:204:2: only the first constant in this group has an explicit type (SA9004)
zeo/proto.go:56:2: only the first constant in this group has an explicit type (SA9004)

staticcheck reports

    ziobtree.go:606:4: Errorf is a pure function but its return value is ignored (SA4017)
    ziobtree.go:626:4: Errorf is a pure function but its return value is ignored (SA4017)
    zlobtree.go:606:4: Errorf is a pure function but its return value is ignored (SA4017)
    zlobtree.go:626:4: Errorf is a pure function but its return value is ignored (SA4017)

run_with_zodb3py2_compat was renamed to run_with_zodb4py2_compat in
nexedi/zodbtools@c59a54ca .

Without the fix go genrate was failing as

    (neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/btree$ go generate
    Traceback (most recent call last):
      File "./testdata/gen-testdata", line 26, in <module>
        from zodbtools.test.gen_testdata import run_with_zodb3py2_compat
    ImportError: cannot import name run_with_zodb3py2_compat

and

    (neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb$ go generate
    Traceback (most recent call last):
      File "./py/pydata-gen-testdata", line 24, in <module>
        from zodbtools.test.gen_testdata import run_with_zodb3py2_compat
    ImportError: cannot import name run_with_zodb3py2_compat

This amends commit fc69e00d (go/zodb/fs1: Fix Python database generator
to work with recent zodbtools).