Chan implementation is going to be moved into C, and this way direct
access to chan internals won't be possible. C implementation will export
dedicated functions for chan tests which will be used.

Prepare for that and factor out retrieving len of chan recv/send queues
into separate functions, which will be adapted at the time of C port.

Nil channel will soon be changed to be represented by underlying NULL C
pointer and there won't be channel object for nil and correspondingly
recv/send queues won't be there. -> Prepare for that and don't check for
recvq/sendq on nil channel.

We provide gpython since 32a21d5b (gpython: Python interpreter with
support for lightweight threads), and golang module, since the
beginning, automatically uses gevent if it was installed via monkey
patching.

This currently pass trivially (since time.now = stdtime.now), but will
be useful once we get time.now implementation into our hands and move it into Cython.

After 9c260fde (time: New package that mirrors Go's time) we have
golang.time.now and golang.time.sleep and it makes it a more
self-dependent system if timing facility is used through golang.time
instead of outside std time module.

For now this is only a "cleanness" change, but will become important once
we start adding pyx-level nogil API to pygolang - there it will be
important to use golang.time.* for correctness.

Starting from e6bea2cf (sync: New package that mirrors Go's sync) there
is sync.WaitGroup and it was using threading.Event to signal that
waitgroup's counter dropped to zero. threading.Event is implemented in
Python's stdlib via threading.Condition, which in turn uses
threading.Lock and list of waiters to implement its functionality.
Which in turn is very similar to what golang.chan does internally with
semaphore and e.g. recv queue.

I noticed this while debugging sync test deadlock (see previous patch)
and suspecting bug in threading.Event for a moment. While it turned
there is no bug in threading.Event, it is better to rely on the common
functionality for similar tasks, and pygolang's channel perfectly
matches here the need to signal an event. By using our own code instead
of stdlib's threading we are likely becoming less bug-prone.

This also brings a small speedup:

    (on i5@1.80GHz)
    name             old time/op  new time/op  delta
    workgroup_empty   239µs ± 1%   220µs ± 1%  -7.62%  (p=0.000 n=10+9)
    workgroup_raise   275µs ± 2%   264µs ± 2%  -3.85%  (p=0.000 n=10+10)

sync.WorkGroup test was doing ctx.done() wait from under test mutex,
something like

    def _(ctx, i):
        with mu:
            ...
            if i == 0:
                raise RuntimeError()    # to cause ctx cancel
            ctx.done().recv()           # i=1 -> wait till ctx is canceled

but it can be a deadlock if T(i=1) runs first and enters
ctx.done().recv() before T(i=0) is run - then T(i=0) will block forever
waiting to lock mu.

This failure was not seen so far, probably because the time to go a new
thread/goroutine is relatively high. However one of upcoming patches,
where go is made faster, revealed this problem and, without the fix,
sync.WorkGroup test was regularly deadlocking.

The problem was there from sync.WorkGroup beginning - from
9ee7ba91 (sync += WorkGroup).

Fix the deadlock by waiting for ctx.done() outside of mu.

9c61f254 (pygopath: Initial draft) has a "TODO py3 support", probably
because https://stackoverflow.com/a/67692 shows different codes for
Python2 and Python3.

However today we have tox coverage for all Python2.7, Python3.6 and
Python3.7 and everything works including gimport tests.

-> Remove the TODO.

Currently gimport tests depend on module-level setup_module /
teardown_module, which require for gimport tests to be in separate file.

Redo the test not to depend on global module-level state and
setup/teardown. This allows e.g. to merge gimport tests into
golang_test.py if/when we want/need to.

Should be in 2aad64bb (golang: Add support for nil channel).

In tests in places where the code checks that something panics, verify
not only that _PanicError is raised, but also what was the argument
passed to panic.

This makes sure that tested code panics in expected place, not just
panics "somewhere".

To keep signal/noise ratio high introduce `panics` which is similar to
`pytest.raises` and asserts that wrapped code panics with expected
argument.

-> Use .[test] to refer to them.
https://stackoverflow.com/a/41398850/9456786

A problem was hit with pytest.fail with raises Failed exception not
being propagated to .wait. As it turned out it was not propagated
because pytest's Failed derives from BaseException, not Exception, and
we were catching only Exception and its children.

Rework the code to propagate all exception types from workers.

Performance change is with noise (it is either a bit faster for one set
of runs, or a bit slower for another set of runs).

It is true for any decorator, that it makes things faster if
def+decorator is used globally instead of at runtime, but for @func it
is especially true since @func, using decorator.decorate, has relatively
high overhead if it is used not only once at program startup, but
instead every time something is run.

In particular moving @func out of WorkGroup.go() make things
considerably faster:

	name             old time/op  new time/op  delta
	workgroup_empty   195µs ± 0%   113µs ± 1%  -41.91%  (p=0.008 n=5+5)
	workgroup_raise   221µs ± 1%   137µs ± 1%  -38.29%  (p=0.008 n=5+5)

See

	https://lab.nexedi.com/kirr/misc/raw/009c4fee/pygolang/prof_workgroup_empty.svg

for bench_workgroup_empty profile, where it is seen that
decorator.decorate was using ~ half of the whole WorkGroup.go() time.

Start adding benchmarks to pygolang.

Measure how much sync chan send/recv take and how much select takes for
synchronous channels. Also measure how much overhead @func adds at both
def and call times, and the overhead of defer compared to try/finally.
For std (non-gevent'ed) Python2.7 we are currently at:

	name         time/op
	go           91.6µs ± 1%
	chan         13.7µs ± 3%
	select       30.1µs ± 4%
	def          55.0ns ± 0%
	func_def     43.6µs ± 0%
	call         63.0ns ± 0%
	func_call    1.06µs ± 0%
	try_finally   136ns ± 1%
	defer        2.33µs ± 0%

@method(cls) was deprecated and removed in favour of @func(cls) in
942ee900 (golang: Deprecate @method(cls) in favour of @func(cls)) and
262f8986 (golang: Kill @method). The test name was overlooked.

With @func being a decorator, the following

	@func(cls)
	def name():
		...

is always processed by python as

	name = func(cls)(def name(): ...)

Before this patch it was leading to name being overridden with None:

	def f():
		print 'hello'

	class C:
		pass

	@func(C)
	def f(c):
		print 'C.f', c

	f()
	Traceback (most recent call last):
	  File "<console>", line 1, in <module>
	TypeError: 'NoneType' object is not callable

We can fix it by returning from `func(cls)(def name(): ...)` the
original `name` object from the calling context.

Unfortunately if `name` was not previously set I did not find a way(*) to
avoid polluting the calling namespace where it is set to what @func(cls)
returns (None) by the hardcoded way how python processes decorators:

	In [2]: c = """
	   ...: @fff
	   ...: def ccc():
	   ...:     return 1
	   ...: """

	In [3]: cc = compile(c, "file", "exec")

	In [4]: dis(cc)
	  2           0 LOAD_NAME                0 (fff)
	              3 LOAD_CONST               0 (<code object ccc at 0x7fafe58d0130, file "file", line 2>)
	              6 MAKE_FUNCTION            0
	              9 CALL_FUNCTION            1
	             12 STORE_NAME               1 (ccc)	<-- NOTE means: ccc = what fff() call returns
	             15 LOAD_CONST               1 (None)
	             18 RETURN_VALUE

At least with no overriding taking place the situation is better now.

NOTE: it is only @func(cls) which potentially pollutes calling
namespace. Just @func (without class) is always clean because by
definition it works as a regular decorator.

(*) there is a very low-level and potentially fragile way to disable
STORE_NAME after CALL_FUNCTION by dynamically patching caller's bytecode
at runtime and replacing STORE_NAME with POP_TOP + NOP...

This reverts commit 469f21a9.

Even though Debian stopped shipping Python3.6 I'm now building it myself
to have test coverage for two latest Python releases.

Implement deadlines / timeouts using timers added recently in 9c260fde
(time: New package that mirrors Go's time).

In the next patch we'll be adding deadlines support with another test.
Move common infrastructure that will be used in all context tests to be
test-module global.

We'll need to mark contexts that miss their deadline with
deadlineExceeded error in a follow-up patch.

In this patch: transform _BaseCtx._cancel to accept an error with which
a context has to be canceled, but for now always use canceled in places
where _cancel is called.

Same story as in Go, but we were lacking documentation notice about this.

Add time.Timer, time.Ticker and convenience functions time.tick,
time.after and time.after_func. These will be used in context to support
deadlines and timeouts.

While at time topic, also provide sleep and now from golang.time, so
that there is no need to import both golang.time and stdlib's time in a
file.

Provide time constants in the module as they are useful to have and
mirror constants provided by Go's time.

Note: timers implementation is very suboptimal for now.

Python adds dirname of run program to sys.path . This way when
golang_test_goleaked.py runs it can import modules located under golang/
just by their name. Until now this was not noticed, but in the next
patch we are going to add golang.time module and if test program is run
with golang/ in sys.path just plain `import time` won't import time from
stdlib and instead import time from golang/ .

Such behaviour can be mitigated by doing `from __future__ import
absolute_import` and we do that, including in golang_test_goleaked.py
(see 81dfefa0 "*: __future__ += absolute_imports; Use unified __future__
everywhere"). However that does not prevent modules - even modules from
stdlib - who are doing `import time` and not doing future
absolute_import to import golang's time instead of stdlib. For example
on PyPy2 threading imports time and then the test breaks:

  Traceback (most recent call last):
    File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/golang_test_goleaked.py", line 24, in <module>
      from golang import go, chan
    File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/__init__.py", line 38, in <module>
      import inspect, threading, collections, random, sys
    File "/usr/lib/pypy/lib-python/2.7/threading.py", line 15, in <module>
      from time import time as _time, sleep as _sleep
    File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/time.py", line 30, in <module>
      from golang import go, chan, select, default, nilchan, panic
  ImportError: cannot import name 'go'

-> Move the test program into a directory different from golang/ to
avoid this trap.

- we are going to introduce golang.time, and from inside there without
  `from __future__ import absolute_imports` it won't be possible to import
  needed stdlib's time.

- we were already doing `from __future__ import print_function`, but
  only in some files.

-> It makes sense to apply updated __future__ usage uniformly.

- correct CHANGELOG title levels: since readme uses "----" for the
  first level and "~~~~" for the second level, "====" was interpreted as
  the third title level and "Pygolang change history" became
  sub-sub-section of "Additional packages and utilities" with section
  for each version conversely becoming first level. It was not very
  noticeable until 0c5f9d06 (readme: Push "Additional packages and
  utilities" into its own section) started to use "~~~~".
  -> Rework CHANGELOG titling to align with the one in README.

- fix minor markup bits in README.

WorkGroup provides way to spawn goroutines that work on a common task
and wait for their completion. It is modelled after
https://godoc.org/golang.org/x/sync/errgroup but is not equal to it.

See WorkGroup docstring for details.

Add sync.Once and sync.WaitGroup.

Add .Context, .background .with_cancel and .with_value. There is no
support for deadline/timeout yet, since Go time package is not yet
mirrored.

Contrary to Go stdlib version, we also support context.merge that takes
https://godoc.org/lab.nexedi.com/kirr/go123/xcontext#hdr-Merging_contexts
for its inspiration.

See https://blog.golang.org/context for overview of contexts.

We are going to add more packages to mirror Go stdlib (e.g. next is
context). Organize a dedicated section for what is additionally provided
besides mirroring Go language and refer to that place only once from the
top-level abstract.

Send/recv on the nil channel block forever; close panics.
If a nil channel is used in select - corresponding case is never selected.

Setting channel to nil is a usual idiom in Go to disable processing some
cases in select. Nil channel is also used as "done" for e.g.
context.Background() - for contexts that can be never canceled.

It was deprecated and scheduled to be removed in 942ee900 (golang:
Deprecate @method(cls) in favour of @func(cls)).

	pypy-gevent runtests: commands[0] | gpython -m pytest gpython/ golang/
	Traceback (most recent call last):
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/bin/gpython", line 10, in <module>
	    sys.exit(main())
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gpython/__init__.py", line 153, in main
	    _ = monkey.patch_all()      # XXX sys=True ?
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 976, in patch_all
	    patch_thread(Event=Event, _warnings=_warnings)
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 178, in ignores
	    return func(*args, **kwargs)
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 588, in patch_thread
	    _patch_existing_locks(threading_mod)
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 492, in _patch_existing_locks
	    if o._RLock__owner is not None:
	AttributeError: 'thread.RLock' object has no attribute '_RLock__owner'

It will be fixed on next Gevent release, however until then the crash is there
and we have to workaround: we can skip patching existing locks as it will be
the same behaviour of next Gevent release, since its just not possible to patch
locks created via instantiated C-level classes:

https://github.com/gevent/gevent/commit/d0e04658

We are doing monkey-patching very early, so it should be safe.

Channel tests are passing on an idle machine. However if there is
another load - e.g. in the presence of simultaneous Firefox start,
channel tests break, for example:

	    def test_select():
	        ...

	        # non-blocking try recv: ok
	        ch = chan()
	        done = chan()
	        def _():
	            for i in range(N):
	                ch.send(i)
	            done.close()
	        go(_)

	        for i in range(N):
	            tdelay()
	            if i % 2:
	                _, _rx = select(
	                        ch.recv,
	                        default,
	                )
	                assert (_, _rx) == (0, i)
	            else:
	                _, _rx = select(
	                        ch.recv_,
	                        default,
	                )
	>               assert (_, _rx) == (0, (i, True))
	E               assert (1, None) == (0, (320, True))
	E                 At index 0 diff: 1 != 0
	E                 Use -v to get the full diff

	golang_test.py:209: AssertionError

The failure here is that it was default case selected, not ch.recv_. The
default was selected because the sending goroutine was not fast to
enqueue next send before we tried to receive. We were trying to make
sure that the sender will be enqueued first via adding 1ms time delay
before trying to receive, but in the presence of concurrent load spikes
that turns out to be not enough.

We could try to fix the test by increasing the time to sleep in tdelay,
make the tests more slow and still not 100% reliable. However we can
change the tests to instead actually wait for the condition that is
semantically required: a sender enqueued on the channel.

Do that everywhere where tdelay was used.

Now tests are faster (it was ~3s total, now it is ~ 0.5s total) and pass
irregardless of whether the machine is idle or otherwise loaded.

Debian dropped Python3.6 from python3-distutils. Without it I'm no
longer being able to run tests on that python version:

	(neo) (z-dev) (g.env) kirr@deco:~/src/tools/go/pygolang$ tox -e py36-thread
	execution failed:  -- Traceback (most recent call last):
	  File "<string>", line 1, in <module>
	ModuleNotFoundError: No module named 'distutils.sysconfig'

	GLOB sdist-make: /home/kirr/src/tools/go/pygolang/setup.py
	py36-thread create: /home/kirr/src/tools/go/pygolang/.tox/py36-thread
	ERROR: invocation failed (exit code 1), logfile: /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log
	ERROR: actionid: py36-thread
	msg: getenv
	cmdargs: '/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread'

	Traceback (most recent call last):
	  File "/usr/lib/python3/dist-packages/virtualenv.py", line 25, in <module>
	    import distutils.sysconfig
	ModuleNotFoundError: No module named 'distutils.sysconfig'
	Running virtualenv with interpreter /usr/bin/python3.6

	ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread (see /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log)', 1)

https://bugs.debian.org/918881#19

PyPy7 always pre-imports the time module. Without the change tests were
failing:

	Traceback (most recent call last):
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/bin/gpython", line 10, in <module>
	    sys.exit(main())
	  File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/site-packages/gpython/__init__.py", line 145, in main
	    '\n\n\t%s\n\nsys.modules:\n\n\t%s' % (bad, sysmodv))
	RuntimeError: gpython: internal error: the following modules are pre-imported, but must be not:

	        ['time']

	sys.modules:

	        ['__future__', '__main__', '__pypy__', '__pypy__._pypydatetime', '__pypy__.builders', '__pypy__.intop', '__pypy__.os', '__pypy__.thread', '__pypy__.time', '_ast', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_continuation', '_csv', '_frozen_importlib', '_frozen_importlib_external', '_imp', '_io', '_locale', '_multibytecodec', '_operator', '_rawffi', '_rawffi.alt', '_signal', '_sre', '_structseq', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'array', 'builtins', 'codecs', 'copyreg', 'encodings', 'encodings.aliases', 'encodings.ascii', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'gc', 'genericpath', 'gpython', 'marshal', 'os', 'os.path', 'posix', 'posixpath', 'pwd', 're', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'sys', 'time', 'unicodedata']

Debian started to ship PyPy3:

	https://packages.debian.org/search?keywords=pypy3