- 26 Jun, 2019 3 commits
-
-
Kirill Smelkov authored
It is true for any decorator, that it makes things faster if def+decorator is used globally instead of at runtime, but for @func it is especially true since @func, using decorator.decorate, has relatively high overhead if it is used not only once at program startup, but instead every time something is run. In particular moving @func out of WorkGroup.go() make things considerably faster: name old time/op new time/op delta workgroup_empty 195µs ± 0% 113µs ± 1% -41.91% (p=0.008 n=5+5) workgroup_raise 221µs ± 1% 137µs ± 1% -38.29% (p=0.008 n=5+5) See https://lab.nexedi.com/kirr/misc/raw/009c4fee/pygolang/prof_workgroup_empty.svg for bench_workgroup_empty profile, where it is seen that decorator.decorate was using ~ half of the whole WorkGroup.go() time.
-
Kirill Smelkov authored
Start adding benchmarks to pygolang. Measure how much sync chan send/recv take and how much select takes for synchronous channels. Also measure how much overhead @func adds at both def and call times, and the overhead of defer compared to try/finally. For std (non-gevent'ed) Python2.7 we are currently at: name time/op go 91.6µs ± 1% chan 13.7µs ± 3% select 30.1µs ± 4% def 55.0ns ± 0% func_def 43.6µs ± 0% call 63.0ns ± 0% func_call 1.06µs ± 0% try_finally 136ns ± 1% defer 2.33µs ± 0%
-
Kirill Smelkov authored
@method(cls) was deprecated and removed in favour of @func(cls) in 942ee900 (golang: Deprecate @method(cls) in favour of @func(cls)) and 262f8986 (golang: Kill @method). The test name was overlooked.
-
- 21 Jun, 2019 1 commit
-
-
Kirill Smelkov authored
With @func being a decorator, the following @func(cls) def name(): ... is always processed by python as name = func(cls)(def name(): ...) Before this patch it was leading to name being overridden with None: def f(): print 'hello' class C: pass @func(C) def f(c): print 'C.f', c f() Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: 'NoneType' object is not callable We can fix it by returning from `func(cls)(def name(): ...)` the original `name` object from the calling context. Unfortunately if `name` was not previously set I did not find a way(*) to avoid polluting the calling namespace where it is set to what @func(cls) returns (None) by the hardcoded way how python processes decorators: In [2]: c = """ ...: @fff ...: def ccc(): ...: return 1 ...: """ In [3]: cc = compile(c, "file", "exec") In [4]: dis(cc) 2 0 LOAD_NAME 0 (fff) 3 LOAD_CONST 0 (<code object ccc at 0x7fafe58d0130, file "file", line 2>) 6 MAKE_FUNCTION 0 9 CALL_FUNCTION 1 12 STORE_NAME 1 (ccc) <-- NOTE means: ccc = what fff() call returns 15 LOAD_CONST 1 (None) 18 RETURN_VALUE At least with no overriding taking place the situation is better now. NOTE: it is only @func(cls) which potentially pollutes calling namespace. Just @func (without class) is always clean because by definition it works as a regular decorator. (*) there is a very low-level and potentially fragile way to disable STORE_NAME after CALL_FUNCTION by dynamically patching caller's bytecode at runtime and replacing STORE_NAME with POP_TOP + NOP...
-
- 24 May, 2019 1 commit
-
-
Kirill Smelkov authored
This reverts commit 469f21a9. Even though Debian stopped shipping Python3.6 I'm now building it myself to have test coverage for two latest Python releases.
-
- 16 May, 2019 7 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Implement deadlines / timeouts using timers added recently in 9c260fde (time: New package that mirrors Go's time).
-
Kirill Smelkov authored
In the next patch we'll be adding deadlines support with another test. Move common infrastructure that will be used in all context tests to be test-module global.
-
Kirill Smelkov authored
We'll need to mark contexts that miss their deadline with deadlineExceeded error in a follow-up patch. In this patch: transform _BaseCtx._cancel to accept an error with which a context has to be canceled, but for now always use canceled in places where _cancel is called.
-
Kirill Smelkov authored
Same story as in Go, but we were lacking documentation notice about this.
-
Kirill Smelkov authored
Add time.Timer, time.Ticker and convenience functions time.tick, time.after and time.after_func. These will be used in context to support deadlines and timeouts. While at time topic, also provide sleep and now from golang.time, so that there is no need to import both golang.time and stdlib's time in a file. Provide time constants in the module as they are useful to have and mirror constants provided by Go's time. Note: timers implementation is very suboptimal for now.
-
Kirill Smelkov authored
Python adds dirname of run program to sys.path . This way when golang_test_goleaked.py runs it can import modules located under golang/ just by their name. Until now this was not noticed, but in the next patch we are going to add golang.time module and if test program is run with golang/ in sys.path just plain `import time` won't import time from stdlib and instead import time from golang/ . Such behaviour can be mitigated by doing `from __future__ import absolute_import` and we do that, including in golang_test_goleaked.py (see 81dfefa0 "*: __future__ += absolute_imports; Use unified __future__ everywhere"). However that does not prevent modules - even modules from stdlib - who are doing `import time` and not doing future absolute_import to import golang's time instead of stdlib. For example on PyPy2 threading imports time and then the test breaks: Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/golang_test_goleaked.py", line 24, in <module> from golang import go, chan File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/__init__.py", line 38, in <module> import inspect, threading, collections, random, sys File "/usr/lib/pypy/lib-python/2.7/threading.py", line 15, in <module> from time import time as _time, sleep as _sleep File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/time.py", line 30, in <module> from golang import go, chan, select, default, nilchan, panic ImportError: cannot import name 'go' -> Move the test program into a directory different from golang/ to avoid this trap.
-
- 14 May, 2019 1 commit
-
-
Kirill Smelkov authored
- we are going to introduce golang.time, and from inside there without `from __future__ import absolute_imports` it won't be possible to import needed stdlib's time. - we were already doing `from __future__ import print_function`, but only in some files. -> It makes sense to apply updated __future__ usage uniformly.
-
- 10 May, 2019 1 commit
-
-
Kirill Smelkov authored
- correct CHANGELOG title levels: since readme uses "----" for the first level and "~~~~" for the second level, "====" was interpreted as the third title level and "Pygolang change history" became sub-sub-section of "Additional packages and utilities" with section for each version conversely becoming first level. It was not very noticeable until 0c5f9d06 (readme: Push "Additional packages and utilities" into its own section) started to use "~~~~". -> Rework CHANGELOG titling to align with the one in README. - fix minor markup bits in README.
-
- 09 May, 2019 1 commit
-
-
Kirill Smelkov authored
-
- 03 May, 2019 3 commits
-
-
Kirill Smelkov authored
WorkGroup provides way to spawn goroutines that work on a common task and wait for their completion. It is modelled after https://godoc.org/golang.org/x/sync/errgroup but is not equal to it. See WorkGroup docstring for details.
-
Kirill Smelkov authored
Add sync.Once and sync.WaitGroup.
-
Kirill Smelkov authored
Add .Context, .background .with_cancel and .with_value. There is no support for deadline/timeout yet, since Go time package is not yet mirrored. Contrary to Go stdlib version, we also support context.merge that takes https://godoc.org/lab.nexedi.com/kirr/go123/xcontext#hdr-Merging_contexts for its inspiration. See https://blog.golang.org/context for overview of contexts.
-
- 02 May, 2019 2 commits
-
-
Kirill Smelkov authored
We are going to add more packages to mirror Go stdlib (e.g. next is context). Organize a dedicated section for what is additionally provided besides mirroring Go language and refer to that place only once from the top-level abstract.
-
Kirill Smelkov authored
Send/recv on the nil channel block forever; close panics. If a nil channel is used in select - corresponding case is never selected. Setting channel to nil is a usual idiom in Go to disable processing some cases in select. Nil channel is also used as "done" for e.g. context.Background() - for contexts that can be never canceled.
-
- 24 Mar, 2019 3 commits
-
-
Kirill Smelkov authored
It was deprecated and scheduled to be removed in 942ee900 (golang: Deprecate @method(cls) in favour of @func(cls)).
-
Kirill Smelkov authored
-
Kirill Smelkov authored
pypy-gevent runtests: commands[0] | gpython -m pytest gpython/ golang/ Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/bin/gpython", line 10, in <module> sys.exit(main()) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gpython/__init__.py", line 153, in main _ = monkey.patch_all() # XXX sys=True ? File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 976, in patch_all patch_thread(Event=Event, _warnings=_warnings) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 178, in ignores return func(*args, **kwargs) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 588, in patch_thread _patch_existing_locks(threading_mod) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 492, in _patch_existing_locks if o._RLock__owner is not None: AttributeError: 'thread.RLock' object has no attribute '_RLock__owner' It will be fixed on next Gevent release, however until then the crash is there and we have to workaround: we can skip patching existing locks as it will be the same behaviour of next Gevent release, since its just not possible to patch locks created via instantiated C-level classes: https://github.com/gevent/gevent/commit/d0e04658 We are doing monkey-patching very early, so it should be safe.
-
- 20 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Channel tests are passing on an idle machine. However if there is another load - e.g. in the presence of simultaneous Firefox start, channel tests break, for example: def test_select(): ... # non-blocking try recv: ok ch = chan() done = chan() def _(): for i in range(N): ch.send(i) done.close() go(_) for i in range(N): tdelay() if i % 2: _, _rx = select( ch.recv, default, ) assert (_, _rx) == (0, i) else: _, _rx = select( ch.recv_, default, ) > assert (_, _rx) == (0, (i, True)) E assert (1, None) == (0, (320, True)) E At index 0 diff: 1 != 0 E Use -v to get the full diff golang_test.py:209: AssertionError The failure here is that it was default case selected, not ch.recv_. The default was selected because the sending goroutine was not fast to enqueue next send before we tried to receive. We were trying to make sure that the sender will be enqueued first via adding 1ms time delay before trying to receive, but in the presence of concurrent load spikes that turns out to be not enough. We could try to fix the test by increasing the time to sleep in tdelay, make the tests more slow and still not 100% reliable. However we can change the tests to instead actually wait for the condition that is semantically required: a sender enqueued on the channel. Do that everywhere where tdelay was used. Now tests are faster (it was ~3s total, now it is ~ 0.5s total) and pass irregardless of whether the machine is idle or otherwise loaded.
-
- 19 Mar, 2019 3 commits
-
-
Kirill Smelkov authored
Debian dropped Python3.6 from python3-distutils. Without it I'm no longer being able to run tests on that python version: (neo) (z-dev) (g.env) kirr@deco:~/src/tools/go/pygolang$ tox -e py36-thread execution failed: -- Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'distutils.sysconfig' GLOB sdist-make: /home/kirr/src/tools/go/pygolang/setup.py py36-thread create: /home/kirr/src/tools/go/pygolang/.tox/py36-thread ERROR: invocation failed (exit code 1), logfile: /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log ERROR: actionid: py36-thread msg: getenv cmdargs: '/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/virtualenv.py", line 25, in <module> import distutils.sysconfig ModuleNotFoundError: No module named 'distutils.sysconfig' Running virtualenv with interpreter /usr/bin/python3.6 ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread (see /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log)', 1) https://bugs.debian.org/918881#19
-
Kirill Smelkov authored
PyPy7 always pre-imports the time module. Without the change tests were failing: Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/bin/gpython", line 10, in <module> sys.exit(main()) File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/site-packages/gpython/__init__.py", line 145, in main '\n\n\t%s\n\nsys.modules:\n\n\t%s' % (bad, sysmodv)) RuntimeError: gpython: internal error: the following modules are pre-imported, but must be not: ['time'] sys.modules: ['__future__', '__main__', '__pypy__', '__pypy__._pypydatetime', '__pypy__.builders', '__pypy__.intop', '__pypy__.os', '__pypy__.thread', '__pypy__.time', '_ast', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_continuation', '_csv', '_frozen_importlib', '_frozen_importlib_external', '_imp', '_io', '_locale', '_multibytecodec', '_operator', '_rawffi', '_rawffi.alt', '_signal', '_sre', '_structseq', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'array', 'builtins', 'codecs', 'copyreg', 'encodings', 'encodings.aliases', 'encodings.ascii', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'gc', 'genericpath', 'gpython', 'marshal', 'os', 'os.path', 'posix', 'posixpath', 'pwd', 're', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'sys', 'time', 'unicodedata']
-
Kirill Smelkov authored
Debian started to ship PyPy3: https://packages.debian.org/search?keywords=pypy3
-
- 15 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Since we already have func (see 5146eb0b "Add support for defer & recover") we can use @func for both plain functions and for methods. For example instead of @func def my_function(...): ... @method(MyClass) <-- NOTE def my_method(self, ...): ... have it as @func def my_function(...): ... @func(MyClass) <-- NOTE def my_method(self, ...): ... which looks more similar to Go and exposes less golang keywords to a user.
-
- 13 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Pymain was not adding e.g. directory of executed file to sys.path, and as the result if there were e.g. 2 files dir/hello.py # imports world dir/world.py running `gpython dir/hello.py` would fail to import world. The case for interactive console was also failing to setup sys.argv as empty, so it was containing ['/path/to/gpython']. It was also, contrary to standard python, unconditionally showing '>>>' prompt even when stdin was not a tty. Fix all that and add a test to cover pymain functionality.
-
- 16 Jan, 2019 6 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Provide gpython interpreter, that sets UTF-8 as default encoding, integrates gevent and puts go, chan, select etc into builtin namespace. Please see details in the documentation added by this patch. pymain was taken from go123@7a476082 (go123: tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft))
-
Kirill Smelkov authored
We describe golang/testdata/ in MANIFEST.in, and that makes testdata to be included into sdist. However testdata does not get installed. Since, we want to be able to run pygolang tests, even on installed package, we need the testdata files to be present there as well. Fix it. See https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files for setuptools-related details.
-
Kirill Smelkov authored
Move version definition from setup.py to golang module so that it is available at runtime with plain golang.__version__ .
-
Kirill Smelkov authored
A comma was missing - this way last element was 'funcgimport', not 'func' and 'gimport', and so `from golang import *` was failing.
-
Kirill Smelkov authored
.../pygolang/golang/strconv_test.py:75: PytestDeprecationWarning: raises(..., 'code(as_a_string)') is deprecated, use the context manager form or use `exec()` directly ... See https://docs.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument
-
- 30 Dec, 2018 1 commit
-
-
Kirill Smelkov authored
Until now it was not included by `python setup.py sdist`. On this topic, also include README.rst explicitly, since even if it is currently implicitly included, that might change some day and break us.
-
- 13 Dec, 2018 4 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
This are functions to decode quotation that was produced by strconv.quote().
-
Kirill Smelkov authored
Move quote implementation from gcompat to strconv. Make quote work on both unicode|bytes string input and produce the same output type. Preserve qq (still remaining in gcompat) to always produce str, since that is used in prints.
-
Kirill Smelkov authored
Stop hiding qq and benchmarking bits in changelog only - add their description into additional sections in readme. Since py.bench description is now in readme, there is no need to keep py.bench example in the changelog.
-