Commit 4d667fa3 authored by Kirill Smelkov's avatar Kirill Smelkov

Fix build for gevent-1.5

Starting from gevent >= 1.5 '*.pxd' files for gevent API are no longer
provided, at least in released gevent wheels. This broke pygolang:

    Error compiling Cython file:
    ------------------------------------------------------------
    ...

    # Gevent runtime uses gevent's greenlets and semaphores.
    # When sema.acquire() blocks, gevent switches us from current to another greenlet.

    IF not PYPY:
        from gevent._greenlet cimport Greenlet
       ^
    ------------------------------------------------------------

    golang/runtime/_runtime_gevent.pyx:28:4: 'gevent/_greenlet.pxd' not found

Since gevent upstream refuses to restore Cython level access[1], let's fix the
build by using gevent bits via Python-level.

Even when used via py import gevent-1.5 brings speed improvement compared to
gevent-1.4 (used via cimport):

	(on i7@2.6GHz, gevent runtime)

                      gevent-1.4   gevent-1.5
                      (cimport)    (py import)

    name              old time/op  new time/op  delta
    pyx_select_nogil  9.47µs ± 0%  8.74µs ± 0%   -7.70%  (p=0.000 n=10+9)
    pyx_go_nogil      14.3µs ± 1%  12.0µs ± 1%  -16.52%  (p=0.000 n=10+10)
    pyx_chan_nogil    7.10µs ± 1%  6.32µs ± 1%  -10.89%  (p=0.000 n=10+10)
    go                16.0µs ± 2%  13.4µs ± 1%  -16.37%  (p=0.000 n=10+10)
    chan              7.50µs ± 0%  6.79µs ± 0%   -9.53%  (p=0.000 n=10+10)
    select            10.8µs ± 1%  10.0µs ± 1%   -6.78%  (p=0.000 n=10+10)

Using gevent-1.5 could have been even faster via cimport (it is still
possible to compile and test against gevent installed in development
mode via `pip install -e` because pxd files are there in gevent worktree
and tarball):

                      gevent-1.5   gevent-1.5
                      (py import)  (cimport)

    name              old time/op  new time/op  delta
    pyx_select_nogil  8.74µs ± 0%  7.90µs ± 1%  -9.60%  (p=0.000 n=9+10)
    pyx_go_nogil      12.0µs ± 1%  11.2µs ± 2%  -6.35%  (p=0.000 n=10+10)
    pyx_chan_nogil    6.32µs ± 1%  5.89µs ± 0%  -6.80%  (p=0.000 n=10+9)
    go                13.4µs ± 1%  12.4µs ± 1%  -7.54%  (p=0.000 n=10+9)
    chan              6.79µs ± 0%  6.42µs ± 0%  -5.47%  (p=0.000 n=10+10)
    select            10.0µs ± 1%   9.4µs ± 1%  -6.39%  (p=0.000 n=10+10)

but we cannot use cimport to access gevent-1.5 universally, since pxd are not
shipped in gevent wheel releases.

In the future we might want to change plain version check into compile time
check whether gevent/_greenlet.pxd is actually present or not and use faster
access if yes. Requesting gevent to be installed in non-binary form
might be also an option worth trying.

However plain version check should be ok for now.

[1] https://github.com/gevent/gevent/issues/1568
parent 2114a560
...@@ -233,6 +233,9 @@ def Extension(name, sources, **kw): ...@@ -233,6 +233,9 @@ def Extension(name, sources, **kw):
pyxenv = kw.get('cython_compile_time_env', {}) pyxenv = kw.get('cython_compile_time_env', {})
pyxenv.setdefault('POSIX', POSIX) pyxenv.setdefault('POSIX', POSIX)
pyxenv.setdefault('PYPY', PYPY) pyxenv.setdefault('PYPY', PYPY)
gverhex = _gevent_version_hex()
if gverhex is not None:
pyxenv.setdefault('GEVENT_VERSION_HEX', gverhex)
kw['cython_compile_time_env'] = pyxenv kw['cython_compile_time_env'] = pyxenv
# XXX hack, because setuptools_dso.Extension is not Cython.Extension # XXX hack, because setuptools_dso.Extension is not Cython.Extension
...@@ -243,3 +246,23 @@ def Extension(name, sources, **kw): ...@@ -243,3 +246,23 @@ def Extension(name, sources, **kw):
ext.cython_compile_time_env = pyxenv ext.cython_compile_time_env = pyxenv
return ext return ext
# _gevent_version_hex returns gevent version in the format of PY_VERSION_HEX.
# None is returned if gevent is not available.
def _gevent_version_hex():
try:
import gevent
except ImportError:
return None
v = gevent.version_info
# https://docs.python.org/3/c-api/apiabiversion.html
rel = {'dev': 0, 'alpha': 0xa, 'beta': 0xb, 'rc': 0xc, 'final': 0xf}
vhex = \
(v.major << (3*8)) | \
(v.minor << (2*8)) | \
(v.micro << (1*8)) | \
(rel[v.releaselevel] << 4) | \
(v.serial << 0)
return vhex
# cython: language_level=2 # cython: language_level=2
# Copyright (C) 2019 Nexedi SA and Contributors. # Copyright (C) 2019-2020 Nexedi SA and Contributors.
# Kirill Smelkov <kirr@nexedi.com> # Kirill Smelkov <kirr@nexedi.com>
# #
# This program is free software: you can Use, Study, Modify and Redistribute # This program is free software: you can Use, Study, Modify and Redistribute
# it under the terms of the GNU General Public License version 3, or (at your # it under the terms of the GNU General Public License version 3, or (at your
...@@ -24,13 +24,16 @@ from __future__ import print_function, absolute_import ...@@ -24,13 +24,16 @@ from __future__ import print_function, absolute_import
# Gevent runtime uses gevent's greenlets and semaphores. # Gevent runtime uses gevent's greenlets and semaphores.
# When sema.acquire() blocks, gevent switches us from current to another greenlet. # When sema.acquire() blocks, gevent switches us from current to another greenlet.
IF not PYPY: # gevent >= 1.5 stopped to provide pxd to its API
# https://github.com/gevent/gevent/issues/1568
#
# on pypy gevent does not compile greenlet.py and semaphore.py citing that
# "there is no greenlet.h on pypy"
IF (GEVENT_VERSION_HEX < 0x01050000) and (not PYPY):
from gevent._greenlet cimport Greenlet from gevent._greenlet cimport Greenlet
from gevent.__semaphore cimport Semaphore from gevent.__semaphore cimport Semaphore
ctypedef Semaphore PYGSema ctypedef Semaphore PYGSema
ELSE: ELSE:
# on pypy gevent does not compile greenlet.py and semaphore.py citing that
# "there is no greenlet.h on pypy"
from gevent.greenlet import Greenlet from gevent.greenlet import Greenlet
from gevent._semaphore import Semaphore from gevent._semaphore import Semaphore
ctypedef object PYGSema ctypedef object PYGSema
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment