Skip to content

C
cython
Commit 9f0b9407 authored by ggellner@encolpuis's avatar ggellner@encolpuis
Browse files
Options

Started tutorial, begun refactoring the compilation section, and updating the limitations.

parent cc9744c1
Show whitespace changes
Inline Side-by-side
Showing with 307 additions and 130 deletions
conf.py
View file @ 9f0b9407
......@@ -40,9 +40,9 @@ copyright = '2008, Stefan Behnel, Robert Bradshaw, William Stein, Gary Furnish,
# other places throughout the built documents.
#
# The short X.Y version.
version = '0.9.6'
version = '0.9.8.1'
# The full version, including alpha/beta/rc tags.
release = '0.9.6.13.1'
release = '0.9.8.1'
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
......
docs/limitations.rst
View file @ 9f0b9407
.. _cython-limitations-label:
*************
Limitations
===========
*************
Unsupported Python features
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Unsupported Python Features
============================
Cython is not quite a full superset of Python. The following restrictions apply:
One of our goals is to make Cython as compatible as possible with standard
Python. This page lists the things that work in Python but not in Cython.
* Function definitions (whether using ``def`` or ``cdef``) cannot be nested within
other function definitions.
* Class definitions can only appear at the top level of a module,
not inside a function.
* The ``import *`` form of import is not allowed anywhere (other forms of the
import statement are fine, though).
* Generators cannot be defined in Cython.
* The ``globals()`` and ``locals()`` functions cannot be used.
.. TODO: this limitation seems to be removed
.. ::
The above restrictions will most likely remain, since removing them would be
difficult and they're not really needed for Cython's intended applications.
.. from module import *
There are also some temporary limitations, which may eventually be lifted, including:
.. This relies on at-runtime insertion of objects into the current namespace and
.. probably will be one of the few features never implemented (as any
.. implementation would be very slow). However, there is the --pre-import option
.. with treats all un-declared names as coming from the specified module, which
.. has the same effect as putting "from module import *" at the top-level of the
.. code. Note: the one difference is that builtins cannot be overriden in this
.. way, as the 'pre-import' scope is even higher than the builtin scope.
Nested def statements
----------------------
Function definitions (whether using ``def`` or ``cdef``) cannot be nested within
other function definitions. ::
def make_func():
def f(x):
return x*x
return f
(work in progress) This relies on functional closures
Generators
-----------
Using the yield keywords. (work in progress) This relies on functional closures
.. TODO Not really a limitation, rather an enchancement proposal
.. Support for builtin types
.. --------------------------
.. Support for statically declaring types such as list and dict and sequence
.. should be provided, and optimized code produced.
.. This needs to be well thought-out, and I think Pyrex has some plans along
.. these lines as well.
Modulo '%' operation on floats
-------------------------------
::
a = b%c
where `b` and `c` are floats will raise the error "Invalid operand types for '%' (float; float)"
This can currently be worked around by putting::
cdef extern from "math.h":
double fmod(double x, double y)
somewhere is the source file and then using::
a = fmod(b,c)
Other Current Limitations
==========================
* The :func:`globals` and :func:`locals` functions cannot be used.
* Class and function definitions cannot be placed inside control structures.
* There is no support for Unicode.
* Special methods of extension types cannot have functioning docstrings.
* The use of string literals as comments is not recommended at present,
because Cython doesn't optimize them away, and won't even accept them in places
......@@ -38,7 +89,7 @@ Behaviour of class scopes
In Python, referring to a method of a class inside the class definition, i.e.
while the class is being defined, yields a plain function object, but in
Cython it yields an unbound method [#]_. A consequence of this is that the
usual idiom for using the ``classmethod`` and ``staticmethod`` functions,
usual idiom for using the :func:`classmethod` and :func:`staticmethod` functions,
e.g.::
class Spam:
......@@ -62,8 +113,8 @@ outside the class, and then assigning the result of ``classmethod`` or
.. rubric:: Footnotes
.. [#] The reason for the different behaviour of class scopes is that
Cython-defined Python functions are PyCFunction objects, not PyFunction
objects, and are not recognised by the machinery that creates a bound
or unbound method when a function is extracted from a class. To get
Cython-defined Python functions are ``PyCFunction`` objects, not
``PyFunction`` objects, and are not recognised by the machinery that creates a
bound or unbound method when a function is extracted from a class. To get
around this, Cython wraps each method in an unbound method object itself
before storing it in the class's dictionary.
docs/overview.rst
View file @ 9f0b9407
......@@ -57,94 +57,6 @@ behind the scenes, just as it is in interpreted Python code. And what's more,
Cython lets you define new built-in Python types just as easily as you can
define new classes in Python.
Sound too good to be true? Read on and find out how it's done.
The Basics of Cython
====================
The fundamental nature of Cython can be summed up as follows: Cython is Python
with C data types.
Cython is Python: Almost any piece of Python code is also valid Cython code.
(There are a few :ref:`cython-limitations-label`, but this approximation will
serve for now.) The Cython compiler will convert it into C code which makes
equivalent calls to the Python/C API.
But Cython is much more than that, because parameters and variables can be
declared to have C data types. Code which manipulates Python values and C
values can be freely intermixed, with conversions occurring automatically
wherever possible. Reference count maintenance and error checking of Python
operations is also automatic, and the full power of Python's exception
handling facilities, including the try-except and try-finally statements, is
available to you -- even in the midst of manipulating C data.
Here's a small example showing some of what can be done. It's a routine for
finding prime numbers. You tell it how many primes you want, and it returns
them as a Python list.
:file:`primes.pyx`:
.. code-block:: none
:linenos:
def primes(int kmax):
cdef int n, k, i
cdef int p[1000]
result = []
if kmax > 1000:
kmax = 1000
k = 0
n = 2
while k < kmax:
i = 0
while i < k and n % p[i] != 0:
i = i + 1
if i == k:
p[k] = n
k = k + 1
result.append(n)
n = n + 1
return result
You'll see that it starts out just like a normal Python function definition,
except that the parameter ``kmax`` is declared to be of type ``int`` . This
means that the object passed will be converted to a C integer (or a
``TypeError.`` will be raised if it can't be).
Lines 2 and 3 use the ``cdef`` statement to define some local C variables.
Line 4 creates a Python list which will be used to return the result. You'll
notice that this is done exactly the same way it would be in Python. Because
the variable result hasn't been given a type, it is assumed to hold a Python
object.
Lines 7-9 set up for a loop which will test candidate numbers for primeness
until the required number of primes has been found. Lines 11-12, which try
dividing a candidate by all the primes found so far, are of particular
interest. Because no Python objects are referred to, the loop is translated
entirely into C code, and thus runs very fast.
When a prime is found, lines 14-15 add it to the p array for fast access by
the testing loop, and line 16 adds it to the result list. Again, you'll notice
that line 16 looks very much like a Python statement, and in fact it is, with
the twist that the C parameter ``n`` is automatically converted to a Python
object before being passed to the append method. Finally, at line 18, a normal
Python return statement returns the result list.
Compiling primes.pyx with the Cython compiler produces an extension module
which we can try out in the interactive interpreter as follows::
>>> import primes
>>> primes.primes(10)
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
See, it works! And if you're curious about how much work Cython has saved you,
take a look at the C code generated for this module.
Language Details
================
For more about the Cython language, see :ref:`language-basics-label`.
Future Plans
============
Cython is not finished. Substantial tasks remaining. See
......
docs/source_files_and_compilation.rst
View file @ 9f0b9407
.. TODO: Rewrite this to be more comprehensive, with examples!
.. _compilation_label:
****************************
Source Files and Compilation
......@@ -8,16 +8,6 @@ Cython source file names consist of the name of the module followed by a
``.pyx`` extension, for example a module called primes would have a source
file named :file:`primes.pyx`.
If your module is destined to live in a package, the source file name should
include the full dotted name that the module will eventually have. For
example, a module called primes that will be installed in a package called
numbers should have a source file called numbers.primes.pyx. This will ensure
that the :attr:`__name__` properties of the module and any classes defined in
it are set correctly. If you don't do this, you may find that pickling doesn't
work, among other problems. It also ensures that the Cython compiler has the
right idea about the layout of the module namespace, which can be important
when accessing extension types defined in other modules.
Once you have written your ``.pyx`` file, there are a couple of ways of turning it
into an extension module. One way is to compile it manually with the Cython
compiler, e.g.::
......@@ -26,12 +16,66 @@ compiler, e.g.::
This will produce a file called :file:`primes.c`, which then needs to be
compiled with the C compiler using whatever options are appropriate on your
platform for generating an extension module. There's a Makefile in the Demos
directory (called Makefile.nodistutils) that shows how to do this for Linux.
platform for generating an extension module. For these options look at the
official Python documentation.
The other, and probably better, way is to use the :mod:`distutils` extension
provided with Cython. See the :file:`setup.py` file in the Demos directory for an
example of how to use it. This method has the advantage of being
cross-platform -- the same setup file should work on any platform where
:mod:`distutils` can compile an
extension module.
provided with Cython. The benifit of this method is that it will give the
platform specific compilation options, acting like a stripped down autotools.
Basic setup.py
===============
The distutils extension provided with Cython allows you to pass ``.pyx`` files
directly to the ``Extension`` constructor in your setup file.
If you have a single Cython file that you want to turn into a compiled
extension, say with filename :file:`example.pyx` the associated :file:`setup.py`
would be::
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("example", ["example.pyx"])]
)
To understand the :file:`setup.py` more fully look at the official
``distutils`` documentation. To compile the extension for use in the
current directory use::
$ python setup.py build_ext --inplace
Cython Files Depending on C Files
===================================
TODO
Multiple Cython Files in a Package
===================================
TODO
Distributing Pyrex modules
===========================
It is strongly recommended that you distribute the generated ``.c`` files as well
as your Cython sources, so that users can install your module without needing
to have Cython available.
It is also recommended that Cython compilation not be enabled by default in the
version you distribute. Even if the user has Cython installed, he probably
doesn't want to use it just to install your module. Also, the version he has
may not be the same one you used, and may not compile your sources correctly.
This simply means that the :file:`setup.py` file that you ship with will just
be a normal distutils file on the generated `.c` files, for the basic example
we would have instead::
from distutils.core import setup
from distutils.extension import Extension
setup(
ext_modules = [Extension("example", ["example.c"])]
)
docs/tutorial.rst 0 → 100644
View file @ 9f0b9407
.. _tutorial_label:
*********
Tutorial
*********
The Basics of Cython
====================
The fundamental nature of Cython can be summed up as follows: Cython is Python
with C data types.
Cython is Python: Almost any piece of Python code is also valid Cython code.
(There are a few :ref:`cython-limitations-label`, but this approximation will
serve for now.) The Cython compiler will convert it into C code which makes
equivalent calls to the Python/C API.
But Cython is much more than that, because parameters and variables can be
declared to have C data types. Code which manipulates Python values and C
values can be freely intermixed, with conversions occurring automatically
wherever possible. Reference count maintenance and error checking of Python
operations is also automatic, and the full power of Python's exception
handling facilities, including the try-except and try-finally statements, is
available to you -- even in the midst of manipulating C data.
Cython Hello World
===================
As Cython can accept almost any valid plain python file (though see the
`exceptions`), one of the hardest things in getting started is just figuring
out how to compile your file.
So lets start with the canonical python hello world::
print "Hello World"
So the first thing to do is rename the file to :file:`helloworld.pyx`. Now we
need to make the :file:`setup.py`, which is like a python Makefile if you are
familiar.::
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)
To use this to build your Cython file use the commandline options::
$ python setup.py build_ext --inplace
Which will leave a file in your local directory called `helloworld.so`. Now to
use this file start the python interpreter and::
>>> import helloworld
"Hello World"
Congratulations! You know know how to build a Cython extension. But So Far
this example doesn't really show us why we would even want to use Cython, so
lets do a more realistic example.
Fibonacci Fun
==============
From the official Python tutorial a simple fibonacci function is defined as::
def fib(n):
"""Print the Fibonacci series up to n."""
a, b = 0, 1
while b < n:
print b,
a, b = b, a + b
Now following the steps for the Hello World example we first rename the file
to have a `.pyx` extension, lets say :file:`fib.pyx`, then we create the
:file:`setup.py` file. Using the file created for the Hello World example, all
that you need to change is the name of the Cython filename, and the resulting
module name, doing this we have::
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("fib", ["fib.pyx"])]
)
Build the extension with the same command used for the helloworld.pyx::
$ python setup.py build_ext --inplace
And use the new extension with::
>>> import fib
>>> fib.fib(2000)
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
Primes
=======
Here's a small example showing some of what can be done. It's a routine for
finding prime numbers. You tell it how many primes you want, and it returns
them as a Python list.
:file:`primes.pyx`: ::
def primes(int kmax):
cdef int n, k, i
cdef int p[1000]
result = []
if kmax > 1000:
kmax = 1000
k = 0
n = 2
while k < kmax:
i = 0
while i < k and n % p[i] != 0:
i = i + 1
if i == k:
p[k] = n
k = k + 1
result.append(n)
n = n + 1
return result
You'll see that it starts out just like a normal Python function definition,
except that the parameter ``kmax`` is declared to be of type ``int`` . This
means that the object passed will be converted to a C integer (or a
``TypeError.`` will be raised if it can't be).
Lines 2 and 3 use the ``cdef`` statement to define some local C variables.
Line 4 creates a Python list which will be used to return the result. You'll
notice that this is done exactly the same way it would be in Python. Because
the variable result hasn't been given a type, it is assumed to hold a Python
object.
Lines 7-9 set up for a loop which will test candidate numbers for primeness
until the required number of primes has been found. Lines 11-12, which try
dividing a candidate by all the primes found so far, are of particular
interest. Because no Python objects are referred to, the loop is translated
entirely into C code, and thus runs very fast.
When a prime is found, lines 14-15 add it to the p array for fast access by
the testing loop, and line 16 adds it to the result list. Again, you'll notice
that line 16 looks very much like a Python statement, and in fact it is, with
the twist that the C parameter ``n`` is automatically converted to a Python
object before being passed to the append method. Finally, at line 18, a normal
Python return statement returns the result list.
Compiling primes.pyx with the Cython compiler produces an extension module
which we can try out in the interactive interpreter as follows::
>>> import primes
>>> primes.primes(10)
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
See, it works! And if you're curious about how much work Cython has saved you,
take a look at the C code generated for this module.
Language Details
================
For more about the Cython language, see :ref:`language-basics-label`.
index.rst
View file @ 9f0b9407
......@@ -11,6 +11,7 @@ Contents:
:maxdepth: 2
docs/overview
docs/tutorial
docs/language_basics
docs/extension_types
docs/sharing_declarations
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7