sharing_declarations.rst 9.25 KB
Newer Older
Robert Bradshaw's avatar
Robert Bradshaw committed
1 2 3 4 5 6 7 8
.. highlight:: cython

.. _sharing-declarations:

********************************************
Sharing Declarations Between Cython Modules
********************************************

9 10 11 12 13
This section describes how to make C declarations, functions and extension
types in one Cython module available for use in another Cython module.
These facilities are closely modeled on the Python import mechanism,
and can be thought of as a compile-time version of it.

Robert Bradshaw's avatar
Robert Bradshaw committed
14 15 16 17 18 19 20 21 22 23 24 25 26

Definition and Implementation files
====================================

A Cython module can be split into two parts: a definition file with a ``.pxd``
suffix, containing C declarations that are to be available to other Cython
modules, and an implementation file with a ``.pyx`` suffix, containing
everything else. When a module wants to use something declared in another
module's definition file, it imports it using the :keyword:`cimport`
statement.

A ``.pxd`` file that consists solely of extern declarations does not need
to correspond to an actual ``.pyx`` file or Python module. This can make it a
mathbunnyru's avatar
mathbunnyru committed
27
convenient place to put common declarations, for example declarations of
28
functions from  an :ref:`external library <external-C-code>` that one
mathbunnyru's avatar
mathbunnyru committed
29
wants to use in several modules.
Robert Bradshaw's avatar
Robert Bradshaw committed
30

31

Robert Bradshaw's avatar
Robert Bradshaw committed
32 33 34 35 36 37 38 39 40 41 42
What a Definition File contains
================================

A definition file can contain:

* Any kind of C type declaration.
* extern C function or variable declarations.
* Declarations of C functions defined in the module.
* The definition part of an extension type (see below).

It cannot contain the implementations of any C or Python functions, or any
mathbunnyru's avatar
mathbunnyru committed
43 44 45
Python class definitions, or any executable statements. It is needed when one
wants to  access :keyword:`cdef` attributes and methods, or to inherit from
:keyword:`cdef` classes defined in this module.
Robert Bradshaw's avatar
Robert Bradshaw committed
46 47 48 49 50 51 52 53

.. note::

    You don't need to (and shouldn't) declare anything in a declaration file
    public in order to make it available to other Cython modules; its mere
    presence in a definition file does that. You only need a public
    declaration if you want to make something available to external C code.

54

Robert Bradshaw's avatar
Robert Bradshaw committed
55 56 57 58 59
What an Implementation File contains
======================================

An implementation file can contain any kind of Cython statement, although there
are some restrictions on the implementation part of an extension type if the
mathbunnyru's avatar
mathbunnyru committed
60
corresponding definition file also defines that type (see below).
Robert Bradshaw's avatar
Robert Bradshaw committed
61
If one doesn't need to :keyword:`cimport` anything from this module, then this
mathbunnyru's avatar
mathbunnyru committed
62
is the only file one needs.
Robert Bradshaw's avatar
Robert Bradshaw committed
63

64

65 66
.. _cimport:

Robert Bradshaw's avatar
Robert Bradshaw committed
67 68 69 70 71 72 73 74 75 76 77 78
The cimport statement
=======================

The :keyword:`cimport` statement is used in a definition or
implementation file to gain access to names declared in another definition
file. Its syntax exactly parallels that of the normal Python import
statement::

    cimport module [, module...]

    from module cimport name [as name] [, name [as name] ...]

79
Here is an example. :file:`dishes.pxd` is a definition file which exports a
80
C data type. :file:`restaurant.pyx` is an implementation file which imports and
Robert Bradshaw's avatar
Robert Bradshaw committed
81
uses it.
mathbunnyru's avatar
mathbunnyru committed
82

Robert Bradshaw's avatar
Robert Bradshaw committed
83 84
:file:`dishes.pxd`::

mathbunnyru's avatar
mathbunnyru committed
85 86 87 88 89 90 91
   cdef enum otherstuff:
       sausage, eggs, lettuce

   cdef struct spamdish:
       int oz_of_spam
       otherstuff filler

Robert Bradshaw's avatar
Robert Bradshaw committed
92 93 94 95 96 97 98 99 100 101 102 103
:file:`restaurant.pyx`::

    cimport dishes
    from dishes cimport spamdish

    cdef void prepare(spamdish *d):
        d.oz_of_spam = 42
        d.filler = dishes.sausage

    def serve():
        cdef spamdish d
        prepare(&d)
104
        print("%d oz spam, filler no. %d" % (d.oz_of_spam, d.filler))
mathbunnyru's avatar
mathbunnyru committed
105

Robert Bradshaw's avatar
Robert Bradshaw committed
106 107 108 109 110 111 112 113 114 115
It is important to understand that the :keyword:`cimport` statement can only
be used to import C data types, C functions and variables, and extension
types. It cannot be used to import any Python objects, and (with one
exception) it doesn't imply any Python import at run time. If you want to
refer to any Python names from a module that you have cimported, you will have
to include a regular import statement for it as well.

The exception is that when you use :keyword:`cimport` to import an extension type, its
type object is imported at run time and made available by the name under which
you imported it. Using :keyword:`cimport` to import extension types is covered in more
mathbunnyru's avatar
mathbunnyru committed
116
detail below.
Robert Bradshaw's avatar
Robert Bradshaw committed
117

mathbunnyru's avatar
mathbunnyru committed
118
If a ``.pxd`` file changes, any modules that :keyword:`cimport` from it may need to be
119
recompiled.  The ``Cython.Build.cythonize`` utility can take care of this for you.
Robert Bradshaw's avatar
Robert Bradshaw committed
120

121 122

Search paths for definition files
Robert Bradshaw's avatar
Robert Bradshaw committed
123 124 125
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When you :keyword:`cimport` a module called ``modulename``, the Cython
126 127 128 129
compiler searches for a file called :file:`modulename.pxd`.
It searches for this file along the path for include files
(as specified by ``-I`` command line options or the ``include_path``
option to ``cythonize()``), as well as ``sys.path``.
Robert Bradshaw's avatar
Robert Bradshaw committed
130

131 132 133
Using ``package_data`` to install ``.pxd`` files in your ``setup.py`` script
allows other packages to cimport items from your module as a dependency.

Robert Bradshaw's avatar
Robert Bradshaw committed
134
Also, whenever you compile a file :file:`modulename.pyx`, the corresponding
135 136 137
definition file :file:`modulename.pxd` is first searched for along the
include path (but not ``sys.path``), and if found, it is processed before
processing the ``.pyx`` file.
Robert Bradshaw's avatar
Robert Bradshaw committed
138

139

mathbunnyru's avatar
mathbunnyru committed
140
Using cimport to resolve naming conflicts
Robert Bradshaw's avatar
Robert Bradshaw committed
141 142 143 144 145 146 147 148
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The :keyword:`cimport` mechanism provides a clean and simple way to solve the
problem of wrapping external C functions with Python functions of the same
name. All you need to do is put the extern C declarations into a ``.pxd`` file
for an imaginary module, and :keyword:`cimport` that module. You can then
refer to the C functions by qualifying them with the name of the module.
Here's an example:
mathbunnyru's avatar
mathbunnyru committed
149

150
:file:`c_lunch.pxd`::
Robert Bradshaw's avatar
Robert Bradshaw committed
151 152

    cdef extern from "lunch.h":
mathbunnyru's avatar
mathbunnyru committed
153
        void eject_tomato(float)
Robert Bradshaw's avatar
Robert Bradshaw committed
154

155
:file:`lunch.pyx`::
Robert Bradshaw's avatar
Robert Bradshaw committed
156 157 158 159 160 161 162 163 164 165

    cimport c_lunch

    def eject_tomato(float speed):
        c_lunch.eject_tomato(speed)

You don't need any :file:`c_lunch.pyx` file, because the only things defined
in :file:`c_lunch.pxd` are extern C entities. There won't be any actual
``c_lunch`` module at run time, but that doesn't matter; the
:file:`c_lunch.pxd` file has done its job of providing an additional namespace
mathbunnyru's avatar
mathbunnyru committed
166
at compile time.
Robert Bradshaw's avatar
Robert Bradshaw committed
167

168

Robert Bradshaw's avatar
Robert Bradshaw committed
169 170 171 172 173
Sharing C Functions
===================

C functions defined at the top level of a module can be made available via
:keyword:`cimport` by putting headers for them in the ``.pxd`` file, for
174
example:
Robert Bradshaw's avatar
Robert Bradshaw committed
175 176 177 178 179

:file:`volume.pxd`::

    cdef float cube(float)

180 181 182 183 184
:file:`volume.pyx`::

    cdef float cube(float x):
        return x * x * x

Robert Bradshaw's avatar
Robert Bradshaw committed
185 186
:file:`spammery.pyx`::

187 188
    from __future__ import print_function

Robert Bradshaw's avatar
Robert Bradshaw committed
189 190 191
    from volume cimport cube

    def menu(description, size):
192 193
        print(description, ":", cube(size),
              "cubic metres of spam")
Robert Bradshaw's avatar
Robert Bradshaw committed
194 195 196 197 198 199 200 201 202 203

    menu("Entree", 1)
    menu("Main course", 3)
    menu("Dessert", 2)

.. note::

    When a module exports a C function in this way, an object appears in the
    module dictionary under the function's name. However, you can't make use of
    this object from Python, nor can you use it from Cython using a normal import
mathbunnyru's avatar
mathbunnyru committed
204
    statement; you have to use :keyword:`cimport`.
Robert Bradshaw's avatar
Robert Bradshaw committed
205

206
.. _sharing_extension_types:
207 208

Sharing Extension Types
Robert Bradshaw's avatar
Robert Bradshaw committed
209 210 211 212 213 214 215 216 217 218 219 220 221 222 223
=======================

An extension type can be made available via :keyword:`cimport` by splitting
its definition into two parts, one in a definition file and the other in the
corresponding implementation file.

The definition part of the extension type can only declare C attributes and C
methods, not Python methods, and it must declare all of that type's C
attributes and C methods.

The implementation part must implement all of the C methods declared in the
definition part, and may not add any further C attributes. It may also define
Python methods.

Here is an example of a module which defines and exports an extension type,
224 225
and another module which uses it:

226
:file:`Shrubbing.pxd`::
227

Robert Bradshaw's avatar
Robert Bradshaw committed
228 229 230
    cdef class Shrubbery:
        cdef int width
        cdef int length
231 232 233

:file:`Shrubbing.pyx`::

Robert Bradshaw's avatar
Robert Bradshaw committed
234 235 236 237 238 239 240 241
    cdef class Shrubbery:
        def __cinit__(self, int w, int l):
            self.width = w
            self.length = l

    def standard_shrubbery():
        return Shrubbery(3, 7)

242
:file:`Landscaping.pyx`::
Robert Bradshaw's avatar
Robert Bradshaw committed
243 244 245 246 247 248

    cimport Shrubbing
    import Shrubbing

    cdef Shrubbing.Shrubbery sh
    sh = Shrubbing.standard_shrubbery()
249
    print("Shrubbery size is %d x %d" % (sh.width, sh.length))
250 251 252 253 254 255 256 257 258

One would then need to compile both of these modules, e.g. using

:file:`setup.py`::

    from distutils.core import setup
    from Cython.Build import cythonize
    setup(ext_modules = cythonize(["Landscaping.pyx", "Shrubbing.pyx"]))

Robert Bradshaw's avatar
Robert Bradshaw committed
259 260 261 262 263 264 265 266 267 268
Some things to note about this example:

* There is a :keyword:`cdef` class Shrubbery declaration in both
  :file:`Shrubbing.pxd` and :file:`Shrubbing.pyx`. When the Shrubbing module
  is compiled, these two declarations are combined into one.
* In Landscaping.pyx, the :keyword:`cimport` Shrubbing declaration allows us
  to refer to the Shrubbery type as :class:`Shrubbing.Shrubbery`. But it
  doesn't bind the name Shrubbing in Landscaping's module namespace at run
  time, so to access :func:`Shrubbing.standard_shrubbery` we also need to
  ``import Shrubbing``.
269 270 271 272 273
* One caveat if you use setuptools instead of distutils, the default
  action when running ``python setup.py install`` is to create a zipped
  ``egg`` file which will not work with ``cimport`` for ``pxd`` files
  when you try to use them from a dependent package.
  To prevent this, include ``zip_safe=False`` in the arguments to ``setup()``.