Updated wrapping_CPlusPlus.html for Cython v0.13

src/userguide/wrapping_CPlusPlus.rst

@@ -9,33 +9,36 @@ Wrapping C++ Classes in Cython
 Overview
 =========
 
-This page aims to get you quickly up to speed so you can wrap C++ interfaces
-with a minimum of pain and 'surprises'.
+Cython v0.13 introduces native support for most of the C++ language. This means that the previous tricks that were used to wrap C++ classes (as described in http://wiki.cython.org/WrappingCPlusPlus_ForCython012AndLower) are no longer useful. 
 
-In the past, Pyrex only supported wrapping of C APIs, and not C++. To wrap
-C++, one had to write a pure-C shim, containing functions for
-constructors/destructors and method invocations. Object pointers were passed
-around as opaque void pointers, and cast to/from object pointers as needed.
-This approach did work, but it got awfully messy and error-prone when trying
-to wrap APIs with large class hierarchies and lots of inheritance.
+Wrapping C++ classes with Cython is now much more straightforward. This document describe in details the new way of wrapping C++ code.
 
-These days, though, Pyrex offers an adequate bare minimum of C++ support,
-which Cython has inherited. The approach described in this document will help
-you wrap a lot of C++ code with only moderate effort. There are some
-limitations, which we will discuss at the end of the document.
+What's new in Cython v0.13 about C++
+---------------------------------------------------
+
+For users of previous Cython versions, here is a brief overview of the main new features of Cython v0.13 regarding C++ support:
+
+* C++ objects can now be dynamically allocated with ``new`` and ``del`` keywords.
+* C++ objects can now be stack-allocated.
+* C++ classes can be declared with the new keyword ``cppclass``.
+* Templated classes are supported.
+* Overloaded functions are supported.
+* Overloading of C++ operators (such as operator+, operator[],...) is supported.
 
 Procedure Overview
-====================
+-------------------
+The general procedure for wrapping a C++ file can now be described as follow:
 
 * Specify C++ language in :file:`setup.py` script
-* Create ``cdef extern from`` blocks and declare classes as 
-  ``ctypedef struct`` blocks
-* Create constructors and destructors
-* Add class methods as function pointers
-* Create Cython wrapper class 
+* Create ``cdef extern from`` blocks with the optional namespace (if exists) and the namespace name as string
+* Declare classes as ``cdef cppclass`` blocks
+* Declare public attributes (variables, methods and constructors) 
+
+A simple Tutorial
+==================
 
 An example C++ API
-===================
+-------------------
 
 Here is a tiny C++ API which we will use as an example throughout this
 document. Let's assume it will be in a header file called
@@ -43,6 +46,7 @@ document. Let's assume it will be in a header file called
 
 .. sourcecode:: c++
 
+    namespace shapes {
         class Rectangle {
         public:
             int x0, y0, x1, y1;
@@ -53,6 +57,7 @@ document. Let's assume it will be in a header file called
             int getArea();
             void move(int dx, int dy);
         };
+    }
     
 and the implementation in the file called :file:`Rectangle.cpp`:
 
@@ -98,11 +103,11 @@ and the implementation in the file called :file:`Rectangle.cpp`:
 This is pretty dumb, but should suffice to demonstrate the steps involved.
 
 Specify C++ language in setup.py
-=================================
+---------------------------------
 
 In Cython :file:`setup.py` scripts, one normally instantiates an Extension
 object. To make Cython generate and compile a C++ source, you just need
-to add a keyword to your Extension construction statement, as in::
+to add the keyword ``language="c++"`` to your Extension construction statement, as in::
 
     ext = Extension(
         "rectangle",                 # name of extension
@@ -114,103 +119,85 @@ to add a keyword to your Extension construction statement, as in::
         cmdclass = {'build_ext': build_ext}
         )
 
-and Cython will generate and compile the :file:`rectangle.cpp` file (from the
+Cython will generate and compile the :file:`rectangle.cpp` file (from the
 :file:`rectangle.pyx`), then it will compile :file:`Rectangle.cpp`
 (implementation of the ``Rectangle`` class) and link both objects files
 together into :file:`rectangle.so`, which you can then import in Python using
 ``import rectangle`` (if you forget to link the :file:`Rectangle.o`, you will
 get missing symbols while importing the library in Python).
 
-With the language="c++" keyword, Cython distutils will generate a C++ file.
 
-Create cdef extern from block
-==============================
+Alternatively, one can also use the ``cython`` command-line utility to generate a C++ ``.cpp`` file, and then compile it into a python extension. C++ mode for the ``cython`` command is turned on with the ``--cplus`` option.
+
+Declaring a C++ class interface
+--------------------------------
 
 The procedure for wrapping a C++ class is quite similar to that for wrapping
 normal C structs, with a couple of additions. Let's start here by creating the
 basic ``cdef extern from`` block::
 
-    cdef extern from "Rectangle.h":
+    cdef extern from "Rectangle.h" namespace "shapes":
 
-This will make the C++ class def for Rectangle available.
+This will make the C++ class def for Rectangle available. Note the namespace declaration.
 
-Declare class as a ctypedef struct
------------------------------------
+Declare class with cdef cppclass
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Now, let's add the Rectangle class to this extern from block -- just copy the
-class def from :file:`Rectangle.h` and adjust for Cython syntax, so now it
-becomes::
+Now, let's add the Rectangle class to this extern from block - just copy the class name from Rectangle.h and adjust for Cython syntax, so now it becomes::
 
-    cdef extern from "Rectangle.h":
-        # known in Cython namespace as 'c_Rectangle' but in C++ as 'Rectangle'
-        ctypedef struct c_Rectangle "Rectangle":
-            int x0, y0, x1, y1
+    cdef extern from "Rectangle.h" namespace "shapes":
+        cdef cppclass Rectangle:
     
-We don't have any way of accessing the constructor/destructor or methods, but
-we'll cover this now.
+Add public attributes
+^^^^^^^^^^^^^^^^^^^^^^
 
-Add constructors and destructors
-----------------------------------
+We now need to declare the attributes for use on Cython::
 
-We now need to expose a constructor and destructor into the Cython
-namespace. Again, we'll be using C name specifications::
-
-    cdef extern from "Rectangle.h":
-        ctypedef struct c_Rectangle "Rectangle":
+    cdef extern from "Rectangle.h" namespace "shapes":
+        cdef cppclass Rectangle:
+            Rectangle(int, int, int, int)
             int x0, y0, x1, y1
-        c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1)
-        void del_Rectangle "delete" (c_Rectangle *rect)
-
-Add class methods
--------------------
+            int getLength()
+            int getHeight()
+            int getArea()
+            void move(int, int)
 
-Now, let's add the class methods. You can circumvent Cython syntax
-limitations by declaring these as function pointers. Recall that in the C++
-class we have:
+Declare a var with the wrapped C++ class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. sourcecode:: c++
+Now, we use cdef to declare a var of the class with the know C++ ``new`` statement::
 
-  int getLength();
-  int getHeight();
-  int getArea();
-  void move(int dx, int dy);
+    cdef Rectangle *rec = new Rectangle(1, 2, 3, 4)
+    cdef int recLength = rec.getLength()
+    ...
+    del rec #delete heap allocated object
 
-So if we convert each of these to function pointers and stick them in our
-extern block, we now get::
+It's also possible to declare a stack allocated object, but it's necessary to have a "default" constructor::
 
-    cdef extern from "Rectangle.h":
-        ctypedef struct c_Rectangle "Rectangle":
-            int x0, y0, x1, y1
-            int getLength()
-            int getHeight()
-            int getArea()
-            void move(int dx, int dy)
-        c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1)
-        void del_Rectangle "delete" (c_Rectangle *rect)
+    cdef extern from "Foo.h":
+        cdef cppclass Foo:
+            Foo()
 
-This will fool Cython into generating C++ method calls even though
-Cython is mostly oblivious to C++.
+    cdef Foo foo
 
-In Pyrex you must explicitly declare these as function pointers, i.e. 
-``(int *getArea)()``.
+Note that, like C++, if the class has only one constructor and it is a default one, it's not necessary to declare it.
 
 Create Cython wrapper class
-=============================
+----------------------------
 
-At this point, we have exposed into our pyx file's namespace a struct which
-gives us access to the interface of a C++ Rectangle type. Now, we need to make
+At this point, we have exposed into our pyx file's namespace the interface of the C++ Rectangle type. Now, we need to make
 this accessible from external Python code (which is our whole point).
 
 Common programming practice is to create a Cython extension type which
 holds a C++ instance pointer as an attribute ``thisptr``, and create a bunch of
 forwarding methods. So we can implement the Python extension type as::
 
-    cdef class Rectangle:
-        cdef c_Rectangle *thisptr      # hold a C++ instance which we're wrapping
+    cdef class PyRectangle:
+        cdef Rectangle *thisptr      # hold a C++ instance which we're wrapping
         def __cinit__(self, int x0, int y0, int x1, int y1):
-            self.thisptr = new_Rectangle(x0, y0, x1, y1)
+            self.thisptr = new Rectangle(x0, y0, x1, y1)
         def __dealloc__(self):
-            del_Rectangle(self.thisptr)
+            del self.thisptr
         def getLength(self):
             return self.thisptr.getLength()
         def getHeight(self):
@@ -229,62 +216,141 @@ attribute access, you could just implement some properties::
         def __set__(self, x0): self.thisptr.x0 = x0
     ...
 
-Caveats and Limitations
-========================
 
-In this document, we have discussed a relatively straightforward way of
-wrapping C++ classes with Cython. However, there are some limitations in
-this approach, some of which could be overcome with clever workarounds (anyone
-here want to share some?), but some of which will require new features in
-Cython.
+Advanced C++ features
+======================
 
-The major limitations I'm most immediately aware of (and there will be many
-more) include:
+We describe here all the C++ features that were not discussed in the above tutorial.
 
 Overloading
 ------------
 
-Presently, it's not easy to overload methods or constructors, but there may be
-a workaround if you try some creative C name specifications
+Overloading is very simple. Just declare the method with different parameters and use any of them::
 
-Access to C-only functions
----------------------------
+    cdef extern from "Foo.h":
+        cdef cppclass Foo:
+            Foo(int)
+            Foo(bool)
+            Foo(int, bool)
+            Foo(int, int)
 
-Whenever generating C++ code, Cython generates declarations of and calls
-to functions assuming these functions are C++ (ie, not declared as extern "C"
-{...} . This is ok if the C functions have C++ entry points, but if they're C
-only, you will hit a roadblock. If you have a C++ Cython module needing
-to make calls to pure-C functions, you will need to write a small C++ shim
-module which:
+Overloading operators
+----------------------
 
-* includes the needed C headers in an extern "C" block
-* contains minimal forwarding functions in C++, each of which calls the
-  respective pure-C function 
+Cython uses C++ for overloading operators::
 
-Inherited C++ methods
-----------------------
+    cdef extern from "foo.h":
+        cdef cppclass Foo:
+            Foo()
+            Foo* operator+(Foo*)
+            Foo* operator-(Foo)
+            int operator*(Foo*)
+            int operator/(int)
+
+    cdef Foo* foo = new Foo()
+    cdef int x
+
+    cdef Foo* foo2 = foo[0] + foo
+    foo2 = foo[0] - foo[0]
+
+    x = foo[0] * foo2
+    x = foo[0] / 1
+
+    cdef Foo f
+    foo = f + &f
+    foo2 = f - f
+
+    del foo, foo2
+
+Nested class declarations
+--------------------------
+C++ allows nested class declaration. Class declarations can also be nested in Cython::
+
+
+    cdef extern from "<vector>" namespace "std":
+        cdef cppclass vector[T]:
+            cppclass iterator:
+                T operator*()
+                iterator operator++()
+                bint operator==(iterator)
+                bint operator!=(iterator)
+            vector()
+            void push_back(T&)
+            T& operator[](int)
+            T& at(int)
+            iterator begin()
+            iterator end()
+            
+    cdef vector[int].iterator iter  #iter is declared as being of type vector<int>::iterator
+            
+Note that the nested class is declared with a ``cppclass`` but without a ``cdef``.
+
+C++ operators not compatible with Python syntax
+------------------------------------------------
+
+Cython try to keep a syntax as close as possible to standard Python. Because of this, certain C++ operators, like the preincrement ``++foo`` or the dereferencing operator ``*foo`` cannot be used with the same syntax as C++. Cython provides functions replacing these operators in a special module ``cython.operator``. The functions provided are:
+
+* ``cython.operator.dereference`` for dereferencing. ``dereference(foo)`` will produce the C++ code ``*foo``
+* ``cython.operator.preincrement`` for pre-incrementation. ``preincrement(foo)`` will produce the C++ code ``++foo``
+* ...
+
+These functions need to be cimported. Of course, one can use a ``from ... cimport ... as`` to have shorter and more readable functions. For example: ``from cython.operator cimport dereference as deref``.
+
+Templates
+----------
+
+Cython uses a bracket syntax for templating. A simple example for wrapping C++ vector::
+
+    from cython.operator cimport dereference as deref, preincrement as inc #dereference and increment operators
+
+    cdef extern from "<vector>" namespace "std":
+        cdef cppclass vector[T]:
+            cppclass iterator:
+                T operator*()
+                iterator operator++()
+                bint operator==(iterator)
+                bint operator!=(iterator)
+            vector()
+            void push_back(T&)
+            T& operator[](int)
+            T& at(int)
+            iterator begin()
+            iterator end()
+
+    cdef vector[int] *v = new vector[int]()
+    cdef int i
+    for i in range(10):
+        v.push_back(i)
+
+    cdef vector[int].iterator it = v.begin()
+    while it != v.end():
+        print deref(it)
+        inc(it)
+
+    del v
+
+Multiple template parameters can be defined as a list, such as [T, U, V] or [int, bool, char]. 
+
+Standard library
+-----------------
+
+Most of the containers of the C++ Standard Library have been declared in pxd files  located in ``/Cython/Includes/libcpp``. These containers are: deque, list, map,  pair,  queue,  set,  stack,  vector.
 
-If you have a class ``Foo`` with a child class ``Bar``, and ``Foo`` has a
-method :meth:`fred`, then you'll have to cast to access this method from
-``Bar`` objects.
 For example::
 
-    class MyClass:
-        Bar *b
-        ...
-        def myfunc(self):
-            ...
-            b.fred()   # wrong, won't work
-            (<Foo *>(self.b)).fred() # should work, Cython now thinks it's a 'Foo'
+    from libcpp.vector cimport vector
 
-It might take some experimenting by others (you?) to find the most elegant
-ways of handling this issue.
+    cdef vector[int] vect
+    cdef int i
+    for i in range(10):
+        vect.push_back(i)
+    for i in range(10):
+        print vect[i]
         
-Advanced C++ features
-----------------------
+The pxd files in ``/Cython/Includes/libcpp`` also work as good examples on how to declare C++ classes.
 
 Exceptions
-^^^^^^^^^^^
+-----------
 
 Cython cannot throw C++ exceptions, or catch them with a try-except statement,
 but it is possible to declare a function as potentially raising an C++
@@ -310,57 +376,51 @@ called, which allows one to do custom C++ to Python error "translations." If
 raise_py_error does not actually raise an exception a RuntimeError will be
 raised.
 
-Templates
-^^^^^^^^^^
-
-Cython does not natively understand C++ templates but we can put them to use
-in some way. As an example consider an STL vector of C ints::
-
-    cdef extern from "some .h file which includes <vector>":
-        ctypedef struct intvec "std::vector<unsigned int>":
-            void (* push_back)(int elem)
-        intvec intvec_factory "std::vector<unsigned int>"(int len)
-
-now we can use the vector like this::
-
-    cdef intvec v = intvec_factory(2)
-    v.push_back(2)
 
-Overloading
-^^^^^^^^^^^^
-
-To support function overloading simply add a different alias to each
-signature, so if you have e.g.
+Caveats and Limitations
+========================
 
-.. sourcecode:: c++
+Access to C-only functions
+---------------------------
 
-    int foo(int a);
-    int foo(int a, int b);
+Whenever generating C++ code, Cython generates declarations of and calls
+to functions assuming these functions are C++ (ie, not declared as extern "C"
+{...} . This is ok if the C functions have C++ entry points, but if they're C
+only, you will hit a roadblock. If you have a C++ Cython module needing
+to make calls to pure-C functions, you will need to write a small C++ shim
+module which:
 
-in your C++ header then interface it like this in your ::
+* includes the needed C headers in an extern "C" block
+* contains minimal forwarding functions in C++, each of which calls the
+  respective pure-C function 
 
-    int fooi "foo"(int)
-    int fooii "foo"(int, int)
+Inherited C++ methods
+----------------------
 
-Operators
-^^^^^^^^^^
+If you have a class ``Foo`` with a child class ``Bar``, and ``Foo`` has a
+method :meth:`fred`, then you'll have to cast to access this method from
+``Bar`` objects.
+For example::
 
-Some operators (e.g. +,-,...) can be accessed from Cython like this::
+    cdef class MyClass:
+        Bar *b
+        ...
+        def myfunc(self):
+            ...
+            b.fred()   # wrong, won't work
+            (<Foo *>(self.b)).fred() # should work, Cython now thinks it's a 'Foo'
 
-    ctypedef struct c_Rectangle "Rectangle":
-        c_Rectangle add "operator+"(c_Rectangle right)
+It might take some experimenting by others (you?) to find the most elegant
+ways of handling this issue.
 
 Declaring/Using References
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+---------------------------
 
 Question: How do you declare and call a function that takes a reference as an argument?
 
-Conclusion
-============
+C++ left-values
+----------------
+
+C++ allows functions returning a reference to be left-values. This is currently not supported in Cython. ``cython.operator.dereference(foo)`` is also not considered a left-value.
 
-A great many existing C++ classes can be wrapped using these techniques, in a
-way much easier than writing a large messy C shim module. There's a bit of
-manual work involved, and an annoying maintenance burden if the C++ library
-you're wrapping is frequently changing, but this recipe should hopefully keep
-the discomfort to a minimum.