overview.html 40.7 KB
Newer Older
William Stein's avatar
William Stein committed
1 2 3 4 5 6 7
<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head>
             
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

             
  <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]">
8
  <title>Cython Language Overview</title>
William Stein's avatar
William Stein committed
9 10 11
</head>
<body>
     
12
<h1>   <hr width="100%">Overview of the Cython Language&nbsp;  <hr width="100%"></h1>
William Stein's avatar
William Stein committed
13 14

   This document informally describes the extensions to the Python language 
15
 made by Cython. Some day there will be a reference manual covering everything 
William Stein's avatar
William Stein committed
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
 in more detail. <br>

  &nbsp;  
<h2> Contents</h2>

     
<ul>

   <li> <a href="#Basics">Basics</a></li>
           <ul>
   <li> <a href="#PyFuncsVsCFuncs">Python functions vs. C functions</a></li>
    <li> <a href="#PyObjParams">Python objects as parameters</a></li>
    <li> <a href="#CVarAndTypeDecls">C variable and type definitions</a></li><li><a href="#AutomaticTypeConversions">Automatic type conversions</a></li>
    <ul>
      <li><a href="#PyToCStringCaveats">Caveats when using a Python string in a C context</a></li>
    </ul>

    <li> <a href="#ScopeRules">Scope rules</a></li>
    <li> <a href="#StatsAndExprs">Statements and expressions</a></li>
                 <ul>
36
   <li> <a href="#ExprSyntaxDifferences">Differences between C and Cython
William Stein's avatar
William Stein committed
37 38 39 40 41 42
expressions<br>
</a></li>
    <li> <a href="#ForFromLoop">Integer for-loops</a></li>
                </ul>
    <li> <a href="#ExceptionValues">Error return values</a></li>
                 <ul>
43
   <li> <a href="#CheckingReturnValues">Checking return values of non-Cython
William Stein's avatar
William Stein committed
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
 functions</a></li>
                </ul>
    <li> <a href="#IncludeStatement">The <tt>include</tt> statement</a></li>
          </ul>
    <li> <a href="#InterfacingWithExternal">Interfacing with External C Code</a></li>
           <ul>
   <li> <a href="#ExternDecls">External declarations</a></li>
                 <ul>
   <li> <a href="#ReferencingHeaders">Referencing C header files</a></li>
    <li> <a href="#StructDeclStyles">Styles of struct/union/enum declaration</a></li>
    <li> <a href="#AccessingAPI">Accessing Python/C API routines</a></li>
    <li> <a href="#CNameSpecs">Resolving naming conflicts - C name specifications</a></li>
                </ul>
    <li> <a href="#PublicDecls">Public declarations</a></li>
          </ul>
    <li> <a href="extension_types.html">Extension Types</a> <font color="#006600">(Section revised in 0.9)</font></li>
60
    <li> <a href="sharing.html">Sharing Declarations Between Cython Modules</a>
William Stein's avatar
William Stein committed
61 62 63 64 65
     <font color="#006600">(NEW in 0.8)</font></li>
    <li> <a href="#Limitations">Limitations</a></li>
           <ul>
   <li> <a href="#Unsupported">Unsupported Python features</a></li>
    <li> <a href="#SemanticDifferences">Semantic differences between Python
66
 and Cython</a></li>
William Stein's avatar
William Stein committed
67 68 69 70 71 72 73 74
          </ul>
    
</ul>

     
<h2>   <hr width="100%"><a name="Basics"></a>Basics   
<hr width="100%"></h2>

75
   This section describes the basic features of the Cython language. The facilities 
William Stein's avatar
William Stein committed
76 77
 covered in this section allow you to create Python-callable functions that 
 manipulate C data structures and convert between Python and C data types. 
78
 Later sections will cover facilities for <a href="#InterfacingWithExternal">wrapping external C code</a>, <a href="extension_types.html">creating new Python types</a> and <a href="sharing.html">cooperation between Cython modules</a>.   
William Stein's avatar
William Stein committed
79 80
<h3> <a name="PyFuncsVsCFuncs"></a>Python functions vs. C functions</h3>

81
   There are two kinds of function definition in Cython:   
William Stein's avatar
William Stein committed
82 83 84 85 86 87 88 89 90 91
<p><b>Python functions</b> are defined using the <b>def</b> statement, as
 in Python. They take Python objects as parameters and return Python objects. 
 </p>

   
<p><b>C functions</b> are defined using the new <b>cdef</b> statement. They
 take either Python objects or C values as parameters, and can return either
 Python objects or C values. </p>

   
92
<p>Within a Cython module, Python functions and C functions can call each other
William Stein's avatar
William Stein committed
93 94
freely, but only Python functions can be called from outside the module by
interpreted Python code. So, any functions that you want to "export" from
95
 your Cython module must be declared as Python functions using <span style="font-weight: bold;">def</span>. </p>
William Stein's avatar
William Stein committed
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261

   
<p>Parameters of either type of function can be declared to have C data types,
 using normal C declaration syntax. For example, </p>

   
<blockquote>        <pre>def spam(int i, char *s):<br>&nbsp;&nbsp;&nbsp; ...</pre>
           <pre>cdef int eggs(unsigned long l, float f):<br>&nbsp;&nbsp;&nbsp; ...</pre>
   </blockquote>

   When a parameter of a Python function is declared to have a C data type, 
 it is passed in as a Python object and automatically converted to a C value, 
 if possible. Automatic conversion is currently only possible for numeric 
types and string types; attempting to use any other type for the parameter 
of a Python function will result in a compile-time error.   
<p>C functions, on the other hand, can have parameters of any type, since 
 they're passed in directly using a normal C function call. </p>

   
<h3> <a name="PyObjParams"></a>Python objects as parameters and return values</h3>

   If no type is specified for a parameter or return value, <i>it is assumed 
 to be a Python object.</i> (Note that this is different from the C convention, 
 where it would default to <tt>int</tt>.) For example, the following defines 
 a C function that takes two Python objects as parameters and returns a Python
 object:   
<blockquote>        <pre>cdef spamobjs(x, y):<br>&nbsp;&nbsp;&nbsp; ...</pre>
   </blockquote>

   Reference counting for these objects is performed automatically according 
 to the standard Python/C API rules (i.e. borrowed references are taken as
 parameters and a new reference is returned).   
<p>The name <b>object</b> can also be used to explicitly declare something 
 as a Python object. This can be useful if the name being declared would otherwise
be taken as the name of a type, for example, </p>

   
<blockquote>        <pre>cdef ftang(object int):<br>&nbsp;&nbsp;&nbsp; ...</pre>
   </blockquote>

   declares a parameter called <tt>int</tt> which is a Python object. You 
can also use <b>object </b>as the explicit return type of a function, e.g. 
  
<blockquote>        <pre>cdef object ftang(object int):<br>&nbsp;&nbsp;&nbsp; ...</pre>
   </blockquote>

   In the interests of clarity, it is probably a good idea to always be explicit 
 about <b>object </b>parameters in C functions.   
<h3> <a name="CVarAndTypeDecls"></a>C variable and type definitions</h3>

   The <b>cdef</b> statement is also used to declare C variables, either
local  or module-level:   
<blockquote>        <pre>cdef int i, j, k<br>cdef float f, g[42], *h</pre>
   </blockquote>

   and C struct, union or enum types:   
<blockquote>        <pre>cdef struct Grail:<br>&nbsp;&nbsp;&nbsp; int age<br>&nbsp;&nbsp;&nbsp; float volume</pre>
           <pre>cdef union Food:<br>&nbsp;&nbsp;&nbsp; char *spam<br>&nbsp;&nbsp;&nbsp; float *eggs</pre>
           <pre>cdef enum CheeseType:<br>&nbsp;&nbsp;&nbsp; cheddar, edam,&nbsp;<br>&nbsp;&nbsp;&nbsp; camembert</pre>
           <pre>cdef enum CheeseState:<br>&nbsp;&nbsp;&nbsp; hard = 1<br>&nbsp;&nbsp;&nbsp; soft = 2<br>&nbsp;&nbsp;&nbsp; runny = 3</pre>
   </blockquote>

   There is currently no special syntax for defining a constant, but you
can  use an anonymous enum declaration for this purpose, for example,   
<blockquote><tt>cdef enum:</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; tons_of_spam = 3</tt></blockquote>

   Note that the words <span style="font-family: monospace;">struct</span>, <span style="font-family: monospace;">union</span> and <span style="font-family: monospace;">enum</span> are used only when <i>defining</i> a type, not when referring to it. For example, to declare a variable pointing 
 to a Grail you would write   
<blockquote>        <pre>cdef Grail *gp</pre>
   </blockquote>

   and <i>not</i>  
<blockquote>        <pre>cdef struct Grail *gp <font color="#ed181e"># WRONG</font></pre>
   </blockquote>

   There is also a <b>ctypedef</b> statement for giving names to types, e.g. 
  
<blockquote>        <pre>ctypedef unsigned long ULong</pre>
           <pre>ctypedef int *IntPtr<br></pre></blockquote>

<h3><a name="AutomaticTypeConversions"></a>Automatic type conversions</h3>

In most situations, automatic conversions will be performed for the
basic numeric and string types when a Python object is used in a
context requiring a C value, or vice versa. The following table
summarises the conversion possibilities.<br>

<br>

<table style="margin-left: auto; margin-right: auto; width: 10%; text-align: left;" border="1" cellpadding="4" cellspacing="0">

  <tbody>
    <tr>
      <th style="vertical-align: top; width: 40%; white-space: nowrap;">C types<br>
      </th>
      <th style="vertical-align: top; width: 150px; white-space: nowrap;">From Python types<br>
      </th>
      <th style="vertical-align: top; width: 150px; white-space: nowrap;">To Python types<br>
      </th>
    </tr>
    <tr>
      <td colspan="1" rowspan="1" style="vertical-align: top; width: 40%; white-space: nowrap;">[unsigned] char<br>
[unsigned] short<br>
      int, long</td>
      <td colspan="1" rowspan="1" style="vertical-align: top; width: 150px; white-space: nowrap;">int, long<br>
      </td>
      <td colspan="1" rowspan="1" style="vertical-align: top; width: 150px; white-space: nowrap;">int<br>
      </td>
    </tr>
    <tr>
    </tr>

    <tr>
      <td colspan="1" rowspan="1" style="vertical-align: top; width: 40%; white-space: nowrap;">unsigned int<br>
unsigned long<br>
      [unsigned] long long<br>

      </td>
      <td colspan="1" rowspan="1" style="vertical-align: top; white-space: nowrap;">int, long<br>
      <br>

      </td>
      <td colspan="1" rowspan="1" style="vertical-align: top; white-space: nowrap;">long<br>
      <br>

      </td>
    </tr>
    <tr>
      
      
      
    </tr>
    <tr>
      <td style="vertical-align: top; width: 40%; white-space: nowrap;">float, double, long double<br>
      </td>
      <td style="vertical-align: top; width: 150px; white-space: nowrap;">int, long, float<br>
      </td>
      <td style="vertical-align: top; width: 150px; white-space: nowrap;">float<br>
      </td>
    </tr>
    <tr>
      <td style="vertical-align: top; width: 40%; white-space: nowrap;">char *<br>
      </td>
      <td style="vertical-align: top; width: 150px; white-space: nowrap;">str<span style="font-style: italic;"></span><br>
      </td>
      <td style="vertical-align: top; width: 150px; white-space: nowrap;">str<br>
      </td>
    </tr>
  </tbody>
</table>

<br>

<h4><a name="PyToCStringCaveats"></a>Caveats when using a Python string in a C context</h4>

You need to be careful when using a Python string in a context expecting a <span style="font-family: monospace;">char *</span>.
In this situation, a pointer to the contents of the Python string is
used, which is only valid as long as the Python string exists. So you
need to make sure that a reference to the original Python string is
held for as long as the C string is needed. If you can't guarantee that
the Python string will live long enough, you will need to copy the C
string.<br>

<br>

262
Cython detects and prevents <span style="font-style: italic;">some</span> mistakes of this kind. For instance, if you attempt something like<br>
William Stein's avatar
William Stein committed
263 264 265

<pre style="margin-left: 40px;">cdef char *s<br>s = pystring1 + pystring2</pre>

266
then Cython will produce the error message "<span style="font-weight: bold;">Obtaining char * from temporary Python value</span>".
William Stein's avatar
William Stein committed
267 268
The reason is that concatenating the two Python strings produces a new
Python string object that is referenced only by a temporary internal
269
variable that Cython generates. As soon as the statement has finished,
William Stein's avatar
William Stein committed
270
the temporary variable will be decrefed and the Python string
271
deallocated, leaving <span style="font-family: monospace;">s</span> dangling. Since this code could not possibly work, Cython refuses to compile it.<br>
William Stein's avatar
William Stein committed
272 273 274 275 276 277 278 279 280 281 282 283

<br>

The solution is to assign the result of the concatenation to a Python variable, and then obtain the char * from that, i.e.<br>

<pre style="margin-left: 40px;">cdef char *s<br>p = pystring1 + pystring2<br>s = p<br></pre>

It is then your responsibility to hold the reference <span style="font-family: monospace;">p</span> for as long as necessary.<br>

<br>

Keep in mind that the rules used to detect such errors are only
284
heuristics. Sometimes Cython will complain unnecessarily, and sometimes
William Stein's avatar
William Stein committed
285 286 287 288 289 290 291 292 293 294
it will fail to detect a problem that exists. Ultimately, you need to
understand the issue and be careful what you do.<br>

<blockquote>
   </blockquote>


     
<h3> <a name="ScopeRules"></a>Scope rules</h3>

295
   Cython determines whether a variable belongs to a local scope, the module 
William Stein's avatar
William Stein committed
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318
 scope, or the built-in scope <i>completely statically.</i> As with Python, 
 assigning to a variable which is not otherwise declared implicitly declares 
 it to be a Python variable residing in the scope where it is assigned. Unlike
 Python, however, a name which is referred to but not declared or assigned
 is assumed to reside in the <i>builtin scope, </i>not the module scope.
Names  added to the module dictionary at run time will not shadow such names.
  
<p>You can use a <b>global</b> statement at the module level to explicitly 
 declare a name to be a module-level name when there would otherwise not be
any indication of this, for example, </p>

   
<blockquote><tt>global __name__</tt> <br>
    <tt>print __name__</tt></blockquote>

   Without the <b>global</b> statement, the above would print the name of 
the builtins module.<br>

  <br>

  Note: A consequence of these rules is that the module-level scope behaves
 the same way as a Python local scope if you refer to a variable before assigning
 to it. In particular, tricks such as the following will <i>not</i> work
319
in  Cython:<br>
William Stein's avatar
William Stein committed
320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349

   
<blockquote>        <pre>try:<br>&nbsp; x = True<br>except NameError:<br>&nbsp; True = 1<br></pre>
  </blockquote>

  because, due to the assignment, the True will always be looked up in the
 module-level scope. You would have to do something like this instead:<br>

   
<blockquote>        <pre>import __builtin__<br>try:<br>  True = __builtin__.True<br>except AttributeError:<br>  True = 1<br></pre>
  </blockquote>

   
<hr width="100%">  
<h3> <a name="StatsAndExprs"></a>Statements and expressions</h3>

   Control structures and expressions follow Python syntax for the most part. 
 When applied to Python objects, they have the same semantics as in Python 
 (unless otherwise noted). Most of the Python operators can also be applied 
 to C values, with the obvious semantics.   
<p>If Python objects and C values are mixed in an expression, conversions 
 are performed automatically between Python objects and C numeric or string 
 types. </p>

   
<p>Reference counts are maintained automatically for all Python objects, and
all Python operations are automatically checked for errors, with appropriate 
 action taken. </p>

   
350
<h4> <a name="ExprSyntaxDifferences"></a>Differences between C and Cython
William Stein's avatar
William Stein committed
351 352 353
expressions</h4>
There
are some differences in syntax and semantics between C expressions and
354
Cython expressions, particularly in the area of C constructs which have
William Stein's avatar
William Stein committed
355 356 357 358 359 360
no direct equivalent in Python.<br>

<ul>
<li>An integer literal without an <span style="font-family: monospace; font-weight: bold;">L</span> suffix is treated as a C constant, and will be truncated to whatever size your C compiler thinks appropriate. With an <span style="font-family: monospace; font-weight: bold;">L</span> suffix, it will be converted to Python long integer (even if it would be small enough to fit into a C int).<br>
    <br>
  </li>
361
<li> There is no <b><tt>-&gt;</tt></b> operator in Cython. Instead of <tt>p-&gt;x</tt>, 
William Stein's avatar
William Stein committed
362 363
 use <tt>p.x</tt></li>
    
364
  &nbsp; <li> There is no <b><tt>*</tt></b> operator in Cython. Instead of 
William Stein's avatar
William Stein committed
365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382
    <tt>*p</tt>,  use <tt>p[0]</tt></li>
    
  &nbsp; <li> There is an <b><tt>&amp;</tt></b> operator, with the same semantics
 as in C.</li>
    
  &nbsp; <li> The null C pointer is called <b><tt>NULL</tt></b>, not 0 (and
     <tt>NULL</tt> is a reserved word).</li>
    
  &nbsp; <li> Character literals are written with a <b>c</b> prefix, for
example:</li>
           <ul>
                <pre>c'X'</pre>
          </ul>
    <li> Type casts are written <b><tt>&lt;type&gt;value</tt></b> , for example:</li>
           <ul>
                <pre>cdef char *p, float *q<br>p = &lt;char*&gt;q</pre>
          </ul>
   <i><b>Warning</b>: Don't attempt to use a typecast to convert between
383
Python  and C data types -- it won't do the right thing. Leave Cython to perform 
William Stein's avatar
William Stein committed
384 385 386 387 388 389 390 391 392 393 394 395
the conversion automatically.</i>  
</ul>

     
<h4> <a name="ForFromLoop"></a>Integer for-loops</h4>

   You should be aware that a for-loop such as   
<blockquote><tt>for i in range(n):</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>

   won't be very fast, even if <tt>i</tt> and <tt>n</tt> are declared as
C integers, because <tt>range</tt> is a Python function. For iterating over 
396
ranges of integers, Cython has another form of for-loop:   
William Stein's avatar
William Stein committed
397 398 399 400
<blockquote><tt>for i from 0 &lt;= i &lt; n:</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>

   If the loop variable and the lower and upper bounds are all C integers, 
401
this form of loop will be much faster, because Cython will translate it into 
William Stein's avatar
William Stein committed
402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448
pure C code.   
<p>Some things to note about the <tt>for-from</tt> loop: </p>

   
<ul>

   <li> The target expression must be a variable name.</li>
    <li> The name between the lower and upper bounds must be the same as
the  target name.</li>
    <li> The direction of iteration is determined by the relations. If they
 are both from the set {<tt>&lt;</tt>, <tt>&lt;=</tt>} then it is upwards;
 if they are both from the set {<tt>&gt;</tt>, <tt>&gt;=</tt>} then it is
downwards. (Any other combination is disallowed.)</li>
    
</ul>

   Like other Python looping statements, <tt>break</tt> and <tt>continue</tt> may be used in the body, and the loop may have an <tt>else</tt> clause. 
 
<h2>   <hr width="100%"></h2>

     
<h3> <a name="ExceptionValues"></a>Error return values</h3>

   If you don't do anything special, a function declared with <b>cdef</b> that does not return a Python object has no way of reporting Python exceptions 
 to its caller. If an exception is detected in such a function, a warning 
message is printed and the exception is ignored.   
<p>If you want a C function that does not return a Python object to be able
 to propagate exceptions to its caller, you need to declare an <b>exception 
 value</b> for it. Here is an example: </p>

   
<blockquote><tt>cdef int spam() except -1:</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>

   With this declaration, whenever an exception occurs inside <tt>spam</tt>, 
 it will immediately return with the value <tt>-1</tt>. Furthermore, whenever 
 a call to <tt>spam</tt> returns <tt>-1</tt>, an exception will be assumed 
 to have occurred and will be propagated.   
<p>When you declare an exception value for a function, you should never explicitly
 return that value. If all possible return values are legal and you can't
reserve one entirely for signalling errors, you can use an alternative form
of exception value declaration: </p>

   
<blockquote><tt>cdef int spam() except? -1:</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>

449
   The "?" indicates that the value <tt>-1</tt> only indicates a <i>possible</i> error. In this case, Cython generates a call to <tt>PyErr_Occurred</tt>if the
William Stein's avatar
William Stein committed
450 451 452 453 454 455 456
exception value is returned, to make sure it really is an error.   
<p>There is also a third form of exception value declaration: </p>

   
<blockquote><tt>cdef int spam() except *:</tt> <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>

457
   This form causes Cython to generate a call to <tt>PyErr_Occurred</tt> after 
William Stein's avatar
William Stein committed
458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486
 <i>every</i> call to spam, regardless of what value it returns. If you have
 a function returning <tt>void</tt> that needs to propagate errors, you will
 have to use this form, since there isn't any return value to test.   
<p>Some things to note: </p>

   
<ul>

   <li> Currently, exception values can only declared for functions returning
 an integer, float or pointer type, and the value must be a <i>literal</i>, 
 not an expression (although it can be negative). The only possible pointer 
 exception value is <tt>NULL</tt>. Void functions can only use the <tt>except 
 *</tt> form.</li>
    <br>
  &nbsp; <li> The exception value specification is part of the signature
of  the function. If you're passing a pointer to a function as a parameter
or  assigning it to a variable, the declared type of the parameter or variable
 must have the same exception value specification (or lack thereof). Here
is an example of a pointer-to-function declaration with an exception value:</li>
           <ul>
                <pre><tt>int (*grail)(int, char *) except -1</tt></pre>
          </ul>
    <li> You don't need to (and shouldn't) declare exception values for functions 
 which return Python objects. Remember that a function with no declared return
 type implicitly returns a Python object.</li>
    
</ul>

     
487
<h4> <a name="CheckingReturnValues"></a>Checking return values of non-Cython 
William Stein's avatar
William Stein committed
488 489 490 491 492 493 494 495 496 497
 functions</h4>

   It's important to understand that the <tt>except</tt> clause does <i>not</i> cause an error to be <i>raised</i> when the specified value is returned. For
example, you can't write something like   
<blockquote>        <pre>cdef extern FILE *fopen(char *filename, char *mode) except NULL <font color="#ed181e"># WRONG!</font></pre>
   </blockquote>

   and expect an exception to be automatically raised if a call to fopen
returns  NULL. The except clause doesn't work that way; its only purpose
is for <i>propagating</i> exceptions that have already been raised, either
498
by a Cython function or a  C function that calls Python/C API routines. To
William Stein's avatar
William Stein committed
499 500 501 502 503 504 505 506 507 508 509
get an exception from a non-Python-aware  function such as fopen, you will
have to check the return value and raise  it yourself, for example,   
<blockquote>        <pre>cdef FILE *p<br>p = fopen("spam.txt", "r")<br>if p == NULL:<br>&nbsp;&nbsp;&nbsp; raise SpamError("Couldn't open the spam file")</pre>
   </blockquote>

     
<h4>   <hr width="100%"></h4>

     
<h4> <a name="IncludeStatement"></a>The <tt>include</tt> statement</h4>

510
   For convenience, a large Cython module can be split up into a number of 
William Stein's avatar
William Stein committed
511 512 513 514 515 516
files which are put together using the <b>include</b> statement, for example 
  
<blockquote>        <pre>include "spamstuff.pxi"</pre>
   </blockquote>

   The contents of the named file are textually included at that point. The 
517
 included file can contain any complete top-level Cython statements, including 
William Stein's avatar
William Stein committed
518 519 520
 other <b>include</b> statements. The <b>include</b> statement itself can 
only appear at the top level of a file.   
<p>The <b>include</b> statement can also be used in conjunction with <a href="#PublicDecls"><b>public</b> declarations</a> to make C functions and
521
 variables defined in one Cython module accessible to another. However, note
William Stein's avatar
William Stein committed
522
 that some of these uses have been superseded by the facilities described
523
in <a href="sharing.html">Sharing Declarations Between Cython Modules</a>,
William Stein's avatar
William Stein committed
524 525 526 527 528 529 530 531
and it is expected that use of the <b>include</b> statement for this purpose
will be phased out altogether in future versions. </p>

   
<h2>   <hr width="100%"><a name="InterfacingWithExternal"></a>Interfacing with External
 C Code   
<hr width="100%"></h2>

532
   One of the main uses of Cython is wrapping existing libraries of C code. 
William Stein's avatar
William Stein committed
533 534 535
This is achieved by using <a href="#ExternDecls">external declarations</a> to declare the C functions and variables from the library that you want to
 use.   
<p>You can also use <a href="#PublicDecls">public declarations</a> to make 
536
 C functions and variables defined in a Cython module available to external 
William Stein's avatar
William Stein committed
537 538
 C code. The need for this is expected to be less frequent, but you might 
want to do it, for example, if you are embedding Python in another application 
539
 as a scripting language. Just as a Cython module can be used as a bridge to
William Stein's avatar
William Stein committed
540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560
allow Python code to call C code, it can also be used to allow C code to
call Python code. </p>

   
<h3> <a name="ExternDecls"></a>External declarations</h3>

   By default, C functions and variables declared at the module level are 
local to the module (i.e. they have the C <b>static</b> storage class). They 
can also be declared <b>extern</b> to specify that they are defined elsewhere,
 for example:   
<blockquote>        <pre>cdef extern int spam_counter</pre>
           <pre>cdef extern void order_spam(int tons)</pre>
   </blockquote>

     
<blockquote>         </blockquote>

     
<h4> <a name="ReferencingHeaders"></a>Referencing C header files</h4>

   When you use an extern definition on its own as in the examples above, 
561
Cython includes a declaration for it in the generated C file. This can cause 
William Stein's avatar
William Stein committed
562 563 564 565
problems if the declaration doesn't exactly match the declaration that will 
be seen by other C code. If you're wrapping an existing C library, for example, 
it's important that the generated C code is compiled with exactly the same 
declarations as the rest of the library.   
566
<p>To achieve this, you can tell Cython that the declarations are to be found
William Stein's avatar
William Stein committed
567 568 569 570 571 572 573 574 575 576 577
 in a C header file, like this: </p>

   
<blockquote>        <pre>cdef extern from "spam.h":</pre>
           <pre>&nbsp;&nbsp;&nbsp; int spam_counter</pre>
           <pre>&nbsp;&nbsp;&nbsp; void order_spam(int tons)</pre>
   </blockquote>

   The <b>cdef extern from</b> clause does three things:   
<ol>

578
   <li> It directs Cython to place a <b>#include</b> statement for the named
William Stein's avatar
William Stein committed
579 580
 header file in the generated C code.<br>
   </li>
581
  &nbsp; <li> It prevents Cython from generating any C code for the declarations
William Stein's avatar
William Stein committed
582 583 584 585 586 587 588
 found in the associated block.<br>
   </li>
  &nbsp; <li> It treats all declarations within the block as though they
started  with <b>cdef extern</b>.</li>
    
</ol>

589 590 591
   It's important to understand that Cython does <i>not</i> itself read the 
C header file, so you still need to provide Cython versions of any declarations 
 from it that you use. However, the Cython declarations don't always have to
William Stein's avatar
William Stein committed
592 593 594 595
exactly match the C ones, and in some cases they shouldn't or can't. In particular:
  
<ol>

596
   <li> Don't use <b>const</b>. Cython doesn't know anything about const,
William Stein's avatar
William Stein committed
597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675
so  just leave it out. Most of the time this shouldn't cause any problem,
although  on rare occasions you might have to use a cast.<sup><a href="#Footnote1"> 1</a></sup><br>
   </li>
  &nbsp; <li> Leave out any platform-specific extensions to C declarations
 such as <b>__declspec()</b>.<br>
   </li>
  &nbsp; <li> If the header file declares a big struct and you only want
to  use a few members, you only need to declare the members you're interested 
in. Leaving the rest out doesn't do any harm, because the C compiler will 
use the full definition from the header file.<br>
     <br>
 In some cases, you might not need <i>any</i> of the struct's members, in
which case you can just put <tt>pass</tt> in the body of the struct declaration,
e.g.<br>
     <br>
     <tt>&nbsp; &nbsp; cdef extern from "foo.h":<br>
 &nbsp; &nbsp; &nbsp; &nbsp; struct spam:<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pass</tt><br>
    <br>
Note that you can only do this inside a <b>cdef extern from</b> block; struct
declarations anywhere else must be non-empty.<br>
    <br>
   </li>
    <li> If the header file uses typedef names such as <b>size_t </b>to refer 
to platform-dependent flavours of numeric types, you will need a corresponding 
     <b>ctypedef</b> statement, but you don't need to match the type exactly,
 just use something of the right general kind (int, float, etc). For example,</li>
           <ol>
                <pre>ctypedef int size_t</pre>
          </ol>
   will work okay whatever the actual size of a size_t is (provided the header 
 file defines it correctly). <br>
  &nbsp; <li> If the header file uses macros to define constants, translate
 them into a dummy <b>enum</b> declaration.<br>
   </li>
  &nbsp; <li> If the header file defines a function using a macro, declare
 it as though it were an ordinary function, with appropriate argument and
result types.</li>
    
</ol>

   A few more tricks and tips:   
<ul>

   <li> If you want to include a C header because it's needed by another
header,  but don't want to use any declarations from it, put <tt><font size="+1">pass</font></tt> in the extern-from block:</li>
    
</ul>

     
<ul>

          <ul>
      <tt>cdef extern from "spam.h":</tt> <br>
      <tt>&nbsp;&nbsp;&nbsp; pass</tt>        </ul>
    
</ul>

     
<ul>

   <li> If you want to include some external declarations, but don't want 
to specify a header file (because it's included by some other header that 
you've already included) you can put <tt>*</tt> in place of the header file 
name:</li>
    
</ul>

     
<blockquote>        <blockquote><tt>cdef extern from *:</tt> <br>
      <tt>&nbsp;&nbsp;&nbsp; ...</tt></blockquote>
   </blockquote>

     
<h4> <a name="StructDeclStyles"></a>Styles of struct, union and enum declaration</h4>

   There are two main ways that structs, unions and enums can be declared 
in C header files: using a tag name, or using a typedef. There are also some
 variations based on various combinations of these.   
676 677 678
<p>It's important to make the Cython declarations match the style used in the
header file, so that Cython can emit the right sort of references to the type
in the code it generates. To make this possible, Cython provides two different
William Stein's avatar
William Stein committed
679 680 681 682 683 684
syntaxes for declaring a struct, union or enum type. The style introduced
above corresponds to the use of a tag name. To get the other style, you prefix
the declaration with <b>ctypedef</b>, as illustrated below. </p>

   
<p>The following table shows the various possible styles that can be found 
685
 in a header file, and the corresponding Cython declaration that you should 
William Stein's avatar
William Stein committed
686 687 688 689
 put in the <b>cdef exern from </b>block. Struct declarations are used as 
an example; the same applies equally to union and enum declarations. </p>

   
690
<p>Note that in all the cases below, you refer to the type in Cython code simply
William Stein's avatar
William Stein committed
691 692 693 694 695 696 697 698
as <tt><font size="+1">Foo</font></tt>, not <tt><font size="+1">struct Foo</font></tt>.
 <br>
  &nbsp;  <table cellpadding="5">
   <tbody>
      <tr bgcolor="#8cbc1c" valign="top">
   <td bgcolor="#8cbc1c">&nbsp;</td>
    <td bgcolor="#ff9933" nowrap="nowrap"><b>C code</b></td>
    <td bgcolor="#66cccc" valign="top"><b>Possibilities for corresponding 
699
Cython  code</b></td>
William Stein's avatar
William Stein committed
700 701 702 703 704 705 706 707 708
    <td bgcolor="#99cc33" valign="top"><b>Comments</b></td>
   </tr>
    <tr bgcolor="#8cbc1c" valign="top">
   <td>1</td>
    <td bgcolor="#ff9900"><tt>struct Foo {</tt> <br>
        <tt>&nbsp; ...</tt> <br>
        <tt>};</tt></td>
    <td bgcolor="#66cccc"><tt>cdef struct Foo:</tt> <br>
        <tt>&nbsp; ...</tt></td>
709
    <td>Cython will refer to the type as <tt>struct Foo </tt>in the generated 
William Stein's avatar
William Stein committed
710 711 712 713 714 715 716 717 718
 C code<tt>.</tt></td>
   </tr>
    <tr bgcolor="#8cbc1c" valign="top">
   <td valign="top">2</td>
    <td bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct {</tt> <br>
        <tt>&nbsp; ...</tt> <br>
        <tt>} Foo;</tt></td>
    <td bgcolor="#66cccc" valign="top"><tt>ctypedef struct Foo:</tt> <br>
        <tt>&nbsp; ...</tt></td>
719
    <td valign="top">Cython will refer to the type simply as <tt>Foo</tt>
William Stein's avatar
William Stein committed
720 721 722 723 724 725 726 727 728 729 730 731
in  the generated C code.</td>
   </tr>
    <tr bgcolor="#8cbc1c" valign="top">
   <td rowspan="2">3</td>
    <td rowspan="2" bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct
foo  {</tt> <br>
        <tt>&nbsp; ...</tt> <br>
        <tt>} Foo;</tt></td>
    <td bgcolor="#66cccc" nowrap="nowrap" valign="top"><tt>cdef struct foo:</tt>       <br>
        <tt>&nbsp; ...</tt> <br>
        <tt>ctypedef foo Foo #optional</tt></td>
    <td rowspan="2" valign="top">If the C header uses both a tag and a typedef 
732
 with <i>different</i> names, you can use either form of declaration in Cython
William Stein's avatar
William Stein committed
733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769
 (although if you need to forward reference the type, you'll have to use
the  first form).</td>
   </tr>
    <tr>
   <td bgcolor="#66cccc"><tt>ctypedef struct Foo:</tt> <br>
        <tt>&nbsp; ...</tt></td>
   </tr>
    <tr bgcolor="#8cbc1c" valign="top">
   <td>4</td>
    <td bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct Foo {</tt> <br>
        <tt>&nbsp; ...</tt> <br>
        <tt>} Foo;</tt></td>
    <td bgcolor="#66cccc" valign="top"><tt>cdef struct Foo:</tt> <br>
        <tt>&nbsp; ...</tt></td>
    <td>If the header uses the <i>same</i> name for the tag and the typedef, 
 you won't be able to include a <b>ctypedef</b> for it -- but then, it's not
necessary.</td>
   </tr>
          </tbody>  </table>
    </p>

   
<h4> <a name="AccessingAPI"></a>Accessing Python/C API routines</h4>

   One particular use of the <b>cdef extern from</b> statement is for gaining 
 access to routines in the Python/C API. For example,   
<blockquote>        <pre>cdef extern from "Python.h":</pre>
           <pre>&nbsp;&nbsp;&nbsp; object PyString_FromStringAndSize(char *s, int len)</pre>
   </blockquote>

   will allow you to create Python strings containing null bytes.   
<p> </p>

   
<hr width="100%">  
<h3> <a name="CNameSpecs"></a>Resolving naming conflicts - C name specifications</h3>

770
   Each Cython module has a single module-level namespace for both Python
William Stein's avatar
William Stein committed
771 772 773
and  C names. This can be inconvenient if you want to wrap some external
C functions  and provide the Python user with Python functions of the same
names.   
774
<p>Cython 0.8 provides a couple of different ways of solving this problem. 
William Stein's avatar
William Stein committed
775 776 777
 The best way, especially if you have many C functions to wrap, is probably 
 to put the extern C function declarations into a different namespace using 
 the facilities described in the section on <a href="sharing.html">sharing 
778
 declarations between Cython modules</a>. </p>
William Stein's avatar
William Stein committed
779 780 781

   
<p>The other way is to use a <b>c name specification</b> to give different 
782
 Cython and C names to the C function. Suppose, for example, that you want 
William Stein's avatar
William Stein committed
783 784 785 786 787 788 789
to wrap an external function called <tt>eject_tomato</tt>. If you declare 
it as </p>

   
<blockquote>        <pre>cdef extern void c_eject_tomato "eject_tomato" (float speed)</pre>
   </blockquote>

790
   then its name inside the Cython module will be <tt>c_eject_tomato</tt>, 
William Stein's avatar
William Stein committed
791 792 793 794 795 796 797 798
whereas its name in C will be <tt>eject_tomato</tt>. You can then wrap it 
with   
<blockquote>        <pre>def eject_tomato(speed):<br>&nbsp; c_eject_tomato(speed)</pre>
   </blockquote>

   so that users of your module can refer to it as <tt>eject_tomato</tt>. 
  
<p>Another use for this feature is referring to external names that happen 
799 800
 to be Cython keywords. For example, if you want to call an external function 
 called <tt>print</tt>, you can rename it to something else in your Cython 
William Stein's avatar
William Stein committed
801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816
module.  </p>

   
<p>As well as functions, C names can be specified for variables, structs, 
 unions, enums, struct and union members, and enum values. For example, </p>

   
<blockquote>        <pre>cdef extern int one "ein", two "zwei"<br>cdef extern float three "drei"<br><br>cdef struct spam "SPAM":<br>&nbsp; int i "eye"</pre>
   <tt>cdef enum surprise "inquisition":</tt> <br>
    <tt>&nbsp; first "alpha"</tt> <br>
    <tt>&nbsp; second "beta" = 3</tt></blockquote>

     
<hr width="100%">  
<h3> <a name="PublicDecls"></a>Public Declarations</h3>

817 818
   You can make C variables and functions defined in a Cython module accessible 
 to external C code (or another Cython module) using the <b><tt>public</tt></b> keyword, as follows:   
William Stein's avatar
William Stein committed
819 820 821 822
<blockquote><tt>cdef public int spam # public variable declaration</tt>       <p><tt>cdef public void grail(int num_nuns): # public function declaration</tt>   <br>
    <tt>&nbsp;&nbsp;&nbsp; ...</tt></p>
  </blockquote>

823
   If there are any <tt>public</tt> declarations in a Cython module, a <b>.h</b> file is generated containing equivalent C declarations for inclusion in other
William Stein's avatar
William Stein committed
824
 C code.   
825 826
<p>Cython also generates a <b>.pxi</b> file containing Cython versions of the
 declarations for inclusion in another Cython module using the <b><a href="#IncludeStatement">include</a></b> statement. If you use this, you
William Stein's avatar
William Stein committed
827 828 829 830 831 832 833 834
 will need to arrange for the module using the declarations to be linked
against  the module defining them, and for both modules to be available to
the dynamic  linker at run time. I haven't tested this, so I can't say how
well it will  work on the various platforms. </p>

   
<blockquote>NOTE: If all you want to export is an extension type, there is
 now a better way -- see <a href="sharing.html">Sharing Declarations Between
835
 Cython Modules</a>.</blockquote>
William Stein's avatar
William Stein committed
836 837 838 839 840

     
<h2>   <hr width="100%">Extension Types   
<hr width="100%"></h2>

841
   One of the most powerful features of Cython is the ability to easily create 
William Stein's avatar
William Stein committed
842 843 844
 new built-in Python types, called <b>extension types</b>. This is a major 
 topic in itself, so there is a&nbsp; <a href="extension_types.html">separate 
 page</a> devoted to it.   
845
<h2>   <hr width="100%">Sharing Declarations Between Cython Modules   
William Stein's avatar
William Stein committed
846 847
<hr width="100%"></h2>

848
   Cython 0.8 introduces a substantial new set of facilities allowing a Cython 
William Stein's avatar
William Stein committed
849
 module to easily import and use C declarations and extension types from another
850
Cython module. You can now create a set of co-operating Cython modules just
William Stein's avatar
William Stein committed
851 852 853 854 855 856 857 858
as easily as you can create a set of co-operating Python modules. There is
a <a href="sharing.html">separate page</a> devoted to this topic.   
<h2>   <hr width="100%"><a name="Limitations"></a>Limitations   
<hr width="100%"></h2>

     
<h3> <a name="Unsupported"></a>Unsupported Python features</h3>

859
   Cython is not quite a full superset of Python. The following restrictions 
William Stein's avatar
William Stein committed
860 861 862 863 864 865 866 867 868 869
 apply:   
<blockquote> <li> Function definitions (whether using <b>def</b> or <b>cdef</b>)
 cannot be nested within other function definitions.<br>
   </li>
  &nbsp; <li> Class definitions can only appear at the top level of a module,
 not inside a function.<br>
   </li>
  &nbsp; <li> The<tt> import *</tt> form of import is not allowed anywhere
 (other forms of the import statement are fine, though).<br>
   </li>
870
  &nbsp; <li> Generators cannot be defined in Cython.<br>
William Stein's avatar
William Stein committed
871 872 873 874 875 876 877
     <br>
   </li>
    <li> The <tt>globals()</tt> and <tt>locals()</tt> functions cannot be 
used.</li>
   </blockquote>

   The above restrictions will most likely remain, since removing them would 
878
 be difficult and they're not really needed for Cython's intended applications. 
William Stein's avatar
William Stein committed
879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897
  
<p>There are also some temporary limitations, which may eventually be lifted, including: 
 </p>

   
<blockquote> <li> Class and function definitions cannot be placed inside
control structures.<br>
   </li>
  &nbsp; <li> In-place arithmetic operators (+=, etc) are not yet supported.<br>
   </li>
  &nbsp; <li> List comprehensions are not yet supported.<br>
   </li>
  &nbsp; <li> There is no support for Unicode.<br>
   </li>
  &nbsp; <li> Special methods of extension types cannot have functioning
docstrings.<br>
     <br>
   </li>
    <li> The use of string literals as comments is not recommended at present,
898
 because Cython doesn't optimize them away, and won't even accept them in
William Stein's avatar
William Stein committed
899 900 901 902
places  where executable statements are not allowed.</li>
   </blockquote>
   
<h3> <a name="SemanticDifferences"></a>Semantic differences between Python
903
 and Cython</h3>
William Stein's avatar
William Stein committed
904 905 906 907 908 909

     
<h4> Behaviour of class scopes</h4>

   In Python, referring to a method of a class inside the class definition, 
 i.e. while the class is being defined, yields a plain function object, but
910
 in Cython it yields an unbound method<sup><font size="-2"><a href="#Footnote2">2</a></font></sup>. A consequence of this is that the
William Stein's avatar
William Stein committed
911 912 913 914 915 916
usual idiom for using the classmethod and staticmethod functions, e.g.   
<blockquote>        <pre>class Spam:</pre>
           <pre>&nbsp; def method(cls):<br>&nbsp;&nbsp;&nbsp; ...</pre>
           <pre>&nbsp; method = classmethod(method)</pre>
   </blockquote>

917
   will not work in Cython. This can be worked around by defining the function 
William Stein's avatar
William Stein committed
918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949
 <i>outside</i> the class, and then assigning the result of classmethod or
 staticmethod inside the class, i.e.   
<blockquote>        <pre>def Spam_method(cls):<br>&nbsp; ...</pre>
           <pre>class Spam:</pre>
           <pre>&nbsp; method = classmethod(Spam_method)</pre>
   </blockquote>

     
<h1>   <hr width="100%"><font size="+0">Footnotes</font>  <hr width="100%"></h1>

   <a name="Footnote1"></a>1. A problem with const could arise if you have 
something like   
<blockquote>        <pre>cdef extern from "grail.h":<br>&nbsp; char *nun</pre>
   </blockquote>

   where grail.h actually contains   
<blockquote>        <pre>extern const char *nun;</pre>
   </blockquote>

   and you do   
<blockquote>        <pre>cdef void languissement(char *s):<br>&nbsp; #something that doesn't change s</pre>
           <pre>...</pre>
           <pre>languissement(nun)</pre>
   </blockquote>

   which will cause the C compiler to complain. You can work around it by 
casting  away the constness:   
<blockquote>        <pre>languissement(&lt;char *&gt;nun)</pre>
   </blockquote>

     
<hr width="100%"><a name="Footnote2"></a>2. The reason for the different behaviour
950
of class scopes is that Cython-defined Python functions are PyCFunction objects,
William Stein's avatar
William Stein committed
951 952
not PyFunction objects, and are not recognised by the machinery that creates
a bound or unbound method when a function is extracted from a class. To get
953
around this, Cython wraps each method in an unbound method object itself before
William Stein's avatar
William Stein committed
954 955 956 957 958 959 960
storing it in the class's dictionary. <br>

  &nbsp; <br>

  <br>

 </body></html>