Learning Cython Programming using Python: Expand your existing legacy applications in C
using Python
Download
Introduction
Cython is a tool that makes writing C extensions to Python as easy as writing Python itself. This is the slogan to which Cython conforms. For those who don't know what I am talking about, writing C extensions to Python from scratch is a fairly difficult process; unless you really understand the Python-C API fully with respect to GIL and garbage collection as well as managing your own reference counting, it's a very difficult process.
I tend to consider Cython to be along these lines: what Jython is to Java and Python, Cython is to C/C++ and Python. It allows us to extend and develop bindings to applications in a really intuitive manner so that we are able to reuse code from levels of the software stack. The Cython compiler compiles the Cython language or even pure Python to a native C Python module, which can be loaded like any Python module via the normal import. It not only generates all the wrapper and boilerplate code, but also commands the Python garbage collector to add all the necessary reference counting code.
What's interesting with the Cython language is that it has native support for understanding C types and is able to juggle them from both languages. It's simply an extension of Python that has additional keywords and some more constructs and which allows you to call into C or Python.
What this book covers
Chapter 1, Cython Won't Bite, will give you an introduction to what Cython is and how it works. It covers setting up your environment and running the "Hello World" application.
Chapter 2, Understanding Cython, will start to get serious with Cython and will discuss how to describe C declarations with respect to Cython along with calling conventions and type conversion.
Chapter 3, Extending Applications, will walk you through comparing the execution of pure Python code with the Cython version of the same code. We also look at extending Tmux, a pure C project, with Cython.
Chapter 4, Debugging Cython, will cover how to use GDB to debug your code and the relative GDB commands. There is also an extensive section on caveats and things to be aware of as well as conventions.
Chapter 5, Advanced Cython, will cover the usage of C++ with Cython, which is just as easy as using C with Cython. We will also work through all the syntax necessary to wrap C++. We will then look into the caveats and more on optimizations, comparing a Python XML parser with a Cython XML parser on large XML files.
Chapter 6, Further Reading, wraps up the book with a final look at some caveats and conventions. Then, we compare Cython against other similar tools like Numba and SWIG, and we will discuss how its used in NumPy and how we can use PyPy and Python 3
What you need for this book
For this book, I used my MacBook and an Ubuntu virtual machine (GDB is too old on Mac OS X for debugging). You will require the following on Mac OS X:
• Xcode
• Cython
• GCC/Clang
• Make
• Python
• Python config
• Python distutils
On Ubuntu, you can install most components via the following:
$ sudo apt-get install build-essential gdb cython
Of course, I will go over this in the introduction, but as long as you have a C compiler and Python and have Python headers installed, you will have everything you need for Cython.
Who this book is for
This book is intended for C developers who like using Python and Python users wanting to implement native C/C++ extensions to Python. As a reader, you can expect to be shown how you can develop applications with Cython, with an emphasis on extending existing systems with help on how you can approach it.
Extending legacy systems can be difficult, but the rewards can be great. Consider very low-level system daemons that we could abstract and extend them and interact with the data from Python in a nice high-level way while leaving all performance-sensitive code alone! This model of development can prove to be efficient and of great return to development time; this can be particularly expensive when it comes to C applications.
It also allows for much more rapid development of the state or logic in a system. There is no need to worry about long data conversion algorithms in C for doing small things and then needing to change it all again
Introduction
Cython is a tool that makes writing C extensions to Python as easy as writing Python itself. This is the slogan to which Cython conforms. For those who don't know what I am talking about, writing C extensions to Python from scratch is a fairly difficult process; unless you really understand the Python-C API fully with respect to GIL and garbage collection as well as managing your own reference counting, it's a very difficult process.
I tend to consider Cython to be along these lines: what Jython is to Java and Python, Cython is to C/C++ and Python. It allows us to extend and develop bindings to applications in a really intuitive manner so that we are able to reuse code from levels of the software stack. The Cython compiler compiles the Cython language or even pure Python to a native C Python module, which can be loaded like any Python module via the normal import. It not only generates all the wrapper and boilerplate code, but also commands the Python garbage collector to add all the necessary reference counting code.
What's interesting with the Cython language is that it has native support for understanding C types and is able to juggle them from both languages. It's simply an extension of Python that has additional keywords and some more constructs and which allows you to call into C or Python.
What this book covers
Chapter 1, Cython Won't Bite, will give you an introduction to what Cython is and how it works. It covers setting up your environment and running the "Hello World" application.
Chapter 2, Understanding Cython, will start to get serious with Cython and will discuss how to describe C declarations with respect to Cython along with calling conventions and type conversion.
Chapter 3, Extending Applications, will walk you through comparing the execution of pure Python code with the Cython version of the same code. We also look at extending Tmux, a pure C project, with Cython.
Chapter 4, Debugging Cython, will cover how to use GDB to debug your code and the relative GDB commands. There is also an extensive section on caveats and things to be aware of as well as conventions.
Chapter 5, Advanced Cython, will cover the usage of C++ with Cython, which is just as easy as using C with Cython. We will also work through all the syntax necessary to wrap C++. We will then look into the caveats and more on optimizations, comparing a Python XML parser with a Cython XML parser on large XML files.
Chapter 6, Further Reading, wraps up the book with a final look at some caveats and conventions. Then, we compare Cython against other similar tools like Numba and SWIG, and we will discuss how its used in NumPy and how we can use PyPy and Python 3
What you need for this book
For this book, I used my MacBook and an Ubuntu virtual machine (GDB is too old on Mac OS X for debugging). You will require the following on Mac OS X:
• Xcode
• Cython
• GCC/Clang
• Make
• Python
• Python config
• Python distutils
On Ubuntu, you can install most components via the following:
$ sudo apt-get install build-essential gdb cython
Of course, I will go over this in the introduction, but as long as you have a C compiler and Python and have Python headers installed, you will have everything you need for Cython.
Who this book is for
This book is intended for C developers who like using Python and Python users wanting to implement native C/C++ extensions to Python. As a reader, you can expect to be shown how you can develop applications with Cython, with an emphasis on extending existing systems with help on how you can approach it.
Extending legacy systems can be difficult, but the rewards can be great. Consider very low-level system daemons that we could abstract and extend them and interact with the data from Python in a nice high-level way while leaving all performance-sensitive code alone! This model of development can prove to be efficient and of great return to development time; this can be particularly expensive when it comes to C applications.
It also allows for much more rapid development of the state or logic in a system. There is no need to worry about long data conversion algorithms in C for doing small things and then needing to change it all again
