ImportError from Cython when linking an external C library

Question

I am trying to reproduce the work of Flavian Coelho related here . He used Cython and the Gnu Science Library (GSL) to get tremendous speedup over Python in random number generation. When I import compiled Cython code into Python (with the import cgibbs ), I get the following error:

  ImportError: dlopen(./cgibbs.so, 2): Symbol not found: _gsl_rng_mt19937 Referenced from: /Users/wesley/scratch/cython/cgibbs.so Expected in: dynamic lookup 

You will notice that the complaint is that the _gsl_rng_mt19937 symbol cannot be found. The function I'm trying to reference is called gsl_rng_mt19937 (there is no leading underscore), and that is how it appears in my .pyx file. I think Cython is somehow causing the problem by adding this underline.

To facilitate troubleshooting, I removed the code and posted it below. My system: Mac OSX 10.7 (Lion), which runs Python 2.7.2 (32-bit), gcc-4.0 (which I used to compile GSL libraries in 32-bit form), GSL 1.15, and Cython v0.15.1.

Here is the contents of cgibbs.pyx:

 #declaring external GSL functions to be used cdef extern from "math.h": double sqrt(double) cdef double Sqrt(double n): return sqrt(n) cdef extern from "gsl/gsl_rng.h": ctypedef struct gsl_rng_type: pass ctypedef struct gsl_rng: pass gsl_rng_type *gsl_rng_mt19937 gsl_rng *gsl_rng_alloc(gsl_rng_type * T) cdef extern from "gsl/gsl_randist.h": double gamma "gsl_ran_gamma"(gsl_rng * r,double,double) double gaussian "gsl_ran_gaussian"(gsl_rng * r,double) cdef gsl_rng *r = gsl_rng_alloc(gsl_rng_mt19937) 

The error will disappear if I comment on the last line of my cgibbs.pyx , but then I cannot use an external library ... Any insights you can offer are appreciated. Thanks!

Answer 1

Thanks to @ChuiTey, I discovered that otool and nm are tools on the Mac that can do what objdump does on Linux. With nm I found that leading underscores are part of the symbol name in the libgsl.a library.

As soon as I realized that the linker (in my case, ld ) was looking for the correct name, it was clear that he simply was not looking for the right place. And as I found out that the header files do not know the location of the libraries to which they link (Duh!), I just need to add the -lgsl parameter to the command that launches the linker (this requires libgsl.a be in the directory where your the linker is looking for libraries - on my machine, /usr/local/lib .)

I also had to move libgsl.dylib from /usr/local/lib because it was compiled for a 64-bit platform and I use 32-bit Python.

There should be an easy way to specify the -lgsl linker option when compiling Cython code with distutils or makefile; for now, I just run gcc-4.0 twice to compile and then link the module. My workflow for compiling a Cython module from the bash command line:

 cython cgibbs.pyx gcc-4.0 -m32 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cgibbs.c -o cgibbs.o gcc-4.0 -bundle -undefined dynamic_lookup -lgsl -arch i386 -g cgibbs.o -o cgibbs.so 

Creates cgibbs.so , a Cython module that can be imported into Python 2.7.

Answer 2

Underlining exported functions is called name mangling . On Windows, the cdecl convention adds one underscore (other conventions have more complex schemes).

You may have incorrectly specified the correct call when creating your GSL library. You should be able to see which names are exported using objdump .

Doesn't GSL already provide SWIG wrappers?

Answer 3

Please take a look at CythonGSL ( https://github.com/twiecki/CythonGSL ) when you want to use the GSL features in Cython.