What is the meaning of this ImportError when importing a .so generated Cython file?

Question

I go through the Cython documentation and build each sample application. I am a little fixated on using the C libraries. After successfully creating the .so file and trying to import it into a python file called test.py, the following error is thrown.

$ python3.2 test.py Traceback (most recent call last): File "test.py", line 12, in <module> from queue import Queue ImportError: dlopen(/Users/jeremy/Development/labs/python/cython_lib_wrapper/queue.so, 2): Symbol not found: _queue_free Referenced from: /Users/jeremy/Development/labs/python/cython_lib_wrapper/queue.so Expected in: flat namespace in /Users/jeremy/Development/labs/python/cython_lib_wrapper/queue.so 

The .so file is next to the test.py file. Thus, it seems that it needs to be found. It runs the latest version of Cython with Python 3.2 on OSX 10.6.

Any ideas?

Edit - Add Build and Output Command

 $ python3.2 setup.py build_ext --inplace running build_ext cythoning queue.pyx to queue.c building 'queue' extension gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -I/Library/Frameworks/Python.framework/Versions/3.2/include/python3.2m -c queue.c -o build/temp.macosx-10.6-intel-3.2/queue.o queue.c: In function '__pyx_f_5queue_5Queue_append': queue.c:627: warning: cast to pointer from integer of different size queue.c: In function '__pyx_f_5queue_5Queue_extend': queue.c:740: warning: cast to pointer from integer of different size queue.c: In function '__pyx_f_5queue_5Queue_peek': queue.c:813: warning: cast from pointer to integer of different size queue.c: In function '__pyx_f_5queue_5Queue_pop': queue.c:965: warning: cast from pointer to integer of different size gcc-4.2 -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -isysroot /Developer/SDKs/MacOSX10.6.sdk -g build/temp.macosx-10.6-intel-3.2/queue.o -o 

Edit 2 - add the comment "otool" in the comments

 queue.so: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0) 

Edit 3 - add output "nm"

 U ___stack_chk_fail U ___stack_chk_guard U _queue_free U _queue_is_empty U _queue_new U _queue_peek_head U _queue_pop_head U _queue_push_tail U dyld_stub_binder 

grep cmd prints this:

 (undefined) external _queue_free (dynamically looked up) 

Answer 1

EDIT:

Oh, you didn’t say that you have code dependency in libcalg. This material needs to be compiled and included when you create a cextension.

Just modify setup.py:

 # setup.py # ... ext_modules = [Extension("queue", ["queue.pyx", "libcalg/queue.c"])] # ... 

---

We can step back and see if we can build a simple example:

I tried the following (3 files, myext.pyx, test.py, setup.py) and it seems to work fine. Of course, I'm on OS X 10.7, so it's not exactly the same as your environment. To eliminate differences, perhaps you can copy them and create them as a health check.

The contents of myext.pyx:

 # myext.pyx def square(x): return x * x 

Content test.py

 # test.py from myext import square print "%d squared is %d"%(4, square(4)) 

Contents setup.py:

 # setup.py from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules = [Extension("myext", ["myext.pyx"])] setup( name = 'Hello world app', cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules ) 

I create a directory containing these 3 files:

 cython_test$ /usr/bin/python setup.py build_ext --inplace running build_ext cythoning myext.pyx to myext.c building 'myext' extension creating build creating build/temp.macosx-10.7-intel-2.7 llvm-gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c myext.c -o build/temp.macosx-10.7-intel-2.7/myext.o llvm-gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -Wl,-F. -arch i386 -arch x86_64 build/temp.macosx-10.7-intel-2.7/myext.o -o /Users/steder/SO/cython_test/myext.so cython_test$ python test.py 4 squared is 16: 

There is a similar otool output in my environment, and DYLD_LIBRARY_PATH is also not set, but nm -m shows the square as defined.

In particular:

 00000000000011d0 (__DATA,__data) non-external ___pyx_k__square 00000000000011e0 (__DATA,__data) non-external ___pyx_mdef_5myext_square 0000000000001218 (__DATA,__bss) non-external ___pyx_n_s__square 0000000000000c80 (__TEXT,__text) non-external ___pyx_pf_5myext_square 

Please give this snapshot and see what nm -m shows in your environment.