Should import order be specified when importing .so?

Question

I get the following import error when trying to load a python module compiled with boost python.

ImportError: /path/to/library/libxml2.so.2: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference 

Strange, I do not see this error if it is a non-standard module for import. ie If I import another module and then this module, it is not possible with an import error. Not sure what is going wrong or how to debug.

Edit: To accurately show the problem:

 $ python -c 'import json, libMYBOOST_PY_LIB' # DOES NOT WORK!!! Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: path/to/xml_library/libxml2.so: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference $ python -c 'import libMYBOOST_PY_LIB, json' # WORKS NOW!!! $ 

Its not just json, several other modules also cause the same problem when importing in front of my module. eg. urllib2

Answer 1

Statement of order import .

As stated in the python link :

As soon as the name of the module is known (unless otherwise indicated, the term “module” refers to both packages and modules), the search for a module or package can begin. The first place is marked by sys.modules , the cache of all modules that were imported earlier. If the module is found there, it is used in step (2) of the import.

Any module can change:

• sys.modules - cache of all previously imported modules
• sys.path - search path for modules

And they can also change import capture:

• sys.meta_path
• sys.path_hooks
• sys.path_importer_cache

Import hooks can provide you with the ability to download modules from zip files, any archive files from the network, etc.

---

 import libMYBOOST_PY_LIB 

This statement intends to modify sys.modules exactly by loading its dependencies into the module cache. It can also modify sys.path . In fact, this is very common for frameworks (e.g. boost , zope , django , requests ...) to send with the batteries on / with a copy of the modules on which they depend.

• django comes with json
• requests comes with urllib3

To know exactly what the library will load, you can use:

 python -v -c 'import libMYBOOST_PY_LIB' 

Answer 2

Problem

The problem with the operating system. The Linux library (a dynamically linked library of shared objects) may depend on other libraries (which may again depend on other libraries, etc.). If these dependent libraries were not resolved correctly, you will get the error you described.

What are shared libraries (sic)

You can create a shared library by taking several object files and linking them together. The compiler saves a lot of metadata when creating a shared library:

• Movement table
• List of exported characters (functions and variables that others can access)
• List of imported characters (functions and variables used by this library from other libraries)
• List of file names of other libraries that can be used to satisfy imported characters

When the library is used, the system loads the library, changes the addresses referenced by the movement table, and then tries to find the imported characters. For them, the system first checks the already loaded libraries. If this does not satisfy all the characters, it tries to find the file names listed in the library and check if a file with that name exists, if it is a valid library and if it exports this character.

Usually the “system” here is a dynamic loader that runs in user space, not kernel space.

How can you check which libraries are used by the program

You can check the contents of the library with commend ldd .

If you want to check the executable executable, try lsof and filter for *.so and also check /proc/[pid]/maps .

How to debug your problem

In your case, hold the program immediately before loading the library (for example, insert a reading from the console or a sleep command). Then check the currently loaded library. You will find that in a good case, a library is already loaded that exports the corresponding symbol. In the event of an error, this library will not load, and the system will try to load the independent library in the next step (for example, another version of the library where the desired character is missing).

Is library order important

Usually not, but it depends on the details. If the same library is needed in a different version, or when the system cannot resolve the shared library in all cases, this can become important. Unfortunately, these problems are quite difficult to debug. On Windows, you have a DLL addon; on Linux, it looks like shared objects. Good luck debugging your problem!