How to import submodules from pandas without matplotlib import?

Question

I am using pandas 0.14.1 on a web server to process reports from an SQL database.

I don't need charting objects, but matplotlib is always imported.

How to import only those modules that I need to do the following?

df = pd.io.sql.frame_query(query, con=conn) df['colname'].apply(somefunc) df.set_index('colname') print df.to_html() 

I constantly have to add the following hack to all report generation scripts:

 import os os.environ['MPLCONFIGDIR'] = '/tmp/' 

Before importing pandas. What can I do to avoid this?

Here is my web server error log when I omit this hack:

 File "/var/www/scripts/myscript.py", line 46, in index\n from pandas.io import sql File "/usr/lib/python2.7/dist-packages/pandas/__init__.py", line 41, in <module>\n from pandas.core.api import * File "/usr/lib/python2.7/dist-packages/pandas/core/api.py", line 9, in <module>\n from pandas.core.groupby import Grouper File "/usr/lib/python2.7/dist-packages/pandas/core/groupby.py", line 15, in <module>\n from pandas.core.frame import DataFrame File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 38, in <module>\n from pandas.core.series import Series File "/usr/lib/python2.7/dist-packages/pandas/core/series.py", line 2524, in <module>\n import pandas.tools.plotting as _gfx File "/usr/lib/python2.7/dist-packages/pandas/tools/plotting.py", line 26, in <module>\n import pandas.tseries.converter as conv File "/usr/lib/python2.7/dist-packages/pandas/tseries/converter.py", line 7, in <module>\n import matplotlib.units as units File "/usr/lib/pymodules/python2.7/matplotlib/__init__.py", line 774, in <module>\n rcParams = rc_params() File "/usr/lib/pymodules/python2.7/matplotlib/__init__.py", line 692, in rc_params\n fname = matplotlib_fname() File "/usr/lib/pymodules/python2.7/matplotlib/__init__.py", line 604, in matplotlib_fname\n fname = os.path.join(get_configdir(), 'matplotlibrc') File "/usr/lib/pymodules/python2.7/matplotlib/__init__.py", line 253, in wrapper\n ret = func(*args, **kwargs) File "/usr/lib/pymodules/python2.7/matplotlib/__init__.py", line 478, in _get_configdir\n raise RuntimeError("Failed to create %s/.matplotlib; consider setting MPLCONFIGDIR to a writable directory for matplotlib configuration data"%h) RuntimeError: Failed to create /var/www/.matplotlib; consider setting MPLCONFIGDIR to a writable directory for matplotlib configuration data 

Additional information: Ubuntu 12.04LTS platform, which has a rather old version of matplotlib. Recent versions fix this error by creating a temporary file . However, it still sucks that matplotlib runs on my web server when I don't need it.

Answer 1

Unfortunately, the answer is to upgrade matplotlib to a version that creates a loadable configuration directory at startup if default locations are not available. It is a pain if you use your Linux distributions (matplotlib v1.1.1). Versions after 1.3.1 should be in order.

Both suggestions in the comments do not fix the problem.

Changing the mpl configuration to use another display driver such as Agg does not stop using matplotlib when trying to create a configuration directory.

Adding an empty matplotlib.py file breaks pandas, because the matplotlib.units module is required to convert the data type.

Therefore, to avoid this, before updating matplotlib, os.environ['MPLCONFIGDIR'] = '/tmp/' hack works fine, but we must remember to put it in every file that pandas uses on our web server. (or create your own custom module that hides all this)