Scipy error: numpy.dtype resized, may indicate binary incompatibility (and the strange behavior associated with it)

Question

I install numpy / scipy / scikit-learn on OS X 10.9.4, and get errors about "resizing numpy.dtype, may indicate binary incompatibility".

Here is what I did to build the repo:

mkvirtualenv thm workon thm pip install numpy scipy pandas ipython # and some other stuff cd /path/to/our/repo # run tests 

Here you will find an excerpt from the corresponding warning (turned into an error, because we use warnings.simplefilter('error') at the beginning of our tests):

 ====================================================================== ERROR: Failure: RuntimeWarning (numpy.dtype size changed, may indicate binary in compatibility) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/loader.py", line 414, in loadTestsFromName addr.filename, addr.module) File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py ", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py ", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/Users/ben/code/thm/alpha/prosper/base/stats/test_auc.py", line 3, in <m odule> import sklearn.metrics File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/ __init__.py", line 6, in <module> from .metrics import (accuracy_score, File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/metrics.py", line 27, in <module> from scipy.spatial.distance import hamming as sp_hamming File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/scipy/spatial/__init__.py", line 90, in <module> from .ckdtree import * File "__init__.pxd", line 155, in init scipy.spatial.ckdtree (scipy/spatial/ckdtree.c:20570) RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility 

I was told that this warning was caused by the fact that scipy was compiled against a different version of numpy than the installed one. However, I installed them all with pips in what I thought was pretty standard, so this should not be a problem, I would have thought.

It seems that although our entire test suite as a whole (via python -m unittest discover ) works with these errors, running separate tests (via python -m unittest <module> ) works fine.

According to the tests, the corresponding version information is given here:

 numpy version 1.9.0 (rev 07601a64cdfeb1c0247bde1294ad6380413cab66) scipy version 0.14.0 (built against numpy 1.9.0) sklearn version 0.15.2 pandas version 0.14.1 

We are pleased to provide additional information upon request!

Answer 1

How did you build sklearn 0.14.1? Did you build it against the same numpy version as for scipy?

The latest versions of scikit-learn, scipy, and numpy have ready-made packages. In particular, scikit-learn 0.15.2 should be binary compatible with numpy 1.7+. I think the same is true for scipy 0.14.0, but you said you created it yourself from the source, which is not what the pip should do by default (it should just install the finished whl package).

Edit: you tried to do:

 pip install -U scipy scikit-learn pandas 

to make sure you use the latest stable whl versions for these packages?

Edit: in the following comment there is an actual answer that works, and probably why this answer was accepted. Namely:

 pip uninstall -y scipy scikit-learn pip install --no-binary scipy scikit-learn