Numpy savez interprets my keys as file names & # 8594; IOError

Question

I use numpy savez as recommended to save numpy arrays. As keys, I use the names of the files from which I downloaded the data. But it looks like savez trying to use file names in some way. What should I do? I would like to avoid deleting files their path names and endings.

 >>> import numpy >>> arrs = {'data/a.text': numpy.array([1,2]), 'data/b.text': numpy.array([3,4]), 'data/c.text': numpy.array([5,6])} >>> numpy.savez('file.npz', **arrs) Traceback (most recent call last): File "<input>", line 1, in <module> File "/usr/lib/python2.6/dist-packages/numpy/lib/io.py", line 305, in savez fid = open(filename,'wb') IOError: [Errno 2] No such file or directory: '/tmp/data/c.text.npy' 

Answer 1

You can encode and decode keys before passing them to savez .

 >>> import numpy >>> import base64 >>> arrs = {'data/a.text': numpy.array([1,2]), 'data/b.text': numpy.array([3,4]), 'data/c.text': numpy.array([5,6])} >>> numpy.savez('file.npz', **dict((base64.urlsafe_b64encode(k), v) for k,v in arrs.iteritems())) >>> npzfile = numpy.load('file.npz') >>> decoded = dict((base64.urlsafe_b64decode(k), v) for k,v in npzfile.iteritems()) >>> decoded {'data/c.text': array([5, 6]), 'data/a.text': array([1, 2]), 'data/b.text': array([3, 4])} 

Answer 2

Savez probably builds temporary files with the names given in the dict. The file name has / . When savez creates the file, it tries to use the given name and extension .npy (ie data/c.txt.py ) as the name of the file in the temp directory. However, the new path leads to a temp subdirectory that temp not exist, which leads to an error.

The solution would be: either replace the slash with something else, or perhaps avoid the file name.

(My previous answer was too complicated and probably wrong.)