How to read numpy 2d array from a string?

Question

How can I read a numpy array from a string? take the line as:

[[ 0.5544 0.4456], [ 0.8811 0.1189]] 

and convert it to an array:

 a = from_string("[[ 0.5544 0.4456], [ 0.8811 0.1189]]") 

where a becomes the object: np.array([[0.5544, 0.4456], [0.8811, 0.1189]])

Update

I am looking for a very simple interface. a method for converting 2D arrays (floats) into a string, and then a method for reading them to restore the array:

arr_to_string(array([[0.5544, 0.4456], [0.8811, 0.1189]])) should return "[[ 0.5544 0.4456], [ 0.8811 0.1189]]"

string_to_arr("[[ 0.5544 0.4456], [ 0.8811 0.1189]]") should return an array([[0.5544, 0.4456], [0.8811, 0.1189]]) object array([[0.5544, 0.4456], [0.8811, 0.1189]])

Ideally, it would be great if arr_to_string had an accuracy parameter that controlled the precision of floating points converted to strings, so that you would not get entries like 0.4444444999999999999999999 .

I can't find anything in numpy docs that does this in both directions. np.save allows you to create a line, but then there is no way to load it back ( np.load only works for files.)

Answer 1

The task is to save not only the data buffer, but also the form and type. np.fromstring reads the data buffer, but as a 1d array; you should get dtype and shape from another place.

 In [184]: a=np.arange(12).reshape(3,4) In [185]: np.fromstring(a.tostring(),int) Out[185]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) In [186]: np.fromstring(a.tostring(),a.dtype).reshape(a.shape) Out[186]: array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) 

The Python object persistence mechanism, executed in time, is pickle , and numpy conforms to socket requirements:

 In [169]: import pickle In [170]: a=np.arange(12).reshape(3,4) In [171]: s=pickle.dumps(a*2) In [172]: s Out[172]: "cnumpy.core.multiarray\n_reconstruct\np0\n(cnumpy\nndarray\np1\n(I0\ntp2\nS'b'\np3\ntp4\nRp5\n(I1\n(I3\nI4\ntp6\ncnumpy\ndtype\np7\n(S'i4'\np8\nI0\nI1\ntp9\nRp10\n(I3\nS'<'\np11\nNNNI-1\nI-1\nI0\ntp12\nbI00\nS'\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\x08\\x00\\x00\\x00\\n\\x00\\x00\\x00\\x0c\\x00\\x00\\x00\\x0e\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x12\\x00\\x00\\x00\\x14\\x00\\x00\\x00\\x16\\x00\\x00\\x00'\np13\ntp14\nb." In [173]: pickle.loads(s) Out[173]: array([[ 0, 2, 4, 6], [ 8, 10, 12, 14], [16, 18, 20, 22]]) 

There is a numpy function here that can read the brine line:

 In [181]: np.loads(s) Out[181]: array([[ 0, 2, 4, 6], [ 8, 10, 12, 14], [16, 18, 20, 22]]) 

You pointed np.save to the string, but you cannot use np.load . The way around this is a step further into the code and use np.lib.npyio.format .

 In [174]: import StringIO In [175]: S=StringIO.StringIO() # a file like string buffer In [176]: np.lib.npyio.format.write_array(S,a*3.3) In [177]: S.seek(0) # rewind the string In [178]: np.lib.npyio.format.read_array(S) Out[178]: array([[ 0. , 3.3, 6.6, 9.9], [ 13.2, 16.5, 19.8, 23.1], [ 26.4, 29.7, 33. , 36.3]]) 

The save line has a header with dtype and shape info:

 In [179]: S.seek(0) In [180]: S.readlines() Out[180]: ["\x93NUMPY\x01\x00F\x00{'descr': '<f8', 'fortran_order': False, 'shape': (3, 4), } \n", '\x00\x00\x00\x00\x00\x00\x00\x00ffffff\n', '@ffffff\ x1a@ \xcc\xcc\xcc\xcc\xcc\xcc#@ffffff*@\x00\x00\x00\x00\x00\ x800@ \xcc\xcc\xcc\xcc\xcc\ xcc3@ \x99\x99\x99\x99\x99\ x197@ffffff :@33333\ xb3=@ \x00\x00\x00\x00\x00\ x80@ @fffff& B@ '] 

If you need a user readable string, you can try json .

 In [196]: import json In [197]: js=json.dumps(a.tolist()) In [198]: js Out[198]: '[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]]' In [199]: np.array(json.loads(js)) Out[199]: array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) 

Going to / from an array list view is the most obvious use of json . Maybe someone wrote a more complex representation of json arrays.

You can also switch to the csv format - there were a lot of questions about reading / writing csv arrays.

---

 '[[ 0.5544 0.4456], [ 0.8811 0.1189]]' 

is a bad string representation for this purpose. It is very similar to str() array, but with , instead of \n . But there is no clean way to parse nested [] , and the missing delimiter is a pain. If it consistently uses , then json can convert it to a list.

np.matrix takes a string of type MATLAB:

 In [207]: np.matrix(' 0.5544, 0.4456;0.8811, 0.1189') Out[207]: matrix([[ 0.5544, 0.4456], [ 0.8811, 0.1189]]) In [208]: str(np.matrix(' 0.5544, 0.4456;0.8811, 0.1189')) Out[208]: '[[ 0.5544 0.4456]\n [ 0.8811 0.1189]]' 

Answer 2

I'm not sure there is an easy way to do this if you don't have commas between the numbers in the internal lists, but if you do, you can use ast.literal_eval :

 import ast import numpy as np s = '[[ 0.5544, 0.4456], [ 0.8811, 0.1189]]' np.array(ast.literal_eval(s)) array([[ 0.5544, 0.4456], [ 0.8811, 0.1189]]) 

EDIT: I haven't tested it very hard, but you can use re to insert commas where you need them:

 import re s1 = '[[ 0.5544 0.4456], [ 0.8811 -0.1189]]' # Replace spaces between numbers with commas: s2 = re.sub('(\d) +(-|\d)', r'\1,\2', s1) s2 '[[ 0.5544,0.4456], [ 0.8811,-0.1189]]' 

and then pass ast.literal_eval :

 np.array(ast.literal_eval(s2)) array([[ 0.5544, 0.4456], [ 0.8811, -0.1189]]) 

(you need to be careful to combine spaces between numbers and spaces between numbers and minus).

Answer 3

For this purpose, Numpy provides a function fromstring . Works with binary or ascii.

http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.fromstring.html

Answer 4

Forward line:

 import numpy as np def array2str(arr, precision=None): s=np.array_str(arr, precision=precision) return s.replace('\n', ',') 

Return to array:

 import re import ast import numpy as np def str2array(s): # Remove space after [ s=re.sub('\[ +', '[', s.strip()) # Replace commas and spaces s=re.sub('[,\s]+', ', ', s) return np.array(ast.literal_eval(s)) 

If you use repr() to convert an array to a string, the conversion will be trivial.