Send multidimensional numpy array over socket

Question

Good afternoon,

I searched for this but did not come up with any answers. I want to send a multidimensional numpy array through a socket. Therefore, I decided to convert it to a string:

However, it destroys the representation of the array:

>>> import numpy as np >>> x = np.array([[0, 1], [2, 3]]) >>> xstring = x.tostring() >>> print xstring >>> print x [[0 1] [2 3]] >>> print xstring >>> nparr = np.fromstring(xstring, dtype=np.uint8) >>> print nparr [0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] 

Anyway, can I somehow convert the string to a string, save its size?

Answer 1

Try this example: -

 import socket import numpy as np from cStringIO import StringIO class numpysocket(): def __init__(self): pass @staticmethod def startServer(): port=7555 server_socket=socket.socket() server_socket.bind(('',port)) server_socket.listen(1) print 'waiting for a connection...' client_connection,client_address=server_socket.accept() print 'connected to ',client_address[0] ultimate_buffer='' while True: receiving_buffer = client_connection.recv(1024) if not receiving_buffer: break ultimate_buffer+= receiving_buffer print '-', final_image=np.load(StringIO(ultimate_buffer))['frame'] client_connection.close() server_socket.close() print '\nframe received' return final_image @staticmethod def startClient(server_address,image): if not isinstance(image,np.ndarray): print 'not a valid numpy image' return client_socket=socket.socket() port=7555 try: client_socket.connect((server_address, port)) print 'Connected to %s on port %s' % (server_address, port) except socket.error,e: print 'Connection to %s on port %s failed: %s' % (server_address, port, e) return f = StringIO() np.savez_compressed(f,frame=image) f.seek(0) out = f.read() client_socket.sendall(out) client_socket.shutdown(1) client_socket.close() print 'image sent' pass 

In this model, the client sends a multidimensional ndarray to the server. There are two functions startServer () and startClient (). startServer takes no arguments, but startClient requires the server address as well as ndarray as arguments. Start the server first, and then start the client. The server starts reading from the buffer only after receiving a message about disconnecting from the client.

Answer 2

Indeed, .tostring only returns the original data. This means that you also need to submit the form and type of the array if they are not known on the other hand.

It may be easier to serialize the array using Pickle:

 import numpy as np from cPickle import dumps, loads x = np.array([[1, 2],[3, 4]], np.uint8) print loads(dumps(x)) # [[1 2] # [3 4]] 

Although for very small arrays, the overhead size can be significant:

 print len(x.tostring()), len(dumps(x)) # 4 171 

For more information on using Pickle, see here.

Answer 3

This is a slightly improvised response for an ajsp response using XML-RPC.

On the server side, when you convert the data, convert the empty data to a string using the .tostring () method . This encodes numy ndarray as a string of bytes. On the client side, when you receive the data, decode it using the '.fromstring ()' method. I wrote two simple functions for this. Hope this is helpful.

1. ndarray2str - Converts numy ndarray to a string of bytes.
2. str2ndarray - Convert binary str back to numy ndarray.

  def ndarray2str(a): # Convert the numpy array to string a = a.tostring() return a 

On the receiver side, data is received as an object 'xmlrpc.client.Binary' . You need to access the data using .data .

  def str2ndarray(a): # Specify your data type, mine is numpy float64 type, so I am specifying it as np.float64 a = np.fromstring(a.data, dtype=np.float64) a = np.reshape(a, new_shape) return a 

Note: The only problem with this approach is that XML-RPC is very slow when sending large arrays. It took me about 4 seconds to send and receive an array of data of size (10, 500, 500, 3) for me.

I am using Python 3.7.4.