ZeroMQ: REQ / REP with big messages

I am trying to send large messages ( 300 MB) on top of -socket. There are several clients, each of which connects to the server via -socket. The server creates a dedicated socket for each client, waits for the request (containing the identifier), and sends the message in two parts using : REP
REQ ZMQ_SENDMORE

  • metadata (~ 1 KB)
  • data (~ 300 MB)

After that, REP-socket closes immediately. Then Context()-instance closes in the parent thread. socket time remains by default (infinitely). Sometimes metadata is sent, but not image data. I tracked calls in ZeroMQ and found that the internal (window) socket was closed before these messages were sent to the network. I thought it would block until there were no messages in the queue. As a workaround, I changed clients to send another request message after receiving the data as confirmation. This works pretty well, but I'm not quite sure if I addressed the main issue. Version ZeroMQ 4.0.4. I use C ++ bindings. Server and clients work in Windows (7 and 10). ZMQ_LINGER
zmq_term()

+4
source share
1 answer

I know this is NOT the answer you want, but I experienced something similar with ZeroMQ on Linux. The main Linux connector is closed and some messages have not passed it.

In my use case, I keep the server open until the user gives me any keyboard input. This is a hack, but for my use case it gives me what I want.

You can also sleep for a second or two before closing the socket. Once again, not very, but it can be a quick fix until it is fixed.

Edited to add: I also send messages with several parts, and one of my message parts is quite large (~ 200 MB).

+1
source

Source: https://habr.com/ru/post/1619966/


All Articles