Ruby AMQP persistent message is deleted after restarting RabbitMQ

Question

I have a ruby ​​script that creates a message using AMQP in RabbitMQ.

# above code sets up config for connecting to RabbitMQ via APMQ AMQP.start(:host => 'localhost') do amq = MQ.new amq.queue('initiate', :durable => true).publish(message_id, :persistent => true) AMQP.stop{ EM.stop } end 

If the RabbitMQ server is restarted, the message is no longer in the initialization queue (or any queue, for that matter). What am I doing wrong that the message is not permanent? I also tried to explicitly create a long-term exchange and bind the queue to this exchange, but the message is still deleted after restarting RabbitMQ.

Answer 1

It's funny that I was just googling for the same problem. RabbitMQ 2.2.0, the default settings. In my case, Ruby clients use rubygem-amqp-0.6.7-3.el5 from EPEL. Long queues associated with the exchange of Durable fanout, posting messages with: persistent => true. Messages lost during server reboot. -Alan

Answer 2

As already mentioned, if you simply mark messages as permanent, they will not necessarily be saved immediately, therefore, if the server shuts down unexpectedly, they may never end up on disk.

So what do you do if you really need the message to be on disk, even if the server crashes?

There are two things you can do. One of them is to wrap the publication in a transaction. When you have completed the transaction, the message will be on disk (if it has not already been delivered to the end user). However, this adds a synchronous call to the server, so it can slow down. If you know that you are going to publish many messages, you can wrap a bunch of publications in a transaction, and then when you commit that you know that they are all on disk.

Another (better performance) alternative is to use publication confirmations. But they are new on server 2.3.1, and I do not think that all Ruby clients support them.

Finally, RabbitMQ will in any case periodically erase persistent messages to disk even in the absence of confirmations, transactions and controlled stops. However, there is an error in 2.2.0, which means that this sometimes does not happen for a long time, so updating to 2.3.1 may be useful.

Answer 3

Yes, Simon is right. About the publisher confirms (described at http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms ), I plan to support them in AMQP 0.8, which will be released soon.

BTW, in the original example, the first argument for publication is the actual data, everything else is specified through options, so publish (message, options), and not publish (message_id, opts).