All geek questions in one place

How to guarantee reliable JMS delivery

I think that a large number (Spring in my case) of applications using JMS can follow this workflow:

Database A ===> Producer ===> JMS Queue ===> Consumer ===> Database B

then reliability is a problem. Say, if when a data record in Database A should always be marked as delivered , when the message contains a data record, it really is consumed and stores the data in Database B Then there are questions:

  • As far as I know, currently the JMS protocol does not define any functions for sending confirmation from consumer to producer , but only to MOM , so the actual consumer-to-producer confirmation methods depend on the JMS provider. Does this mean that there is no way to develop a mechanism for such confirmation, which can work, as a rule, for all JMS products (ActiveMQ, WebSphere MQ and Jboss MQ)?

  • Consider a dimming scenario, then it makes the messages in the queue just evaporate, so do you need to resend them? or different JMS products can get what is left because the messages are serialized so that the missing message can only be caused by transaction management or async / sync configuration, but not because the application server is down?

+5
source share
3 answers

JMS guarantees the delivery of a message by nature, if the message is sent, then it will be delivered to the consumer, if there is, no matter what happens, MOM is designed to ensure this fact. In any case, delivery is not required, which means it is being processed.

Reliability is provided through various mechanisms:

  • the first is to store the message in a queue (queue AND the message should be marked as constant, which is the default value), which ensures that the message will not be lost in case of a system interruption.
  • then you have a confirmation and retry policy, the message will be stored in the queue until the consumer confirms this and in the case of a transaction session is not delivered until the consumer processes the message or the maximum retry is reached. After that, the error message can be redirected to the dead letter queue for analysis.

To ensure consistency between the two data sources, you must use transaction XA, at least on the producer side (you have at least 2 resources implied in the transaction database A and JMS queue) to ensure that the message will not be sent in the queue, if the commit in database A fails or the database will not be updated if the message is not queued. Message consumption must also be met to ensure re-delivery in the event of a rollback.

Within the transaction boundaries, both the consumer and the producer will never be included, since this contradicts the asynchronous nature of the messaging system, you cannot block resources on the producer's side until the consumer processes the message, because you have no guarantee on when this will happen .

NB: if your database does not support XA (or to improve performance), and if you have only 2 resources that are implied in the transaction (in the database and in the JMS queue), you can take a look at logging the last transaction of resources

+3
source

1) From my experience with queue managers (MQ Series, ActiveMQ, and HornetQ), I have never needed such recognition between producer / consumer. Also, the environment I was dealing with, traffic was about 50/60 million per day of objects in several queues. And all the queues are saved.

2) In my case, using the save mechanism in the queue manager was quite sufficient to handle the dimming script. I used drive persistence on the MQ Series and HornetQ.

However, sometimes, in order to count the number of messages, we developed some mechanisms for comparing database A with base BB to ensure that messages were also used. I do not know if the JMS architecture should provide such a mechanism, because such a task can reduce performance.

This is something - from my point of view - that you should measure in your system architecture how important it is to match this information, because it is not so easy to save.

Sincerely.

+2
source

If I understand your question, this is similar to the case for JTA / XA transactions (as long as your DB / JMS providers support them). Spring TX managers can help supplier agnostics (more) suppliers.

FYI, I use Apache Camel for this type of thread that has pretty good error handling between producers / consumers.

+1
source

Source: https://habr.com/ru/post/1205812/

More articles:


All Articles