MQ queue with multiple consumers, but only one active

Question

We have one MQ queue that receives messages from an external system outside our control. Our incoming message processing system is critical and needs to be up and running 27x7 no matter what.

The procedure for processing incoming messages is also not subject to discussion, which means that we must process them in the exact order in which they were received.

To make sure our system is 100% accessible, we deployed our system to a bunch of physical machines capable of handling these messages.

As soon as the messages reached our system, we created a mechanism to make sure that message processing does not fail, and also receives some performance enhancement as a result of parallel processing. For us, the advantage in performance is good, but it is rather a side effect, since our main goal is high availability while ensuring the correct processing order.

My thoughts were that on each machine, the MDB could process incoming messages, but only one active consumer at a time.

We use Webshere MQ as a JMS provider and WebShere Application Server 8.5 to deploy our application.

The problem with several consumers who listen to the same queue does not seem to be a workable solution, because when messages arrive in bulk, they will be cyclically transmitted to all consumers, and there is no way to control how this happens and messages easily exit the sequence.

When I manually stopped all the listeners, but obviously the messages were processed in order. But manually turning off and starting such listeners is certainly not a HA solution.

We could implement monitoring processes to test the system and shut them down or run them as needed, but it still looks too weak for me. In fact, we want all listeners to work, but only one will receive messages. If this happens for some reason, then the other one sitting there will become active and begin to process messages.

We initially looked at using a topic rather than a queue, but this is due to other issues, as shown below:

• we cannot control the source of our messages
• the large volume of messages that we have will put us in trouble with our subscribers, who must be durable, and when you return back, you will have to deal with many waiting messages.
• input queues are already part of the cluster, and a lot of work will be required to change the entire infrastructure.

In any case, in my opinion, this should be an existing template for situations like this. Any help, suggestion would be very helpful.

The solution does not have to be a specific MQ, any idea is welcome.

Thanks in advance

Answer 1

Create the second queue, we will call it the "managing queue". In this queue, put one message, we will call it a token. Change the application processing as follows:

• Listen to the management queue for the message.
• Get a token from the management queue under synchronization.
• Put the same token message back into the management queue, also under synchronization.
• Process a transaction from a regular input queue, also under synchronization.
• COMMIT messages.
• Loop.

COMMIT completes the transaction in the input queue and makes the token available to other MDBs. No input queue processing can occur except for MDB, which has a token under synchronization. However, you can have any number of MDBs waiting for a token. The failure of any of them allows others to instantly capture.

No need to use XA, by the way. Single Phase COMMIT WMQ works great with this.

Answer 2

When applications try to use a queue through their MDB listeners, we can restrict them by defining a queue using DEFSOPT (Exclusive). This ensures that only one application can consume messages from this queue.

If we want to limit only one instance of the application, define it as NOSHARE. Thus, one instance of one application can receive messages in a queue at a time. Others will receive a turn when the current one releases the castle.

Answer 3

in my opinion, synchronization of several consumers is not a big problem and is the most effective solution. I don’t know where the processing result should be written (maybe the JMS queue again?), But I would try to use the lightweitght agent up to this point. You can use timestamps or use a counter over JMS to maintain order. Consumers can run in parallel and then send messages to the support queue. Than a single agent can order them using a queuebrowser and then a transaction. This agent must be "wathdogged".

Alessandro