Is there an enterprise message queue that can discard duplicate messages (the first value remains)?

Question

I am looking for a message queue search with these requirements. Could not find him; perhaps the closest was the rabbitmq-lvc plugin (but I need the first value in the line to lock in and stay in front). Does anyone know the technology to support them?

• message queue - FIFO
• If a duplicate message is in the queue, the message queue itself either rejects or deletes it.

For example, manufacturers put these three messages (each with a discriminator value) in the queue in this sequence: M1 (discriminator = 7654), M2 (discriminator = 2435), M3 (discriminator = 7654). Now I want the message queue to see that M3 has the same discriminator value as M1, and thus discards / rejects M3. Consumers receive only: M1, M2.

Thanks tom

Answer 1

I do not know other transports, but I know that WebSphere MQ does not do this, and I believe that the explanation will be applied as a whole for the category. I would be very surprised to find that any message transport really provides this. Here are a few reasons:

• Asynchronous messages must be atomic. Different providers create their own communication capabilities (communication between two or more messages), but generally, proximity to messages should be avoided. Your use case not only requires transport to deal with message proximity, but it does so at an indefinite interval between related messages.
• The message payload is a blob. For performance reasons, WMQ does not affect the payload of messages, with the exception of compression or code page conversion. All that requires parsing the message payload is work for WebSphere Message Broker, DataPower, or WebSphere ESB. I expect that any message transport that is alleged to be executed will encounter similar problems, as payload analysis leads to longer codes and non-linear performance degradation. The exception is message properties, but WMQ uses them only for selection, and I expect this to be the case.
• The operation is stateless. As a transport, the state of the application can be stored in a constant message, but the state of the transport layer should not depend on the state of the application in different units of work. Again, the ESB product type is best suited if you want to delegate the management of some state of the application to the messaging layer, and especially when such management covers many units of work.
• Guaranteed delivery. WMQ was designed to never lose your permanent message. If the application explicitly sets the expiration date, the message may disappear because the sender said that this is normal. If the message is not permanent, it can disappear, but only in exceptional condition and, again, because the sender said that this is normal. The use case you described may cause the message to go away not because the sender said that everything was in order, or even because the recipient said that everything was in order, but because of interaction with some unrelated third party, which accidentally beat you in line with a duplicate value. What if this first message has an unacceptable problem with the header or code page and rolls back? What should I do if as an attacker I spew garbage messages with all possible 4-digit values ​​for the discriminator?

As I said, I do not know other messaging products, so there may be something there that meets your requirements, and if so, I will be interested to read about it. However, in the event that no one answers, this post may shed light on the reasons why.