Azure Service Bus - Round Robin Theme for Multiple Services

Question

Here's the script:

Publisher #1 ═══╗ ╔═══ Round Robin ═══╦═══ Subscriber #1 (Service 1) ║ ║ ╚═══ Subscriber #2 (Service 1) ╠═══ Topic ═══╣ ║ ║ ╔═══ Subscriber #3 (Service 2) Publisher #2 ═══╝ ╚═══ Round Robin ═══╩═══ Subscriber #4 (Service 2) 

I have one message that needs to be processed by several subscribers, but only for each service (for each service several instances will be executed).

Message # 1 must be processed by subscribers # 1 and # 3. Message No. 2 should be processed by subscribers No. 2 and No. 4. Message No. 3, subscribers No. 1 and No. 3 again. In principle, each message should be cyclically distributed across each of the load-balanced services that subscribe to each message, grouped by each service that connects. Is this possible without creating multiple themes?

Even if it’s not a cool robin per-say, I would like to use all my efforts to load balance between several services. Is it possible?

Answer 1

1. Themes

Themes are a publishing / distribution mechanism that will send a message once per subscription (subscriber).

Subscribing to a topic is like a virtual queue that receives copies of messages sent to a topic. Messages are received from the subscription identically to how they are received from the queue ...

Subscriptions support the same patterns that were described earlier in this section regarding queues: a competing customer, temporary isolation, load balancing, and load balancing.

Source: MSDN Article

To achieve your scenario, you need to reuse topic subscriptions among competing consumers (service instances).

 Publisher #1 ═══╗ ╔═══ Subscription 1 ═══╦═══ Service 1-instance 1 ║ ║ ╚═══ Service 1-instance 2 ╠═══ Topic ═══╣ ║ ║ ╔═══ Service 2-instance 1 Publisher #2 ═══╝ ╚═══ Subscription 2 ═══╩═══ Service 2-instance 2 

A. Create a topic subscription

 string connectionString = "<Secret>" var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString); if (!namespaceManager.SubscriptionExists("TestTopic", "Inventory")) { namespaceManager.CreateSubscription("TestTopic", "Inventory"); } 

B. Listen to an existing subscription

 MessagingFactory factory = MessagingFactory.Create(uri, tokenProvider); MessageReceiver receiver = factory.CreateMessageReceiver("TestTopic/subscriptions/Inventory"); 

2. Queues

Using multiple Queues may also be appropriate for your specific scenario. Each Queue , having several competing consumers (instances), delivers the message only once to the first client, who requests and processes it successfully.

Then the design becomes:

 Publisher #1 ═══╗ ╔═══ Service 1 Queue ═══╦═══ Subscriber #1 (Service 1) ║ ║ ╚═══ Subscriber #2 (Service 1) ╠═══ASB═══╣ ║ ║ ╔═══ Subscriber #3 (Service 2) Publisher #2 ═══╝ ╚═══ Service 2 Queue ═══╩═══ Subscriber #4 (Service 2) 

Answer 2

I would like to use all my efforts to load the balance between several services. Is it possible?

From your description, it seems that you are trying to load a balance between multiple instances of any given service, and not one service. You get this out of the box with a competing consumer template that supports ASB. Not exactly what you need to work on.

The problem I'm trying to solve is that I want every consumer to consume messages on one topic. Is this possible with ASB?

Yes it is possible. If you have a common theme with a subscription to a service (not an instance) and contains a rule that is always evaluated as true, SqlFilter ( 1 = 1 ). Then each subscriber will receive a copy of this message. In essence, you will send a message to all your services. Thanks to a competing consumer, only one instance of each service will receive this message.

To target a specific service, you must create an additional rule that will filter the message by property (standard or custom property). For example, this could be the name of a service.

If you don't want to focus on middleware for your microservices, you can look into the framework that will do it for you. Such a framework usually has additional capabilities. See NServiceBus or MassTransit, for example.

Full caveat - I am working on NServiceBus and its ASB vehicle.