Message Queuing in the ASP.Net Web Interface

Question

I am developing a single-page client-side application (SPA) using AngularJS and ASP.Net WebAPI.

One of the features of SPA includes downloading a large CSV file, processing it on the server and returning the output to the user.

Obviously, this kind of calculation cannot be done on the Internet, and so I implemented UploadController, which is responsible for receiving the file, and PollingController, which is responsible for notifying the user when the calculation is complete.

The client application monitors the PollingController every few seconds.

I have no experience in message queues, but my gut tells me that they are required in this situation.

How do you recommend implementing this functionality in a non-blocking, efficient way?

Examples will be highly appreciated

Answer 1

In the past, I used message-based core service bus modules.

You are writing an application (running as a Windows service) that listens for messages sent via the event bus.

Your interface can publish these messages on the bus.

The most popular basis for this in .NET is NServiceBus , however, it has recently become commercial. You can also look at MassTransit , although this has very poor documentation.

The workflow you are doing:

• The MVC application accepts the download and places it in some directory accessible through the Windows service
• The MVC application publishes a "UploadReady" message.
• The service receives the message, processes the file, and updates some state for the polling controller.
• The polling controller observes a change in this state. Usually DB record, etc.

A good bit about using a structure like this is that if your service is omitted or redistributed, any processing can queue and resume, so you won't have any downtime.

Answer 2

For lengthy operations, you need a separate Windows Service application (or the worker role if it is Windows Azure). IIS can kill ASP.NET processes when disposing of the pool, and your operation will not complete.

The message queue is primarily intended for communication. You can use it between the website and the working parts. But it is not required there if your data is not supercritical. You can establish a connection using the database, cache, file system, or 100 other ways :)

You can use SignalR to notify the client of completed processing.