Scalability CQRS + EventSourcing

I am trying to use CQRS and EventSorcing in my new project. I follow how Greg Young proposed several years ago (implementation by Mark Nijhof - http://cre8ivethought.com/blog/2009/11/12/cqrs--la-greg-young/ ). And I have some questions regarding the scalability of this solution.

Some points were mentioned in this article by Mark Nijhof. But now the problem is the Denormalizer part, which is responsible for updating the reporting database. I want to make this part asynchronous, so after publishing events on the bus, I want to immediately return control. We suggested that Denormalizer could be implemented as a standalone web service (WCF), which will process incoming events and make updates to the report database in synchronization mode with batches. It looks like this could be a bottleneck, so we also want to add some scalability at the moment - a cluster solution. But in the case of a cluster, we cannot control the sequence of reports on database updates (or we must implement some strange and, I believe, logical logic that will check the versions of objects in the report database). Another problem is the stability of the solution: in case of failure, we will lose updates in the denormalizer, since we do not save them anywhere). So, now I'm looking for a solution to this problem (Denormalizer scalability), any thoughts are welcome!

+6
source share
1 answer

To get started, you will definitely want the denormalizer to be placed in a separate process. From there, you can publish the domain to your messaging infrastructure with events occurring in the domain. One simple strategy to help speed up denormalization is to break things down by message / event type. In other words, you can create a separate queue for each type of message, and then bind the denormalizer (using the message bus) to the corresponding events. The advantage of this is that you do not have messages folding one after another - everything starts working in parallel. The only places where you might have a conflict are tables that listen to several types. However, you now distribute the load between many endpoints.

As long as you use some kind of messaging infrastructure, you will not lose event messages when trying to denormalize. Instead, after a number of retry attempts, the message will be considered β€œpoison” and move to the error queue. Just monitor the error queue for problems. Once the message is in the error queue, you can check your logs to see why they are there, fix the problem, and then move it.

Another consideration is that the example of Mark Nijhof is somewhat old. There are several CQRS features, as well as tips in the DDD / CQRS of the Google Group .

+4
source

Source: https://habr.com/ru/post/885803/


All Articles