All geek questions in one place

Parallel recordings for event sources at the top of Kafka

I was considering using Apache Kafka as an event store in the event source configuration. Published events will be associated with specific resources delivered on a topic related to the type of resource and placed in sections by resource identifier. So, for example, creating a resource of type Folder and id 1 will lead to the FolderCreate event, which will be delivered to the "folder" topic in the section specified by bypassing identifier 1 by the total number of sections in the topic. Although I do not know how to handle concurrent events that make the log inconsistent.

The simplest scenario will have two simultaneous actions that can invalidate each other, for example, one to update the folder, and the other to destroy the same folder. In this case, the section for this topic may contain an invalid sequence [FolderDestroy, FolderUpdate]. This situation is often captured by event versions as described here , but Kafka does not support this feature.

What can be done to ensure consistency in the Kafka magazine itself in these cases?

+4
source share
1 answer

I think it’s possible to use Kafka to search for aggregate sources (in the sense of DDD) or “resources”. Some notes:

  • , ( ) . , Kafka ack = all, , . , / , .
  • , , , , , ( ) , , .
  • - , .
  • . ( ) .

https://issues.apache.org/jira/browse/KAFKA-2260 1 , , , .

Kafka, , . , 4 - KTable, , , .

+3

Source: https://habr.com/ru/post/1678521/

More articles:


All Articles