How to implement a microservice event-driven architecture using Spring Cloud Stream Kafka and a service database

Question

I am trying to implement an event driven architecture to handle distributed transactions. Each service has its own database and uses Kafka to send messages to inform other microservices about operations.

Example:

Order service -------> | Kafka |------->Payment Service | | Orders MariaDB DB Payment MariaDB Database 

The order receives an order request. He must save the new Order in his database and publish a message so that the Payment Service understands that he must charge for the item:

private orderBusiness orderBusiness;

 @PostMapping public Order createOrder(@RequestBody Order order){ logger.debug("createOrder()"); //a.- Save the order in the DB orderBusiness.createOrder(order); //b. Publish in the topic so that Payment Service charges for the item. try{ orderSource.output().send(MessageBuilder.withPayload(order).build()); }catch(Exception e){ logger.error("{}", e); } return order; } 

These are my doubts:

• Steps a.- (saving to the order database) and b.- (posting the message) should be performed atomically in the transaction. How can i achieve this?
• This is due to the previous one: I am sending a message with: orderSource.output (). send (MessageBuilder.withPayload (order) .build ()); These operations are asynchronous, and ALWAYS returns true, regardless of whether the Kafka broker works. How can I find out that the message has reached the Kafka broker?

Answer 1

Steps a.- (saving to the order database) and b.- (posting the message) should be performed in a transaction, atomically. How can i achieve this?

Kafka does not currently support the transactions (and therefore also rollbacks or commits) that you need to synchronize. In short: you cannot do what you want. This will change in the near future when the KIP-98 is merged, but it may take some time. In addition, even with transactions in Kafka, an atomic transaction through two systems is a very difficult task, everything that follows will be improved only thanks to the support of transactions in Kafka, it still will not solve your problem. To do this, you will need to study biphasic adoption in your system.

You can get a little closer by adjusting the manufacturer’s properties, but in the end you will have to choose between at least once or at most once for one of your systems (MariaDB or Kafka).

Let's start with what you can do at Kafka to ensure message delivery, and down we will dive into your options for the overall flow of processes and what are the consequences.

Guaranteed delivery

You can configure how many brokers should acknowledge receipt of your messages before the request is returned to you with the acks parameter: setting this to everything you tell the broker to wait until all replicas confirm your message before returning, reply to you. This still does not guarantee 100% that your message will not be lost, because it has only been written in the page cache, and there are theoretical scenarios when the broker fails before it remains on disk, where the message may still be lost. But this is as good a guarantee as you are going to get. You can also reduce the risk of data loss by reducing the interval at which brokers force fsync to disk (underlined text and / or flush.ms), but please keep in mind that these values ​​can result in large performance penalties.

In addition to these settings, you will need to wait until your Kafka manufacturer returns a response to your request and checks if an exception has occurred. This is related to the second part of your question, so I will go further. If the answer is clear, you can be as confident as possible so that your data gets into Kafka and starts to worry about MariaDB.

All that we have examined so far is only about how to ensure that Kafka receives your messages, but you also need to write data to MariaDB, and this can also fail, which would make it necessary to recall the message that you have already sent to Kafke - and this you cannot do.

So, basically, you need to choose one system in which you deal better with duplicate / missing values ​​(depending on whether you redirect partial failures), and this will affect the order in which you do.

Option 1

In this option, you initialize the transaction in MariaDB, then send a message to Kafka, wait for a response, and if the transfer was successful, you complete the transaction in MariaDB. If sending to Kafka has failed, you can cancel the transaction in MariaDB and everything will be a dandy. If, however, sending to Kafka is successful and your MariaDB commit fails for some reason, then there is no way to return a message from Kafka. This way you will either skip the message in MariaDB or have a duplicate message in Kafka if you resend it later.

Option 2

This is largely the opposite, but you are probably better off deleting a message written in MariaDB, depending on your data model.

Of course, you can mitigate both approaches by following unsuccessful attempts and repeating them only later, but all this is more connected with a more complex problem.

Personally, I would go with approach 1, since the likelihood of a commit failure should be slightly less than the transfer itself, and do some cheating on the other side of Kafka.

---

This is due to the previous: I am sending a message from :. OrderSource.output () send (MessageBuilder.withPayload (order) .build ()); These operations are asynchronous, and ALWAYS returns true, regardless of whether the Kafka broker is down. How can I find out that the message has reached the Kafka broker?

Now, first of all, I admit that I am not familiar with Spring, so this may not be useful to you, but the following code fragment illustrates one way to check for exceptions. By calling a flash, you block until all the packages have finished (and failed or were successful), and then check the results.

 Producer<String, String> producer = new KafkaProducer<>(myConfig); final ArrayList<Exception> exceptionList = new ArrayList<>(); for(MessageType message : messages){ producer.send(new ProducerRecord<String, String>("myTopic", message.getKey(), message.getValue()), new Callback() { @Override public void onCompletion(RecordMetadata metadata, Exception exception) { if (exception != null) { exceptionList.add(exception); } } }); } producer.flush(); if (!exceptionList.isEmpty()) { // do stuff } 

Answer 2

I believe that a suitable way to implement Event Sourcing is that Kafka can be populated directly from events pushed by a plugin that reads from an RDBMS binary, for example using Confluent BottledWater ( https://www.confluent.io/blog / bottled-water-real-time-integration-of-postgresql-and-kafka / ) or more actively Debezium ( http://debezium.io/ ). Microservice consumption can then listen to these events, consume them, and act in accordance with their respective databases, ultimately compatible with the RDBMS database.

Take a look at my complete answer for a guide: fooobar.com/questions/1014851 / ...