All geek questions in one place

Kafka: What happens when the entire Kafka cluster is down?

We test Producer and Consumer using Kafka. A few questions:

  • What happens when all brokers do not work and they do not respond at all?

  • Does the producer need to continue to ping Kafka brokers to find out when it will be online? Or is there a more elegant way for a Producer app to know?

  • How does Zookeeper help with all this? What if ZK also works?

+5
source share
2 answers

If one or more brokers do not work, the producer will try again for a certain period of time (depending on the settings). And during this time, one or more consumers will not be able to read anything until the relevant brokers are reached.

But if the cluster shuts down for a longer period than your total retry period, you may need to find a way to resend these failed messages.

+3
source

If all brokers are completed, your cluster will receive nothing. I believe that the Producer will also fail, because the heart beat from the zookeeper will not be accepted. I am sure that if zookeeper does not work, your system will no longer work. This is one of Kafka’s weaknesses; he needs a zoker to work.

+1
source

Source: https://habr.com/ru/post/1266029/

More articles:


All Articles