In the following case, a very strange flow occurs.
I have 6 consumer processes connected to a cluster of 3 brokers. 3 consumers (set A) subscribe to a specific set of topics (topics C), and the remaining 3 consumers (set B) subscribe to another set of topics (topics D). All consumers belong to the same consumer group. All topics have a replication rate of 3 and are divided into 24 sections. Each consumer is automatically assigned 8 sections of the topic that are supposed to be used. I set the destination template round round.
If I kill a user from set A, topics C are redistributed successfully from other consumers from set A. Each consumer in set A now subscribes to 12 sections from topics C.
But the consumers in set B are going crazy. All topics and all sections in D are assigned to any one user from B, and the remaining consumers remain inactive. All 24 sections of all topics in set D are assigned to one process.
I need some insight on how to debug this and what could be the problem?
I believe that only consumers in topic C are affected. Consumer set B should remain unaffected and definitely not assign all sections to a single process
Kafka's version is 0.11
source share