So, I have a cluster of 6 machines. These computers have 2 IP addresses (open and closed). Private IPs use a high-speed network (1 Gbit / s), and the public uses a slower network (120 Mbit / s).
When setting up my kafka / zookeeper cluster, I set up a private IP as a broker ip. Now my configuration is fine, as I see the output saying that all my brokers are connected to the correct IP
INFO Connecting to zookeeper on 10.11.10.2:2181,10.11.10.3:2181,10.11.10.4:2181,10.11.10.5:2181,10.11.10.6:2181,10.11.10.7:2181 (kafka.server.KafkaServer)
Now that I tested the manufacturer’s performance, I could get a bandwidth of more than 120 Mbps.
bin/kafka-producer-perf-test.sh
When I used nload to check how much data goes through each interface, I found that data is coming in and out through a public IP interface (slower network interface). I double-checked the zookeeper and kafka logs, but they both say that my brokers are on a private IP.
Any ideas on what is the issue or what do I need to change?
source
share