Kafka message spoiled by master, but replica looks good

Question

Kafka version : 0.9

We have a java application trying to read from Kafka. but we see data corruption in kafka messages

The following is a Java application error:

org.apache.kafka.common.KafkaException: Error deserializing key/value for partition varnish_raw_logs-31 at offset 568565 Caused by: org.apache.kafka.common.record.InvalidRecordException: Record is corrupt (stored crc = 1835858402, computed crc = 4025136930) 

So, I decided to see which node belongs to section 31 and see what the data looks like using the kafka dump log. you see below that the value 31 belongs to node -4 and node -2 has its own replica.

 Topic: varnish_raw_logs Partition: 31 Leader: 4 Replicas: 4,2 Isr: 4,2 

Now, when I use the dump to view the logs from node 4 and node 2. I see that the data is valid in the replica node (node ​​-2), but corrupted in master.

So my question . What is the main reason for a wizard (node ​​-4) to enter this state? While you can argue, we should skip errors related to a specific event. I do not want to miss these messages because I have a good message in the replica.