Kafka Rejection Request with High Level API

Question

I would like to use the Kafka high-level API, and at the same time, I would like to turn off automatic offsets. I tried to achieve this by following these steps.

1) auto.commit.enable = false
2) offsets.storage = kafka
3) dual.commit.enabled = false

I created an offset manager that periodically creates an offsetcommit request for kafka and captures the offset.

However, I have the following questions

1) Does the high-level API automatically extract the offset from the kafka repository and initialize itself with this offset? Or should I use a simple user API to achieve this?

2) Is kafka-based storage maintained for offsets in all brokers? Or is it supported by only one broker?

Answer 1

I created an offset manager that periodically creates an offsetcommit request for kafka and captures the offset.

You do not need to do this if you are using a high-level consumer that provides you methods for fixing manual offsets, javadoc (in the Manual Offset Control section) provides you with examples of how to do this.

1) Does the high-level API automatically extract the offset from the kafka repository and initialize itself with this offset? Or should I use a simple user API for this?

A high-level user will take care of getting the last recorded offset when it is rebooted so that you can resume consumption from where you left off.

2) kafka ? ?

Kafka __consumer_offsets, 3 50 . , 3 . , offset offsets.