How to change the starting offset for a topic?

Question

Can I change the initial offset for a new topic? I would like to create a new topic and start reading at offset 10000 . How?

Answer 1

You can do this with the zookeeper shell. Kafka uses zookeeper to track consumer biases.

Go to the kafka bin directory and call the zookeeper shell (my version of kafka is 0.8.0)

 ./zookeeper-shell.sh localhost:2181 

Now use the zookeeper get command

 get /consumers/consumer_group_id/offsets/topic/0 

it shows something like

 2043 cZxid = 0x4d ctime = Wed Mar 18 03:56:32 EDT 2015 ... 

Here 2043 is the maximum absorption. Set the desired value using the zookeeper set command

 set /consumers/consumer_group_id/offsets/topic/0 10000 

The path is framed as follows: users / [consumer_group_id] / offsets / [topic] / [partition_id].
You will need to replace the appropriate consumer group, topic and section identifier.

* Also, since you mentioned this new instance of kafka, I'm not sure if consumers will be connected and consumer groups created.

Answer 2

Since corrections kafka 0.9 are stored in the subject. To change the offset, use the seek () method:

 public void seek(TopicPartition partition, long offset) 

Overrides the sample offsets that the consumer will use in the next poll(timeout) . If this API is called for the same section more than once, the last offset will be used for the next poll (). Note that you can lose data if this API is arbitrarily used in the middle of consumption, before resetting the sample offset