Kineis: What is the best / safest way to shut down a worker?

Question

I am using AWS Kinesis Client Library .

I need a way to disable the Kinesis Worker thread during deployment, so I stop at a breakpoint, not in the middle processRecords().

I see a logical shutdown present in Worker.java, but it is closed.

I need a breakpoint and idempotency to be important to me, and I don’t want to kill the process in the middle of the game.

[EDIT]

Thanks to @CaptainMurphy, I noticed that it Worker.javaprovides a method shutdown()that safely closes the worker and LeaseCoordinator. What he does not do is challenge the shutdown()task at IRecordProcessor. He abruptly stops IRecordProcessor, not worrying about the condition.

I understand that Idempotency between breakpoints is not guaranteed by KCL, and the developer should make the design fault-tolerant, but I feel that it IRecordProcessormust be turned off correctly before it LeaseCoordinatorstops regardless.

Answer 1

The write processor shutdown method is actually called when you call the shutdown of the Worker. You can track it back from the ShutdownTask class, which is created by the ShardConsumer class, which is closed by Worker.

, - , , checkpointer . . Records():

for(Record currRecord : records)
{
    someProcessSingleRecordMethod(currRecord)
    if(shutdown) 
    { 
        checkpointer.checkpoint(currRecord.getSequenceNumber()); 
        return; 
    } 
}

shutdown true.

, Kinesis - " " , , . " " Kinesis.

Answer 2

1.7.1 (. ), , , IShutdownNotificationAware .

• , IShutdownNotificationAware IRecordProcessor. shutdownRequested(IRecordProcessorCheckpointer checkpointer). : shutdown IRecordProcessor , TERMINATE

• Future<Void> shutdown = worker.requestShutdown();
  shutdown.get(); // wait for shutdown complete
  

PS: Kinesis 1.7.4 , . 1.7.4 .