Kafka is not really structured for this. To understand why, review the design documentation here .
To provide a one-time confirmation, you need to create some external tracking system for your application, where you explicitly write confirmations and implement locks on the transaction ID to ensure that things are processed only once. Estimated implementation costs, such as a system, are extremely high and are one of the main reasons large transactional systems require relatively exotic equipment and may have lower scalability than systems like Kafka.
If you don't need strong longevity semantics, you can use the group APIs to keep track of when the last message was read. This ensures that each message is read at least once. Please note: because the group API does not provide you with the ability to explicitly track your own processing logic, your actual processing guarantees in this scenario are rather weak. Schemes that rely on idempotent processing are common in this environment.
Alternatively, you can use the poorly named SimpleConsumer API (which is pretty complicated to use), which allows you to explicitly track the timestamps in your application. This is the highest level of processing guarantee that can be achieved using the Kafka native API, since it allows you to track your applications with their own data processing, which are read from the queue.
source share