Fixed stream card with timeout without deletion

Question

Imagine a use case where events are broadcast for each user, but only the first week of events is interesting. During this period, state logic takes place using mapWithState. After this period, user inbound events should be ignored.

When the user state accepts the memory, it makes sense to change it after the user weekly period to a simple marker already seen.

If an event arrives for this user a week or later after his first event, it is easy to change the state to this already seen marker.

But if no events appear after this week, the state will never change to this already seen marker, and the state will continue to occupy memory forever.

As far as I understand, adding a timeout (to the user state) will not help, since you are not allowed to change the state for the timeout state (it makes sense, since it will be deleted).

Is there an easy way to achieve this use case?

Answer 1

From what I understand, Spark 2.2 mapGroupsWithState has richer timeouts that can be used not only to delete a state, but also to change it (check here ).