DynamoDB sequentially reads for the Global Secondary Index

Question

Why can't I get consistent reads for global secondary indexes?

I have the following setup:

Table: tblUsers (id as hash)

Global secondary index: tblUsersEmailIndex (email as a hash, identifier as an attribute)

Global secondary index: tblUsersUsernameIndex (username as a hash, id as an attribute)

I request indexes to check if a given email or username is present, so I am not duplicating the user.

Now the problem is that I cannot perform consistent reads for index queries. But why not? This is one of the few times when I really need updated data.

According to AWS documentation:

Global secondary index queries only support consistency.

Changes to the table data apply to global secondary indexes within a fraction of a second under normal conditions. However, in some unlikely failure scenarios, large propagation delays can occur. Because of this, your applications must anticipate and handle situations where a query on the global secondary index returns results that do not match the latter .

But how do I deal with this situation? How can I make sure that this email or username is not already present in db?

Answer 1

You probably already went through this: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html

The short answer is that you cannot do what you want to do with the global secondary indexes (i.e. always consistency in the end).

The solution here would be to have a separate w / attribute table that interests you as a key, and do sequential reads there. You will need to make sure that you update this when you insert new entities, and you also have to worry about the extreme case where the insertions there are successful, but not in the main table (i.e. you need to make sure that they are synchronized)

Another solution would be to scan the entire table, but that would probably be redundant if the table was large.

Why do you care if someone creates 2 accounts with the same email address? You can simply use the username as the primary hash key and simply not use the uniqueness of the email.

Answer 2

Depending on your situation and taking into account all the alternatives, it may be acceptable to add an automatic retry if you did not find anything in GSI for the first time in order to circumvent the lack of strictly agreed readings. I did not even think about it until I came across road barriers with other options, and then realized that it was simple and did not cause any problems for our specific use case.

  public async getByToken(token: string): Promise<TokenResponse> { let tokenResponse: TokenResponse; let tries = 1; while (tries <= 2) { // Can't perform strongly consistent read on GSI so we have to do this to insure the token doesn't exist let item = await this.getItemByToken(token); if (item) return new TokenResponse(item); if (tries == 1) await this.sleep(1000); tries++; } return tokenResponse; } 

Since we don’t care about performance for someone sending a nonexistent token (which should never happen), we solve the problem without any performance degradation (except for a possible delay of 1 second after creating the token).). If you just created a token, you would not need to convert it back to the data you just transferred. But if you happen to do this, we will process it transparently.