5+ transactional counter per second in the Google App Engine data warehouse

Question

I am developing a tournament version of the game in which I expect more than 1000 simultaneous players. When the tournament begins, players will be eliminated fairly quickly (perhaps more than 5 per second), but the process will slow down as the tournament progresses. Depending on when the player is expelled from the tournament, a certain amount of points is awarded. For example, the player who throws the first one receives nothing, and the player who takes the 500th place receives 1 point, and the winner of the first place receives 200 points. Now I would like to reward and display the number of points immediately after the player has been removed.

The problem is that when I insert a new row into the data store after the player has been deleted, the row object must be in a separate entity group, so I would not hit the gae storage limit of 1-5 records per second for 1 group of persons. I also need to be able to read and write the number of lines in a row so that I can correctly determine the win for all players that will be eliminated.

What would be the best way to implement a datamodel to support this?

Answer 1

Since there are a limited number of players, competition problems within a few seconds are unlikely to be supported for a very long time, so you have two options:

• Just ignore the problem. Exclusion clusters will occur, but as long as this is not a stable situation, the replay mechanics for transactions ensures that all of them are executed.
• When someone leaves, record it yourself and update the status of the tournament by assigning ranks asynchronously. This means that you cannot immediately tell them about your ranking, but rather you should make an asynchronous response or poll it.

I would suggest that the first one, to be honest: even if half of your 1000 tournament people left in the first 5 minutes - an incredibly unlikely event - you still watched less than 2 beats per second. In fact, any spikes will be smaller and shorter than that.

It should be borne in mind that due to the fact that transactions are performed, transactions of the same group of entities that meet together will be resolved in a semi-random order, that is, this is not a strict FIFO queue. If you require this, you will have to force it yourself, although this is far from a trivial matter in a distributed system of any kind.

Answer 2

existing comments and answers are very well suited to a particular question.

at a higher level, check out this publication and the open source library from the james team. they faced a similar problem and eventually developed a scalable scoreboard based on a data warehouse that efficiently processes both updates and requests for arbitrary pages.