Memcache and ultimate consistency

Question

I am working on a small project to learn about the Google App Engine, the project is in Java and has Customer objects, Client instances can have a policy. Each client has its own group of individuals, so transactions can be used to change the client.

The main page of the site is the list of Clients, when a new client is added, the list of clients is displayed again.

Since each client is in its own group of entities, there are times when a newly added client does not appear in the new client list, updating the client list after a few seconds, and the client will appear. A similar problem exists when deleting clients, you delete the client, but it appears in the general list for a few seconds. I understand that this can be expected in the Google App Engine due to the possible sequence provided by the data store.

So, I tried to work around this problem using memcache to keep recently added or recently added clients deleted. The code I'm using is below.

public List<Customer> getCustomers() { List<Customer> cachedCustomers = myCache.getCached(); List<Customer> recentlyDeleted = myCache.getDeleted(); // Calls the real datastore. List<Customer> dbCustomers = customerDao.getCustomerList(); Set<Customer> allCustomers = new HashSet<Customer>(); // Add cached first as these are most the most up todate. allCustomers.addAll(cachedCustomers); allCustomers.addAll(dbCustomers); allCustomers.removeAll(recentlyDeleted); List<Customer> allList = new ArrayList<Customer>(); allList.addAll(allCustomers); Collections.sort(allList); return allList; } 

I ask here because I think that the way I do it does not feel the “right” way to do it and wants to hear from those who know the best ways to get around the problems that the constant sequence creates.

Answer 1

What you do is recommendations . Therefore, I believe that this is the right way to do this.

Also, I did a Google and GitHub search for a library that could handle this for you, but couldn't find it. Thus, for this caching method, the last insertions and deletions look “correct”, I suggest you write a library that handles this for any constant class that you want.

In addition, I suggest reading this post , where Ikai Lan, developer of the Google App Engine application developer program, explains how fetching and inserting in HR works and what are the performance indicators compared to the Master / Slave data warehouse.

Answer 2

With HRD, in the case of 99.9%, the results of your write operation should be visible within a few seconds. Therefore, in a typical web application, if you serve a web page as a result of an operation, the time before the user performs the next operation should be sufficient to record the entries.

So, if you create a results page from live data objects in the application, you should be fine. You should also be fine if you redirect the user, for example. client page where you read the client by ID (as agreed).

If you need to query, and you want the results to appear in the query issued immediately after the operation, I think that there is no way to get around memcache or use some tricks to record the presence / absence of just modified objects.