Slow memcache performance in Google App Engine

Question

Memcache is one of those things where the solution can be absolutely any, and no one ever gives a decent answer, perhaps because it does not exist. So I am not looking for a direct answer, but maybe just something to make me move in the right direction.

For a typical request, here is my info on AppStats:

So, from a general 440 ms request, I spend 342 ms in memcache. And here I decided that memcache should be lightning fast. I have to do something wrong.

Looking at my memcache statistics in my admin console, I have the following:

Hit count: 3848 Miss count: 21382 Hit ratio: 15% 

I am not an expert on this, but I'm sure 15% is terrible.

The typical query above is too detailed to explain, but basically I create and place a new object that also updates and places the parent object, which also updates and places any users associated with the parent object.

In all this, I always get the key, never ask. Oh, and I use NDB, so all the main memcache files are processed automatically . Therefore, I never touch memcache on my own in my code.

Any ideas?

Edit: here is a breakdown of my request

So, I have only 2 datastore and 2 puts. The rest is automatically processed by memcache material. Why does he work so hard? Would I be better off manually processing this material?

Answer 1

Let's take a closer look at your data. Seven memcache entries took as long as two storage files. This actually proves that memcache, for example, is 3.5 times faster than Datastore.

If a typical request to your application requires updates of at least three database objects, followed by updating more objects (associated with users), you cannot make this operation "lightning fast." Memcache helps when you read records much more often than you write them. If the number of reads and writes in the user record is at the same level, you should consider disabling the cache for this model.

You can also try asynchronous operations and task queues . From your description, it looks like you are trying to update the object first and update its parent only after the update is completed, because it is natural. You can run them at the same time; it will probably require some refactoring, but it's worth it.

Secondly, perhaps updating “all connected users” may be. deferred to a task generated in the background; Task queues have a very convenient interface for this. "Linked Users" will not be updated immediately, but they probably do not need to! However, the delay in your request will be less.