Invalidity of multiple grouped cache keys

Question

I have a TicketType model that has about 500 instances.

It changes only a few times a week.

But if it changes, I need to invalidate all cached values ​​that used the old TicketTypes.

Unfortunately, some cache keys are not fixed. They contain the calculated data.

I see these solutions:

Use the version argument and update the version value for the post save TicketType message handler.

Use a common prefix for all cache keys that are based on TicketType. Then invalidate all cache keys in the post save signal handler.

I think there is a third, and better way ...

Example:

TicketType is a tree. TicketTypes visibility is related to permissions. Two users can see the tree differently if they have different permissions. We cache it according to permissions. User permissions become serialized and hashed. The cache key is created by creating a line containing the hash and the fixed part:

 hash_key='ticket-type-tree--%s' % hashed_permissions 

If the TicketType tree changes, we must be sure that the old data is not loaded from the cache. Active revocation is not required if old data is not used.

Answer 1

You can use ticket modification time as part of the cache key.

 hash_key = 'ticket-type-tree--%s-%s' % (hashed_permissions, tree.lastmodified) 

You can add DateTimeField with auto_now=True . If getting the modification time from db is too expensive, you can also cache it.

Typically, a cache update in the post_save signal handler is great. If you do not want to constantly have consistent data and want to pay the additional cost of transactions.

Answer 2

Use redis to cache your models.

The way I will cache my instances will be as follows:

1 - Make sure you get one item at a time. For example: Model.objects.get (foo = 'bar'), and each time you use the foo attribute to get the model from the database. This will be used to ensure that data becomes invalid later.

2-Override save () method and make sure it saves data to the cache using the foo attribute.

eg:

 class Model(model.Model): foo = models.CharField() bar = models.CharField() def save(self, *args, **kwargs): redis.set(foo, serialize_model()) super(Model, self).save(*args, **kwargs) def serialize_model(): return serilized_object 

3-Override get method to get the serialized object before deleting the database.

eg:

 class Model(model.Model): ... def get(self, *args, **kwargs): if redis.get(self.foo): return redis.get(self.foo) else: return super(Model).get(*args, **kwargs) 

4. Override the delete method to remove the cache if the instance was deleted or deleted.

eg

 class Model(model.Model): ... def delete(self,*args, **kwargs): redis.delete(self.foo) super(Model, self).delete(*args, **kwargs) 

Replace the model class with your model, in this case it will be a ticket type

One thing, I assume that you will not touch the database outside of your Django application. If you use raw sql anywhere else, this will not work.

Look for redis functions on your website, they have a function to remove, install and retrieve. If you use a different caching method. See how to install, receive, and remove.

Answer 3

Well, basically you're the problem - it's just the expressiveness of the cache key. When you need to do something complex, like hashing a set to get a key, it should be a hint that something is missing.

In your case, I believe that the “permission set” object is simply missing. You could call it a group, a role (like in RBAC) ... That's why I asked you if the sets are duplicate - actually your hash key is just a way to recreate the identifier of the installed object that does not exist.

So the solution would be:

• create a role model, with M2M from users and M2M to permissions (which, as I understand it, are related to your TicketTypes)
• use an event handler to catch TicketType persistence.
  • selection of all influencing roles (through permissions)
  • generate keys (something like ticket-TREEID-ROLEID type) and invalidate them

Two concluding observations:

• Sometimes cache.clear () is the solution, especially if you are not using the cache for anything else.
• You say that your SQL query query is huge when navigating a tree. If you have not already done so, you can simply optimize it with prefetch and select_related (see the document).

Answer 4

In the TicketType tracking signal handler:
a) generate a key depending on the permissions of all users and invalid keys
b) generate a key for each permutation (permission) (if you can calculate them) and revoke the keys
c) use a second memcached instance to store only this cache and clear it (easiest)

PS: Pro tip will update caches, not just cancel them. However, an uncaught exception in the django signal can be a problem, so be tired