I have a database with files that you can search, view and have multiple copies on multiple servers.
I cache the search, browse the pages and server locations (URLs). Say I'm deleting a file, what's a good way to cancel all requests, view data and URLs for this file? Or if the file server is down and I need to invalidate all the URLs pointing to that server?
Essentially, I'm looking for something similar to memcache-tags , but with standard memcache and php components. (Without the need to change anything on the web server itself). I need some kind of relationship in many ways (one server has many files and one file has several servers) between the keys, but it seems that it cannot find a suitable way to achieve this. In some situations, the acceptable level of obsolete caches is acceptable (minor updates, etc.), but in some cases it is not (usually deleted, but the server does not work), where I need to invalidate all cache elements that contain links to it.
Some approaches that I looked at:
Namespaces :
$ns_key = $memcache->get("foo_namespace_key"); // if not set, initialize it if($ns_key===false) $memcache->set("foo_namespace_key", rand(1, 10000)); // cleverly use the ns_key $my_key = "foo_".$ns_key."_12345"; $my_val = $memcache->get($my_key); //To clear the namespace do: $memcache->increment("foo_namespace_key");
- Limits the cache key to a single namespace
Approach to element caching :
$files = array('file1','file2'); // Cache all files as single entries foreach ($files as $file) { $memcache->set($file.'_key'); } $search = array('file1_key','file2_key'); // Retrieve all items found by search (typically cached as file ids) foreach ($search as $item) { $memcache->get($item); }
- It gives a problem if the file server is turned off and all keys containing the URLs of this server must be invalid (requires a large number of small cache elements, which requires a large number of cache requests) - interrupts any chance of caching full objects and results.
Tag Effect :
class KeyEnabled_Memcached extends Zend_Cache_Backend_Memcached { private function getTagListId() { return "MyTagArrayCacheKey"; } private function getTags() { if(!$tags = $this->_memcache->get($this->getTagListId())) { $tags = array(); } return $tags; } private function saveTags($id, $tags) { // First get the tags $siteTags = $this->getTags(); foreach($tags as $tag) { $siteTags[$tag][] = $id; } $this->_memcache->set($this->getTagListId(), $siteTags); } private function getItemsByTag($tag) { $siteTags = $this->_memcache->get($this->getTagListId()); return isset($siteTags[$tag]) ? $siteTags[$tag] : false; } /** * Save some string datas into a cache record * * Note : $data is always "string" (serialization is done by the * core not by the backend) * * @param string $data Datas to cache * @param string $id Cache id * @param array $tags Array of strings, the cache record will be tagged by each string entry * @param int $specificLifetime If != false, set a specific lifetime for this cache record (null => infinite lifetime) * @return boolean True if no problem */ public function save($data, $id, $tags = array(), $specificLifetime = false) { $lifetime = $this->getLifetime($specificLifetime); if ($this->_options['compression']) { $flag = MEMCACHE_COMPRESSED; } else { $flag = 0; } $result = $this->_memcache->set($id, array($data, time()), $flag, $lifetime); if (count($tags) > 0) { $this->saveTags($id, $tags); } return $result; } /** * Clean some cache records * * Available modes are : * 'all' (default) => remove all cache entries ($tags is not used) * 'old' => remove too old cache entries ($tags is not used) * 'matchingTag' => remove cache entries matching all given tags * ($tags can be an array of strings or a single string) * 'notMatchingTag' => remove cache entries not matching one of the given tags * ($tags can be an array of strings or a single string) * * @param string $mode Clean mode * @param array $tags Array of tags * @return boolean True if no problem */ public function clean($mode = Zend_Cache::CLEANING_MODE_ALL, $tags = array()) { if ($mode==Zend_Cache::CLEANING_MODE_ALL) { return $this->_memcache->flush(); } if ($mode==Zend_Cache::CLEANING_MODE_OLD) { $this->_log("Zend_Cache_Backend_Memcached::clean() : CLEANING_MODE_OLD is unsupported by the Memcached backend"); } if ($mode==Zend_Cache::CLEANING_MODE_MATCHING_TAG) { $siteTags = $newTags = $this->getTags(); if(count($siteTags)) { foreach($tags as $tag) { if(isset($siteTags[$tag])) { foreach($siteTags[$tag] as $item) { // We call delete directly here because the ID in the cache is already specific for this site $this->_memcache->delete($item); } unset($newTags[$tag]); } } $this->_memcache->set($this->getTagListId(),$newTags); } } if ($mode==Zend_Cache::CLEANING_MODE_NOT_MATCHING_TAG) { $siteTags = $newTags = $this->getTags(); if(count($siteTags)) { foreach($siteTags as $siteTag => $items) { if(array_search($siteTag,$tags) === false) { foreach($items as $item) { $this->_memcache->delete($item); } unset($newTags[$siteTag]); } } $this->_memcache->set($this->getTagListId(),$newTags); } } } }
- It is not possible to control which keys are invalid when, due to the pop-up key of the internal memcache, you can delete the tag key, which in turn will invalidate a large number of valid valid keys (which will still exist).
- Problems writing concurrency
Two-stage caching system :
// Having one slow, and one fast cache mechanism where the slow cache is reliable storage containing a copy of tag versions $cache_using_file['tag1'] = 'version1'; $cache_using_memcache['key'] = array('data' = 'abc', 'tags' => array('tag1' => 'version1');
- Potential bottleneck using disk / mysql etc. for slow cache
- Problems writing concurrency