The fastest database engine for caching?

Question

I use MySQL for my main database, where I store the actual objects. When an object is rendered using a template, rendering takes a lot of time.

Because of this, I decided to cache the generated HTML. Right now I am storing a cache in files, named appropriate, and it works much faster. However, I know that this is not the best way to do this.

I need a (preferably key) database to store my cache. I cannot use the caching proxy because I still need to handle cached HTML. Is there such a database with a PHP interface?

Edit: If I use memcached and I cache about a million pages, will I end in RAM?

Edit 2: And again, I have a lot of HTML for caching (gigabytes).

Answer 1

If I use memcached and I cache a million pages, will I not exit RAM?

Memcached

memcached is also a true solid product (for example, redis more), used on all major sites to maintain their performance. Almost all active tweets (which are retrieved by the user) are stored in memcached for insane performance.

If you want to be fast, you must have an active dataset in memory. But yes, if the data set is larger than your available memory, you should (always have to store data in a permanent data store, since memcached is volatile) store data in a permanent data store, for example, mysql. When it is not available in memory, you will try to extract it from the data store and cache its memcache for future use (with the expire header).

Redis

I really like redis because it is an extended key storage with insane performance

Redis is an extended keystore. It is similar to memcached, but the data set is not mutable, and the values ​​can be strings, just like memcached, but also lists, sets, and ordered sets. All these data types can manipulate atomic operations to push / pop elements, add / remove elements, make a server-side connection, intersection, difference between sets, and so on. Redis supports a variety of sorting options.

Redis has a VM , so you do not need a separate persistent data store. I really like redis because of all available (power :)?). This tutorial with simon willison displays (a lot) of the original power that redis has.

Speed

Redis is pretty fast! , 110000 SETs / second, 81000 GETs / second in the Linux field for entry-level. Check tests .

Fixation

Redis is more actively developing. 8 hours ago antirez (redis) did something against memcached on November 12, the last commit .

Install Redis

Redis is insanely easy to install. He has no dependencies. You only need to do:

make ./redis-server redis.conf #start redis 

to compile redis (Awesome :)?).

Install memcached

Memcached has a dependency (libevent), which makes install difficult.

 wget http://memcached.org/latest tar -zxvf memcached-1.xxtar.gz cd memcached-1.xx ./configure make && make test sudo make install 

not entirely true, because memcached has a libevent dependency, and ./configure does not run libevent . But then again they are packages that are cool but require root to install.

Answer 2

Redis pretty fast: 110,000 sets / second

If speed is a concern, why use a network layer?

According to: http://tokutek.com/downloads/mysqluc-2010-fractal-trees.pdf

• InnoDB inserts ... 43,000 records per second AT ITS PEAK *;
• TokuDB Attachments ... 34,000 records per second AT ITS PEAK *;
• G-WAN KV inserts .... 100,000,000 records per second

(*) after several thousand inserts, performance is severely degraded for InnoDB and TokuDB, which end up writing to disk when their cache, system cache, and disk controller cache are full. See PDF for an interesting discussion of the problems caused by the InnoDB database index topology (which greatly disrupts locality, while the Fractals topology is much better ... but still not linear).

Answer 3

To clarify the answers in logical representations:

• Flat files run as fast as the media used (DISK or RAM)
• An environment that caches MRU elements in RAM (most used)
• The solution has an intelligent / fast hash index for all locations (what SQL systems are based on)

This combination will give you the best solution you are looking for.

For the argument as a whole, a flat file or not — except for the MEMORY ONLY solution — all engines use some form of flat file. The magic knows where your data is, and the setup reads to pull the data back most optimally. In the 80s at IBM, we used a fixed size flat record length file that was not optimized for disk space; it was optimized for I / O. Indexes were then based on Record Length * ROWID.

Now, when you need, your maximum productivity for scaling is the introduction of a smart combination - we accept more than 1 million companies with more than 10 pages per company - 10 million files, plus js, css and images.

Theory 1). You know that your limitation is RAM - the dynamic contents of the reel on the disc whenever possible, and discard features like hit counts. Use NGINX or configure APACHE highly (or, as we did, create our own web servers since 2001) - the whole concept is using RAM for YOURSELF USE and a very intelligent disk-based content search - usually URIs are ok.

Theory 2) - Trend analysis and user forecasting - I spent years studying and developing systems that track trends. If I know that the user will go along the path A, B, C, D - then when he gets to B, I have already pre-programmed C and D. If I know that the user will go A, B, but can go E, then D. You have the choice to pre-cache C and E or to pre-fetch RAM RAM and manually select C or E when the user selects this.

The web server that we developed along with some accounting systems that I have developed over the years integrates Theory 2 into the prefetch with Smart Caching combinations. We also save the contents to disk in deflation mode - therefore, the transport layer simply pushes the contents onto the stack, since 99% of browsers support deflated streams. (It’s faster to rephrase 1% before sending than to deflate 99% of the time)

According to MEMCACHED and SWAP - Disk speed is your enemy, however, linking the kernel to managing this enemy is an epic failure! If you want to outperform MEMCACHED performance, learn how to tune RAM DISK and save your deflated HOT requested items!

** DISCLAIMER: All this assumes that you have enough bandwidth, that your Infrastructure / Users bandwidth is not your bottleneck, but your servers. @ 3FINC

Answer 4

http://memcached.org/ + http://php.net/manual/en/book.memcache.php

Answer 5

Flat files are technically the fastest, but if you are looking for something with a PHP interface and just screams, look at postgres.

http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL#Raw_Speed

For memory caching, see memcached

http://memcached.org/

* Edit: from your edit ... (redundant yes) ... if you cache this volume in memory, you will have problems. Look at postgres columnar table queries or a quasi-standard flat file solution.

Answer 6

As far as I know, using the file system is actually the fastest way to cache visualized templates without resorting to storing them in memory. Any database will simply add overhead and make it all slower compared.

Answer 7

I would use memcached or APC . Depending on whether caching between servers is required. Memcached is the daemon you are connecting to, where APC is actually located inside the PHP instance (a little faster). Both of them store a cache in memory, so that it quickly flashes.

Answer 8

In fact, storing the cache in files is the fastest way to do this. But if you are really interested in placing them in a database, you can check out MongoDB . MongoDB is a document-oriented database, therefore there are no server-side joins, therefore it is faster than mysql (1. with php 2. there are many tests on the Internet).