Sphinx performance after downtime

Question

I am writing a web application for which I use Sphinx to search for about a million documents.

The performance is excellent: a typical request takes only 0.05 seconds, but if the requests are not executed for several hours, then for several requests this takes much longer - up to 1000 times longer, and then returns to normal. The query log is as follows:

[Wed Mar 7 17:23:55.937 2012] 0.221 sec [Wed Mar 7 17:32:00.726 2012] 0.012 sec [Wed Mar 7 17:32:00.984 2012] 0.052 sec [Wed Mar 7 17:32:01.416 2012] 0.222 sec [Thu Mar 8 09:15:10.418 2012] 10.147 sec [Thu Mar 8 09:16:00.560 2012] 48.262 sec [Thu Mar 8 09:16:55.429 2012] 54.153 sec [Thu Mar 8 09:17:54.454 2012] 0.012 sec [Thu Mar 8 09:17:54.713 2012] 0.052 sec [Thu Mar 8 09:17:55.141 2012] 0.218 sec 

I assume that my busy server changes Sphinx memory when not in use or something like that.

How can i solve this?

I reviewed scripted fake requests every minute, but this seems like a pretty ugly hack.

Answer 1

How fast are the drives on this server?

I assume that this is due to the need to return to disk for indexes. After access, the OS will cache files several times.

It might be worth considering an SSD. A small SSD - large enough for sphinx indices - is relativistically cheap these days.

If this is really a memory replacement, it also suggests that you have slow drives. But this is also something to solve. Can you add more memory to the server? (or even place the swap partition on a newly installed SSD :)

btw, find out if there is an exchange with something like Munin (or Cacti, etc.)