Your observed performance issue on initial request is probably one of the following issues (in a rough order of probability):
1) Your application / web service has some overhead for initializing on the first request (i.e. allocating memory, setting up connection pools, resolving DNS, ...).
2) The requested indexes or data has not yet been stored in memory, so they must be loaded.
3) The query optimizer may take a little longer for the first request because it compares the execution of the plan for the query template.
It would be very useful to test the request using the mongo shell and isolate whether the service data is associated with MongoDB or your web service (and not with the synchronization of both, as you did).
The following are some notes related to MongoDB.
Caching
MongoDB does not have "caching" time for documents in memory. It uses memory mapped files for disk I / O, and documents in memory are based on your active requests (documents / indexes you recently downloaded), as well as available memory. The operating system virtual memory manager is responsible for caching and will usually follow the least used algorithm (LRU) to decide which pages will be replaced from memory.
Memory usage
The expected behavior is that over time, MongoDB will grow to use all free memory to store the active working dataset.
Looking at your provided db.stats() numbers (and assuming this is your only database), it looks like your database is about 1 GB in size, so you should be able to store everything in your total 10Gb if:
- there are other processes competing for memory
- you rebooted the
mongod server and these documents / indexes have not yet been requested.
In MongoDB 2.2, there is a new touch command that you can use to load indexes or documents into memory after restarting the server. This should be used only at initial start-up to “warm up” the server, since otherwise you could uselessly force the actual “active” data out of memory.
On a Linux system, for example, you can use the top command and you should see that:
- virtual bytes / VSIZE will be the size of the entire database
- If the server does not have other running processes, the resident bytes / RSIZE will be the shared memory of the machine (including the contents of the file system cache).
mongod should not use swap (since files are mapped to memory)
You can use the mongostat tool to get a quick view of mongod activity .. or it’s more useful to use a service like MMS to track metrics over time.
Query optimizer
MongoDB The query optimizer compares the execution of the plan for the query template every 1000 writes, and then caches the "winning" query plan until the next optimizer starts .. or you explicitly call explain() in this query.
This should be simple to check: run your request in the mongo shell using .explain() and look at ms timeouts, as well as the number of index entries and documents to check. Timing for explanation () is not the actual time it will take for the requests to complete, as it includes the costs of comparing plans. Typical execution will be much faster .. and you can search for slow queries in your mongod .
By default, MongoDB will log all requests more slowly than 100 ms, so this provides a good starting point for finding queries for optimization. You can adjust the slow ms value with the --slowms option or with the Database Profiler commands.
Further reading in the MongoDB documentation: