How much memory should the cache system use in Windows?

Question

I am developing a client / server application in which large pieces of data are stored on the server, such as large images or video files that are requested by the client, and I need to create a client cache system in memory to store several of these large data to speed up the process. To be clear, each individual image or video is not so big, but the total size of all of them can be really big.

But I ran into the “how much should I cache” problem and wondered if there were any golden rules in Windows about which strategy I should adopt. Caching is done on the client, I do not need to cache on the server.

• Should I stay at x% of global memory usage at all times? And how much will it be? What happens if another program starts up and takes up a lot of memory, should I clear the cache?
• Should I ask how much free memory is available before caching and use a fixed percentage of this memory for my needs?
• I hope I do not need to go there, but I have to ask the user how much memory he wants to allocate to my application? If so, how can I calculate the default value for this property and for those who will never use this parameter?

Answer 1

Instead of creating your own caching algorithms, why don't you write the data to a file with the FILE_ATTRIBUTE_TEMPORARY attribute and use your own client computer.

Although this approach means that you are using a file, if there is memory in the system, then the file will never leave the cache and remain in memory all the time.

Some advantages:

• You do not need to write code.
• The system cache takes into account all other running processes. It would be inappropriate for you to take it upon yourself.
• In 64-bit Windows, the system can use all available cache memory for it. In a 32-bit Delphi process, you are limited to 32-bit address space.
• Even if your cache is full and your files are flushed to disk, accessing the local disk is much faster than querying the database and then transferring files over the network.

Answer 2

It depends on what other software is running on the server. First, I can configure it manually. Design a system that can use a certain amount of memory. If possible, build it so that you can change this value while it is running.

If you have these features, you can try some settings to see what works best. I do not know any golden rules, but I would understand that you must set a percentage of the total memory or the total available memory with a certain minimum amount of memory in order to be free for the system at any time. If you save miminum, say 500 MB for the server OS, you can use the rest or 90% of the rest of your cache. But these numbers depend on the version of the OS and other applications running on the server.

I think it’s best to set up numbers from the outside and create a management tool that allows you to set the values ​​manually first. Then, if you find out what works best, you can subtract the formulas to calculate these values ​​and integrate them into your management tool. This tool should not be an integral part of the cache program itself (which is likely to be a service without a GUI).

Answer 3

A server application typically requires resources reserved for its own use by its administrator. I don’t care about the behavior of other users, I would like to be a “polite” application, so it should allow the size of the cache, etc. Configured by an administrator who is the only one who knows how to properly configure his systems (usually ...)

The default values ​​should in any case take into account how much memory is available in general, especially on 32-bit systems with less than 4 GB of memory (as long as Delphi provides only 32-bit applications) in order to leave something free for operating systems systems and avoids switching too often. It is also recommended that you request the user to select it during configuration.

If the application runs only on the server, a value between 40 and 75% of the available memory may be fine (depending on how much memory is required outside the cache), but ask the user again, since it is almost impossible to find out what others might need. applications. You can also have min cache size and maximum cache size, start by highlighting a smaller value, and then grow if necessary and reduce it if necessary.

On a 32-bit system, this is a kind of memory usage that can help use PAE / AWE to access more than 3 GB of memory.

Update : you can also control the monitoring / skipping of the cache and calculate which cache size will best suit the needs of the user (it may be too small but too large), as well as user recommendations about this.

Answer 4

Questions:

• Can a single image be requested by multiple clients? Or can one image be requested multiple times in a short amount of time?

• How short is the interval?

• Is the network speed really high? The higher the speed of the hard drive? If you have a normal network, the hard drive will be able to read files from the disk and deliver them over the network in real time. Especially since Windows already does good caching, so the most recent files are already in the cache.

• The main task of the computer on which the server application is running is to start the server? Or just a regular computer, also used for other tasks? In other words, is it a dedicated server or a regular workstation / desktop?

but I have to ask the user how much memory he wants to allocate to my application?

I would definitely go there !!! If the user believes that the server application is not an important application, it is likely to give it a low priority (low cache). In addition, he believes that this is the most important application for work, it will allow the application to allocate all the RAM it needs, to the detriment of other less important applications.

Just deliver the application with this parameter set by default to an acceptable value (which will be approximately x% of the total RAM). I will use as 70% of the total RAM if the main purpose of the computer is to store this server application and about 40-50% if its purpose is a public computer.

Answer 5

Honestly, the questions you ask will not be my main concern. I would be more worried about how efficient my cache is. If your files are really that big, how much can you cache? And if your client server application has many users, what are the chances that your cache will cache something that someone else will use?

It might be worth the analysis before you burn too much time on the subtle details.