Delphi: FastMM virtual memory management link?

Question

I recently had a problem (see my last question) that made me take a closer look at memory management in a Delphi application. After my first research, I have two questions.

I started playing with FastMMUsageTracker and noticed the following. When I open the file that will be used by the application (which also creates a form, etc.), there is a significant discrepancy between the variation of the available virtual memory for the application and the change in the "FastMM4" memory.

Firstly, I am a bit confused by the terminology: why is there some kind of memory allocated by FastMM and some “system-reserved” (and reserved) memory? Since FastMM is a memory manager, why is the system responsible for allocating some memory?

Also, how can I get more detailed information about which objects / structures were allocated by this memory? The VM graph is useful only when displaying the amount of memory that is "distributed across the system", "reserved system" or "allocated FastMM", but there is no reference to the actual objects that require this memory. Is it possible, for example, to get a report in the middle of execution, similar to what FastMM creates when the application is closed? FastMM obviously stores this information somewhere.

As a bonus for me, if people can recommend a good link (book, website) on this issue, it will also be very appreciated. There are tons of information on the net, but it is usually very specific for specific cases and expert-oriented.

Thanks!

PS: This is not about finding leaks, no problem, just trying to better understand memory management and be proactive in the future, as our application uses more and more memory.

Answer 1

Some of your questions are easy. Well, anyway, one of them!

Why is there some FastMM-allocated memory and some "system" (and reserved) memory? Since FastMM is a memory manager, why is the system responsible for allocating some memory?

The code you write in Delphi is only part of what is being done in your process. You use third-party libraries as a DLL, primarily the Windows API. Each time you create a Delphi form, for example, there are many windows behind it that consume memory. This memory is not allocated by FastMM, and I suppose this is what is called "system" in your question.

However, if you want to go deeper, this very quickly becomes an extremely complex topic. If you want to delve deeper into the implementation of Windows memory management, I think you need to consult a serious reference source. I offer Windows Internals from Mark Russinovich, David Solomon and Alex Ionescu.

Answer 2

Firstly, I am a bit confused by the terminology: why is there some kind of memory allocated by FastMM and some “system-reserved” (and reserved) memory? Since FastMM is a memory manager, why is the system responsible for allocating some memory?

Where do you think FastMM gets memory allocation? Of course, this comes from the system.

When the application starts, FastMM receives a block of memory from the system. When you request the use of some memory (using GetMem, New, or TSomething.Create), FastMM tries to pass it to you from this first start block. If this is not enough, FastMM requests more (in one block, if possible) from the system, and returns part of this to you. When you free something, FastMM does not return this memory to the OS, because it shows that you will use it again. It just means that it is not used internally. He is also trying to rebuild unused blocks so that they are as contiguous as possible, so as to try not to return to the OS for more unnecessary use. (However, this reorganization is not always possible, and in the case when you finish the fragmentation of memory with such things as multiple resizing of dynamic arrays, many objects create and free, etc.)

In addition to the memory that FastMM manages in your application, the system allocates space for the stack and heap. Each process receives a mega stack of space when it starts, as a place to place variables. This stack (and heap) can dynamically grow as needed.

When your application shuts down, all of the allocated memory is returned to the OS. (Perhaps this is not so immediately in the task manager, but it is.)

Is it possible, for example, to get a report, average execution, similar to what FastMM generates when the application closes?

As far as I can judge. Since FastMM stores it somewhere, this does not necessarily mean that there is access to it at runtime from outside the memory manager. You can look at the source of FastMMUsageTracker to find out how the information is retrieved (using GetMemoryManagerState and GetMemoryMap in the RefreshSnapshot method). A source is also available for FastMM4; You can see which public methods are available.

FastMM’s own documentation (in the form of readme files, FastMMOOptions.inc comments and FastMM4_FAQ.txt file) is somewhat useful in explaining how it works and what debugging options (and information) are available.

Answer 3

For the detailed memory card that the process uses, try VMMAP from www.sysinternals.com (also co-authored with Mark Russinovich, mentioned in David's answer). It also allows you to see what is stored in some places (type control-T when selecting a detailed line).

Caution: your process has a lot more memory than you think. You may need to read the book first.