How to analyze excessive memory consumption (PageFileUsage) in a Delphi application?

Question

This is a continuation of this question: What could explain the difference in memory usage reported by FastMM or GetProcessMemoryInfo?

My Delphi XE uses a very large amount of memory, which sometimes leads to an exception from memory. I'm trying to understand why and what causes the use of this memory, and while FastMM reports low memory usage, when I query TProcessMemoryCounters.PageFileUsage, I can clearly see that the application uses most of the memory.

I would like to understand what causes this problem, and I would like to advise how to deal with it:

• Is there any way to find out what is in this memory and where was it allocated?
• Is there any tool to track memory usage by line / procedure in a Delphi application?
• Any general recommendations on how to deal with such a problem?

EDIT 1 . Here are two screenshots of FastMMUsageTracker indicating that the system has allocated memory.

• Before starting the process:

• Upon completion of the process:

Legend: light red - highlighted by FastMM, and gray color highlighted by the system.

I would like to understand what makes the system use up so much memory. Perhaps, understanding what is contained in this memory, or which line of code or procedure really caused this selection.

EDIT 2 . I would prefer not to use the full version of AQTime for several reasons:

• I use several virtual machines for development, and their licensing system is PITA (I am already a registered user of TestComplete)
• The LITE version does not provide enough information, and I will not spend money without indicating that the FULL version will provide me with valuable information.

Any other suggestions?

Answer 1

Another problem might be heap fragmentation. This means that you have enough free space, but all the free blocks are small. You can see it visually using the original version of FastMM and use FastMMUsageTracker.pas as suggested.

Answer 2

You need a profiler, but even this will not be enough in many places and cases. In addition, in your case, you will need a full-featured AQTime, rather than the lite version that comes with Delphi XE and XE2. (AQTIME is extremely expensive and annoyingly node-locked, so don't think I'm a smartbear shill.)

The fact is that people often mistakenly accept the AQTime Allocation Profiler as the only way to find leaks. He can also tell you where your memory goes, at least within the instrument. While working and using a large amount of memory, I click Run → Get Results.

Here is one of my applications - this is a profile in AQTime with its distribution profile, showing which class allocates the number of instances on the heap and how much memory they use. Since you are reporting low usage of the Delphi heap using FastMM, this suggests that most of AQTime's capabilities for parsing by the name of the delphi class will also be useless to you. However, using AQTime events and triggers, you can figure out which areas of your application are causing "memory usage", and when it happens, what are the costs. Real-time AQTime may be enough to help you narrow down the cause, even if it cannot find for you which function call causes the most memory usage.

Column names include an "Object Name", which includes such things: * All delphi classes, as well as their instance counter and heap usage. * Virtual memory blocks allocated using Win32 calls.

It can detect the allocations of the Delphi and C / C ++ libraries on the heap and can see certain memory allocations of the Windows-API level.

Pay attention to the live number of objects, the amount of memory from the heap used.

Usually I try to determine the cost of memory for a specific operation, measuring the memory usage of the heap before and immediately after some expensive operation, but before clearing (freeing) the memory from this expensive operation. I can set event points inside AQTime, and when any particular method hits or the flag is turned on by me, I can measure it before and after the values, and then compare them.

Only FastMM cannot even determine the non-delphi distribution or heap distribution that is not managed by FastMM. Therefore, AQTime is not limited.