A simple C # application creates a huge number of page errors. What for?

You have a simple C # console application that imports text data into SQL.

It takes about 300K in memory and 80% in CPU. 2 GB of RAM is available at any time, but at the same time, a page error shows 500 thousand.

The application is 32-bit, and the OS is either W2000 or 32-bit XP, and .NET 3.5

Anyone can explain what could be the problem, and how can I investigate this further?

EDIT: Now I'm sure the page errors are related to disk I / O (read). I commented on a part of SQL, and reading a blank disk generates only that number.

EDIT2: On average 200 crashes / sec and 4000 soft crashes / sec.

I wonder if there will be a similar appearance on W2008

+4
source share
4 answers

First, how do you measure the memory that the application uses? If you look at the "working set", then only the part that is in physical memory. You should also take a look at “Virtual Machine Size” (or “Commit Size”), where the actual virtual memory is occupying your process.

If the Windows kernel settings manager believes that your application is inactive or needs to be left to give other processes more power, it may decide to reduce the size of the working set. If the size of the working set is smaller than what your application should work on, you can easily see many page errors because it is just a race between the balance manager and the application. Typically, the balance manager monitors memory usage and can also increase the size of the working set accordingly. However, this can be prevented in certain circumstances, such as low physical free memory, high I / O (cache stress in physical memory), low process priority, application background / foreground state, etc.

It may just be the behavior of the .NET garbage collector due to the fact that a huge number of small blocks of memory get allocated and allocated in a very short time, causing a load on the allocation and freeing of memory. "VM size" can remain roughly the same size, but behind the scenes it can constantly allocate / free memory, causing continuous page errors.

It is also known that the DLLs used by this process also take into account process statistics. Not your application, but only one of the COM or .NET library you are using can cause this behavior. You can infer the actual culprit by changing the behavior of your application (for example, by deleting the database access code and leaving only the object allocation code behind) to see which component is really causing interruptions.

EDITOR: About your question about the effect of GC on kinked memory: the CLR actually dynamically grows the heap and brings memory back to the OS as needed . This does not happen synchronously. The GC works behind the scenes and frees up memory in large chunks to prevent application hindering. Say you select many small objects and release them almost immediately. This causes many links to remain in the memory for a moment before being released. It is easy to imagine that it becomes like a head between a garbage collector and a memory allocation code. While the GC ultimately catches up, the required new memory must be satisfied from the "new memory", not the old one, because the old one has not yet been freed. Since the actual memory we are working on situations around, the balance manager may not think about giving our process more memory, because we are on the verge of always around the same physical amount of memory, but we constantly do not need “newly allocated memory” "more memory", therefore page errors.

+5
source

Page errors are normal. The memory is unloaded, and when you access it, this causes the page to crash, and the system returns it back. This is by design.

I have an application running on my machine right now with 500 million page errors. Nothing to worry about!

+4
source

Page Errors Mean Memory Problems

Consider increasing memory if you have excessive page errors.

You have a large working set size.

A working set is a set of pages of memory loaded into RAM. This is measured by the Process \ Working Set. A high value may indicate that you have downloaded multiple assemblies.

The Process \ Working Set does not have a specific threshold for viewing, although a high or fluctuating value may indicate a lack of memory. A high or fluctuating value, accompanied by a high page error rate, clearly indicates that there is not enough memory on your server.

Further reading:

Check the memory in the System Resources section of the following MSDN article: http://msdn.microsoft.com/en-us/library/ff647791.aspx#scalenetchapt15_topic9

Please indicate the code to investigate.

0
source

I am testing the possible answer to this question in my application. Break your work set into smaller pieces and work with pieces.

For example, I have a large list of objects (9000-30000). If I break this list into a piece of 500 or so, it should support 500 objects in memory while I work them.

You need to increase or decrease the size of your piece until you can work with it fast enough for the OS to save it in memory. This is a theory that I have not yet fully tested. But it should work.

0
source

Source: https://habr.com/ru/post/1340623/


All Articles