Entity data query and memory leak

Question

I load a lot of data into a loop, but after some operations I delete them, but I see that the memory allocation is growing very quickly, a few seconds and 1 GB, so how can I clear it after each iteration?

using (var contex = new DB) { var inputs = contex.AIMRInputs.Where(x => x.Input_Type == 1); foreach (var input in inputs) { var data = contex.Values.Where(x => x.InputID == input.InputID).OrderBy(x => x.TimeStamp).ToList(); if (data.Count == 0) continue; foreach (var value in data) { Console.WriteLine(Value.property); } data.Clear(); } } 

Answer 1

The first thing you can do is turn off change tracking because you are not changing any data in your code. This prevents attachment of loaded objects to the context:

For DbContext (EF> = 4.1):

 var inputs = contex.AIMRInputs.AsNoTracking() .Where(x => x.Input_Type == 1); 

and

 var data = contex.Values.AsNoTracking() .Where(x => x.InputID == input.InputID) .OrderBy(x => x.TimeStamp) .ToList(); 

Edit

For EF 4.0, you can leave your queries as they are, but add the following as the first two lines in the using block:

 contex.AIMRInputs.MergeOption = MergeOption.NoTracking; contex.Values.MergeOption = MergeOption.NoTracking; 

This disables change tracking for the ObjectContext .

Edit 2

... especially referring to @James Reategui's comment below that AsNoTracking reduces memory footprint:

This is often true (for example, in the model / query of this question), but not always! In fact, using AsNoTracking can be counterproductive with respect to memory usage.

What does AsNoTracking do when objects materialize in memory?

• First: it does not bind the object to the context and therefore does not create entries in the context state manager. These entries consume memory. When using POCOs, records contain a snapshot of the property values ​​of the object when it was first loaded / attached to the context - basically a copy of all the (scalar) properties in addition to the object itself. Thus, the total memory is (approximately) twice as large as the size of the object when AsNoTracking not applied.

• Second: On the other hand, when entities are not context bound, EF cannot take advantage of matching identities between key values ​​and object reference identifiers. This means that objects with the same key will be implemented several times, which will lead to additional memory, but without using AsNoTracking EF ensures that the object materializes only once per key value.

The second point becomes especially important when related objects are loaded. A simple example:

Let's say we have an Order and Customer object, and an order has one Order.Customer customer. Let's say that the Order object is 10 bytes in size, and the Customer object is 20 bytes in size. Now we run this query:

 var orderList = context.Orders .Include(o => o.Customer).Take(3).ToList(); 

And let all 3 loaded orders have the same customer. Since we have not disabled EF tracking, we materialize:

• 3 order objects = 3x10 = 30 bytes
• 1 customer object = 1x20 = 20 bytes (because the context recognizes that the customer is the same for all 3 orders, he only materializes one customer object)
• 3 snapshots with initial values ​​= 3x10 = 30 bytes
• 1 client snapshot record with initial values ​​= 1x20 = 20 bytes

Amount: 100 bytes

(For simplicity, I assume that context records with copied property values ​​are the same size as the entities themselves.)

Now we run the request with change tracking disabled:

 var orderList = context.Orders.AsNoTracking() .Include(o => o.Customer).Take(3).ToList(); 

Materialized data:

• 3 order objects = 3x10 = 30 bytes
• 3 (!) Client objects = 3x20 = 60 bytes (no matching identifiers = several objects per key, all three client objects will have the same property values, but they still remain three objects in memory)
• No snapshot entries.

Amount: 90 bytes

So, using AsNoTracking , in this case the request consumed 10 bytes less memory.

Now, the same calculation with 5 orders ( Take(5) ), again all orders have the same customer:

Without AsNoTracking :

• 5 orders of objects = 5x10 = 50 bytes
• 1 client object = 1x20 = 20 bytes
• 5 snapshot records with initial values ​​= 5x10 = 50 bytes
• 1 client snapshot record with initial values ​​= 1x20 = 20 bytes

Amount: 140 bytes

With AsNoTracking :

• 5 orders of objects = 5x10 = 50 bytes
• 5 (!) Client objects = 5x20 = 100 bytes
• No snapshot entries.

Amount: 150 bytes

This time using AsNoTracking was 10 bytes more expensive.

The numbers above are very rough, but somewhere break-even point, where using AsNoTracking may require more memory.

The difference in memory consumption between using AsNoTracking or not depends heavily on the request, relationships in the model and the specific data loaded by the request. For example: AsNoTracking it will always be better to use memory when the orders in the example above all (or mostly) have different customers.

Conclusion: AsNoTracking is primarily a means of improving query performance, rather than memory usage. In many cases, it also consumes less memory. But don't be surprised if a particular request requires more memory with AsNoTracking . In the end, you should measure the amount of memory for a solid decision in favor of or against AsNoTracking .

Answer 2

Part if the problem here may be regarding the DataContext. Many of them cache information or store additional information as requests are made, and therefore its memory area will increase over time. At first I would check with the profiler, but if this is your problem, you may need to recreate a new datacontext after each X request (experiment with different X values ​​to see what works best).

I would also like to note that most people have a great memory. You must be sure that you are using more memory than is really acceptable before you begin to do these types of optimizations. The GC will also begin to flush memory more aggressively, as you have less free memory to work with. This does not bother premature optimization (and you shouldn't either).