What is the most efficient way to delete a large number of objects (10.000+) in Core Data?

Question

The way I try to remove multiple sets from 10.000+ NSManagedObjects is just too memory intensive (about 20 MB in real time) and my application is reset. Here is the implementation of the delete method:

+ (void)deleteRelatedEntitiesInManagedObjectContext:(NSManagedObjectContext *)context { NSFetchRequest *fetch = [[NSFetchRequest alloc] init]; [context setUndoManager:nil]; [fetch setEntity:[NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context]]; [fetch setIncludesPropertyValues:NO]; NSError *error = nil; NSArray *entities = [context executeFetchRequest:fetch error:&error]; NSInteger deletedCount = 0; for (NSManagedObject *item in entities) { [context deleteObject:item]; deletedCount++; if (deletedCount == 500) { [context save:&error]; deletedCount = 0; } } if (deletedCount != 0) { [context save:&error]; } } 

I tried: -setFetchBatchSize, but there was even more memory.

What would be a more memory efficient way to do this?

Answer 1

EDIT : just watched 2015 WWDC “What's New in Core Data” (it's always the first video I watch, but I'm very busy this year) and they announced a new API: NSBatchDeleteRequest , which should be much more efficient than any previous decision.

---

Efficiency has several meanings and most often means some kind of compromise. Here, I am assuming that you just want to contain memory on deletion.

Master data has many performance parameters that go beyond any single SO issue.

How memory is managed depends on the settings for the managed objects and the fetchRequest context. View documents to see all options. In particular, you should keep this in mind.

Also, keep in mind the performance aspect. This type of operation must be performed in a separate thread.

Also, note that the rest of your object graph will also come into play (due to the way CoreData handles the removal of related objects.

Regarding memory consumption, in MOC, in particular, pay attention to two properties. Although there are many, it is in no way close to comprehensive. If you want to see what happens, NSLog is your MOC before and after each save operation. In particular, log registeredObjects and deletedObjects.

• The MOC has a list of registered objects. By default, it does not save registered objects. However, if preservesRegisteredObjects is set to YES, it will retain all registered objects.

• For deletion, in particular, setPropagatesDeletesAtEndOfEvent tells the MOC how to handle related objects. If you want them to be processed with preservation, you need to set this value to NO. Otherwise, it will wait for the current event to complete.

• If you have really large objects, consider using fetchLimit. While errors do not take up a lot of memory, they still accept some, and many thousands are not insignificant at the same time. This means more sampling, but you will limit the amount of memory

• Also think that anytime you have large internal loops, you should use your own autostart pool.

• If this MOC has a parent, saving only moves these changes to the parent. In this case, if you have a parent MOC, you just make it grow.

To limit memory, consider this (not necessarily the best for your case - many Core Data options - only you know what is best for your situation, based on all the parameters that you use elsewhere.

I wrote a category in NSManagedObjectContext that I use to save when I want to make sure the save goes to a repository very similar to this. If you do not use the MOC hierarchy, you do not need it, but ... there is no reason NOT to use the hierarchy (if you are not tied to the old iOS).

 - (BOOL)cascadeSave:(NSError**)error { __block BOOL saveResult = YES; if ([self hasChanges]) { saveResult = [self save:error]; } if (saveResult && self.parentContext) { [self.parentContext performBlockAndWait:^{ saveResult = [self.parentContext cascadeSave:error]; }]; } return saveResult; } 

I changed your code a bit ...

 + (void)deleteRelatedEntitiesInManagedObjectContext:(NSManagedObjectContext *)context { NSFetchRequest *fetch = [[NSFetchRequest alloc] init]; [context setUndoManager:nil]; [fetch setEntity:[NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context]]; [fetch setIncludesPropertyValues:NO]; [fetch setFetchLimit:500]; NSError *error = nil; NSArray *entities = [context executeFetchRequest:fetch error:&error]; while ([entities count] > 0) { @autoreleasepool { for (NSManagedObject *item in entities) { [context deleteObject:item]; } if (![context cascadeSave:&error]) { // Handle error appropriately } } entities = [context executeFetchRequest:fetch error:&error]; } } 

Answer 2

At the time of inspiration, I deleted [fetch setIncludesPropertyValues:NO]; and that was good. From the docs:

During normal sampling (includePropertyValues ​​is YES), the Basic data retrieves the object identifier and property data for the corresponding records, fills the line cache with information, and returns the managed object as errors (see returnObjectsAsFaults). These errors are managed by objects, but all of their property data is still in the line cache until an error is generated. When the error is completed, Core Data retrieves data from the row cache - there is no need to return to the database.

I managed to reduce the allocated live bytes to ~ 13 MB, which is better.

Answer 3

NSBatchDeleteRequest Worked for me; reduced the time to delete managed objects by 5 times without a burst of memory.

Answer 4

This will be an interesting test, try Magical Record . There is a truncate method which should be very efficient (I used it on data sets of up to 3000 records without problems. It is interesting to see how it processes 10,000.

I would simply not use it only for this function, if you have not tried it, you need to. This makes working with Core Data much easier and with much less code.

Hope this helps.

Answer 5

I have not tested it in any way, but if memory is your main problem, you can try to encapsulate these batches of 500 deletions into an additional autorun pool. Perhaps this context: save creates quite a few auto-implemented objects that are not freed until you finish the loop of the loop of the loop. With 10,000+ entries, he could add quite well.

Answer 6

If you don't want to use a different API, try another NSFetchRequest function, fetchLimit , perhaps in combination with fetchOffset . I saw this on one of the iTunes U iPad courses in the example with a heavy number of crunches with Core Data.

 NSInteger limit = 500; NSInteger count = [context countForFetchRequest:fetch error:nil]; [fetch setFetchLimit:limit]; for (int i=0; i < count/limit+1; i++) { // do the fetch and delete and save } 

You can customize fetchLimit to meet your memory needs.