Core DataNameKeyPath section with link attribute performance issue

Question

I have a master data model with three objects:
Person , Group , Photo with the relationship between them as follows:

• Person <-----------> Group (ratio from one to a large number)
• Face <-------------> Photo (one to one)

When I fetch using the NSFetchedResultsController in a UITableView , I want to group Person objects using the Group entity name attribute.

For this, I use sectionNameKeyPath:@"group.name" .

The problem is that when I use the attribute from the Group relation, the NSFetchedResultsController retrieves everything in advance in small batches of 20 (I have setFetchBatchSize: 20 ) instead of fetching batches while I scroll through the tableView .

If I use an attribute from a Person object (for example, sectionNameKeyPath:@"name" ) to create partitions, everything works fine: NSFetchResultsController loads small batches of 20 objects when scrolling.

The code I use to create the NSFetchedResultsController is:

 - (NSFetchedResultsController *)fetchedResultsController { if (_fetchedResultsController) { return _fetchedResultsController; } NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init]; NSEntityDescription *entity = [NSEntityDescription entityForName:[Person description] inManagedObjectContext:self.managedObjectContext]; [fetchRequest setEntity:entity]; // Specify how the fetched objects should be sorted NSSortDescriptor *groupSortDescriptor = [[NSSortDescriptor alloc] initWithKey:@"group.name" ascending:YES]; NSSortDescriptor *personSortDescriptor = [[NSSortDescriptor alloc] initWithKey:@"birthName" ascending:YES selector:@selector(localizedStandardCompare:)]; [fetchRequest setSortDescriptors:[NSArray arrayWithObjects:groupSortDescriptor, personSortDescriptor, nil]]; [fetchRequest setRelationshipKeyPathsForPrefetching:@[@"group", @"photo"]]; [fetchRequest setFetchBatchSize:20]; NSError *error = nil; NSArray *fetchedObjects = [self.managedObjectContext executeFetchRequest:fetchRequest error:&error]; if (fetchedObjects == nil) { NSLog(@"Error Fetching: %@", error); } _fetchedResultsController = [[NSFetchedResultsController alloc] initWithFetchRequest:fetchRequest managedObjectContext:self.managedObjectContext sectionNameKeyPath:@"group.name" cacheName:@"masterCache"]; _fetchedResultsController.delegate = self; return _fetchedResultsController; } 

This is what I get in the Tools if I create sections based on "group.name" without any interaction with the user interface of the application:

And this is what I get (with a bit of scrolling over the UITableView) if sectionNameKeyPath is zero:

Please help me in this matter?

EDIT 1:

It seems that I am getting inconsistent results from the simulator and the tools: when I asked this question, the application started in the simulator after about 10 seconds (according to Time Profiler) using the above code.

But today, using the same code as above, the application runs in the simulator for 900 ms, even if it does a temporary prefetch for all objects and does not block the user interface.

I added some fresh screenshots:

EDIT 2: I reset the simulator and the results are intriguing: after the import operation and exit from the application, the first launch looked like this: After a bit of scrolling: Now this is what happens in the second run: After the fifth run:

EDIT 3: Running the application for the seventh time and eight times, I get the following:

Answer 1

Almost a year after I posted this question, I finally found the culprits that allow this behavior (which has changed a bit in Xcode 6):

• As for the inconsistent sampling time: I used the cache while I was back and forth with the simulator opening, closing, and resetting.

• As for the fact that everything was pre-selected in small batches without scrolling (in Xcode 6 Core Data Instruments, this is no longer the case), now this is one, large sample that takes whole seconds):

It seems that setFetchBatchSize is not working properly with parent/child contexts . The question was published back in 2012, and it seems that it is still there http://openradar.appspot.com/11235622 .

To solve this problem, I created another independent context with NSMainQueueConcurrencyType and set its persistence coordinator be the same as my other contexts .

Read more about issue # 2 here: fooobar.com/questions/313016 / ...

Answer 2

This is your stated goal: “I need Person objects that need to be grouped by section using a group of relationship objects, a name attribute, and NSFetchResultsController to select in small batches as they scroll, rather than up as they are now.”

The answer is a bit complicated, primarily because of how NSFetchedResultsController sections NSFetchedResultsController built, and how this affects the behavior of the selection.

TL DR; To change this behavior, you will need to change the way NSFetchedResultsController partitions are created.

What's happening?

When an NSFetchedResultsController given a NSFetchedResultsController query (fetchLimit and / or fetchBatchSize), several things happen.

If sectionNameKeyPath not specified, it does exactly what you expect. The fetch function returns an array of proxy results with "real" objects for the first fetchBathSize element. For example, if you have setFetchBatchSize up to 2, and your predicate corresponds to 10 items in the repository, the results contain the first two objects. Other objects will be retrieved separately as they become available. This provides a smooth playback of paginated answers.

However, when the sectionNameKeyPath parameter is sectionNameKeyPath , the resulting result controller should do a little more. To compute sections, he needs to access this key path for all objects in the results. He lists 10 points in the results in our example. The first two have already been received. The remaining 8 will be selected during the listing to get the key value needed to create the section information. If you have many results for your sample query, this can be very inefficient. There are a number of public errors regarding this feature:

NSFetchedResultsController initially takes too long to configure partitions

NSFetchedResultsController ignores the fetchLimit property

NSFetchedResultsController, table index and buyback performance

... and a few more. When you think about it, it makes sense. Creating NSFetchedResultsSectionInfo objects requires that the recipient of the results display each value in the results for sectionNameKeyPath , combine them with a unique combination of values ​​and use this information to create the correct number of NSFetchedResultsSectionInfo objects, set the name and title of the index, know how many objects the section contains in the results, etc. .d. There is no such way to handle the general use case. With that in mind, your tool tracks may make more sense.

How can you change that?

You can create your own NSFetchedResultsController , which provides an alternative strategy for creating NSFetchedResultsSectionInf o objects, but you may run into some of the same problems. For example, if you use the existing fetchedObjects functions to access the elements of the selection results, you will encounter the same behavior when accessing objects that are errors. For your implementation, you will need a strategy to solve this problem (it is possible, but it depends on your needs and requirements).

Oh god no. What about a temporary hack that just makes it work a little better, but doesn't fix the problem?

Changing the data model will not change the behavior described above, but may slightly affect performance. Batch updates will not have a significant impact on this behavior, and in fact will not play well with the selected result controller. However, it might be much more useful for you to set relationshipKeyPathsForPrefetching instead to include your "group" relationships, which can greatly improve crash behavior and behavior. Another strategy may be to perform a different sample to batch crash these objects before trying to use the selected result controller, which will more efficiently populate the various levels of Core Data data caches in memory.

The NSFetchedResultsController cache is primarily for section information. This prevents the need for a full recount of sections at each change (at best), but it can actually make the initial sample for assembly of sections occupied much longer. You will need to experiment to make sure the cache is worth your use.

If your main problem is that these operations with the main data block user interaction , you can unload them from the main stream. NSFetchedResultsController can be used in the context of a private queue (background) , which will prevent the locking of user interfaces of Core Data operations.

Answer 3

Based on my experience, the way to achieve your goal is to denormalize your model. In particular, you can add the group attribute to your Person object and use this attribute as sectionNameKeyPath . So, by creating Person , you must also pass the group to which it belongs.

This denormalization process is correct because it avoids fetching related group objects, since this is not necessary. The disadvantage may be that if you change the name of the group, all persons associated with this name must change, on the contrary, you may have incorrect values.

A key aspect here is the following. You should keep in mind that Core Data is not a relational database. The model should not be developed as a database scheme where normalization can be performed, but it should be developed in terms of the presentation and use of data in the user interface.

Change 1

I can’t understand your comment, could you explain better?

What I found very intriguing is that even if the application is doing a full prefetch in the simulator, the application loads 900ms (with 5000 objects) on the device, despite the simulator, where it loads a lot slower.

In any case, I would be interested to know more about your Photo organization. If you pre-extract the photo, overall performance may be affected.

Do you need to pre-select a Photo in your table view? Are they thumbs (small photos)? Or ordinary images? Are you using an external storage flag ?

Adding an additional attribute (e.g. group ) to the Person object cannot be a problem. Updating the value of this attribute when changing the name object of the group object is not a problem if you run it in the background. In addition, starting with iOS 8, you have a batch update as described in Batch Database Updates .