Truncated Kernel NSData Objects

Question

I store arrays of doubles in an NSData * object, which is saved as a binary property in the Core Data (SQLite) data model. I do this to store selected data for graphical display in an iPhone application. Sometimes when a binary contains more than 300 doubles, not all duplicates are stored on disk. When I leave and restart my application, there can be no more than 25 data points that have been saved or up to 300.

Using NSSQLitePragmasOption with synchronous = FULL, and that can make a difference. It’s hard to say because the error is intermittent.

Given the warnings about performance issues resulting from using synchronous = FULL, I'm looking for tips and pointers.

Thanks.

[[ Edit: here is the code. ]]

The goal (still unrealized) of -addToCache: is to add every new binding to the cache, but periodically update the data object with an error (error?).

From Data.m

 @dynamic dataSet;  // NSData * attribute of Data entity

  - (void) addDatum: (double_t) datum
     {
     DLog (@ "- [Data addDatum:% f]", datum);
     [self addToCache: datum];
     }

 - (void) addToCache: (double_t) datum
     {
     if (cache == nil)
         {
         cache = [NSMutableData dataWithData: [self dataSet]];
         [cache retain];
         }
     [cache appendBytes: & datum length: sizeof (double_t)];
     DLog (@ "- [Data addToCache:% f] ... [cache length] =% d; cache =% p", datum, [cache length], cache);
     [self flushCache];
     }

 - (void) wrapup
     {
     DLog (@ "- [Data wrapup]");
     [self flushCache];
     [cache release];
     cache = nil;
     DLog (@ "[self isFault] =% @", [self isFault]? @ "YES": @ "NO");  // [self isFault] is always NO.
     }

 - (void) flushCache
     {
     DLog (@ "flushing cache to store");
     [self setDataSet: cache];
     DLog (@ "- [Data flushCache:] [[self dataSet] length] =% d", [[self dataSet] length]);
     }

 - (double *) bytes
     {
     return (double *) [[self dataSet] bytes];
     }

 - (NSInteger) count
     {
     return [[self dataSet] length] / sizeof (double);
     }

 - (void) dump
     {
     ALog (@ "Dump Data");
     NSInteger numDataPoints = [self count];
     double * data = (double *) [self bytes];
     ALog (@ "numDataPoints =% d", numDataPoints);
     for (int i = 0; i 

Answer 1

I tried to get the behavior as if my Core Data object could have an NSMutableData attribute. For this, my NSManagedObject (called Data) had the NSData attribute and the ISMutableData ivar attribute. My application takes sample data from the sensor and adds each data point to the dataset - that’s why I need this design.

NSMutableData was added to each new data point, and then the NSData attribute was set to NSMutableData.

I suspect that since the NSData pointer did not change (although its contents were), Core Data did not evaluate the amount of changes. Calling -hasChanged on NSManagedObjectContext showed that changes had occurred, and calling -updatedObjects even listed the Data object as changed. But the evidence that was written seems to have been truncated (sometimes).

To get around this, I changed the situation a bit. New data points are still added to NSMutableData, but the NSData attribute is set only after the selection is complete. This means that it is likely that a failure can lead to truncated data - but, for the most part, this work seems to have solved the problem.

Caution emptor: the error has always been intermittent, so it is possible that it still exists, but is simply harder to manifest.