AVCaptureDeviceOutput does not call the captureOutput delegate method

Question

I am creating an iOS application (my first) that still processes the video on the fly. To dive into this, I followed suit from Apple's AV * documentation.

The process includes setting the input (camera) and output. The output works with a delegate, which in this case is the controller itself (it matches and implements the necessary method).

The problem I am facing is that the delegate method is never called. The code below is an implementation of the controller, and it has several NSLogs. I can see the message "running", but the "delegate method called" is never displayed.

This code is within a controller that implements the AVCaptureVideoDataOutputSampleBufferDelegate protocol.

- (void)viewDidLoad { [super viewDidLoad]; // Initialize AV session AVCaptureSession *session = [AVCaptureSession new]; if ([[UIDevice currentDevice] userInterfaceIdiom] == UIUserInterfaceIdiomPhone) [session setSessionPreset:AVCaptureSessionPreset640x480]; else [session setSessionPreset:AVCaptureSessionPresetPhoto]; // Initialize back camera input AVCaptureDevice *camera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo]; NSError *error = nil; AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:camera error:&error]; if( [session canAddInput:input] ){ [session addInput:input]; } // Initialize image output AVCaptureVideoDataOutput *output = [AVCaptureVideoDataOutput new]; NSDictionary *rgbOutputSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCMPixelFormat_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]; [output setVideoSettings:rgbOutputSettings]; [output setAlwaysDiscardsLateVideoFrames:YES]; // discard if the data output queue is blocked (as we process the still image) //[output addObserver:self forKeyPath:@"capturingStillImage" options:NSKeyValueObservingOptionNew context:@"AVCaptureStillImageIsCapturingStillImageContext"]; videoDataOutputQueue = dispatch_queue_create("VideoDataOutputQueue", DISPATCH_QUEUE_SERIAL); [output setSampleBufferDelegate:self queue:videoDataOutputQueue]; if( [session canAddOutput:output] ){ [session addOutput:output]; } [[output connectionWithMediaType:AVMediaTypeVideo] setEnabled:YES]; [session startRunning]; NSLog(@"started"); } - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { NSLog(@"delegate method called"); CGImageRef cgImage = [self imageFromSampleBuffer:sampleBuffer]; self.theImage.image = [UIImage imageWithCGImage: cgImage ]; CGImageRelease( cgImage ); } 

Note. I am creating iOS 5.0 as a target.

Edit:

I found a question that, although it asks for a solution to another problem, does exactly what my code should do. I copied the code from this question verbatim to an empty xcode application, added NSLogs to the captureOutput function, and did not receive the call. Is this a configuration problem? Is something missing?

Answer 1

Your session is a local variable. Its scope is limited to viewDidLoad . Since this is a new project, I can confidently say that you are using ARC. In this case, this object will not leak and, therefore, will continue to live as it would in the related question, rather, the compiler will ensure that the object is released before viewDidLoad .

Therefore, your session is not running because it no longer exists.

(aside: self.theImage.image = ... is unsafe, because it performs the UIKit action of the main queue, you probably want dispatch_async , which is before dispatch_get_main_queue() )

So, selective corrections:

 @implementation YourViewController { AVCaptureSession *session; } - (void)viewDidLoad { [super viewDidLoad]; // Initialize AV session session = [AVCaptureSession new]; if ([[UIDevice currentDevice] userInterfaceIdiom] == UIUserInterfaceIdiomPhone) [session setSessionPreset:AVCaptureSessionPreset640x480]; else /* ... etc ... */ } - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { NSLog(@"delegate method called"); CGImageRef cgImage = [self imageFromSampleBuffer:sampleBuffer]; dispatch_sync(dispatch_get_main_queue(), ^{ self.theImage.image = [UIImage imageWithCGImage: cgImage ]; CGImageRelease( cgImage ); }); } 

Most people advocate using underscores at the beginning of instance variable names now, but I skipped it for simplicity. You can use the Xcode built into the refactor tool to fix this after you confirm the diagnosis is correct.

I moved CGImageRelease inside the block sent to the main queue to ensure that its lifetime extends beyond its capture in UIImage . I can’t immediately find the documentation confirming that CoreFoundation objects have a service life that automatically extends when captured in a block.

Answer 2

I found another reason why the didOutputSampleBuffer delegate didOutputSampleBuffer cannot be called - saving to the file and receiving the output connections of the sample buffer are mutually exclusive. In other words, if your session already has AVCaptureMovieFileOutput and then you add AVCaptureVideoDataOutput , only the AVCaptureFileOutputRecordingDelegate delegate methods are AVCaptureFileOutputRecordingDelegate .

For reference only, I could not find anywhere in the AV Foundation design documentation an explicit description of this limitation, but Apple support confirmed this several years ago, as noted in this SO answer .

One way to solve the problem is to delete AVCaptureMovieFileOutput completely and manually write the recorded frames to a file in the didOutputSampleBuffer delegate didOutputSampleBuffer , along with processing custom buffers. You can find these two . Useful answers.

Answer 3

In my case, there is a problem because I call

 if ([_session canAddOutput:_videoDataOutput]) [_session addOutput:_videoDataOutput]; 

before i call

 [_session startRunning]; 

I'm just starting to call addOutput: after startRunning

Hope this helps someone.