- copyPixelBufferForItemTime: itemTimeForDisplay: null

Question

The problem is when I compile my application using iOS9 sdk, when my application tries to get CVPixelBufferReffrom AVPlayerItemVideoOutputusing the function - copyPixelBufferForItemTime:itemTimeForDisplay:, from time to time I get an empty value when the video and all instances are created.

with iOS 8, my application worked fine, but with iOS9 I give me this problem, even the version of my application located in the downloadable app store that was compiled with the iOS 8 SDK gives me the same problem when installed in IOS9.

When a problem occurs and I get null getCVPixelBufferRefif I press the home button and the application goes into the background, when I open the application again and becomes active, the instance AVPlayerItemVideoOutputthat gave me null CVPixelBufferRefstarts working fine, and the problem is solved.

Here is a youtube video where I am replicating a problem:

https://www.youtube.com/watch?v=997zG08_DMM&feature=youtu.be

Here is a sample code to instantiate all elements:

NSURL *url ;
url = [[NSURL alloc] initFileURLWithPath:[_mainVideo objectForKey:@"file"]];

NSDictionary *pixBuffAttributes = @{(id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange)};
_videoOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes];
_myVideoOutputQueue = dispatch_queue_create("myVideoOutputQueue", DISPATCH_QUEUE_SERIAL);
[_videoOutput setDelegate:self queue:_myVideoOutputQueue];

_player = [[AVPlayer alloc] init];


// Do not take mute button into account
NSError *error = nil;
BOOL success = [[AVAudioSession sharedInstance]
                setCategory:AVAudioSessionCategoryPlayback
                error:&error];
if (!success) {
   // NSLog(@"Could not use AVAudioSessionCategoryPlayback", nil);
}

asset = [AVURLAsset URLAssetWithURL:url options:nil];


if(![[NSFileManager defaultManager] fileExistsAtPath:[[asset URL] path]]) {
   // NSLog(@"file does not exist");
}

NSArray *requestedKeys = [NSArray arrayWithObjects:kTracksKey, kPlayableKey, nil];

[asset loadValuesAsynchronouslyForKeys:requestedKeys completionHandler:^{

    dispatch_async( dispatch_get_main_queue(),
                   ^{
                       /* Make sure that the value of each key has loaded successfully. */
                       for (NSString *thisKey in requestedKeys)
                       {
                           NSError *error = nil;
                           AVKeyValueStatus keyStatus = [asset statusOfValueForKey:thisKey error:&error];
                           if (keyStatus == AVKeyValueStatusFailed)
                           {
                               [self assetFailedToPrepareForPlayback:error];
                               return;
                           }
                       }

                       NSError* error = nil;
                       AVKeyValueStatus status = [asset statusOfValueForKey:kTracksKey error:&error];
                       if (status == AVKeyValueStatusLoaded)
                       {
                           //_playerItem = [AVPlayerItem playerItemWithAsset:asset];


                           [_playerItem addOutput:_videoOutput];
                           [_player replaceCurrentItemWithPlayerItem:_playerItem];
                           [_videoOutput requestNotificationOfMediaDataChangeWithAdvanceInterval:ONE_FRAME_DURATION];

                           /* When the player item has played to its end time we'll toggle
                            the movie controller Pause button to be the Play button */
                           [[NSNotificationCenter defaultCenter] addObserver:self
                                                                    selector:@selector(playerItemDidReachEnd:)
                                                                        name:AVPlayerItemDidPlayToEndTimeNotification
                                                                      object:_playerItem];

                           seekToZeroBeforePlay = NO;

                           [_playerItem addObserver:self
                                         forKeyPath:kStatusKey
                                            options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew
                                            context:AVPlayerDemoPlaybackViewControllerStatusObservationContext];

                           [_player addObserver:self
                                     forKeyPath:kCurrentItemKey
                                        options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew
                                        context:AVPlayerDemoPlaybackViewControllerCurrentItemObservationContext];

                           [_player addObserver:self
                                     forKeyPath:kRateKey
                                        options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew
                                        context:AVPlayerDemoPlaybackViewControllerRateObservationContext];


                           [self initScrubberTimer];

                           [self syncScrubber];


                       }
                       else
                       {
                         //  NSLog(@"%@ Failed to load the tracks.", self);
                       }
                   });
}];

Here is a code example giving me a zero pixel buffer

CVPixelBufferRef pixelBuffer =
[_videoOutput
 copyPixelBufferForItemTime:[_playerItem currentTime]
itemTimeForDisplay:nil];

NSLog(@"the pixel buffer is %@", pixelBuffer);
NSLog (@"the _videoOutput is %@", _videoOutput.description);
CMTime dataTime = [_playerItem currentTime];
//NSLog(@"the current time is %f", dataTime);
return pixelBuffer;

Answer 1

I had the same problem and found the answer in this thread: https://forums.developer.apple.com/thread/27589#128476

, , , . :

func retrievePixelBufferToDraw() -> CVPixelBuffer? {
  guard let videoItem = player.currentItem else { return nil }
  if videoOutput == nil || self.videoItem !== videoItem {
    videoItem.outputs.flatMap({ return $0 as? AVPlayerItemVideoOutput }).forEach {
      videoItem.remove($0)
    }
    if videoItem.status != AVPlayerItemStatus.readyToPlay {
      // see https://forums.developer.apple.com/thread/27589#128476
      return nil
    }

    let pixelBuffAttributes = [
      kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
      ] as [String: Any]

    let videoOutput = AVPlayerItemVideoOutput.init(pixelBufferAttributes: pixelBuffAttributes)
    videoItem.add(videoOutput)
    self.videoOutput = videoOutput
    self.videoItem = videoItem
  }
  guard let videoOutput = videoOutput else { return nil }

  let time = videoItem.currentTime()
  if !videoOutput.hasNewPixelBuffer(forItemTime: time) { return nil }
  return videoOutput.copyPixelBuffer(forItemTime: time, itemTimeForDisplay: nil)
}

Answer 2

, , : 64- ios 9.0 , arm64.

arm64 .

Answer 3

* , , , , .

: . , AVPlayerItemVideoOutput alloc , , , , . / . 1 AVPlayerItemVideoOutput , 1 .

hasNewPixelBufferForItemTime - , , .

//////////////////////
 if (g_TexturePointer)
{
    if([plug.playerOutput hasNewPixelBufferForItemTime:[plug.player currentTime]])
    {
        pbuffer = [plug.playerOutput copyPixelBufferForItemTime:plug.player.currentItem.currentTime itemTimeForDisplay:nil];
    } ... .. . (No need to show the rest.)

!

Answer 4

, , , ​​AVFoundation, , 1 , , - AVPlayer, . , , . , () , AVPlayer , , .

However, AVPlayerItemVideoOutput will not supply more pixel buffers after the player finishes playing. Therefore, you should probably protect yourself from this case, remembering whether you have already received pixel buffers. Otherwise, your player will play the loop randomly.

In the class interface:

@property (nonatomic) BOOL videoOutputHadPixelBuffer;

And then before trying to copy the pixel buffer:

if (![self.videoOutput hasNewPixelBufferForItemTime:self.player.currentTime] && !self.videoOutputHadPixelBuffer)
{
    [self restartPlayer]; // call your custom restart routine where you create a new AVPlayer object
}

self.videoOutputHadPixelBuffer = YES; // guard against missing pixel buffers after playback finished