Repeating scene elements in iOS YUV video output

Question

I capture video and process the resulting YUV frames. The output is as follows:

Although it is usually displayed on the phone screen. But my peer gets it like this img above. Each element is repeated and shifted by some value horizontally and vertically.

My captured video is 352x288, and my YPixelCount = 101376, UVPixelCount = YPIXELCOUNT / 4

Any hint to solve this or a starting point to understand how to handle YUV video clips on iOS?

NSNumber* recorderValue = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange]; [videoRecorderSession setSessionPreset:AVCaptureSessionPreset352x288]; 

And this is the captureOutput function

 - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection{ if(CMSampleBufferIsValid(sampleBuffer) && CMSampleBufferDataIsReady(sampleBuffer) && ([self isQueueStopped] == FALSE)) { CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); CVPixelBufferLockBaseAddress(imageBuffer,0); UInt8 *baseAddress[3] = {NULL,NULL,NULL}; uint8_t *yPlaneAddress = (uint8_t *)CVPixelBufferGetBaseAddressOfPlane(imageBuffer,0); UInt32 yPixelCount = CVPixelBufferGetWidthOfPlane(imageBuffer,0) * CVPixelBufferGetHeightOfPlane(imageBuffer,0); uint8_t *uvPlaneAddress = (uint8_t *)CVPixelBufferGetBaseAddressOfPlane(imageBuffer,1); UInt32 uvPixelCount = CVPixelBufferGetWidthOfPlane(imageBuffer,1) * CVPixelBufferGetHeightOfPlane(imageBuffer,1); UInt32 p,q,r; p=q=r=0; memcpy(uPointer, uvPlaneAddress, uvPixelCount); memcpy(vPointer, uvPlaneAddress+uvPixelCount, uvPixelCount); memcpy(yPointer,yPlaneAddress,yPixelCount); baseAddress[0] = (UInt8*)yPointer; baseAddress[1] = (UInt8*)uPointer; baseAddress[2] = (UInt8*)vPointer; CVPixelBufferUnlockBaseAddress(imageBuffer,0); } } 

Is there something wrong with the above code?

Answer 1

Your code doesn't look so bad. I see two errors and one potential problem:

• Invalid uvPixelCount . The YUV 420 format means that there is color information for each block of size 2 by 2 pixels. So the correct score is:

   uvPixelCount = (width / 2) * (height / 2); 

  You are writing something about yPixelCount / 4 , but I do not see this in your code.

• UV information alternates, that is, the second plane alternates with U and value V. Or in another way: there is a value U for all even byte addresses and a value V for all odd byte addresses. If you really need to separate U and V information, memcpy will not do.

• There may be additional bytes after each pixel line. You should use CVPixelBufferGetBytesPerRowOfPlane(imageBuffer, 0) to get the number of bytes between two lines. As a result, one memcpy will not do. Instead, you need to copy each pixel line separately to get rid of extra bytes between the lines.

All this explains only part of the resulting image. The remaining parts are probably related to the differences between your code and the expected recipient. Have you written anything about this? Do different partners need separate values ​​of U and V? Is this also a 4: 2: 0 compression? Are you also using a video range, not a full range?

If you provide more information, I can give you more advice.