ReplayKit video constantly skips frames in Xamarin

Question

I have a very simple video project that worked great in Swift, but the same code ported to an empty project in Xamarin creates a video that constantly skips frames every few seconds. The code starts with ViewDidLoad and stops through UIButton The following is the write code:

 RPScreenRecorder rp = RPScreenRecorder.SharedRecorder; AVAssetWriter assetWriter; AVAssetWriterInput videoInput; public override void ViewDidLoad() { base.ViewDidLoad(); StartScreenRecording(); } public void StartScreenRecording() { VideoSettings videoSettings = new VideoSettings(); NSError wError; assetWriter = new AVAssetWriter(videoSettings.OutputUrl, AVFileType.AppleM4A, out wError); videoInput = new AVAssetWriterInput(AVMediaType.Video, videoSettings.OutputSettings); videoInput.ExpectsMediaDataInRealTime = true; assetWriter.AddInput(videoInput); if (rp.Available) { rp.StartCaptureAsync((buffer, sampleType, error) => { if (buffer.DataIsReady) { if (assetWriter.Status == AVAssetWriterStatus.Unknown) { assetWriter.StartWriting(); assetWriter.StartSessionAtSourceTime(buffer.PresentationTimeStamp); } if (assetWriter.Status == AVAssetWriterStatus.Failed) { return; } if (sampleType == RPSampleBufferType.Video) { if (videoInput.ReadyForMoreMediaData) { videoInput.AppendSampleBuffer(buffer); } } } }); } } public void StopRecording() { rp.StopCapture((error) => { if (error == null) { assetWriter.FinishWriting(() => { }); } }); } 

And this is what the VideoSettings file looks like:

 public class VideoSettings { public string VideoFilename => "render"; public string VideoFilenameExt = "mp4"; public nfloat Width { get; set; } public nfloat Height { get; set; } public AVVideoCodec AvCodecKey => AVVideoCodec.H264; public NSUrl OutputUrl { get { return GetFilename(VideoFilename,VideoFilenameExt); } } private NSUrl GetFilename(string filename, string extension) { NSError error; var docs = new NSFileManager().GetUrl(NSSearchPathDirectory.DocumentDirectory, NSSearchPathDomain.User, null, true, out error).ToString() + filename + 1 + "." + extension; if (error == null) { return new NSUrl(docs); } return null; } public AVVideoSettingsCompressed OutputSettings { get { return new AVVideoSettingsCompressed { Codec = AvCodecKey, Width = Convert.ToInt32(UIScreen.MainScreen.Bounds.Size.Width), Height = Convert.ToInt32(UIScreen.MainScreen.Bounds.Size.Height) }; } } } 

Answer 1

TL DR: you need a try {} finally {} block to help the GC release your sample buffers in a timely manner:

 try { // Do stuff with the buffer. } finally { buffer.Dispose (); } 

Long story:

This is because the GC is not fast enough to realize that the CMSampleBuffer needs to be freed, and you have run out of fetch buffers, which is why you see the delay, because until a new buffer is available, a snapshot of this actual frame .

Also don't worry about calling Dispose () Xamarin does the right thing, so no extra checks are needed.

 public void Dispose () { this.Dispose (true); GC.SuppressFinalize (this); } protected virtual void Dispose (bool disposing) { if (this.invalidate.IsAllocated) { this.invalidate.Free (); } if (this.handle != IntPtr.Zero) { CFObject.CFRelease (this.handle); this.handle = IntPtr.Zero; } } 

So your code should look something like this:

 if (rp.Available) { // TODO: Also note that we are not using the Async version here rp.StartCapture((buffer, sampleType, error) => { try { if (buffer.DataIsReady) { if (assetWriter.Status == AVAssetWriterStatus.Unknown) { assetWriter.StartWriting (); assetWriter.StartSessionAtSourceTime (buffer.PresentationTimeStamp); } if (assetWriter.Status == AVAssetWriterStatus.Failed) { return; } if (sampleType == RPSampleBufferType.Video) { if (videoInput.ReadyForMoreMediaData) { videoInput.AppendSampleBuffer (buffer); } } } } finally { buffer.Dispose (); } }, null); }