Using CMSampleTimingInfo, CMSampleBuffer and AudioBufferList from a raw PCM stream

Question

I get a raw PCM stream from the reference implementation of the Google WebRTC website (hook inserted into VoEBaseImpl::GetPlayoutData ). The sound appears to be linear PCM signed with int16, but when recording using AssetWriter it retains a highly distorted and higher level sound file.

I assume this is an error somewhere with input parameters, most likely with regard to converting stereo-int16 to AudioBufferList, and then to CMSampleBuffer. Is there a problem with the code below?

 void RecorderImpl::RenderAudioFrame(void* audio_data, size_t number_of_frames, int sample_rate, int64_t elapsed_time_ms, int64_t ntp_time_ms) { OSStatus status; AudioChannelLayout acl; bzero(&acl, sizeof(acl)); acl.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo; AudioStreamBasicDescription audioFormat; audioFormat.mSampleRate = sample_rate; audioFormat.mFormatID = kAudioFormatLinearPCM; audioFormat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked; audioFormat.mFramesPerPacket = 1; audioFormat.mChannelsPerFrame = 2; audioFormat.mBitsPerChannel = 16; audioFormat.mBytesPerPacket = audioFormat.mFramesPerPacket * audioFormat.mChannelsPerFrame * audioFormat.mBitsPerChannel / 8; audioFormat.mBytesPerFrame = audioFormat.mBytesPerPacket / audioFormat.mFramesPerPacket; CMSampleTimingInfo timing = { CMTimeMake(1, sample_rate), CMTimeMake(elapsed_time_ms, 1000), kCMTimeInvalid }; CMFormatDescriptionRef format = NULL; status = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, &audioFormat, sizeof(acl), &acl, 0, NULL, NULL, &format); if(status != 0) { NSLog(@"Failed to create audio format description"); return; } CMSampleBufferRef buffer; status = CMSampleBufferCreate(kCFAllocatorDefault, NULL, false, NULL, NULL, format, (CMItemCount)number_of_frames, 1, &timing, 0, NULL, &buffer); if(status != 0) { NSLog(@"Failed to allocate sample buffer"); return; } AudioBufferList bufferList; bufferList.mNumberBuffers = 1; bufferList.mBuffers[0].mNumberChannels = audioFormat.mChannelsPerFrame; bufferList.mBuffers[0].mDataByteSize = (UInt32)(number_of_frames * audioFormat.mBytesPerFrame); bufferList.mBuffers[0].mData = audio_data; status = CMSampleBufferSetDataBufferFromAudioBufferList(buffer, kCFAllocatorDefault, kCFAllocatorDefault, 0, &bufferList); if(status != 0) { NSLog(@"Failed to convert audio buffer list into sample buffer"); return; } [recorder writeAudioFrames:buffer]; CFRelease(buffer); } 

For reference, the sampling rate that I get from WebRTC on the iPhone 6S + / iOS 9.2 is 48 kHz with 480 samples to call this hook, and I get data every 10 ms.

Answer 1

As a result, I opened the audio file that was generated in Audacity, and saw that half of it fell out in each frame, as shown in this rather bizarre form:

Changing acl.mChannelLayoutTag to kAudioChannelLayoutTag_Mono and changing audioFormat.mChannelsPerFrame to 1 solved the problem, and now the sound quality is excellent. Hooray!

Answer 2

First of all, congratulations on the fact that you need to create an audio CMSampleBuffer from scratch. For the majority, they are not created and not destroyed, but transferred to immaculate and mysterious from CoreMedia and AVFoundation .

presentationTimeStamp in your time information is in a single millisecond, which cannot represent your 48 kHz sample positions in time.

Instead of CMTimeMake(elapsed_time_ms, 1000) try CMTimeMake(elapsed_frames, sample_rate) , where elapsed_frames is the number of frames that you previously recorded.

This explains the distortion, but not the step, so make sure the AudioStreamBasicDescription matches your AVAssetWriterInput setting. It's hard to say without seeing your AVAssetWriter code.

ps Search for writeAudioFrames - if it is asynchronous, you will have rights issues with audio_data .

pps it looks like you are CMFormatDescriptionRef .