Problem:
During each playback, the sound is between 1-2 seconds behind the video.
Setup:
Assets are downloaded using AVURLAssets from the media stream.
To write a composition, I use AVMutableCompositions and AVMutableCompositionTracks with asymmetric timelines. Audio and video are transferred to the device. The timeline for audio is 44100; the timeline for the video is 600.
Playback is performed using AVPlayer.
Attempted Solution:
- Using
videoAssetTrack.timeRange for [composition insertTimeRange] . - Using
CMTimeRangeMake(kCMTimeZero, videoAssetTrack.duration); - Using
CMTimeRangeMake(kCMTimeZero, videoAssetTrack.timeRange.duration);
Code:
+(AVMutableComposition*)overlayAudio:(AVURLAsset*)audioAsset withVideo:(AVURLAsset*)videoAsset { AVMutableComposition* mixComposition = [AVMutableComposition composition]; AVAssetTrack* audioTrack = [self getTrackFromAsset:audioAsset withMediaType:AVMediaTypeAudio]; AVAssetTrack* videoTrack = [self getTrackFromAsset:videoAsset withMediaType:AVMediaTypeVideo]; CMTime duration = videoTrack.timeRange.duration; AVMutableCompositionTrack* audioComposition = [self composeTrack:audioTrack withComposition:mixComposition andDuration:duration andMedia:AVMediaTypeAudio]; AVMutableCompositionTrack* videoComposition = [self composeTrack:videoTrack withComposition:mixComposition andDuration:duration andMedia:AVMediaTypeVideo]; [self makeAssertionAgainstAudio:audioComposition andVideo:videoComposition]; return mixComposition; } +(AVAssetTrack*)getTrackFromAsset:(AVURLAsset*)asset withMediaType:(NSString*)mediaType { return [[asset tracksWithMediaType:mediaType] objectAtIndex:0]; } +(AVAssetExportSession*)configureExportSessionWithAsset:(AVMutableComposition*)composition toUrl:(NSURL*)url { AVAssetExportSession* exportSession = [[AVAssetExportSession alloc] initWithAsset:composition presetName:AVAssetExportPresetHighestQuality]; exportSession.outputFileType = @"com.apple.quicktime-movie"; exportSession.outputURL = url; exportSession.shouldOptimizeForNetworkUse = YES; return exportSession; } -(IBAction)playVideo { [avPlayer pause]; avPlayerItem = [AVPlayerItem playerItemWithAsset:mixComposition]; avPlayer = [[AVPlayer alloc]initWithPlayerItem:avPlayerItem]; avPlayerLayer =[AVPlayerLayer playerLayerWithPlayer:avPlayer]; [avPlayerLayer setFrame:CGRectMake(0, 0, 305, 283)]; [avPlayerLayer setVideoGravity:AVLayerVideoGravityResizeAspectFill]; [playerView.layer addSublayer:avPlayerLayer]; [avPlayer seekToTime:kCMTimeZero]; [avPlayer play]; }
Comments:
I am not very versed in the structure of AVFoundation. It is very likely that I am simply misusing the fragments I provided. (i.e. why "insertTimeRange" for the composition?)
I can provide any other information necessary for resolution, including debug asset tracking property values, network telemetry, streaming information, etc.
source share