Video doesn't spin using AVMutableVideoCompositionLayerInstruction

Question

I am trying to combine the two videos that I get after recording using the camera as a UIImagePickerController. I managed to merge the video into one, but I have some problems with the orientation of the video.

As I understand it, with the UIImagePickerController is that all the videos are captured in the landscape, which means that the videos recorded in the portrait are rotated 90 °.

After each recording, I add a new video to the array

func imagePickerController(picker: UIImagePickerController!, didFinishPickingMediaWithInfo info:NSDictionary) { let tempImage = info[UIImagePickerControllerMediaURL] as NSURL videos.append(tempImage) let pathString = tempImage.relativePath self.dismissViewControllerAnimated(true, completion: {}) } 

Then, when I want to join, I go through each video and create an instruction and add the instruction to another array

 var composition = AVMutableComposition() let trackVideo:AVMutableCompositionTrack = composition.addMutableTrackWithMediaType(AVMediaTypeVideo, preferredTrackID: CMPersistentTrackID()) let trackAudio:AVMutableCompositionTrack = composition.addMutableTrackWithMediaType(AVMediaTypeAudio, preferredTrackID: CMPersistentTrackID()) var insertTime = kCMTimeZero for i in 0...(videos.count-1){ let moviePathUrl = videos[i] let sourceAsset = AVURLAsset(URL: moviePathUrl, options: nil) let tracks = sourceAsset.tracksWithMediaType(AVMediaTypeVideo) let audios = sourceAsset.tracksWithMediaType(AVMediaTypeAudio) if tracks.count > 0{ var videoDuration = CMTimeRangeMake(kCMTimeZero, sourceAsset.duration); let assetTrack:AVAssetTrack = tracks[0] as AVAssetTrack let assetTrackAudio:AVAssetTrack = audios[0] as AVAssetTrack trackVideo.insertTimeRange(videoDuration, ofTrack: assetTrack, atTime: insertTime, error: nil) trackAudio.insertTimeRange(videoDuration, ofTrack: assetTrackAudio, atTime: insertTime, error: nil) //Rotate var rotater = AVMutableVideoCompositionLayerInstruction(assetTrack: assetTrack) rotater.setTransform(assetTrack.preferredTransform, atTime: insertTime) rotater.setOpacity(0.0, atTime: CMTimeAdd(insertTime, sourceAsset.duration)) instructions.append(rotater) //Resize var resizer = AVMutableVideoCompositionLayerInstruction(assetTrack: assetTrack) resizer.setCropRectangle(CGRectMake(0, 0, 300, 300), atTime: insertTime) instructions.append(resizer) insertTime = CMTimeAdd(insertTime, sourceAsset.duration) } } 

When I create all the instructions, I add them to the main statement and create an export session.

 var instruction = AVMutableVideoCompositionInstruction(); instruction.timeRange = CMTimeRangeMake(kCMTimeZero, insertTime); instruction.layerInstructions = instructions; var mainCompositionInst = AVMutableVideoComposition() mainCompositionInst.instructions = NSArray(object: instruction) mainCompositionInst.frameDuration = CMTimeMake(1, 60); mainCompositionInst.renderSize = CGSizeMake(300, 300); var exporter = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetHighestQuality) exporter.videoComposition = mainCompositionInst; 

What am I missing?

Answer 1

Try using trackVideo in the initializer to instruct your layer to use AVMutableCompositionTrack trackID , not the source trackID resource

Update:

You only need one AVMutableVideoCompositionLayerInstruction , so declare it before the loop with the AVMutableCompositionTrack parameter as the parameter. Then, at each iteration of the loop, set the necessary properties of the layer instruction (transform, crop rect) for the current video ad you are working with. You control how video content on the composition track should be displayed at each insertion time.

At the end, put a single layerInstruction in the command array and use this in the video composition.

 var layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: trackVideo) for i in 0...(videos.count-1){ //...your other code here layerInstruction.setTransform(assetTrack.preferredTransform, atTime: insertTime) layerInstruction.setCropRectangle(CGRectMake(0, 0, 300, 300), atTime: insertTime) } var instruction = AVMutableVideoCompositionInstruction(); instruction.timeRange = CMTimeRangeMake(kCMTimeZero, insertTime); instruction.layerInstructions = NSArray(object: layerInstruction); var mainCompositionInst = AVMutableVideoComposition() mainCompositionInst.instructions = NSArray(object: instruction) 

Answer 2

You have two layers. You need to apply the rotation instruction to both layers in the composition. What are you doing here, apply the rotation instruction to only one of them. Get a link to both elements in the video composition and apply separate instructions to the two layers.

Answer 3

You can just use this simple code, it works for me:

  var assetVideoTrack = (sourceAsset.tracksWithMediaType(AVMediaTypeVideo)).last as! AVAssetTrack var compositionVideoTrack = (composition.tracksWithMediaType(AVMediaTypeVideo)).last as! AVMutableCompositionTrack if (assetVideoTrack.playable && compositionVideoTrack.playable) { compositionVideoTrack.preferredTransform = assetVideoTrack.preferredTransform } 

Answer 4

I found a solution in the Flutter plugin project.

 - (CGAffineTransform)fixTransform:(AVAssetTrack*)videoTrack { CGAffineTransform transform = videoTrack.preferredTransform; if (transform.tx == 0 && transform.ty == 0) { NSInteger rotationDegrees = (NSInteger)round(radiansToDegrees(atan2(transform.b, transform.a))); NSLog(@"TX and TY are 0. Rotation: %ld. Natural width,height: %f, %f", (long)rotationDegrees, videoTrack.naturalSize.width, videoTrack.naturalSize.height); if (rotationDegrees == 90) { NSLog(@"Setting transform tx"); transform.tx = videoTrack.naturalSize.height; transform.ty = 0; } else if (rotationDegrees == 270) { NSLog(@"Setting transform ty"); transform.tx = 0; transform.ty = videoTrack.naturalSize.width; } } return transform; } // set layerInstruction [firstVideoLayerInstruction setTransform:[self fixTransform:firstVideoAssetTrack] atTime:kCMTimeZero]; 

Flutter VideoPlayerPlugin