Merging mp4 clips with mp4parser makes sound per video

Question

I am developing an application that combines mp4 clips using the mp4parser library (isoparser-1.0-RC-27.jar and aspectjrt-1.8.0.jar). When two clips are combined, they become one clip, but as more clips are added to it, the output of mp4 is audio per video.

Here is the code:

Movie[] clips = new Movie[2]; //location of the movie clip storage File mediaStorageDir = new File(Environment.getExternalStoragePublicDirectory( Environment.DIRECTORY_PICTURES), "TestMerge"); //Build the two clips into movies Movie firstClip = MovieCreator.build(first); Movie secondClip = MovieCreator.build(second); //Add both movie clips clips[0] = firstClip; clips[1] = secondClip; //List for audio and video tracks List<Track> videoTracks = new LinkedList<Track>(); List<Track> audioTracks = new LinkedList<Track>(); //Iterate all the movie clips and find the audio and videos for (Movie movie: clips) { for (Track track : movie.getTracks()) { if (track.getHandler().equals("soun")) audioTracks.add(track); if (track.getHandler().equals("vide")) videoTracks.add(track); } } //Result movie from putting the audio and video together from the two clips Movie result = new Movie(); //Append all audio and video if (videoTracks.size() > 0) result.addTrack(new AppendTrack(videoTracks.toArray(new Track[videoTracks.size()]))); if (audioTracks.size() > 0) result.addTrack(new AppendTrack(audioTracks.toArray(new Track[audioTracks.size()]))); //Output the resulting movie to a new mp4 file String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date()); String outputLocation = mediaStorageDir.getPath()+timeStamp; Container out = new DefaultMp4Builder().build(result); FileChannel fc = new RandomAccessFile(String.format(outputLocation), "rw").getChannel(); out.writeContainer(fc); fc.close(); //Now set the active URL to play as the combined videos! setURL(outputLocation); } 

My guess is that when you add more clips, the synchronization of the video in the audio is confused, because if the two longer clips are combined, then the audio / video is fine. In any case, to prevent this poor synchronization of video and audio in several small clips, or did someone find a solution for this using mp4parser ?? FFMpeg is another solution that I am considering, but did not find anyone to use it for this.

EDIT: I found that the sound is usually longer than the video, so this causes the final resulting video to be so biased as more and more clips are added to the click. I'm going to solve by grinding audio samples

Answer 1

I was able to fix this problem using the technique with the above editing. The trick is to keep track of how many bonds are combined, and remove samples from the end of the audio track of the last clip. As the final mp4 output grows with more clips, you need to delete more and more. This is partly due to the difference in synchronization of audio and video tracks, since the audio track can be 1020 ms and the video can be 1000 ms, with 5 clicks added, then you will have an offset of about 100 ms for audio and video lengths, so you must compensate for this .

Answer 2

just putting the code in the answer to Lucas above:

1.

 LinkedList<Track> videoTracks = new LinkedList<>(); LinkedList<Track> audioTracks = new LinkedList<>(); double[] audioDuration = {0}, videoDuration = {0}; for (Movie m : clips) { for (Track t : m.getTracks()) { if (t.getHandler().equals("soun")) { for (long a : t.getSampleDurations()) audioDuration[0] += ((double) a) / t.getTrackMetaData().getTimescale(); audioTracks.add(t); } else if (t.getHandler().equals("vide")) { for (long v : t.getSampleDurations()) videoDuration[0] += ((double) v) / t.getTrackMetaData().getTimescale(); videoTracks.add(t); } } adjustDurations(videoTracks, audioTracks, videoDuration, audioDuration); } 

2.

 private void adjustDurations(LinkedList<Track> videoTracks, LinkedList<Track> audioTracks, double[] videoDuration, double[] audioDuration) { double diff = audioDuration[0] - videoDuration[0]; //nothing to do if (diff == 0) { return; } //audio is longer LinkedList<Track> tracks = audioTracks; //video is longer if (diff < 0) { tracks = videoTracks; diff *= -1; } Track track = tracks.getLast(); long[] sampleDurations = track.getSampleDurations(); long counter = 0; for (int i = sampleDurations.length - 1; i > -1; i--) { if (((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale()) > diff) { break; } diff -= ((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale()); audioDuration[0] -= ((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale()); counter++; } if (counter == 0) { return; } track = new CroppedTrack(track, 0, track.getSamples().size() - counter); //update the original reference tracks.removeLast(); tracks.addLast(track); }