使用mp4parser合并mp4剪辑会导致视频后面的音频不同步

Question

我正在开发一个应用程序，使用mp4parser库（isoparser-1.0-RC-27.jar和aspectjrt-1.8.0.jar）合并mp4剪辑。当两个剪辑合并时，它们变成了一个单一的剪辑，但是当更多的剪辑添加到其中时，输出的mp4文件会出现音频落后于视频的情况。

以下是代码：

    Movie[] clips = new Movie[2];

    //location of the movie clip storage
    File mediaStorageDir = new File(Environment.getExternalStoragePublicDirectory(
            Environment.DIRECTORY_PICTURES), "TestMerge");

    //Build the two clips into movies
    Movie firstClip = MovieCreator.build(first);
    Movie secondClip = MovieCreator.build(second);

    //Add both movie clips
    clips[0] = firstClip;
    clips[1] = secondClip;

    //List for audio and video tracks
    List<Track> videoTracks = new LinkedList<Track>();
    List<Track> audioTracks = new LinkedList<Track>();

    //Iterate all the movie clips and find the audio and videos
    for (Movie movie: clips) {
        for (Track track : movie.getTracks()) {
            if (track.getHandler().equals("soun")) 
                audioTracks.add(track);                
            if (track.getHandler().equals("vide"))
                videoTracks.add(track);
        }
    }

    //Result movie from putting the audio and video together from the two clips
    Movie result = new Movie();

    //Append all audio and video
    if (videoTracks.size() > 0)
        result.addTrack(new AppendTrack(videoTracks.toArray(new Track[videoTracks.size()])));

    if (audioTracks.size() > 0) 
        result.addTrack(new AppendTrack(audioTracks.toArray(new Track[audioTracks.size()])));

    //Output the resulting movie to a new mp4 file
    String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
    String outputLocation = mediaStorageDir.getPath()+timeStamp;
    Container out = new DefaultMp4Builder().build(result);
    FileChannel fc = new RandomAccessFile(String.format(outputLocation), "rw").getChannel();
    out.writeContainer(fc);
    fc.close();

    //Now set the active URL to play as the combined videos!
    setURL(outputLocation);
}

我的猜测是随着越来越多的剪辑被添加，视频和音频的同步会被搞乱。因为如果两个较长的剪辑合并，那么音频/视频就没有问题。有没有办法防止在多个小剪辑中出现这种糟糕的音视频同步问题，或者有没有人使用mp4parser找到了解决方案？另一个我正在考虑的解决方案是FFMpeg，但是我还没有找到其他人用它来完成这个问题的方法。

编辑： 我已经发现音频通常比视频长，因此当越来越多的剪辑被添加到一个剪辑中时，这就导致最终生成的视频偏移很多。我将通过裁剪音频样本来解决这个问题。

- Lucas Crawford

Answer 1

只需将代码放在上面Lucas的答案中：

1.

LinkedList<Track> videoTracks = new LinkedList<>();
            LinkedList<Track> audioTracks = new LinkedList<>();
            double[] audioDuration = {0}, videoDuration = {0};
            for (Movie m : clips) {
                for (Track t : m.getTracks()) {
                    if (t.getHandler().equals("soun")) {
                        for (long a : t.getSampleDurations()) audioDuration[0] += ((double) a) / t.getTrackMetaData().getTimescale();
                        audioTracks.add(t);
                    } else if (t.getHandler().equals("vide")) {
                        for (long v : t.getSampleDurations()) videoDuration[0] += ((double) v) / t.getTrackMetaData().getTimescale();
                        videoTracks.add(t);
                    }
                }

                adjustDurations(videoTracks, audioTracks, videoDuration, audioDuration);
            }

2.

private void adjustDurations(LinkedList<Track> videoTracks, LinkedList<Track> audioTracks, double[] videoDuration, double[] audioDuration) {
    double diff = audioDuration[0] - videoDuration[0];

    //nothing to do
    if (diff == 0) {
        return;
    }

    //audio is longer
    LinkedList<Track> tracks = audioTracks;

    //video is longer
    if (diff < 0) {
        tracks = videoTracks;
        diff *= -1;
    }

    Track track = tracks.getLast();
    long[] sampleDurations = track.getSampleDurations();
    long counter = 0;
    for (int i = sampleDurations.length - 1; i > -1; i--) {
        if (((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale()) > diff) {
            break;
        }
        diff -= ((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale());
        audioDuration[0] -= ((double) (sampleDurations[i]) / track.getTrackMetaData().getTimescale());
        counter++;
    }

    if (counter == 0) {
        return;
    }

    track = new CroppedTrack(track, 0, track.getSamples().size() - counter);

    //update the original reference
    tracks.removeLast();
    tracks.addLast(track);
}

- mtuulu

---

只有一个建议，在计算样本持续时间时，应该除以轨道时间刻度以获得实际持续时间。 - Manish Kumar

Answer 2

我能够通过上述编辑中的技巧解决这个问题。诀窍在于跟踪合并的剪辑数量，并从最近添加的剪辑的音频轨道末尾删除样本。随着更多剪辑被合并，生成的输出mp4会变得越来越大，因此需要从末尾剥离更多内容。这部分是由于音频和视频轨道的时间差异造成的，因为音频轨道可能是1020毫秒，而视频是1000毫秒，如果添加了5个剪辑，则音频与视频长度的偏移量约为100毫秒，因此必须进行补偿。

- Lucas Crawford

---

请添加裁剪音轨的代码，以便其他人可以找到您的解决方案。谢谢！ - Episodex

我之前写过这个，让我找出我的解决方案。 - Lucas Crawford

谢谢您的关注 :). 我已经找到了如何在技术上裁剪音频，但仍然在选择正确的样本数量方面遇到问题。我想知道您是否找到了解决方法。 - Episodex

问题在于音频比视频提前，因此删除音频样本是无济于事的。只需删除足够数量的视频样本即可使音频与视频同步，但不足以被人眼察觉。 - Lucas Crawford