Java - Xuggle - 获取视频帧的最佳方法

Question

我一周以来一直在使用xuggle，编写了一个方法用于从视频中获取帧，但如果视频很长，该方法会花费太多时间:

public static void getFrameBySec(IContainer container, int videoStreamId, IStreamCoder videoCoder, IVideoResampler resampler, double sec) 
{ 
    BufferedImage javaImage = new BufferedImage(videoCoder.getWidth(), videoCoder.getHeight(), BufferedImage.TYPE_3BYTE_BGR); 
    IConverter converter = ConverterFactory.createConverter(javaImage, IPixelFormat.Type.BGR24); 
    IPacket packet = IPacket.make(); 
    while(container.readNextPacket(packet) >= 0) 
    { 
        if (packet.getStreamIndex() == videoStreamId) 
        { 
            IVideoPicture picture = IVideoPicture.make(videoCoder.getPixelType(), videoCoder.getWidth(), videoCoder.getHeight()); 
            int offset = 0; 
            while(offset < packet.getSize()) 
            { 
                int bytesDecoded = videoCoder.decodeVideo(picture, packet, offset); 
                if (bytesDecoded < 0) 
                    throw new RuntimeException("got error decoding video"); 
                offset += bytesDecoded; 
                if (picture.isComplete()) 
                { 
                    IVideoPicture newPic = picture; 
                    if (resampler != null) 
                    { 
                        newPic = IVideoPicture.make(resampler.getOutputPixelFormat(), picture.getWidth(), picture.getHeight()); 

                        if (resampler.resample(newPic, picture) < 0) 
                            throw new RuntimeException("could not resample video from"); 
                    } 
                    if (newPic.getPixelType() != IPixelFormat.Type.BGR24) 
                            throw new RuntimeException("could not decode video as RGB 32 bit data in"); 

                    javaImage = converter.toImage(newPic); 
                    try 
                    { 
                        double seconds = ((double)picture.getPts()) / Global.DEFAULT_PTS_PER_SECOND; 
                        if (seconds >= sec && seconds <= (sec +(Global.DEFAULT_PTS_PER_SECOND ))) 
                        { 

                            File file = new File(Config.MULTIMEDIA_PATH, "frame_" + sec + ".png"); 
                            ImageIO.write(javaImage, "png", file); 
                            System.out.printf("at elapsed time of %6.3f seconds wrote: %s \n", seconds, file); 
                            return; 
                        } 
                    } 
                    catch (Exception e) 
                    { 
                        e.printStackTrace(); 
                    } 
                } 
            } 
        } 
        else 
        { 
            // This packet isn't part of our video stream, so we just 
            // silently drop it. 
        } 
    } 
    converter.delete(); 
} 

你知道一种更好的方法来做这件事吗？

- Mark Design

Answer 1

使用seekKeyFrame选项。您可以使用此函数在视频文件中寻找任何时间（时间以毫秒为单位）。

double timeBase = 0;
int videoStreamId = -1;

private void seekToMs(IContainer container, long timeMs) {
    if(videoStreamId == -1) {
        for(int i = 0; i < container.getNumStreams(); i++) {
            IStream stream = container.getStream(i);
            IStreamCoder coder = stream.getStreamCoder();
            if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
                videoStreamId = i;
                timeBase = stream.getTimeBase().getDouble();
                break;
            }
        }
    }

    long seekTo = (long) (timeMs/1000.0/timeBase);
    container.seekKeyFrame(videoStreamId, seekTo, IContainer.SEEK_FLAG_BACKWARDS);
}

然后您可以使用经典的while(container.readNextPacket(packet) >= 0)方法将图像保存到文件中。

注意：它不会寻找精确时间，而是近似时间，因此您仍需要浏览数据包（但比以前少得多）。

- svenkapudija

---

在这种情况下，“timeMs”或“seekTO”相对于什么？我不太明白。如何相对于当前的pts跳转？[IVideoPicture.getPts()] - KarlP

Answer 2

仅从你的代码中阅读，我发现可以进行一些优化。

首先，你需要在第一次阅读整个文件后，创建一个字节偏移和时间的索引。然后函数可以通过给定的时间查找对应的字节偏移，你可以在该偏移处解码视频并执行其余的代码。

另一种选择是使用你的方法，每次读取整个文件，但不要调用所有的resampler、newPic和Java图像转换器代码，而是首先检查秒数是否匹配。如果匹配，则将图像转换为新的pic以显示。

所以

if(picture.isComplete()){

try { 

  double seconds = ((double)picture.getPts()) / Global.DEFAULT_PTS_PER_SECOND; 
  if (seconds >= sec && seconds <= (sec +(Global.DEFAULT_PTS_PER_SECOND ))) 
  {
    Resample image
    Convert Image
    Do File stuff
  }

- Michael