安卓更改视频文件分辨率

Question

我正在编写一个Android应用程序，可以将给定视频的分辨率改变为给定的尺寸、比特率和音频采样率。我正在使用API级别18中提供的内置MediaCodec和MediaMuxer类，并且基本上遵循BigFlake.com (http://bigflake.com/mediacodec/)中的示例，但我在使它们平滑运行方面遇到了一些问题。

现在，当调用MediaCodec类上的dequeueInputBuffer时，我会得到一个IllegalStateException异常。我知道这是一种全能型异常，但希望有人可以查看下面的代码并让我知道哪里出了问题？

更新：

事实证明，dequeueInputBuffer调用的问题在于分辨率。由于480 x 360不是16的倍数，所以dequeueInputBuffer抛出了IllegalStateException异常。将目标分辨率更改为512 x 288解决了该问题。

现在，我有一个队列queueInputBuffer方法的问题。这个调用给我带来了和之前完全相同的IllegalStateException异常，但是现在原因不同了。

有趣的是，我已经查看了BigFlake.com上的示例，甚至重新实现了它，但我仍然在这一行上得到相同的异常。有人知道发生了什么吗？

顺便说一下，我已经删除了旧代码并使用最新的代码更新了此帖子。

谢谢！

package com.mikesappshop.videoconverter;

import android.media.MediaCodec;
import android.media.MediaCodecInfo;
import android.media.MediaCodecList;
import android.media.MediaExtractor;
import android.media.MediaFormat;
import android.media.MediaMuxer;
import android.util.Log;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;

/**
 * Created by mneill on 11/3/15.
 */

public class VideoConverter {

    // Interface

    public interface CompletionHandler {
        void videoEncodingDidComplete(Error error);
    }

    // Constants

    private static final String TAG = "VideoConverter";
    private static final boolean VERBOSE = true;           // lots of logging

    // parameters for the encoder
    private static final String MIME_TYPE = "video/avc";    // H.264 Advanced Video Coding
    private static final int FRAME_RATE = 15;               // 15fps
    private static final int CAPTURE_RATE = 15;               // 15fps
    private static final int IFRAME_INTERVAL = 10;          // 10 seconds between I-frames
    private static final int CHANNEL_COUNT = 1;
    private static final int SAMPLE_RATE = 128000;
    private static final int TIMEOUT_USEC = 10000;

    // size of a frame, in pixels
    private int mWidth = -1;
    private int mHeight = -1;

    // bit rate, in bits per second
    private int mBitRate = -1;

    // encoder / muxer state
    private MediaCodec mDecoder;
    private MediaCodec mEncoder;
    private MediaMuxer mMuxer;
    private int mTrackIndex;
    private boolean mMuxerStarted;

    /**
     * Starts encoding process
     */
    public void convertVideo(String mediaFilePath, String destinationFilePath, CompletionHandler handler) {

        // TODO: Make configurable
//        mWidth = 480;
//        mHeight = 360;
        mWidth = 512;
        mHeight = 288;
        mBitRate = 500000;

        try {

            if ((mWidth % 16) != 0 || (mHeight % 16) != 0) {
                Log.e(TAG, "Width or Height not multiple of 16");
                Error e = new Error("Width and height must be a multiple of 16");
                handler.videoEncodingDidComplete(e);
                return;
            }

            // prep the decoder and the encoder
            prepareEncoderDecoder(destinationFilePath);

            // load file
            File file = new File(mediaFilePath);
            byte[] fileData = readContentIntoByteArray(file);

            // fill up the input buffer
            fillInputBuffer(fileData);

            // encode buffer
            encode();

        } catch (Exception ex) {
            Log.e(TAG, ex.toString());
            ex.printStackTrace();

        } finally {

            // release encoder and muxer
            releaseEncoder();
        }
    }

    /**
     * Configures encoder and muxer state
     */

    private void prepareEncoderDecoder(String outputPath) throws Exception {

        // create decoder to read in the file data
        mDecoder = MediaCodec.createDecoderByType(MIME_TYPE);

        // create encoder to encode the file data into a new format
        MediaCodecInfo info = selectCodec(MIME_TYPE);
        int colorFormat = selectColorFormat(info, MIME_TYPE);

        MediaFormat format = MediaFormat.createVideoFormat(MIME_TYPE, mWidth, mHeight);
        format.setInteger(MediaFormat.KEY_COLOR_FORMAT, colorFormat);
        format.setInteger(MediaFormat.KEY_BIT_RATE, mBitRate);
        format.setInteger(MediaFormat.KEY_FRAME_RATE, FRAME_RATE);
        format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, IFRAME_INTERVAL);

        mEncoder = MediaCodec.createByCodecName(info.getName());
        mEncoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
        mEncoder.start();

        // Create a MediaMuxer for saving the data
        mMuxer = new MediaMuxer(outputPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);

        mTrackIndex = -1;
        mMuxerStarted = false;
    }

    /**
     * Releases encoder resources.  May be called after partial / failed initialization.
     */
    private void releaseEncoder() {

        if (VERBOSE) Log.d(TAG, "releasing encoder objects");

        if (mEncoder != null) {
            mEncoder.stop();
            mEncoder.release();
            mEncoder = null;
        }

        if (mMuxer != null) {
            mMuxer.stop();
            mMuxer.release();
            mMuxer = null;
        }
    }

    private void fillInputBuffer(byte[] data) {

        boolean inputDone = false;
        int processedDataSize = 0;
        int frameIndex = 0;

        Log.d(TAG, "[fillInputBuffer] Buffer load start");

        ByteBuffer[] inputBuffers = mEncoder.getInputBuffers();

        while (!inputDone) {

            int inputBufferIndex = mEncoder.dequeueInputBuffer(10000);
            if (inputBufferIndex >= 0) {

                if (processedDataSize >= data.length) {

                    mEncoder.queueInputBuffer(inputBufferIndex, 0, 0, computePresentationTime(frameIndex), MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                    inputDone = true;
                    Log.d(TAG, "[fillInputBuffer] Buffer load complete");

                } else {

                    ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];

                    int limit = inputBuffer.capacity();
                    int pos = frameIndex * limit;
                    byte[] subData = new byte[limit];
                    System.arraycopy(data, pos, subData, 0, limit);

                    inputBuffer.clear();
                    inputBuffer.put(subData);

                    Log.d(TAG, "[encode] call queueInputBuffer");
                    mDecoder.queueInputBuffer(inputBufferIndex, 0, subData.length, computePresentationTime(frameIndex), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                    Log.d(TAG, "[encode] did call queueInputBuffer");

                    Log.d(TAG, "[encode] Loaded frame " + frameIndex + " into buffer");

                    frameIndex++;
                }
            }
        }
    }

    private void encode() throws Exception {

        // get buffer info
        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

        // start encoding
        ByteBuffer[] encoderOutputBuffers = mEncoder.getOutputBuffers();

        while (true) {

            int encoderStatus = mEncoder.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);

            if (encoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {

                // no output available yet
                if (VERBOSE) Log.d(TAG, "no output available, spinning to await EOS");
                break;

            } else if (encoderStatus == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {

                // not expected for an encoder
                encoderOutputBuffers = mEncoder.getOutputBuffers();

            } else if (encoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {

                // should happen before receiving buffers, and should only happen once
                if (!mMuxerStarted) {

                    MediaFormat newFormat = mEncoder.getOutputFormat();
                    Log.d(TAG, "encoder output format changed: " + newFormat);

                    // now that we have the Magic Goodies, start the muxer
                    mTrackIndex = mMuxer.addTrack(newFormat);
                    mMuxer.start();
                    mMuxerStarted = true;
                }

            } else if (encoderStatus > 0) {

                ByteBuffer encodedData = encoderOutputBuffers[encoderStatus];

                if (encodedData == null) {
                    throw new RuntimeException("encoderOutputBuffer " + encoderStatus + " was null");
                }

                if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
                    if (VERBOSE) Log.d(TAG, "ignoring BUFFER_FLAG_CODEC_CONFIG");
                    bufferInfo.size = 0;
                }

                if (bufferInfo.size != 0) {

                    if (!mMuxerStarted) {
                        throw new RuntimeException("muxer hasn't started");
                    }

                    // adjust the ByteBuffer values to match BufferInfo (not needed?)
                    encodedData.position(bufferInfo.offset);
                    encodedData.limit(bufferInfo.offset + bufferInfo.size);

                    mMuxer.writeSampleData(mTrackIndex, encodedData, bufferInfo);
                    if (VERBOSE) Log.d(TAG, "sent " + bufferInfo.size + " bytes to muxer");
                }

                mEncoder.releaseOutputBuffer(encoderStatus, false);

                if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                    if (VERBOSE) Log.d(TAG, "end of stream reached");
                    break;      // out of while
                }
            }
        }
    }

    private byte[] readContentIntoByteArray(File file) throws Exception
    {
        FileInputStream fileInputStream = null;
        byte[] bFile = new byte[(int) file.length()];

        //convert file into array of bytes
        fileInputStream = new FileInputStream(file);
        fileInputStream.read(bFile);
        fileInputStream.close();

        return bFile;
    }

    /**
     * Returns the first codec capable of encoding the specified MIME type, or null if no
     * match was found.
     */
    private static MediaCodecInfo selectCodec(String mimeType) {
        int numCodecs = MediaCodecList.getCodecCount();
        for (int i = 0; i < numCodecs; i++) {
            MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
            if (!codecInfo.isEncoder()) {
                continue;
            }
            String[] types = codecInfo.getSupportedTypes();
            for (int j = 0; j < types.length; j++) {
                if (types[j].equalsIgnoreCase(mimeType)) {
                    return codecInfo;
                }
            }
        }
        return null;
    }

    private static int selectColorFormat(MediaCodecInfo codecInfo, String mimeType) {
        MediaCodecInfo.CodecCapabilities capabilities = codecInfo.getCapabilitiesForType(mimeType);
        for (int i = 0; i < capabilities.colorFormats.length; i++) {
            int colorFormat = capabilities.colorFormats[i];
            if (isRecognizedFormat(colorFormat)) {
                return colorFormat;
            }
        }

        return 0;   // not reached
    }

    private static boolean isRecognizedFormat(int colorFormat) {
        switch (colorFormat) {
            // these are the formats we know how to handle for this test
            case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar:
            case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420PackedPlanar:
            case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar:
            case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420PackedSemiPlanar:
            case MediaCodecInfo.CodecCapabilities.COLOR_TI_FormatYUV420PackedSemiPlanar:
                return true;
            default:
                return false;
        }
    }

    /**
     * Generates the presentation time for frame N, in microseconds.
     */
    private static long computePresentationTime(int frameIndex) {
        return 132 + frameIndex * 1000000 / FRAME_RATE;
    }
}

- miken.mkndev

Answer 1

唯一让我感到困惑的是你正在执行以下操作：

MediaCodecInfo info = selectCodec(MIME_TYPE);
int colorFormat = selectColorFormat(info, MIME_TYPE);

然后是这个：

mEncoder = MediaCodec.createEncoderByType(MIME_TYPE);

不要这样做：

mEncoder = MediaCodec.createByCodecName(info.getName());

（参见此示例）

我不知道为什么这会导致失败，或者为什么失败要到解码输入缓冲区时才体现出来。但是如果你从编解码器中选择颜色格式，你应该确保使用该编解码器。

其他注意事项：

- 你正在记录mEncoder.configure()的异常，但没有停止执行。请在其中加入throw new RuntimeException(ex)。 - 你正在记录但忽略encode()中的异常...为什么？去掉try/catch以确保故障明显并停止编码过程。 - 你可以省略KEY_SAMPLE_RATE和KEY_CHANNEL_COUNT，它们是用于音频的。 - 对于同时对视频进行编解码的视频转换，最好通过Surface传递数据，但看起来你已经将YUV数据存储在文件中。希望你已经选择了与编码器兼容的YUV格式。

- fadden

---

感谢@fadden的回答，但不幸的是这并不是解决我的问题的方法。我已经在我的帖子中更新了修复方法，但我会给你点赞，因为这是很好的信息。我已经修复了你指出的许多问题，但我仍然有问题:( - miken.mkndev

你使用的是什么设备？安卓系统的版本是多少？编解码器名称是什么（来自info.getName()）？使用了什么颜色格式？你将多少数据放入缓冲区？在configure()和抛出异常的那一行之间，logcat中是否有任何奇怪的东西？ - fadden

没关系。我放弃了这个想法，因为我意识到一些API需要23级，而我需要支持16级，所以无论如何都不能使用这个实现。最终，我选择了使用FFMPEG集成，使用了这个框架https://github.com/WritingMinds/ffmpeg-android-java。 - miken.mkndev

Answer 2

在函数fillInputBuffer中，为什么在else部分将数据加入解码器缓冲区？

Log.d(TAG, "[encode] call queueInputBuffer");
mDecoder.queueInputBuffer(inputBufferIndex, 0, subData.length, computePresentationTime(frameIndex), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
Log.d(TAG, "[encode] did call queueInputBuffer");

使用mEncoder替代mDecoder，它会正常工作。

- Mohan