OnFrameAvailable回调未到达

Question

我是新手开发安卓应用，正在尝试使用SurfaceTexture来使用相机。但是OnFrameAvailable()回调没有被调用……请给我提供一个解决方案。以下是代码。

这里缺少什么？我不确定是否正确调用了setOnFrameListener()。

package com.example.cameratest;

import com.example.test.R;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.view.Menu;
import android.view.View;


import android.graphics.SurfaceTexture;
import android.graphics.SurfaceTexture.OnFrameAvailableListener;
import android.hardware.Camera;
import android.media.MediaCodec;
import android.media.MediaCodecInfo;
import android.media.MediaFormat;
import android.media.MediaMuxer;
import android.opengl.*;

import android.util.Log;
import android.view.Surface;


import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import java.util.concurrent.locks.ReentrantLock;

public class MainActivity extends Activity implements OnFrameAvailableListener {
    private static final String TAG = "CameraToMpegTest";
    private static final boolean VERBOSE = true;           // lots of logging
    // where to put the output file (note: /sdcard requires WRITE_EXTERNAL_STORAGE permission)
    private static final long DURATION_SEC = 8;
    // camera state
    private Camera mCamera;
    private static SurfaceTexture mSurfaceTexture;
    private int[] mGlTextures = null;
    private Object mFrameSyncObject = new Object();
    private boolean mFrameAvailable = false;

    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    public void startCamera(View v) {
        try {

            this.initCamera(0);
            this.StartCamera();


        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    private void StartCamera() {

        try {

            mCamera.startPreview();

            long startWhen = System.nanoTime();
            long desiredEnd = startWhen + DURATION_SEC * 1000000000L;

            int frameCount = 0;

            while (System.nanoTime() < desiredEnd) {
// Feed any pending encoder output into the muxer.

                awaitNewImage();
            }
        } finally {
// release everything we grabbed
            releaseCamera();

        }
    }

    /**
     * Stops camera preview, and releases the camera to the system.
     */
    private void releaseCamera() {
        if (VERBOSE) Log.d(TAG, "releasing camera");
        if (mCamera != null) {
            mCamera.stopPreview();
            mCamera.release();
            mCamera = null;
        }
    }

    private void initCamera(int cameraId) {

        mCamera = Camera.open(cameraId);
        if (mCamera == null) {
            Log.d(TAG, "No front-facing camera found; opening default");
            mCamera = Camera.open();    // opens first back-facing camera
        }
        if (mCamera == null) {
            throw new RuntimeException("Unable to open camera");
        }

        Camera.Parameters parms = mCamera.getParameters();
        parms.setPreviewSize(640, 480);
        mGlTextures = new int[1];
        GLES20.glGenTextures(1, mGlTextures, 0);


        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, mGlTextures[0]);


        GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
                GLES20.GL_NEAREST);
        GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
                GLES20.GL_LINEAR);
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S,
                GLES20.GL_CLAMP_TO_EDGE);
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T,
                GLES20.GL_CLAMP_TO_EDGE);
        mSurfaceTexture = new SurfaceTexture(mGlTextures[0]);
        try {
            mCamera.setPreviewTexture(mSurfaceTexture);
        } catch (IOException e) {
// TODO Auto-generated catch block
            e.printStackTrace();
        }
        mSurfaceTexture.setOnFrameAvailableListener(MainActivity.this);


    }

    public void awaitNewImage() {
        final int TIMEOUT_MS = 4500;
        synchronized (mFrameSyncObject) {
            while (!mFrameAvailable) {
                try {
// Wait for onFrameAvailable() to signal us.  Use a timeout to avoid
// stalling the test if it doesn't arrive.
                    if (VERBOSE) Log.i(TAG, "Waiting for Frame in Thread");
                    mFrameSyncObject.wait(TIMEOUT_MS);
                    if (!mFrameAvailable) {
// TODO: if "spurious wakeup", continue while loop
                        throw new RuntimeException("Camera frame wait timed out");
                    }
                } catch (InterruptedException ie) {
// shouldn't happen
                    throw new RuntimeException(ie);
                }
            }
            mFrameAvailable = false;
        }


    }

    @Override
    public void onFrameAvailable(SurfaceTexture st) {
        if (VERBOSE) Log.d(TAG, "new frame available");
        synchronized (mFrameSyncObject) {
            if (mFrameAvailable) {
                throw new RuntimeException("mFrameAvailable already set, frame could be dropped");
            }
            mFrameAvailable = true;
            mFrameSyncObject.notifyAll();
        }
    }
}

- Nehal Shah

---

我也遇到了同样的问题。你有没有偶然找到解决方案？ - Subhransu

在我的理解中，onFrameAvailable 应该与线程一起使用。这样我就没有遇到问题了。 - Nehal Shah

Answer 1

我认为在您的OnFrameAvailable()回调后，您需要调用SurfaceTeture.updateTextImage()来告诉相机“我已经使用了您的上一帧，请给我另一帧”。（抱歉，我的英语无法提供更好的解释）

- cescom

Answer 2

    @Override   
    public void onFrameAvailable(SurfaceTexture surfaceTexture) {
        ...
        surfaceTexture.updateTexImage();
    }

我遇到了同样的问题，看起来我忘记调用updateTexImage()方法了。

- Ariel Yust

---

你不应该在OpenGL线程之外的任何地方（例如渲染期间）调用updateTexImage()。https://developer.android.com/reference/android/graphics/SurfaceTexture.html - Rupert Rawnsley

@RupertRawnsley onFrameAvailable() 在OpenGL线程上运行。一旦您使相机监听SurfaceTexture更改，那么您就可以在OpenGL线程上通过此回调接收这些更改，您可以安全地调用surfaceTexture.updateTexImage();并在EglSurfaces上绘制任何内容。请查看google/grafika/以获取示例。 - Ariel Yust

文档中没有提到这一点。实际上，grafika示例中的三个onFrameAvailable调用都没有使用GL线程，并且对于此实现，该实现的注释说您需要在表面所有者线程（在此情况下为GL）上调用updateTexImage。 - Rupert Rawnsley

@RupertRawnsley，文档没有提到很多事情，实际上Grafika正在使用Gl-Thread，它使用UI-Thread作为GL-Thread，在任何情况下，如果您在打开相机并设置onFrameAvailable监听器之前调用offscreenSurface.makeCurrent()，则可以在onFrameAvailable()内调用updateTexImage()。请不要相信我，只需尝试并查看其是否有效 :) - Ariel Yust

Answer 3

使用方法setOnFrameAvailableListener(@Nullable final OnFrameAvailableListener listener, @Nullable Handler handler)替换setOnFrameAvailableListener(@Nullable OnFrameAvailableListener listener)。

在您的情况下，您可以将代码修改为：

frameUpdateThread = new HandlerThread("frameUpdateThread");
frameUpdateThread.start();
mSurfaceTexture.setOnFrameAvailableListener(MainActivity.this, Handler(frameUpdateThread.getLooper()));

- zyzhang

Answer 4

在我看来，onFrameAvailable 应该与线程一起使用。这样我就不会遇到问题，并确保在接收到帧后调用 updatetextImage。

- Nehal Shah