在相机流上绘制文本或图像（GLSL）

Question

在相机流上绘制文本或图像（GLSL）

androidopengl-esglsl

5

我有一个基于grafika的示例的直播应用程序，我通过RTMP发送我的视频源以进行实时广播。

现在，我想通过在视频流上叠加文本或标志来为我的视频添加水印。我知道可以使用GLSL过滤器来完成此操作，但是根据我提供的示例，我不知道如何实现它。

我尝试使用Alpha混合，但似乎两种纹理格式不兼容（一种是TEXTURE_EXTERNAL_OES，另一种是TEXTURE_2D），我只得到一个黑色帧作为返回值。

编辑：

我基于Kickflip API：编写了我的代码。

class CameraSurfaceRenderer implements GLSurfaceView.Renderer {
    private static final String TAG = "CameraSurfaceRenderer";
    private static final boolean VERBOSE = false;

    private CameraEncoder mCameraEncoder;

    private FullFrameRect mFullScreenCamera;
    private FullFrameRect mFullScreenOverlay;     // For texture overlay

    private final float[] mSTMatrix = new float[16];
    private int mOverlayTextureId;
    private int mCameraTextureId;

    private boolean mRecordingEnabled;

    private int mFrameCount;

    // Keep track of selected filters + relevant state
    private boolean mIncomingSizeUpdated;
    private int mIncomingWidth;
    private int mIncomingHeight;
    private int mCurrentFilter;
    private int mNewFilter;

    boolean showBox = false;


    /**
     * Constructs CameraSurfaceRenderer.
     * <p>
     * @param recorder video encoder object
     */
    public CameraSurfaceRenderer(CameraEncoder recorder) {
        mCameraEncoder = recorder;

        mCameraTextureId = -1;
        mFrameCount = -1;

        SessionConfig config = recorder.getConfig();
        mIncomingWidth = config.getVideoWidth();
        mIncomingHeight = config.getVideoHeight();
        mIncomingSizeUpdated = true;        // Force texture size update on next onDrawFrame

        mCurrentFilter = -1;
        mNewFilter = Filters.FILTER_NONE;

        mRecordingEnabled = false;
    }


    /**
     * Notifies the renderer that we want to stop or start recording.
     */
    public void changeRecordingState(boolean isRecording) {
        Log.d(TAG, "changeRecordingState: was " + mRecordingEnabled + " now " + isRecording);
        mRecordingEnabled = isRecording;
    }

    @Override
    public void onSurfaceCreated(GL10 unused, EGLConfig config) {
        Log.d(TAG, "onSurfaceCreated");
        // Set up the texture blitter that will be used for on-screen display.  This
        // is *not* applied to the recording, because that uses a separate shader.
        mFullScreenCamera = new FullFrameRect(
                new Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_EXT));
        // For texture overlay:
        GLES20.glEnable(GLES20.GL_BLEND);
        GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
        mFullScreenOverlay = new FullFrameRect(
                  new Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_2D));
        mOverlayTextureId = GlUtil.createTextureWithTextContent("hello!");
        mOverlayTextureId = GlUtil.createTextureFromImage(mCameraView.getContext(), R.drawable.red_dot);
        mCameraTextureId = mFullScreenCamera.createTextureObject();

        mCameraEncoder.onSurfaceCreated(mCameraTextureId);
        mFrameCount = 0;
    }

    @Override
    public void onSurfaceChanged(GL10 unused, int width, int height) {
        Log.d(TAG, "onSurfaceChanged " + width + "x" + height);
    }

    @Override
    public void onDrawFrame(GL10 unused) {
        if (VERBOSE){
            if(mFrameCount % 30 == 0){
                Log.d(TAG, "onDrawFrame tex=" + mCameraTextureId);
                mCameraEncoder.logSavedEglState();
            }
        }

        if (mCurrentFilter != mNewFilter) {
            Filters.updateFilter(mFullScreenCamera, mNewFilter);
            mCurrentFilter = mNewFilter;
            mIncomingSizeUpdated = true;
        }

        if (mIncomingSizeUpdated) {
            mFullScreenCamera.getProgram().setTexSize(mIncomingWidth, mIncomingHeight);
            mFullScreenOverlay.getProgram().setTexSize(mIncomingWidth, mIncomingHeight);
            mIncomingSizeUpdated = false;
            Log.i(TAG, "setTexSize on display Texture");
        }

        // Draw the video frame.
        if(mCameraEncoder.isSurfaceTextureReadyForDisplay()){
            mCameraEncoder.getSurfaceTextureForDisplay().updateTexImage();
            mCameraEncoder.getSurfaceTextureForDisplay().getTransformMatrix(mSTMatrix);
            //Drawing texture overlay:
            mFullScreenOverlay.drawFrame(mOverlayTextureId, mSTMatrix);
            mFullScreenCamera.drawFrame(mCameraTextureId, mSTMatrix);
        }
        mFrameCount++;
    }

    public void signalVertialVideo(FullFrameRect.SCREEN_ROTATION isVertical) {
        if (mFullScreenCamera != null) mFullScreenCamera.adjustForVerticalVideo(isVertical, false);
    }

    /**
     * Changes the filter that we're applying to the camera preview.
     */
    public void changeFilterMode(int filter) {
        mNewFilter = filter;
    }

    public void handleTouchEvent(MotionEvent ev){
        mFullScreenCamera.handleTouchEvent(ev);
    }

}

这是在屏幕上呈现图像的代码（GLSurfaceView），但实际上这并没有覆盖在视频上。如果我没记错，这是在CameraEncoder中完成的。

问题是，将CameraSurfaceRenderer中的代码复制到CameraEncoder中（它们在滤镜方面的代码相似）并不能提供覆盖的文本/图像。

- M Rajoy

是的，那就是我想表达的意思，抱歉。 - M Rajoy

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- James Poag · Answer 1

纹理对象使用GL_OES_EGL_image_external OpenGL ES扩展定义的GL_TEXTURE_EXTERNAL_OES纹理目标，这限制了纹理的使用方式。每次绑定纹理时，必须将其绑定到GL_TEXTURE_EXTERNAL_OES目标而不是GL_TEXTURE_2D目标。此外，任何从纹理采样的OpenGL ES 2.0着色器都必须声明其使用此扩展，例如，使用“#extension GL_OES_EGL_image_external：require”指令。这样的着色器还必须使用samplerExternalOES GLSL采样器类型访问纹理。

https://developer.android.com/reference/android/graphics/SurfaceTexture.html

请贴出您用于执行 alpha 混合的代码，我可以尝试修复它。

我可能会覆盖Texture2dProgram并将其传递给FullFrame渲染器。它具有使用GL_TEXTURE_EXTERNAL_OES扩展进行呈现的示例代码。基本上，@Override draw函数，调用基本实现，绑定您的水印并绘制。

那应该位于相机和视频编码器之间。