从CALayer（AVPlayerLayer）创建OpenGL纹理

Question

我有一个AVPlayerLayer，我希望能够创建一个OpenGL纹理。我对OpenGL纹理很熟悉，甚至可以将CGImageRef转换为OpenGL纹理。下面的代码似乎应该可以正常工作，但是我只得到了纯黑色。我做错了什么？我需要在CALayer / AVPlayerLayer上设置任何属性吗？

CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();

int width = (int)[layer bounds].size.width;
int height = (int)[layer bounds].size.height;

CGContextRef context = CGBitmapContextCreate(NULL,
                                 width,
                                 height,
                                 8,
                                 width * 4,
                                 colorSpace,
                                 kCGImageAlphaPremultipliedLast);

CGColorSpaceRelease(colorSpace);

if (context== NULL) {
    ofLog(OF_LOG_ERROR, "getTextureFromLayer: failed to create context 1");
    return;
}

[[layer presentationLayer] renderInContext:context];

CGImageRef cgImage = CGBitmapContextCreateImage(context);

int bytesPerPixel   = CGImageGetBitsPerPixel(cgImage)/8;
if(bytesPerPixel == 3) bytesPerPixel = 4;

GLubyte *pixels     = (GLubyte *) malloc(width * height * bytesPerPixel);

CGContextRelease(context);
context = CGBitmapContextCreate(pixels,
                                width,
                                height,
                                CGImageGetBitsPerComponent(cgImage),
                                width * bytesPerPixel,
                                CGImageGetColorSpace(cgImage),
                                kCGImageAlphaPremultipliedLast);

if(context == NULL) {
    ofLog(OF_LOG_ERROR, "getTextureFromLayer: failed to create context 2");
    free(pixels);
    return;
}

CGContextDrawImage(context, CGRectMake(0.0, 0.0, width, height), cgImage);

int glMode;
switch(bytesPerPixel) {
    case 1:
        glMode = GL_LUMINANCE;
        break;
    case 3: 
        glMode = GL_RGB;
        break;
    case 4: 
    default:
        glMode = GL_RGBA; break;
}

if(texture.bAllocated() == false || texture.getWidth() != width || texture.getHeight() != height) {
    NSLog(@"getTextureFromLayer: allocating texture %i, %i\n", width, height);
    texture.allocate(width, height, glMode, true);
}

// test texture
//        for(int i=0; i<width*height*4; i++) pixels[i] = ofRandomuf() * 255;

texture.loadData(pixels, width, height, glMode);

CGContextRelease(context);
CFRelease(cgImage);
free(pixels);

附注：变量“texture”是一个C++ opengl（-es兼容）纹理对象，我知道它是有效的。如果我取消注释“测试纹理”的循环，用随机噪声填充纹理，我可以看到它，所以问题肯定出在之前。

更新

针对Nick Weaver的回复，我尝试了不同的方法，但现在始终从copyNextSampleBuffer中获取NULL，并且状态为3（AVAssetReaderStatusFailed）。我错过了什么吗？

变量

AVPlayer                *videoPlayer;
AVPlayerLayer           *videoLayer;
AVAssetReader           *videoReader;
AVAssetReaderTrackOutput*videoOutput;

初始化

    videoPlayer = [[AVPlayer alloc] initWithURL:[NSURL fileURLWithPath:[NSString stringWithUTF8String:videoPath.c_str()]]];

    if(videoPlayer == nil) {
        NSLog(@"videoPlayer == nil ERROR LOADING %s\n", videoPath.c_str());
    } else {
        NSLog(@"videoPlayer: %@", videoPlayer);
        videoLayer = [[AVPlayerLayer playerLayerWithPlayer:videoPlayer] retain];
        videoLayer.frame = [ThreeDView instance].bounds;
        // [[ThreeDView instance].layer addSublayer:videoLayer]; // test to see if it's loading and running

        AVAsset *asset = videoPlayer.currentItem.asset;
        NSArray *tracks = [asset tracksWithMediaType:AVMediaTypeVideo];
        NSDictionary *settings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA], (NSString*)kCVPixelBufferPixelFormatTypeKey, nil];

        videoReader = [[AVAssetReader alloc] initWithAsset:asset error:nil];
        videoOutput = [[AVAssetReaderTrackOutput alloc] initWithTrack:[tracks objectAtIndex:0] outputSettings:settings];
        [videoReader addOutput:videoOutput];
        [videoReader startReading];
    }

绘制循环

    if(videoPlayer == 0) {
        ofLog(OF_LOG_WARNING, "Shot::drawVideo: videoPlayer == 0");
        return;
    }


    if(videoOutput == 0) {
        ofLog(OF_LOG_WARNING, "Shot::drawVideo: videoOutput == 0");
        return;
    }

    CMSampleBufferRef sampleBuffer = [videoOutput copyNextSampleBuffer];

    if(sampleBuffer == 0) {
        ofLog(OF_LOG_ERROR, "Shot::drawVideo: sampleBuffer == 0, status: %i", videoReader.status);
        return;
    }


    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CFRelease(sampleBuffer);

    CVPixelBufferLockBaseAddress(imageBuffer,0); 

    unsigned char *pixels = ( unsigned char *)CVPixelBufferGetBaseAddress(imageBuffer); 

    int width = CVPixelBufferGetWidth(imageBuffer); 
    int height = CVPixelBufferGetHeight(imageBuffer);

    if(videoTexture.bAllocated() == false || videoTexture.getWidth() != width || videoTexture.getHeight() != height) {
        NSLog(@"Shot::drawVideo() allocating texture %i, %i\n", width, height);
        videoTexture.allocate(width, height, GL_RGBA, true);
    }

    videoTexture.loadData(pixels, width, height, GL_BGRA);

    CVPixelBufferUnlockBaseAddress(imageBuffer,0);

- memo

---

你正在传输受FairPlay保护的HLS媒体吗？如果是的话，我认为试图获取解密数据的引用不起作用是有意为之的设计：http://stackoverflow.com/q/42839831/1097106 - schmittsfn

Answer 1

我认为iOS4：如何将视频文件用作OpenGL纹理？对你的问题会有所帮助。

- Nick Weaver

---

谢谢，这是一个不同的方法，但我现在也尝试过了。我总是从copyNextSampleBuffer得到NULL，我已经在我的问题中添加了新代码。我有什么遗漏吗？ - memo

如果您的视频格式为HTTP Live Streaming，则没有简单的方法从中获取单个帧。 - kolyuchiy