我需要将我从OTVideoFrame类获取的YUV帧转换为CVPixelBuffer。
该类提供了视频帧中包含三个元素-y,u,v,每个元素在索引位置0,1,2的平面数组。
@property (nonatomic, retain) NSPointerArray *planes
以及视频帧的格式
@property (nonatomic, retain) OTVideoFormat *format
其中包含像宽度、高度、每行字节数等帧属性。
我需要为接收到的OTVideoFrame图像添加滤镜,我已经尝试过以下答案:
这两个链接中有Objective-C的解决方案,但我想用swift来实现。第二个链接中的一个答案是用swift编写的,但缺少关于答案引用的YUVFrame结构的一些信息。
我收到的格式是NV12。
以下是我一直在尝试做的事情,但我不知道该如何继续:
/**
* Calcualte the size of each plane from OTVideoFrame.
*
* @param frame The frame to render.
* @return tuple containing three elements for size of each plane
*/
fileprivate func calculatePlaneSize(forFrame frame: OTVideoFrame)
-> (ySize: Int, uSize: Int, vSize: Int){
guard let frameFormat = frame.format
else {
return (0, 0 ,0)
}
let baseSize = Int(frameFormat.imageWidth * frameFormat.imageHeight) * MemoryLayout<GLubyte>.size
return (baseSize, baseSize / 4, baseSize / 4)
}
/**
* Renders a frame to the video renderer.
*
* @param frame The frame to render.
*/
func renderVideoFrame(_ frame: OTVideoFrame) {
let planeSize = calculatePlaneSize(forFrame: frame)
let yPlane = UnsafeMutablePointer<GLubyte>.allocate(capacity: planeSize.ySize)
let uPlane = UnsafeMutablePointer<GLubyte>.allocate(capacity: planeSize.uSize)
let vPlane = UnsafeMutablePointer<GLubyte>.allocate(capacity: planeSize.vSize)
memcpy(yPlane, frame.planes?.pointer(at: 0), planeSize.ySize)
memcpy(uPlane, frame.planes?.pointer(at: 1), planeSize.uSize)
memcpy(vPlane, frame.planes?.pointer(at: 2), planeSize.vSize)
let yStride = frame.format!.bytesPerRow.object(at: 0) as! Int
// multiply chroma strides by 2 as bytesPerRow represents 2x2 subsample
let uStride = frame.format!.bytesPerRow.object(at: 1) as! Int
let vStride = frame.format!.bytesPerRow.object(at: 2) as! Int
let width = frame.format!.imageWidth
let height = frame.format!.imageHeight
var pixelBuffer: CVPixelBuffer? = nil
var err: CVReturn;
err = CVPixelBufferCreate(kCFAllocatorDefault, Int(width), Int(height), kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, nil, &pixelBuffer)
if (err != 0) {
NSLog("Error at CVPixelBufferCreate %d", err)
fatalError()
}
}
参考这两个链接的指导,我尝试创建像素缓冲区,但每次都卡在这一点上,因为此后Objective-C代码的转换与Swift 3中的不相似。