使用AVFramework在iOS上捕获图像

Question

我正在使用这段代码捕获图像

#pragma mark - image capture

// Create and configure a capture session and start it running
- (void)setupCaptureSession 
{
    NSError *error = nil;

    // Create the session
    AVCaptureSession *session = [[AVCaptureSession alloc] init];

    // Configure the session to produce lower resolution video frames, if your 
    // processing algorithm can cope. We'll specify medium quality for the
    // chosen device.
    session.sessionPreset = AVCaptureSessionPresetMedium;

    // Find a suitable AVCaptureDevice
    AVCaptureDevice *device = [AVCaptureDevice
                           defaultDeviceWithMediaType:AVMediaTypeVideo];

    // Create a device input with the device and add it to the session.
    AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device 
                                                                    error:&error];
    if (!input)
    {
        NSLog(@"PANIC: no media input");
    }
    [session addInput:input];

    // Create a VideoDataOutput and add it to the session
    AVCaptureVideoDataOutput *output = [[AVCaptureVideoDataOutput alloc] init];
    [session addOutput:output];

    // Configure your output.
    dispatch_queue_t queue = dispatch_queue_create("myQueue", NULL);
    [output setSampleBufferDelegate:self queue:queue];
    dispatch_release(queue);

    // Specify the pixel format
    output.videoSettings = 
    [NSDictionary dictionaryWithObject:
    [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] 
                            forKey:(id)kCVPixelBufferPixelFormatTypeKey];


    // If you wish to cap the frame rate to a known value, such as 15 fps, set 
    // minFrameDuration.

    // Start the session running to start the flow of data
    [session startRunning];

    // Assign session to an ivar.
    [self setSession:session];
}




// Delegate routine that is called when a sample buffer was written
- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
   fromConnection:(AVCaptureConnection *)connection
{ 
    NSLog(@"captureOutput: didOutputSampleBufferFromConnection");

    // Create a UIImage from the sample buffer data
    UIImage *image = [self imageFromSampleBuffer:sampleBuffer];

    //< Add your code here that uses the image >
    [self.imageView setImage:image];
    [self.view setNeedsDisplay];
}


// Create a UIImage from sample buffer data
- (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer 
{
    NSLog(@"imageFromSampleBuffer: called");
    // Get a CMSampleBuffer's Core Video image buffer for the media data
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 
    // Lock the base address of the pixel buffer
    CVPixelBufferLockBaseAddress(imageBuffer, 0); 

    // Get the number of bytes per row for the pixel buffer
    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); 

    // Get the number of bytes per row for the pixel buffer
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
    // Get the pixel buffer width and height
    size_t width = CVPixelBufferGetWidth(imageBuffer); 
    size_t height = CVPixelBufferGetHeight(imageBuffer); 

    // Create a device-dependent RGB color space
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 

    // Create a bitmap graphics context with the sample buffer data
    CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, 
                                             bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
    // Create a Quartz image from the pixel data in the bitmap graphics context
    CGImageRef quartzImage = CGBitmapContextCreateImage(context); 
    // Unlock the pixel buffer
    CVPixelBufferUnlockBaseAddress(imageBuffer,0);


    // Free up the context and color space
    CGContextRelease(context); 
    CGColorSpaceRelease(colorSpace);

    // Create an image object from the Quartz image
    UIImage *image = [UIImage imageWithCGImage:quartzImage];

    // Release the Quartz image
    CGImageRelease(quartzImage);

    return (image);
}

-(void)setSession:(AVCaptureSession *)session
{
    NSLog(@"setting session...");
    self.captureSession=session;
}

捕获代码可用。但是！ 我需要更改两件事： - 在我的视图中从相机获取视频流。 - 每隔一段时间（例如5秒）获取图像。 请帮帮我，这可以如何实现？

- Oleg

---

感谢分享详细代码，对我的工作有很大帮助。+1 - zolio

Answer 1

添加以下行

output.minFrameDuration = CMTimeMake(5, 1);

在评论下方

 // If you wish to cap the frame rate to a known value, such as 15 fps, set
 // minFrameDuration.

但在这之上

[session startRunning];

编辑

使用以下代码预览相机输出。

AVCaptureVideoPreviewLayer *previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:session];
UIView *aView = self.view;
CGRect videoRect = CGRectMake(0.0, 0.0, 320.0, 150.0);
previewLayer.frame = videoRect; // Assume you want the preview layer to fill the view.
[aView.layer addSublayer:previewLayer];

Edit 2: 好吧，没问题。

苹果提供了一种设置minFrameDuration的方法，具体请见此处。

现在，可以使用以下代码来设置帧时长。

AVCaptureConnection *conn = [output connectionWithMediaType:AVMediaTypeVideo];

if (conn.supportsVideoMinFrameDuration)
    conn.videoMinFrameDuration = CMTimeMake(5,1);
if (conn.supportsVideoMaxFrameDuration)
    conn.videoMaxFrameDuration = CMTimeMake(5,1);

- Ilanchezhian

---

谢谢。如何在我的视图中设置从相机播放的视频流？我应该添加另一个AVCaptureOutput吗？ - Oleg

顺便提一下，此方法已被弃用。 - Oleg

使用 AVCaptureVideoPreviewLayer 播放预览。请查看我的更新答案。 - Ilanchezhian

好的。很清楚。但是对于已弃用的setMinFrameDuration该怎么办？ - Oleg

现在可以通过AVCaptureConnection来设置持续时间。请参考我的编辑2。 - Ilanchezhian

嗨@Aadhira，我明白如何设置fps。我有一个小问题，我想计算fps和丢帧率，我该怎么做？ - iosLearner

Answer 2

注意：AVCaptureOutput 的回调会被发布到您指定的调度队列中。我看到你在这个回调中执行了UI更新，这是错误的。您应该只在主队列中执行它们。例如：

请注意- AVCaptureOutput的回调是在您指定的调度队列中发布的。我注意到您在此回调中执行了UI更新，这是不正确的。您应该只在主队列中执行它们。例如：

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
    didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
       fromConnection:(AVCaptureConnection *)connection
{ 
    NSLog(@"captureOutput: didOutputSampleBufferFromConnection");
    // Create a UIImage from the sample buffer data
    UIImage *image = [self imageFromSampleBuffer:sampleBuffer];
    dispatch_async(dispatch_get_main_queue(), ^{
    //< Add your code here that uses the image >
        [self.imageView setImage:image];
        [self.view setNeedsDisplay];
    }
} 

- Eugene Dudnyk

---

你是个救命恩人！这个问题导致我的ImageView无法从相机更新，谢谢！ - Jim True

Answer 3

这里是一个Swift版本的imageFromSampleBuffer函数:

func imageFromSampleBuffer(sampleBuffer:CMSampleBuffer!) -> UIImage {
    let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
    CVPixelBufferLockBaseAddress(imageBuffer, 0)

    let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
    let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
    let width = CVPixelBufferGetWidth(imageBuffer)
    let height = CVPixelBufferGetHeight(imageBuffer)

    let colorSpace = CGColorSpaceCreateDeviceRGB()

    let bitmapInfo:CGBitmapInfo = [.ByteOrder32Little, CGBitmapInfo(rawValue: CGImageAlphaInfo.PremultipliedFirst.rawValue)]
    let context = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, bitmapInfo.rawValue)

    let quartzImage = CGBitmapContextCreateImage(context)
    CVPixelBufferUnlockBaseAddress(imageBuffer, 0)

    let image = UIImage(CGImage: quartzImage!)
    return image
}

以下是适用于我的视频设置：

videoDataOutput = AVCaptureVideoDataOutput()
videoDataOutput?.videoSettings = [kCVPixelBufferPixelFormatTypeKey:Int(kCVPixelFormatType_32BGRA)]
            videoDataOutput?.setSampleBufferDelegate(self, queue: queue)

- Timuçin