AVAssetWriter的问题

Question

我尝试使用AVAssetWriter将CGImages写入文件，从图像创建视频。在模拟器上，我已经成功地用三种不同的方法实现了这一点，但是在运行iOS 4.3的iPhone 4上，每种方法都失败了。所有这些都与像素缓冲有关。

我的第一种方法是根据需要创建像素缓冲区，而不使用池。那可以工作，但是在设备上太占用内存了。

我的第二种方法是使用推荐的AVAssetWriterInputPixelBufferAdaptor，然后使用CVPixelBufferPoolCreatePixelBuffer从适配器的pixelBufferPool中提取像素缓冲区。

这在模拟器上也可以工作，但在设备上失败，因为适配器的像素缓冲区池从未分配。我没有收到任何错误消息。

最后，我尝试使用CVPixelBufferPoolCreate创建自己的像素缓冲区池。在模拟器上也可以工作，但是在设备上，直到我尝试使用appendPixelBuffer附加像素缓冲区时才会出现失败。

我在网上找到了非常少的信息。我基于我发现的示例来编写代码，但是几天来都没有进展。如果有任何人在使用AVAssetWriter成功完成此操作方面有经验，请看一下并告诉我是否有任何异常。

注意：您将看到注释掉的尝试块。

首先，设置

- (BOOL) openVideoFile: (NSString *) path withSize:(CGSize)imageSize {
size = CGSizeMake (480.0, 320.0);//imageSize;

NSError *error = nil;
videoWriter = [[AVAssetWriter alloc] initWithURL:
                              [NSURL fileURLWithPath:path] fileType:AVFileTypeQuickTimeMovie
                                                          error:&error];
if (error != nil)
    return NO;

NSDictionary *videoCleanApertureSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                            [NSNumber numberWithDouble:size.width], AVVideoCleanApertureWidthKey,
                                            [NSNumber numberWithDouble:size.height], AVVideoCleanApertureHeightKey,
                                            [NSNumber numberWithInt:10], AVVideoCleanApertureHorizontalOffsetKey,
                                            [NSNumber numberWithInt:10], AVVideoCleanApertureVerticalOffsetKey,
                                            nil];


NSDictionary *videoAspectRatioSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                          [NSNumber numberWithInt:1], AVVideoPixelAspectRatioHorizontalSpacingKey,
                                          [NSNumber numberWithInt:1],AVVideoPixelAspectRatioVerticalSpacingKey,
                                          nil];



NSDictionary *codecSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                               //[NSNumber numberWithInt:960000], AVVideoAverageBitRateKey,
                              // [NSNumber numberWithInt:1],AVVideoMaxKeyFrameIntervalKey,
                               videoCleanApertureSettings, AVVideoCleanApertureKey,
                               videoAspectRatioSettings, AVVideoPixelAspectRatioKey,
                               //AVVideoProfileLevelH264Main31, AVVideoProfileLevelKey,
                               nil];

NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                               AVVideoCodecH264, AVVideoCodecKey,
                               codecSettings,AVVideoCompressionPropertiesKey,
                               [NSNumber numberWithDouble:size.width], AVVideoWidthKey,
                               [NSNumber numberWithDouble:size.height], AVVideoHeightKey,
                               nil];
writerInput = [[AVAssetWriterInput
                                    assetWriterInputWithMediaType:AVMediaTypeVideo
                                    outputSettings:videoSettings] retain];
NSMutableDictionary * bufferAttributes = [[NSMutableDictionary alloc] init];
[bufferAttributes setObject: [NSNumber numberWithInt: kCVPixelFormatType_32ARGB]
                   forKey: (NSString *) kCVPixelBufferPixelFormatTypeKey];
[bufferAttributes setObject: [NSNumber numberWithInt: 480]
                   forKey: (NSString *) kCVPixelBufferWidthKey];
[bufferAttributes setObject: [NSNumber numberWithInt: 320]
                   forKey: (NSString *) kCVPixelBufferHeightKey];


//NSDictionary *bufferAttributes = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:kCVPixelFormatType_32ARGB], kCVPixelBufferPixelFormatTypeKey, nil];
//[bufferAttributes setObject: [NSNumber numberWithInt: 640]
//                   forKey: (NSString *) kCVPixelBufferWidthKey];
//[bufferAttributes setObject: [NSNumber numberWithInt: 480]
//                   forKey: (NSString *) kCVPixelBufferHeightKey];
adaptor = [[AVAssetWriterInputPixelBufferAdaptor
            assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput
            sourcePixelBufferAttributes:nil] retain];

//CVPixelBufferPoolCreate (kCFAllocatorSystemDefault,NULL,(CFDictionaryRef)bufferAttributes,&pixelBufferPool);
//Create buffer pool
NSMutableDictionary*     attributes;
attributes = [NSMutableDictionary dictionary];

int width = 480;
int height = 320;

[attributes setObject:[NSNumber numberWithInt:kCVPixelFormatType_32ARGB] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey];
[attributes setObject:[NSNumber numberWithInt:width] forKey: (NSString*)kCVPixelBufferWidthKey];
[attributes setObject:[NSNumber numberWithInt:height] forKey: (NSString*)kCVPixelBufferHeightKey];
CVReturn theError = CVPixelBufferPoolCreate(kCFAllocatorDefault, NULL, (CFDictionaryRef) attributes, &pixelBufferPool);                                           


NSParameterAssert(writerInput);
NSParameterAssert([videoWriter canAddInput:writerInput]);
[videoWriter addInput:writerInput];

writerInput.expectsMediaDataInRealTime = YES;

//Start a session:
[videoWriter startWriting];
[videoWriter startSessionAtSourceTime:kCMTimeZero];

buffer = NULL;
lastTime = kCMTimeZero;
presentTime = kCMTimeZero;

return YES;
}

接下来，介绍两种方法：一种是向写入器追加内容，另一种是创建像素缓冲区以进行追加。

- (void) writeImageToMovie:(CGImageRef)image 
{
    if([writerInput isReadyForMoreMediaData])
    {
//          CMTime frameTime = CMTimeMake(1, 20);
//          CMTime lastTime=CMTimeMake(i, 20); //i is from 0 to 24 of the loop above
//          CMTime presentTime=CMTimeAdd(lastTime, frameTime);

        buffer = [self pixelBufferFromCGImage:image];
        BOOL success = [adaptor appendPixelBuffer:buffer withPresentationTime:presentTime];
        if (!success) NSLog(@"Failed to appendPixelBuffer");
        CVPixelBufferRelease(buffer);

        presentTime = CMTimeAdd(lastTime, CMTimeMake(5, 1000));
        lastTime = presentTime;
    }
    else
    {
        NSLog(@"error - writerInput not ready");
    }
}

- (CVPixelBufferRef)pixelBufferFromCGImage:(CGImageRef)image
{
CVPixelBufferRef pxbuffer;
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                         [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                         [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                         nil];
if (pixelBufferPool == NULL) NSLog(@"pixelBufferPool is null!");
CVReturn status = CVPixelBufferPoolCreatePixelBuffer (NULL, pixelBufferPool, &pxbuffer); 
/*if (pxbuffer == NULL) {
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, size.width,
                                      size.height, kCVPixelFormatType_32ARGB, (CFDictionaryRef) options, 
                                      &pxbuffer);

}*/
//NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);


CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
//NSParameterAssert(pxdata != NULL);

CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, size.width,
                                             size.height, 8, 4*size.width, rgbColorSpace, 
                                             kCGImageAlphaNoneSkipFirst);
//NSParameterAssert(context);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
CGContextDrawImage(context, CGRectMake(90, 10, CGImageGetWidth(image), 
                                       CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);

CVPixelBufferUnlockBaseAddress(pxbuffer, 0);

return pxbuffer;
}

- listing boat

---

哇，这太令人失望了。没有人有头绪吗？ - listing boat

Answer 1

我已经找到了解决此问题的方法。

如果您想让AVAudioPlayer和AVAssetWriter正确地配合使用，必须拥有一个可“混音”（mixable）的音频会话类别。

您可以使用一个可混音的类别，例如AVAudioSessionCategoryAmbient。

但是，我需要使用AVAudioSessionCategoryPlayAndRecord。

您可以通过实现以下内容来将任何类别设置为可混合：

```objc [[AVAudioSession sharedInstance] setCategory:yourDesiredCategory withOptions:AVAudioSessionCategoryOptionMixWithOthers error:nil]; ```

OSStatus propertySetError = 0;

UInt32 allowMixing = true;

propertySetError = AudioSessionSetProperty (
                       kAudioSessionProperty_OverrideCategoryMixWithOthers,  // 1
                       sizeof (allowMixing),                                 // 2
                       &allowMixing                                          // 3
                   );

- listing boat

---

我不太明白这个解决方案如何与您的原始代码相关。 - ninjudd

Answer 2

首先，在创建适配器对象时，您需要传递一些bufferAttributes：

    NSDictionary *bufferAttributes = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey, nil];

AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
                                                 assetWriterInputPixelBufferAdaptorWithAssetWriterInput:_videoWriterInput
                                                 sourcePixelBufferAttributes:bufferAttributes];

然后，删除对 CVPixelBufferPoolCreate 的调用，因为适配器对象已经创建了一个像素缓冲池，所以只需调用这个即可：

                CVPixelBufferRef pixelBuffer = NULL;
            CVPixelBufferPoolCreatePixelBuffer(NULL, adaptor.pixelBufferPool, &pixelBuffer);
            CVPixelBufferLockBaseAddress(pixelBuffer, 0);

            // ...fill the pixelbuffer here

            CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

            CMTime frameTime = CMTimeMake(frameCount,(int32_t) 30);

            BOOL res = [adaptor appendPixelBuffer:pixelBuffer withPresentationTime:frameTime];
            CVPixelBufferRelease(pixelBuffer);
            CFRelease(sampleBuffer);

我认为应该可以了，我曾经遇到过类似的错误，通过按照这里所示的方式创建适配器和像素缓冲区来解决它。

- Adrian Demetrescu

---

除了上面的建议，您还应该测试一下在代码中调用startSessionAtSourceTime后像素缓冲区是否成功创建。您的代码应该检查“if（adaptor.pixelBufferPool == nil）”，然后在进入编码帧循环之前处理失败情况。 - MoDJ