在iOS上使用OpenAL进行声音捕获

Question

我正在尝试使用OpenAL在iOS上进行声音捕获（我正在编写一个跨平台库，因此避免使用特定于iOS的录制声音方式）。 开箱即用的OpenAL捕获无法正常工作，但是已经存在已知的解决方法：在开始捕获之前打开输出上下文。这个解决方案对我在iOS 5.0上奏效了。

然而，在iOS 5.1.1上，这种解决方法只对我尝试录制的第一个样本有帮助。（在开始捕获之前，我将我的AudioSession切换到PlayAndRecord，并打开默认输出设备。录制完样本后，我关闭设备并将会话切换回之前的状态。） 对于第二个样本，重新打开输出上下文无法解决问题，也无法捕获任何声音。

是否有已知的方法来解决这个问题？

// Here's what I do before starting the recording
oldAudioSessionCategory = [audioSession category];
[audioSession setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];
[audioSession setActive:YES error:nil];
// We need to have an active context. If there is none, create one.
if (!alcGetCurrentContext()) {
    outputDevice = alcOpenDevice(NULL);
    outputContext = alcCreateContext(outputDevice, NULL);
    alcMakeContextCurrent(outputContext);
}

// Capture itself
inputDevice = alcCaptureOpenDevice(NULL, frequency, FORMAT, bufferSize);
....
alcCaptureCloseDevice(inputDevice);

// Restoring the audio state to whatever it had been before capture
if (outputContext) {
    alcDestroyContext(outputContext);
    alcCloseDevice(outputDevice);
}
[[AVAudioSession sharedInstance] setCategory:oldAudioSessionCategory 
                                 error:nil];

- gogabr

---

可能更新一下比较合适。我最终在AudioUnits的基础上重新实现了OpenAL捕获功能，放弃了苹果的实现。 - gogabr

你的实现是否作为开源代码可用？这对于其他试图达到同样目标的人将会非常有帮助。 - Bijoy Thangaraj

@bijoy-thangaraj 抱歉，我不知怎么错过了你的请求。我正在添加代码。 - gogabr

Answer 1

以下是我用来模拟捕获扩展的代码。 一些注释： 1. 在整个项目中，OpenKD用于线程原语等。您可能需要替换这些调用。 2. 我必须对启动捕获时的延迟进行斗争。因此，我不断地读取声音输入并在不需要时将其丢弃。（例如，建议使用此类解决方案：here。）这反过来又需要捕捉onResignActive通知，以释放麦克风的控制权。您可能或可能不想使用这样的修补程序。 3. 与alcGetIntegerv(device, ALC_CAPTURE_SAMPLES, 1, &res)不同，我必须定义一个单独的函数alcGetAvailableSamples。

简而言之，这段代码不太可能在您的项目中直接使用，但希望您可以根据自己的需求进行调整。

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <KD/kd.h>
#include <AL/al.h>
#include <AL/alc.h>

#include <AudioToolbox/AudioToolbox.h>
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>

#include "KD/kdext.h"

struct InputDeviceData {
    int id;
    KDThreadMutex *mutex;
    AudioUnit audioUnit;
    int nChannels;
    int frequency;
    ALCenum format;
    int sampleSize;
    uint8_t *buf;
    size_t bufSize;    // in bytes
    size_t bufFilledBytes;  // in bytes
    bool started;
};

static struct InputDeviceData *cachedInData = NULL;

static OSStatus renderCallback (void                        *inRefCon,
                                AudioUnitRenderActionFlags  *ioActionFlags,
                                const AudioTimeStamp        *inTimeStamp,
                                UInt32                      inBusNumber,
                                UInt32                      inNumberFrames,
                                AudioBufferList             *ioData);
static AudioUnit getAudioUnit();
static void setupNotifications();
static void destroyCachedInData();
static struct InputDeviceData *setupCachedInData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples);
static struct InputDeviceData *getInputDeviceData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples);

/** I only have to use NSNotificationCenter instead of CFNotificationCenter
 *  because there is no published name for WillResignActive/WillBecomeActive
 *  notifications in CoreFoundation.
 */
@interface ALCNotificationObserver : NSObject
- (void)onResignActive;
@end
@implementation ALCNotificationObserver
- (void)onResignActive {
    destroyCachedInData();
}
@end

static void setupNotifications() {
    static ALCNotificationObserver *observer = NULL;
    if (!observer) {
        observer = [[ALCNotificationObserver alloc] init];
        [[NSNotificationCenter defaultCenter] addObserver:observer selector:@selector(onResignActive) name:UIApplicationWillResignActiveNotification object:nil];
    }
}

static OSStatus renderCallback (void                        *inRefCon,
                                AudioUnitRenderActionFlags  *ioActionFlags,
                                const AudioTimeStamp        *inTimeStamp,
                                UInt32                      inBusNumber,
                                UInt32                      inNumberFrames,
                                AudioBufferList             *ioData) {
    struct InputDeviceData *inData = (struct InputDeviceData*)inRefCon;

    kdThreadMutexLock(inData->mutex);
    size_t bytesToRender = inNumberFrames * inData->sampleSize;
    if (bytesToRender + inData->bufFilledBytes <= inData->bufSize) {
        OSStatus status;
        struct AudioBufferList audioBufferList; // 1 buffer is declared inside the structure itself.
        audioBufferList.mNumberBuffers = 1;
        audioBufferList.mBuffers[0].mNumberChannels = inData->nChannels;
        audioBufferList.mBuffers[0].mDataByteSize = bytesToRender;
        audioBufferList.mBuffers[0].mData = inData->buf + inData->bufFilledBytes;
        status = AudioUnitRender(inData->audioUnit, 
                                 ioActionFlags, 
                                 inTimeStamp, 
                                 inBusNumber, 
                                 inNumberFrames, 
                                 &audioBufferList);
        if (inData->started) {
            inData->bufFilledBytes += bytesToRender;
        }
    } else {
        kdLogFormatMessage("%s: buffer overflow", __FUNCTION__);
    }
    kdThreadMutexUnlock(inData->mutex);

    return 0;
}

static AudioUnit getAudioUnit() {
    static AudioUnit audioUnit = NULL;

    if (!audioUnit) {
        AudioComponentDescription ioUnitDescription;

        ioUnitDescription.componentType          = kAudioUnitType_Output;
        ioUnitDescription.componentSubType       = kAudioUnitSubType_VoiceProcessingIO;
        ioUnitDescription.componentManufacturer  = kAudioUnitManufacturer_Apple;
        ioUnitDescription.componentFlags         = 0;
        ioUnitDescription.componentFlagsMask     = 0;

        AudioComponent foundIoUnitReference = AudioComponentFindNext(NULL,
                                                                     &ioUnitDescription);
        AudioComponentInstanceNew(foundIoUnitReference,
                                  &audioUnit);

        if (audioUnit == NULL) {
            kdLogMessage("Could not obtain AudioUnit");
        }
    }

    return audioUnit;
}

static void destroyCachedInData() {
    OSStatus status;
    if (cachedInData) {
        status = AudioOutputUnitStop(cachedInData->audioUnit);
        status = AudioUnitUninitialize(cachedInData->audioUnit);
        free(cachedInData->buf);
        kdThreadMutexFree(cachedInData->mutex);
        free(cachedInData);
        cachedInData = NULL;
    }
}

static struct InputDeviceData *setupCachedInData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples) {
    static int idCount = 0;
    OSStatus status;
    int bytesPerFrame = (format == AL_FORMAT_MONO8) ? 1 :
                        (format == AL_FORMAT_MONO16) ? 2 :
                        (format == AL_FORMAT_STEREO8) ? 2 :
                        (format == AL_FORMAT_STEREO16) ? 4 : -1;
    int channelsPerFrame = (format == AL_FORMAT_MONO8) ? 1 :
                           (format == AL_FORMAT_MONO16) ? 1 :
                           (format == AL_FORMAT_STEREO8) ? 2 :
                           (format == AL_FORMAT_STEREO16) ? 2 : -1;
    int bitsPerChannel = (format == AL_FORMAT_MONO8) ? 8 :
                         (format == AL_FORMAT_MONO16) ? 16 :
                         (format == AL_FORMAT_STEREO8) ? 8 :
                         (format == AL_FORMAT_STEREO16) ? 16 : -1;

    cachedInData = malloc(sizeof(struct InputDeviceData));
    cachedInData->id = ++idCount;
    cachedInData->format = format;
    cachedInData->frequency = frequency;
    cachedInData->mutex = kdThreadMutexCreate(NULL);
    cachedInData->audioUnit = audioUnit;
    cachedInData->nChannels = channelsPerFrame;
    cachedInData->sampleSize = bytesPerFrame;
    cachedInData->bufSize = bufferSizeInSamples * bytesPerFrame;
    cachedInData->buf = malloc(cachedInData->bufSize);
    cachedInData->bufFilledBytes = 0;
    cachedInData->started = FALSE;

    UInt32 enableOutput        = 1;    // to enable output
    status = AudioUnitSetProperty(audioUnit,
                                  kAudioOutputUnitProperty_EnableIO,
                                  kAudioUnitScope_Input,
                                  1,
                                  &enableOutput, sizeof(enableOutput));

    struct AudioStreamBasicDescription basicDescription;
    basicDescription.mSampleRate = (Float64)frequency;
    basicDescription.mFormatID = kAudioFormatLinearPCM;
    basicDescription.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    basicDescription.mBytesPerPacket = bytesPerFrame;
    basicDescription.mFramesPerPacket = 1;
    basicDescription.mBytesPerFrame = bytesPerFrame;
    basicDescription.mChannelsPerFrame = channelsPerFrame;
    basicDescription.mBitsPerChannel = bitsPerChannel;
    basicDescription.mReserved = 0;

    status = AudioUnitSetProperty(audioUnit, 
                                  kAudioUnitProperty_StreamFormat, // property key 
                                  kAudioUnitScope_Output,        // scope
                                  1,                             // 1 is output
                                  &basicDescription, sizeof(basicDescription));      // value

    AURenderCallbackStruct renderCallbackStruct;
    renderCallbackStruct.inputProc = renderCallback;
    renderCallbackStruct.inputProcRefCon = cachedInData;
    status = AudioUnitSetProperty(audioUnit, 
                                  kAudioOutputUnitProperty_SetInputCallback, // property key 
                                  kAudioUnitScope_Output,        // scope
                                  1,                             // 1 is output
                                  &renderCallbackStruct, sizeof(renderCallbackStruct));      // value

    status = AudioOutputUnitStart(cachedInData->audioUnit);

    return cachedInData;
}

static struct InputDeviceData *getInputDeviceData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples) {
    if (cachedInData && 
        (cachedInData->frequency != frequency ||
         cachedInData->format != format ||
         cachedInData->bufSize / cachedInData->sampleSize != bufferSizeInSamples)) {
            kdAssert(!cachedInData->started);
            destroyCachedInData();
        }
    if (!cachedInData) {
        setupCachedInData(audioUnit, frequency, format, bufferSizeInSamples);
        setupNotifications();
    }

    return cachedInData;
}


ALC_API ALCdevice* ALC_APIENTRY alcCaptureOpenDevice(const ALCchar *devicename, ALCuint frequency, ALCenum format, ALCsizei buffersizeInSamples) {
    kdAssert(devicename == NULL);    

    AudioUnit audioUnit = getAudioUnit();
    struct InputDeviceData *res = getInputDeviceData(audioUnit, frequency, format, buffersizeInSamples);
    return (ALCdevice*)res->id;
}

ALC_API ALCboolean ALC_APIENTRY alcCaptureCloseDevice(ALCdevice *device) {
    alcCaptureStop(device);
    return true;
}

ALC_API void ALC_APIENTRY alcCaptureStart(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to start a stale AL capture device");
        return;
    }
    cachedInData->started = TRUE;
}

ALC_API void ALC_APIENTRY alcCaptureStop(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to stop a stale AL capture device");
        return;
    }
    cachedInData->started = FALSE;
}

ALC_API ALCint ALC_APIENTRY alcGetAvailableSamples(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to get sample count from a stale AL capture device");
        return 0;
    }
    ALCint res;
    kdThreadMutexLock(cachedInData->mutex);
    res = cachedInData->bufFilledBytes / cachedInData->sampleSize;
    kdThreadMutexUnlock(cachedInData->mutex);
    return res;
}

ALC_API void ALC_APIENTRY alcCaptureSamples(ALCdevice *device, ALCvoid *buffer, ALCsizei samples) {    
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to get samples from a stale AL capture device");
        return;
    }
    size_t bytesToCapture = samples * cachedInData->sampleSize;
    kdAssert(cachedInData->started);
    kdAssert(bytesToCapture <= cachedInData->bufFilledBytes);

    kdThreadMutexLock(cachedInData->mutex);
    memcpy(buffer, cachedInData->buf, bytesToCapture);
    memmove(cachedInData->buf, cachedInData->buf + bytesToCapture, cachedInData->bufFilledBytes - bytesToCapture);
    cachedInData->bufFilledBytes -= bytesToCapture;
    kdThreadMutexUnlock(cachedInData->mutex);
}

- gogabr

Answer 2

我找到了一种让苹果的OpenAL工作的方法。 在我的原始代码片段中，您需要在alcDestroyContext（outputContext）之前调用 alcMakeContextCurrent（NULL）。

- gogabr