定期从mex文件向MATLAB发送数据

3

我现在正在使用MATLAB完全编写的数据采集工具。我的同事希望我用MATLAB编写这个工具,以便他们可以扩展和修改它。

这个软件需要从两个连接的USB摄像头中获取图片。这些相机的API是用C ++编写的,并且有文档记录-> 这里

问题在于:当我编写一个抓取图片的mex文件时,它会包括相机的初始化和配置加载,这需要很长时间。如果我想通过这种方式抓取图片,MATLAB需要超过1秒钟才能完成任务。一旦初始化完成,相机就能够录制和发送100 fps。我需要能够将每个记录的图像发送回MATLAB。因为所需的记录会话大约需要12小时,我们需要一个具有轻微后处理的实时屏幕。

是否可能在mex文件中生成一个循环,该循环向MATLAB发送数据,然后等待来自MATLAB的返回信号并继续进行?这样,我就可以初始化相机并定期将图像发送到MATLAB。

我是C++初学者,很可能我不理解为什么这不可能的基本概念。

感谢您提供任何建议或资源,我可以查看。

请在下面找到使用Basler提供的Pylon API初始化相机的代码。

// Based on the Grab_MultipleCameras.cpp Routine from Basler
/*
This routine grabs one frame from 2 cameras connected
via two USB3 ports. It directs the Output to MATLAB.
*/

// Include files to use the PYLON API.
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
// Include Files for MEX Generation
#include <matrix.h>
#include <mex.h>   

// Namespace for using pylon objects.
using namespace Pylon;

// We are lazy and use Basler USB namespace
using namespace Basler_UsbCameraParams;

// Standard namespace
using namespace std;

// Define Variables Globally to be remembered between each call
// Filenames for CamConfig
const String_t filenames[] = { "NodeMapCam1.pfs","NodeMapCam2.pfs" };

// Limits the amount of cameras used for grabbing.
static const size_t camerasToUse = 2;

// Create an array of instant cameras for the found devices and 
// avoid exceeding a maximum number of devices.
CBaslerUsbInstantCameraArray cameras(camerasToUse);

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  // Automagically call PylonInitialize and PylonTerminate to ensure the pylon runtime system.
  // is initialized during the lifetime of this object
  PylonAutoInitTerm autoInitTerm;

  try
  {
    // Get the transport layer factory
    CTlFactory& tlFactory = CTlFactory::GetInstance();

    // Get all attached devices and exit application if no device or USB Port is found.
    DeviceInfoList_t devices;
    ITransportLayer *pTL = dynamic_cast<ITransportLayer*>(tlFactory.CreateTl(BaslerUsbDeviceClass));
    if (pTL == NULL)
    {
      throw RUNTIME_EXCEPTION("No USB transport layer available.");
    }

    if (pTL->EnumerateDevices(devices) == 0)
    {
      throw RUNTIME_EXCEPTION("No camera present.");
    }

    // Create and attach all Pylon Devices. Load Configuration
    for (size_t i = 0; i < cameras.GetSize(); ++i)
    {
      cameras[i].Attach(tlFactory.CreateDevice(devices[i]));
    }

    // Open all cameras.
    cameras.Open();

    // Load Configuration and execute Trigger
    for (size_t i = 0; i < cameras.GetSize(); ++i)
    {
      CFeaturePersistence::Load(filenames[i], &cameras[i].GetNodeMap());
    }
    if (cameras[0].IsOpen() && cameras[1].IsOpen())
    {
      mexPrintf("\nCameras are fired up and configuration is applied\n");
      // HERE I WOULD LIKE TO GRAB PICTURES AND SEND THEM
      // PERIODICALLY TO MATLAB.
    }
  }
  catch (GenICam::GenericException &e)
  {
    // Error handling
    mexPrintf("\nAn exception occured:\n");
    mexPrintf(e.GetDescription());
  }

  return;
}
3个回答

1
我遇到了同样的问题,想在Matlab中使用mex API与Basler相机配合。这里的贡献和提示确实帮助了我提出一些想法。但是,有一个比之前提出的解决方案更简单的方法。不需要将相机指针返回给Matlab,因为对象将在多个mex调用之间保留在内存中。以下是我使用新的mex C++ API编程的工作代码。祝你玩得开心。
这是可以使用mex编译的C++文件:
#include <opencv2/core/core.hpp>
#include <opencv2/opencv.hpp>
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <chrono>
#include <string>
using namespace matlab::data;
using namespace std;
using namespace Pylon;
using namespace Basler_UsbCameraParams;
using namespace GenApi;
using namespace cv;
using matlab::mex::ArgumentList;

class MexFunction : public matlab::mex::Function{
    matlab::data::ArrayFactory factory;
    double Number = 0;
    std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
    std::ostringstream stream;
    Pylon::CInstantCamera* camera;
    INodeMap* nodemap;
    double systemTime;
    double cameraTime;
public:

    MexFunction(){}

    void operator()(ArgumentList outputs, ArgumentList inputs) {

        try {
        Number = Number + 1;

        if(!inputs.empty()){

            matlab::data::CharArray InputKey = inputs[0];

            stream << "You called: " << InputKey.toAscii() << std::endl;
            displayOnMATLAB(stream);

            // If "Init" is the input value
            if(InputKey.toUTF16() == factory.createCharArray("Init").toUTF16()){

                // Important: Has to be closed
                PylonInitialize();
                IPylonDevice* pDevice = CTlFactory::GetInstance().CreateFirstDevice();
                camera = new CInstantCamera(pDevice);

                nodemap = &camera->GetNodeMap();

                camera->Open();

                camera->RegisterConfiguration( new CSoftwareTriggerConfiguration, RegistrationMode_ReplaceAll, Cleanup_Delete);

                CharArray DeviceInfo = factory.createCharArray(camera -> GetDeviceInfo().GetModelName().c_str());

                stream << "Message: Used Camera is " << DeviceInfo.toAscii() << std::endl;
                displayOnMATLAB(stream);
            }


            // If "Grab" is  called
            if(InputKey.toUTF16() == factory.createCharArray("Grab").toUTF16()){
                    static const uint32_t c_countOfImagesToGrab = 1;
                    camera -> StartGrabbing(c_countOfImagesToGrab);
                    CGrabResultPtr ptrGrabResult;
                    Mat openCvImage;
                    CImageFormatConverter formatConverter;
                    CPylonImage pylonImage;
                    while (camera -> IsGrabbing()) {
                        camera -> RetrieveResult(5000, ptrGrabResult, TimeoutHandling_ThrowException);
                        if (ptrGrabResult->GrabSucceeded()) {
                            formatConverter.Convert(pylonImage, ptrGrabResult);

                            Mat openCvImage = cv::Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(), CV_8UC1,(uint8_t *)pylonImage.GetBuffer(), Mat::AUTO_STEP);

                            const size_t rows = openCvImage.rows;
                            const size_t cols = openCvImage.cols;

                            matlab::data::TypedArray<uint8_t> Yp = factory.createArray<uint8_t>({ rows, cols });

                            for(int i = 0 ;i < openCvImage.rows; ++i){

                                for(int j = 0; j < openCvImage.cols; ++j){

                                    Yp[i][j] = openCvImage.at<uint8_t>(i,j);
                                }
                            }
                            outputs[0] =  Yp;
                        }
                    }
            }

            // if "Delete"
            if(InputKey.toUTF16() == factory.createCharArray("Delete").toUTF16()){
                camera->Close();
                PylonTerminate();

                stream << "Camera instance removed" << std::endl;
                displayOnMATLAB(stream);
                Number = 0;
                //mexUnlock();
            }
        }

        // ----------------------------------------------------------------
        stream << "Anzahl der Aufrufe bisher: " << Number << std::endl;
        displayOnMATLAB(stream);
        // ----------------------------------------------------------------

        }
        catch (const GenericException & ex) {
            matlabPtr->feval(u"disp", 0, std::vector<Array>({factory.createCharArray(ex.GetDescription()) }));
        }
    }

    void displayOnMATLAB(std::ostringstream& stream) {
        // Pass stream content to MATLAB fprintf function
        matlabPtr->feval(u"fprintf", 0,
                         std::vector<Array>({ factory.createScalar(stream.str()) }));
        // Clear stream buffer
        stream.str("");
    }
};


这个 mex 文件可以通过以下命令从 Matlab 中调用:
% Initializes the camera. The camera parameters can also be loaded here.
NameOfMexFile('Init');

% Camera image is captured and sent back to Matlab
[Image] = NameOfMexFile('Grab');

% The camera connection has to be closed.
NameOfMexFile('Delete');


欢迎对此代码进行优化和改进。代码的效率仍存在问题,图像采集需要约0.6秒。这主要是由于需要将cv::mat图像转换为TypedArray,以便将其返回到Matlab。请参见两个循环中的此行:Yp[i][j] = openCvImage.at<uint8_t>(i,j); 我还没有想出如何使其更加高效。此外,该代码无法用于将多个图像返回到Matlab。
也许有人有想法或提示可以使从cv::mat到Matlab数组类型的转换更快。我已经在另一篇帖子中提到了这个问题。请参见这里:如何使用Mex C++ API将Opencv图像cv::mat返回到Matlab

1

不要将数据发送到MATLAB,而是应该使您的mex文件存储与相机相关的设置,以便它不会在每个调用中进行初始化。一种方法是为mex文件使用两种调用模式。一个是“init”调用,另一个是获取数据的调用。在MATLAB中的伪代码如下:

cameraDataPtr = myMex('init');
while ~done
   data = myMex('data', cameraDataPtr);
end

在您的mex文件中,您应该将相机设置存储在一个跨调用持久的内存中。一种方法是使用C++中的“new”。您应该将此内存指针作为int64类型返回给MATLAB,如上面的代码中所示的cameraDataPtr。当需要“data”时,您应该以cameraDataPtr作为输入,并将其转换回相机设置。例如,在C++中,如果有一个存储所有与相机相关数据的CameraSettings对象,则粗略的伪代码如下:
if prhs[0] == 'init' { // Use mxArray api to check this
  cameraDataPtr = new CameraSettings; // Initialize and setup camera
  plhs[0] = createMxArray(cameraDataPtr); // Use mxArray API to create int64 from pointer
  return;
} else {
   // Need data
   cameraDataPtr = getCameraDataPtr(plhs[1]);
   // Use cameraDataPtr after checking validity to get next frame
}

这个方法可行是因为mex文件一旦被加载,就会一直保留在内存中,直到你清除它们。当mex文件从内存中卸载时,你应该使用mexAtExit函数释放相机资源。如果你的mex文件只在这里使用,也可以使用“static”将相机设置存储在c++中。这样可以避免编写一些mxArray处理代码来返回你的c++指针。
如果你将对这个mex文件的调用封装到MATLAB对象中,你可以更轻松地控制初始化和运行时过程,并向用户呈现更好的API。

非常好的答案,非常感谢。它对我帮助很大! - Julian Zimmermann

1
您可以循环发送图像回到MATLAB,但您希望它在工作区中是什么样子(多个2D图像,一个巨大的3D/4D数组,单元格等)?我认为您正在寻找的解决方案是一个有状态的MEX文件,它可以通过'init''new'命令启动,然后使用'capture'命令重复调用已初始化的相机。

这里有一个如何在我的GitHub上实现此操作的示例。从class_wrapper_template.cpp开始,并根据您的命令(newcapturedelete等)进行修改。以下是其核心的一个粗略且未经测试的示例(也可在Gist.GitHub上镜像):

// pylon_mex_camera_interface.cpp
#include "mex.h"
#include <vector>
#include <map>
#include <algorithm>
#include <memory>
#include <string>
#include <sstream>

////////////////////////  BEGIN Step 1: Configuration  ////////////////////////
// Include your class declarations (and PYLON API).
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>

// Define class_type for your class
typedef CBaslerUsbInstantCameraArray class_type;

// List actions
enum class Action
{
    // create/destroy instance - REQUIRED
    New,
    Delete,
    // user-specified class functionality
    Capture
};

// Map string (first input argument to mexFunction) to an Action
const std::map<std::string, Action> actionTypeMap =
{
    { "new",        Action::New },
    { "delete",     Action::Delete },
    { "capture",    Action::Capture }
}; // if no initializer list available, put declaration and inserts into mexFunction

using namespace Pylon;
using namespace Basler_UsbCameraParams;

const String_t filenames[] = { "NodeMapCam1.pfs","NodeMapCam2.pfs" };
static const size_t camerasToUse = 2;
/////////////////////////  END Step 1: Configuration  /////////////////////////

// boilerplate until Step 2 below
typedef unsigned int handle_type;
typedef std::pair<handle_type, std::shared_ptr<class_type>> indPtrPair_type; // or boost::shared_ptr
typedef std::map<indPtrPair_type::first_type, indPtrPair_type::second_type> instanceMap_type;
typedef indPtrPair_type::second_type instPtr_t;

// getHandle pulls the integer handle out of prhs[1]
handle_type getHandle(int nrhs, const mxArray *prhs[]);
// checkHandle gets the position in the instance table
instanceMap_type::const_iterator checkHandle(const instanceMap_type&, handle_type);

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {

    // static storage duration object for table mapping handles to instances
    static instanceMap_type instanceTab;

    if (nrhs < 1 || !mxIsChar(prhs[0]))
        mexErrMsgTxt("First input must be an action string ('new', 'delete', or a method name).");

    char *actionCstr = mxArrayToString(prhs[0]); // convert char16_t to char
    std::string actionStr(actionCstr); mxFree(actionCstr);

    for (auto & c : actionStr) c = ::tolower(c); // remove this for case sensitivity

    if (actionTypeMap.count(actionStr) == 0)
        mexErrMsgTxt(("Unrecognized action (not in actionTypeMap): " + actionStr).c_str());

    // If action is not 'new' or 'delete' try to locate an existing instance based on input handle
    instPtr_t instance;
    if (actionTypeMap.at(actionStr) != Action::New && actionTypeMap.at(actionStr) != Action::Delete) {
        handle_type h = getHandle(nrhs, prhs);
        instanceMap_type::const_iterator instIt = checkHandle(instanceTab, h);
        instance = instIt->second;
    }

    //////// Step 2: customize each action in the switch in mexFuction ////////
    switch (actionTypeMap.at(actionStr))
    {
    case Action::New:
    {
        if (nrhs > 1 && mxGetNumberOfElements(prhs[1]) != 1)
            mexErrMsgTxt("Second argument (optional) must be a scalar, N.");

        handle_type newHandle = instanceTab.size() ? (instanceTab.rbegin())->first + 1 : 1;

        // Store a new CBaslerUsbInstantCameraArray in the instance map
        std::pair<instanceMap_type::iterator, bool> insResult = 
            instanceTab.insert(indPtrPair_type(newHandle, std::make_shared<class_type>(camerasToUse)));

        if (!insResult.second) // sanity check
            mexPrintf("Oh, bad news.  Tried to add an existing handle."); // shouldn't ever happen
        else
            mexLock(); // add to the lock count

        // return the handle
        plhs[0] = mxCreateDoubleScalar(insResult.first->first); // == newHandle

        // Get all attached devices and exit application if no device or USB Port is found.
        CTlFactory& tlFactory = CTlFactory::GetInstance();
        // Check if cameras are attached
        ITransportLayer *pTL = dynamic_cast<ITransportLayer*>(tlFactory.CreateTl(BaslerUsbDeviceClass));
        // todo: some checking here... (pTL == NULL || pTL->EnumerateDevices(devices) == 0)

        // Create and attach all Pylon Devices. Load Configuration
        CBaslerUsbInstantCameraArray &cameras = *instance;
        DeviceInfoList_t devices;
        for (size_t i = 0; i < cameras.GetSize(); ++i) {
          cameras[i].Attach(tlFactory.CreateDevice(devices[i]));
        }

        // Open all cameras.
        cameras.Open();

        // Load Configuration and execute Trigger
        for (size_t i = 0; i < cameras.GetSize(); ++i) {
          CFeaturePersistence::Load(filenames[i], &cameras[i].GetNodeMap());
        }

        if (cameras[0].IsOpen() && cameras[1].IsOpen()) {
            mexPrintf("\nCameras are fired up and configuration is applied\n");

        break;
    }
    case Action::Delete:
    {
        instanceMap_type::const_iterator instIt = checkHandle(instanceTab, getHandle(nrhs, prhs));
        (instIt->second).close(); // may be unnecessary if d'tor does it
        instanceTab.erase(instIt);
        mexUnlock();
        plhs[0] = mxCreateLogicalScalar(instanceTab.empty()); // just info
        break;
    }
    case Action::Capture:
    {
        CBaslerUsbInstantCameraArray &cameras = *instance; // alias for the instance

        // TODO: create output array and capture a frame(s) into it
        plhs[0] = mxCreateNumericArray(...);
        pixel_type* data = (pixel_type*) mxGetData(plhs[0]);
        cameras[0].GrabOne(...,data,...);
        // also for cameras[1]?
        }
    }
    default:
        mexErrMsgTxt(("Unhandled action: " + actionStr).c_str());
        break;
    }
    ////////////////////////////////  DONE!  ////////////////////////////////
}

// See github for getHandle and checkHandle

这句话的意思是:这个想法就是你只需要调用一次来进行初始化。保留了HTML,不做解释。
>> h = pylon_mex_camera_interface('new');

然后您将在MATLAB循环中调用它以获取帧:

>> newFrame{i} = pylon_mex_camera_interface('capture', h);

完成后:

>> pylon_mex_camera_interface('delete', h)

你应该使用MATLAB类进行包装。可以轻松地从cppclass.m派生。有关派生类示例,请参见pqheap.m

哇,非常感谢。你的努力和出色的回答!这对我帮助很大。 - Julian Zimmermann
@JulianCarpenter 没问题。我相信会有错误,等你解决了实际语法后,我很乐意将其整合进去。 - chappjc
我可以问一个跟进的问题吗?我尝试实现你的代码,但是遇到了一些困难...在这行代码 CBaslerUsbInstantCameraArray &cameras = instance; 中,你正在使用名为 instance 的智能指针初始化对象引用....我的编译器对此抱怨。我使用 Visual Studio 2015,并得到以下输出:error C2440: 'initializing': cannot convert from 'instPtr_t' to 'class_type &' ...如果你能指点我 - ;) - 找到解决方案的正确方向,那就太棒了! - Julian Zimmermann
@JulianCarpenter 我的错误,instance 的类型是 shared_ptr<class_type> 而不仅仅是 class_type。当使用实例时,您必须取消引用 shared_ptr。请尝试使用 CBaslerUsbInstantCameraArray&cameras = *instance;。希望我能编译您的代码...如果失败了,请尝试直接使用它,例如 (*instance)[i].Attach() - chappjc
非常感谢!通过对解除引用的指针进行索引,您刚刚拯救了我的一天。相机在多次调用中保持打开状态。当我完成整个项目后,我会上传并给您链接!再次感谢。 - Julian Zimmermann
@JulianCarpenter 太好了!请上传最终产品。如果您不介意的话,我希望您能链接回问题和我的GitHub文件夹。谢谢! - chappjc

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接