使用已知的内参矩阵和外参矩阵进行立体视觉三维点计算

3
我已经成功地使用两个相机的内部矩阵计算了旋转、平移,并且从左右相机得到了校正后的图像。现在,我想知道如何计算图像中一个点的三维坐标,比如这些绿色点。我看了方程,但它需要基线,而我不知道如何计算基线。您能向我展示使用给定信息(R、T和内部矩阵)计算绿点的三维坐标的过程吗?
FYI: 1. 我还有基础矩阵和本质矩阵,以备我们需要。 2. 原始图像大小为960 x 720。校正后的图像为925 x 669。 3. 左图中的绿点坐标为(562, 185),右图中的绿点坐标为(542, 185)。

enter image description here

1个回答

8
术语“基线”通常只是指翻译。由于您已经有旋转、平移和内参矩阵(我们将它们称为R,TK),因此您可以进行三角测量,不需要基础或本质矩阵(它们可以用于提取R,T等,但您已经拥有它们)。您实际上也不需要对图像进行校正,因为它并不会太大地改变三角测量过程。有许多三角测量的方法,每种方法都有其优缺点,并且有许多库来实现它们。因此,在这里我可以做的只是给您概述问题和潜在解决方案,以及指向资源的指针,您可以直接使用它们,或者作为编写自己代码的灵感来源。
  • Formalization and solution outlines. Let's formalize what we are after here. You have a 3d point X, with two observations x_1 and x_2 respectively in the left and right images. If you backproject them, you obtain two rays:

    ray_1=K^{1}x_1
    rat_2=R*K^{-1}x_2+T  //I'm assuming that [R|T] is the pose of the second camera expressed in the referential of the first camera 
    

    Ideally, you'd want those two rays to meet at point X. Since in practice we always have some noise (discretization noise, rounding errors and so on) the two rays wont meet at X, so the best answer would be a point Q such that

    Q=argmin_X {d(X,ray_1)^2+d(X,ray_2)^2}
    

    where d(.) denotes the Euclidian distance between a line and a point. You can solve this problem as a regular least squares problem, or you can just take the geometric approach (called midpoint) of considering the line segment l that is perpendicular to both ray_1 and ray_2, and take its middle as your solution. Another quick and dirty way is to use the DLT. Basically, you re-write the constrains (i.e. X should be as close as possible to both rays) as a linear system AX=0 and solve it with SVD.

    Usually, the geometric (midpoint) method is the less precise. The DLT based one, while not the most stable numerically, usually produces acceptable results.

  • Ressources that present in depth formalization

    Hartley-Zisserman's book of course! Chapter 12. A simple DLT-based method, which is the one used in opencv (both in the calibration and sfm modules) is explained on page 312. It is very easy to implement, it shouldn't take more that 10 minutes in any language.

    Szeliski'st book. It has an intersting discussion on triangulation in the chapter on SFM, but is not as straight-forward or in depth as Hartley-Zisserman's.

  • Code. You can use the triangulation methods from opencv, either from the calib3d module, or from the contribs/sfm module. Both use the DLT, but the code from the SFM module is more easily understandable (the calib3d code has a lot of old-school C code which is not very pleasant to read). There is also another lib, called openGV, which has a few interesting methods for triangulation.

    cv::triangulatePoints

    cv::sfm::triangulatePoints

    OpenGV

    The openGV git repo doesn't seem very active, and I'm not a big fan of the design of the library, but if I remember correctly (feel free to tell me otherwise) it offers methods other that the DLT for triangulations.

    Naturally, those are all written in C++, but if you use other languages, finding wrappers or similar libraries wont be difficult (with python you still have opencv wrappers, and MATLAB has a bundle module, etc.).


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接