OpenCV Charuco 相机标定不能正确消除畸变

Question

我在进行charuco相机校准时遇到了问题。据我所知，我做的一切都是正确的，但最终的畸变图像比预期的要严重得多。使用4x4的棋盘确实有效，但这时矫正的区域太小了，所以我需要使用7x7的棋盘来解决问题。如果有人能看出我做错了什么，那么帮助将非常感激，我现在有些困惑。以下是情况：

我有四个摄像头的设置，每个摄像头都需要进行校准。对于每个摄像头，我拥有charuco板7x7_1000的11张图片，因此总共有44张图片。

这些是所有相机的原始图像：

https://i.ibb.co/RD6q6kH/Contact-Sheet-001.png

从我在教程中理解的内容来看，charuco相机校准不需要看到所有标记（这就是charuco板的全部意义）。因此，就我所知，源图像是可以的。

我检索每个相机集合的标记和插值棋盘角点，并将其馈送给v2.aruco.calibrateCameraCharuco函数。一切似乎都很好，如下图所示：

https://i.ibb.co/NZvjrRf/f0e0c9665f5911e9bc2b4cedfb69585c-cam01-original-markers.png

然后我调用cv2.undistort函数，但结果并不是我期望的：

https://i.ibb.co/H2GM9tD/f0e0c9665f5911e9bc2b4cedfb69585c-cam01-original-recified.png

这是我根据教程中的示例编写的代码：

def draw_charuco_board( filename, board, size=(2000, 2000) ):
    imboard = board.draw(size)
    cv2.imwrite(filename, imboard)


def detect_charuco_corners( full_board_image_gray, board ):
    parameters = cv2.aruco.DetectorParameters_create()
    return cv2.aruco.detectMarkers(full_board_image_gray, board.dictionary, parameters=parameters)


def charuco_camera_calib( board, filename_glob_pattern, do_flip=False, flip_axis=0 ):
    """
    calibrates the camera using the charuco board
    @see https://docs.opencv.org/trunk/d9/d6a/group__aruco.html#ga54cf81c2e39119a84101258338aa7383
    @see https://github.com/opencv/opencv_contrib/blob/master/modules/aruco/samples/calibrate_camera_charuco.cpp
    """
    charuco_corners = []
    charuco_ids = []
    calib_corners = []
    calib_ids = []
    fns = glob.glob(filename_glob_pattern)
    size = None
    for fn in fns:
        image = cv2.imread(fn, flags=cv2.IMREAD_UNCHANGED)
        image_size = tuple(image.shape[:2][::-1])
        if size is None:
            size = image_size
        elif not image_size == size:
            raise RuntimeError( "charuco_camera_calib:images are not the same size. previous: {} last: {}\n\tlast image: {}".format(size,image_size,fn))
        image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        if do_flip:
            image = cv2.flip(image, flip_axis)
        corners, ids, _ = detect_charuco_corners( image, board )
        charuco_corners.append(np.array(corners))
        charuco_ids.append(np.array(ids))

        if len(corners):
            # refine the detection
            for i, corner in enumerate(corners):
                criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_COUNT, 30, 0.1)
                cv2.cornerSubPix(image, corner,
                                winSize = (10,10),
                                zeroZone = (-1,-1),
                                criteria = criteria)
            # interpolate to find all the chessboard corners
            retval, chessboard_corners, chessboard_ids = cv2.aruco.interpolateCornersCharuco( corners, ids, image, board )
            if chessboard_corners is not None and chessboard_ids is not None:
                calib_corners.append(np.array(chessboard_corners))
                calib_ids.append(np.array(chessboard_ids))
        else:
            raise RuntimeError( "charuco_camera_calib:could not get any markers from image: {}".format(fn))

    retval, camera_matrix, dist_coeffs, rvecs, tvecs = cv2.aruco.calibrateCameraCharuco( np.array(calib_corners), np.array(calib_ids), board, size, None, None )

    new_camera_matrix, roi = cv2.getOptimalNewCameraMatrix(
        camera_matrix, dist_coeffs, size, 1, size
    )
    # map_x, map_y = cv2.initUndistortRectifyMap(
    #     camera_matrix, dist_coeffs, None, new_camera_matrix, size, cv2.CV_32FC1
    # )

    for i, fn in enumerate(fns):
        image = cv2.imread(fn, flags=cv2.IMREAD_UNCHANGED)
        image = cv2.aruco.drawDetectedMarkers(image, charuco_corners[i], charuco_ids[i])
        image = cv2.drawChessboardCorners(image, (6,6), calib_corners[i], True)
        image = cv2.undistort(image, camera_matrix, dist_coeffs)
        # image = cv2.remap(image, map_x, map_y, cv2.INTER_LINEAR)
        new_fn = fn.replace(".png", "_recified.png")
        cv2.imwrite(new_fn, image)

    return camera_matrix, dist_coeffs

该板块是使用以下方式创建的：

charuco_board = calibrate.create_charuco_board( dict_name=cv2.aruco.DICT_7X7_1000, squares_x=7, squares_y=7, square_length=40, marker_size=30 )

charuco_camera_calib被用glob模式对每个相机的所有图像进行调用，图片翻转被关闭，因为图片已经正确定位。

如我所说，我非常感激任何形式的帮助，因为我不知道出了什么问题。

Jonathan

*) tutorial_aruco_calibration

- Jochemspek

Answer 1

好的，我终于解决了它。问题是输入图像太“稀疏”了，如果我可以这么说的话。Charuco棋盘应该覆盖相机的整个视场，而不仅仅是中心。将这些图像添加到混合中解决了问题。

- Jochemspek