问题描述
我试图了解相机校准/3D 重建并面临 cv::fisheye::distort/undistortPoints 函数的奇怪行为。我希望鱼眼模型沿着将它连接到主点 (cx,cy) 的射线移动一个点，但是，它没有。此外，functions cv::fisheye::distortPoints 和 cv::fisheye::undistortPoints 并不互逆（正如人们所期望的那样）。
以下代码创建了一个具有畸变系数和不畸变的相机矩阵，然后将任意点畸变回来。相机内在参数和失真系数的值取自公共数据集。
cv::Mat camera_matrix = cv::Mat::zeros(3,3,CV_64F);
camera_matrix.at<double>(0,0) = 190.9784;
camera_matrix.at<double>(1,1) = 190.9733;
camera_matrix.at<double>(0,2) = 254.9317;
camera_matrix.at<double>(1,2) = 256.8974;
camera_matrix.at<double>(2,2) = 1;
std::cout << "Camera matrix: \n" << camera_matrix << "\n" <<std::endl;
cv::Mat distortion_coefficients(4,1,CV_64F);
distortion_coefficients.at<double>(0) = 0.003482;
distortion_coefficients.at<double>(1) = 0.000715;
distortion_coefficients.at<double>(2) = -0.0020532;
distortion_coefficients.at<double>(3) = 0.000203;
std::cout << "distortion coefficients\n"<< distortion_coefficients<< "\n" << std::endl;
cv::Mat original_point(1,CV_64FC2);
original_point.at<cv::Point2d>(0).x= 7.7;
original_point.at<cv::Point2d>(0).y= 9.9;
cv::Mat undistorted,distorted;
cv::fisheye::undistortPoints(original_point,undistorted,camera_matrix,distortion_coefficients,cv::Mat(),camera_matrix);
cv::fisheye::distortPoints(undistorted,distorted,distortion_coefficients);
std:: cout << "Original point: " << original_point.at<cv::Point2d>(0).x << " " << original_point.at<cv::Point2d>(0).y << std::endl;
std:: cout << "Undistorted point: " << undistorted.at<cv::Point2d>(0).x << " " << undistorted.at<cv::Point2d>(0).y<< std::endl;
std:: cout << "distorted point: " << distorted.at<cv::Point2d>(0).x << " " << distorted.at<cv::Point2d>(0).y;
结果是
Camera matrix:
[190.9784,254.9317;
0,190.9733,256.8974;
0,1]
distortion coefficients
[0.003482;
0.000715;
-0.0020532;
0.000203]
Original point: 7.7 9.9
Undistorted point: 8905.69 8899.45
distorted point: 464.919 466.732
靠近左上角的点被移到右下角。
cv::fisheye::undistortimage 正在处理数据集图像 - 曲线被转回线。
我错过了什么？
解决方法
你错过了两件事。
- 您使用的参数不正确 鱼眼::扭曲点（）函数。你需要标准化的点。 来自docs，但不是很清楚：
请注意，该函数假定未失真点的相机内在矩阵为恒等。这意味着如果你想用 undistortPoints() 变换回不失真的点，你必须将它们与 P−1 相乘。
- 您需要意识到并非失真图像中的所有点都最终出现在未失真图像中。外推只能在一定程度上起作用，超过某个点时，未失真和重新失真不会互为逆。
为了对点进行归一化，您首先需要对这些点进行均质化（转换为 3d）并将逆矩阵与每个均质化点相乘以获得归一化点。
您可以对新的相机矩阵使用 fisheye::estimateNewCameraMatrixForUndistortRectify 来调整源和目标中有效像素之间的平衡。但是，如果您希望未失真的点与未失真图像中的点相匹配，则需要将这个新的相机矩阵用于 undistortImage 中的 Knew。
cv::Mat k = cv::Mat::zeros(3,3,CV_64F);
k.at<double>(0,0) = 190.9784;
k.at<double>(1,1) = 190.9733;
k.at<double>(0,2) = 254.9317;
k.at<double>(1,2) = 256.8974;
k.at<double>(2,2) = 1;
std::cout << "Camera matrix: \n" << k << "\n" <<std::endl;
cv::Mat d(4,1,CV_64F);
d.at<double>(0) = 0.003482;
d.at<double>(1) = 0.000715;
d.at<double>(2) = -0.0020532;
d.at<double>(3) = 0.000203;
std::cout << "Distortion coefficients\n"<< d<< "\n" << std::endl;
cv::Mat points_original(1,4,CV_64FC2);
points_original.at<cv::Point2d>(0).x= 7.7;
points_original.at<cv::Point2d>(0).y= 9.9;
points_original.at<cv::Point2d>(1).x= 30;
points_original.at<cv::Point2d>(1).y= 30;
points_original.at<cv::Point2d>(2).x= 40;
points_original.at<cv::Point2d>(2).y= 40;
points_original.at<cv::Point2d>(3).x= 50;
points_original.at<cv::Point2d>(3).y= 50;
cv::Mat nk;
// float balance = 1;
// fisheye::estimateNewCameraMatrixForUndistortRectify(k,d,cv::Size(512,512),Mat::eye(3,CV_64FC1),nk,balance);
nk = k;
std::cout << "New Camera matrix: \n" << nk << "\n" <<std::endl;
cv::Mat points_undistorted,points_redistorted;
cv::fisheye::undistortPoints(points_original,points_undistorted,k,cv::Mat(),nk);
// {x,y} -> {x,y,1}
std::vector<Point3d> points_undistorted_homogeneous;
convertPointsToHomogeneous(points_undistorted,points_undistorted_homogeneous);
Mat cam_intr_inv = nk.inv();
for(int i=0;i<points_undistorted_homogeneous.size();++i){
Mat p(Size(1,3),CV_64FC1);
p.at<double>(0,0) = points_undistorted_homogeneous[i].x;
p.at<double>(1,0) = points_undistorted_homogeneous[i].y;
p.at<double>(2,0) = points_undistorted_homogeneous[i].z;
Mat q = cam_intr_inv*p;
points_undistorted_homogeneous[i].x = q.at<double>(0,0);
points_undistorted_homogeneous[i].y = q.at<double>(1,0);
points_undistorted_homogeneous[i].z = q.at<double>(2,0);
}
std::vector<Point2d> points_undistorted_normalized;
convertPointsFromHomogeneous(points_undistorted_homogeneous,points_undistorted_normalized);
fisheye::distortPoints(points_undistorted_normalized,points_redistorted,d);
for(int i = 0;i<points_original.size().width;++i){
std:: cout << "Original point: " << points_original.at<cv::Point2d>(i) << "\n";
std:: cout << "Undistorted point: " << points_undistorted.at<cv::Point2d>(i) << "\n";
std:: cout << "Redistorted point: " << points_redistorted.at<cv::Point2d>(i) << "\n\n";
}
结果：
Original point: [7.7,9.9]
Undistorted point: [8905.69,8899.45]
Redistorted point: [463.048,464.816]
Original point: [30,30]
Undistorted point: [8125.4,8196.15]
Redistorted point: [461.864,465.638]
Original point: [40,40]
Undistorted point: [7775.49,7846.24]
Redistorted point: [461.725,465.582]
Original point: [50,50]
Undistorted point: [-3848.98,-3886.37]
Redistorted point: [50,50]
如您所见，角球的前 40 分失败了。也许您可以通过再次校准并在图像边缘（以不同角度）添加更多棋盘角来改善这一点。