Q matrix for repjectImageTo3D function in opencv

Question

I am doing a project in opencv to detect an obstacle to the blind using stereo calibration. I correctly calculated the map of inconsistencies. Now, to find the distance from the obstacle from the camera, I want its 3D coordinates [X, Y, Z], which I suppose can be found using the reprojectImageTo3D () function, but I don’t have a Q matrix to use in of this function, because the Q-matrix that I get from stereoRectify () comes in zero, probably because I used pre-calibrated images. Although I have internal and external parameters of my camera. So my question is, how can I manually create a Q-matrix for direct use in the repjectImageTo3D () function if I know the focal length, baseline and everything else about my camera? What is the main format of the matrix Q?

Answer 1

The shape of the Q-matrix is ​​defined as follows:

In this image, c _x and c _y are the coordinates of the main point in the left camera (if you made a stereo match with the dominant left camera) c ' _x is the x-coordinate of the main point in the right camera (c _x and c' _x will be the same if you specify the flag CV_CALIB_ZERO_DISPARITY for stereoRectify() ), f is the focal length, and T _x is the length of the baseline (possibly a negative length of the baseline, this is a transfer from one optical center to another I think).

I would suggest taking a look at the OpenCV Training book for more information. It is still based on an earlier C interface, but it explains quite well the underlying theory and where I got the Q-matrix form.

Answer 2

if you want to create the Q matrix directly:

 cv::Mat Q; Q.at<double>(0,0)=1.0; Q.at<double>(0,1)=0.0; Q.at<double>(0,2)=0.0; Q.at<double>(0,3)=-160; //cx Q.at<double>(1,0)=0.0; Q.at<double>(1,1)=1.0; Q.at<double>(1,2)=0.0; Q.at<double>(1,3)=-120; //cy Q.at<double>(2,0)=0.0; Q.at<double>(2,1)=0.0; Q.at<double>(2,2)=0.0; Q.at<double>(2,3)=348.087; //Focal Q.at<double>(3,0)=0.0; Q.at<double>(3,1)=0.0; Q.at<double>(3,2)=1.0/95; //1.0/BaseLine Q.at<double>(3,3)=0.0; //cx - cx' 

But you have to calibrate both cameras, and then get the Q matrix from cv :: stereoRectify. Be careful, read the Q-matrix as double values.

Answer 3

A brief description of the three-dimensional reconstruction for stereo vision is given below in the following figure.
You can get the Q matrix and better understand it in the process.

Answer 4

could you help answer that with reprojectImageTo3D () we will have a world coordinate (X, Y, Z). But where exactly is the coordinate of the origin of the world? I ask because I would like to do a 360-degree scan with a stereo camera in order to restore the entire object to world coordinate. thanks