Search for rotation matrices between two cameras for "Stereorectify"

Question

So, I have a depth map and the external properties and interiors of the camera. I want to return 3D points and surface normals. I am using the ReprojectImageTo3D function. In the stereo straightening function, to find Q, how to get the rotation matrix between the coordinate systems of the 1st and 2nd cameras? I have an individual rotation matrix and translation vector, but how do I get it "between cameras?" .Also it will give me 3D points. Is there a way to generate surface normals?

Answer 1

Given that you have an external matrix of both cameras, can't you just take the inverse external matrix of camera 1 times the external matrix of camera 2? Also, for a direct connection between the two cameras, take a look at the main matrix (or, more specifically, the main matrix). See if you can find a copy of Hartley and Sisserman's Multiple Geometry of Views.

As for the surface normals, you can calculate them yourself by calculating the cross products at the corners of the triangles. However, first you need a restored cloud of 3D points.