How to match the depth of the frame with the color frame WITHOUT Kinect

Question

I am trying to display a depth frame in a color frame without kinect. I previously acquired images using Kinect, and now, based on the image of depth, where I can clearly see the shape of a person’s body, I want to match both color and depth without using the kinect MapDephtFrametoColorFrame method (I cannot apply this method without using Kinect )

How to do it?

I thought about acquiring the depth points, where they are 255 (threshold value), and then matches the [x, y] points with the color, but I have no results.

Thank you in advance

Answer 1

Earlier, I found an article indicating how to do this ( here is the link )

Here's the pseudo code from the site (and all parameters are on the website too):

P3D.x = (x_d - cx_d) * depth(x_d,y_d) / fx_d P3D.y = (y_d - cy_d) * depth(x_d,y_d) / fy_d P3D.z = depth(x_d,y_d) P3D' = R.P3D + T P2D_rgb.x = (P3D'.x * fx_rgb / P3D'.z) + cx_rgb P2D_rgb.y = (P3D'.y * fy_rgb / P3D'.z) + cy_rgb 

Like you, I'm trying to map the depth to rgb data, however after using this algorithm I still stick to some misalignments where the color matching seems to be slightly biased in other directions, which varies from image to image in my dataset.

Hope that at least helps you in the right direction.