Get depth in color position, Kinect SDK

Question

I am looking for a way (as quickly as possible) to get the corresponding color pixel depth from a Kinect camera.

I found the MapDepthFrameToColorFrame function. But it only gives me color in a certain position of depth, I want the opposite.

The reason I want it is because I can click on the position in the RGB image and get the position for that pixel.

Is there a way to do this faster than iterating over all the results from a MapDepthFrameToColorFrame?

Answer 1

The problem is that not every color pixel will have a depth assigned to it due to the location of cameras and infrared emitters. If you come across Kinect and raise your left hand in front of your chest, you have a section of your chest that will not have depth data because your hand is blocking the IR emitter. Since the IR camera is closer to the RGB camera, you will usually not have a problem with depth to RGB; if Kinect can see IR, it can see RGB.

To answer your question: I don't know how to do this because the RGB index does not give you enough information to calculate the depth index. Have you tried using MapDepthFrameToColorFrame and the loop as you mentioned? If you don’t hit the maniac (or the computer), I bet that the application can handle this just fine. I would not waste time optimizing until you are sure that this will cause a problem.

If you want, you can try to reduce the size of the loop by checking the subset of the color coordinates returned by the function. Get the RGB pixel coordinates (x, y), then check the depth coordinates somewhere near (x, y). Maybe (x-20, y-20) to (x + 20, y + 20). If this cannot be found, you can check the rest of the array. Check it out a bit and you will probably find a good range for checking around the requested pixel, which you will get in most cases without many cycles.