Kinect: from color space to world coordinates

Question

I track the ball using rgb data from kinect. After that, I look at the corresponding depth data. Both of them work great. Now I want to have the actual world coordinates x, y, z (for example, skeletal space) instead of the values ​​x_screen, y_screen and depth. Unfortunately, the methods provided by kinect sdk ( http://msdn.microsoft.com/en-us/library/hh973078.aspx ) do not help me. Basically, I need the function "NuiImageGetSkeletonCoordinatesFromColorPixel", but I do not exist. All functions basically go in the opposite direction.

I know that this is possible, possibly with openNI, but I cannot use it for other reasons.

Is there a function that does this for me, or do I need to do the conversion myself? If I have to do it myself, how would I do it? I sketched a small diagram http://i.imgur.com/ROBJW8Q.png - do you think this would work?

Answer 1

Check out CameraIntrinsics.

typedef struct _CameraIntrinsics { float FocalLengthX; float FocalLengthY; float PrincipalPointX; float PrincipalPointY; float RadialDistortionSecondOrder; float RadialDistortionFourthOrder; float RadialDistortionSixthOrder; } CameraIntrinsics; 

You can get it from ICoordinateMapper::GetDepthCameraIntrinsics .

Then, for each pixel (u,v,d) in the depth space, you can get the coordinate in world space by doing the following:

 x = (u - principalPointX) / focalLengthX * d; y = (v - principalPointY) / focalLengthY * d; z = d; 

For a color space pixel, you first need to find the corresponding depth space pixel, which you should use ICoordinateMapper::MapCameraPointTodepthSpace . Since not every color pixel has an associated pixel of depth (1920x1080 versus 512x424), you cannot have the color dot region of full HD.