Kinect v2: Spatial Resolution / Depth Resolution / Camera Calibration

Question

For my application, I analyzed the spatial resolution of Kinect v2.

To analyze the spatial resolution, I recorded a perpendicular and flat plane at a given distance and converted the depth map of the plane into a point cloud. Then I compare the point with my neighbors, calculating the Euclidean distance.

By calculating the Euclidean distance for this case (1 meter between the plane and the ring), the resolution is close to 3 mm between the points. For an airplane with a distance of 2 meters, I got a resolution of up to 3 mm.

Comparing this with the literature, I think my results are pretty bad.

For example, Yang et al. received an average resolution of 4 mm for an airplane with a distance of 4 meters to kinect ( Evaluating and improving the accuracy of Kinect depth for Windows v2 )

Here is an example of my flat point cloud (2 meters away from my kinect):

Has anyone made any observation regarding the spatial resolution of Kinect v2 or the idea why my solution is bad?

In my opinion, I think something wrong happened when converting my depth image to world coordinates. Therefore, the code is disabled here:

%normalize image points by multiply inverse of K u_n=(u(:)-c_x)/f_x; v_n=(v(:)-c_y)/f_y; % u,v are uv-coordinates of my depth image %calc radial distortion r=sqrt(power(u_n,2)+power(v_n,2)); radial_distortion =1.0 + radial2nd * power(r,2) + radial4nd * power(r,4) + radial6nd * power(r,6); %apply radial distortion to uv-coordinates u_dis=u_n(:).*radial_distortion; v_dis=v_n(:).*radial_distortion; %apply cameramatrix to get undistorted depth point x_depth=u_dis*f_x+c_x; y_depth=v_dis*f_y+c_y; %convert 2D to 3D X=((x_depth(:)-c_x).*d(:))./f_x; Y=((y_depth(:)-c_y).*d(:))./f_y; Z=d; % d is the given depth value at (u,v) 

EDIT: So far, I have also tried to include points directly from the coordinate mapper without further calibration steps.

The results related to resolution are still the same. Does anyone compare the results?

Answer 1

@JavaNullPointer, the way you convert your information to 3D using Kinect v2, the community is not well received.

Also, the calculations you do largely follow the work of Nicholas Berrus - http://burrus.name/index.php/Research/KinectCalibration

For Kinect v2, there is still a lot of information on how to do this. However, the new SDK features preserve the Kinect calibration table space.

The procedure is quite simple:

1- you need to save this information in a table - https://msdn.microsoft.com/en-us/library/windowspreview.kinect.coordinatemapper.getdepthframetocameraspacetable.aspx

2- once you save this information to a file, you can convert your depth points (2D) into 3D camera space.

Here is the code you should use:

 // Get the depth for this pixel ushort depth = frameData[y * depthFrameDescription.Height + x]; // Get the value from the x/y table PointF lutValue = this.cameraSpaceTable[y * depthFrameDescription.Height + x]; // create the CameraSpacePoint for this pixel // values are in meters so convert CameraSpacePoint csp = new CameraSpacePoint(); csp.X = lutValue.X * depth * 0.001f; csp.Y = lutValue.Y * depth * 0.001f; csp.Z = depth * 0.001f; 

Also, take a lot:

https://msdn.microsoft.com/en-us/library/windowspreview.kinect.coordinatemapper.mapdepthframetocameraspace.aspx

or

https://msdn.microsoft.com/en-us/library/windowspreview.kinect.coordinatemapper.mapdepthframetocameraspaceusingibuffer.aspx

In addition, the depths, infrared streams of the bodyindex are aligned (the same resolution), so you really do. If you need to get color points, you should also keep this mapping. All of this information is available on the Kinect MSDN 2.0 website.

I hope you can save this information and then repeat this spatial resolution test.