This document fully reuses the Azure Kinect calibration functions to transform points between the coordinate systems of each sensor on the Femto Bolt device. Applications needing to transform entire images can leverage accelerated operations provided in Image Transformation.
Retrieving Calibration Data
Device calibration data is needed to perform the coordinate system transforms. The calibration data is stored in the k4a_calibration_t data type. This is obtained from the device using the k4a_device_get_calibration() function. The calibration data is different for each device, or for different operating modes on the same device. So k4a_device_get_calibration() takes the depth_mode and color_resolution as input parameters.
OpenCV Compatibility
The calibration parameters are compatible with OpenCV. See OpenCV documentation for details on the calibration parameters for each camera. Also see the SDK OpenCV compatibility sample which demonstrates converting the k4a_calibration_t type to corresponding OpenCV data structures.
Coordinate Transformation Functions
The diagram below shows the different coordinate systems for Femto Bolt and the functions used to transform between them. The 3D coordinate systems of the gyroscope and accelerometer are omitted from the diagram for simplicity.
Note on lens distortion: The 2D coordinates always reference the distorted images in the SDK. The SDK distortion correction sample demonstrates correcting image distortion. In general, 3D points are never affected by lens distortion.
Converting Between 3D Coordinate Systems
The function k4a_calibration_3d_to_3d() uses the external calibration of the cameras to transform a 3D point from the source coordinate system to the destination coordinate system. The source and destination can be set to any of the four 3D coordinate systems – color camera, depth camera, gyroscope, or accelerometer. If source and destination are the same, the input 3D point is returned unmodified as output.
Converting Between 2D and 3D Coordinate Systems
The function k4a_calibration_3d_to_2d() transforms a 3D point from the source coordinate system to the 2D pixel coordinate of the destination camera. This function is commonly referred to as the projection function. While the source can be set to any of the four 3D coordinate systems, the destination must be the depth or color camera. If source and destination are different, the input 3D point is first transformed into the 3D coordinate system of the destination camera using k4a_calibration_3d_to_3d(). With the 3D point represented in the destination camera coordinates, the corresponding 2D pixel coordinate is computed using the internal calibration of the destination camera. If the 3D point falls outside the field of view of the destination camera, the valid value is set to 0.
The function k4a_calibration_2d_to_3d() transforms the 2D pixel coordinate of the source camera to a 3D point in the coordinate system of the destination camera. The source must be the color or depth camera. The destination can be set to any of the four 3D coordinate systems. In addition to the 2D pixel coordinate, the depth value of the pixel in the source camera image (in mm) is also required as input to the function. One way to derive depth values in the color camera geometry is using the function k4a_transformation_depth_image_to_color_camera(). The function uses the internal calibration of the source camera to compute the 3D ray emanating from the camera focus through the specified pixel coordinate. The exact 3D point location along this ray is then looked up using the depth value. This operation is commonly referred to as the unprojection function. If source and destination cameras are different, the function transforms the 3D point into the coordinate system of the destination via k4a_calibration_3d_to_3d(). If the 2D pixel coordinate falls outside the field of view of the source camera, the valid value is set to 0.
Converting Between 2D Coordinate Systems
The function k4a_calibration_2d_to_2d() transforms a 2D pixel coordinate from the source camera to a 2D pixel coordinate in the destination camera. The source and destination must be set to the color or depth camera. The function requires the depth value of the pixel in the source camera image (in mm) to be provided as input. One way to derive depth values in the color camera geometry is using the function k4a_transformation_depth_image_to_color_camera(). It calls k4a_calibration_2d_to_3d() to transform into a 3D point in the source camera system. It then calls k4a_calibration_3d_to_2d() to transform into the 2D pixel coordinate in the destination camera image. If either k4a_calibration_2d_to_3d() or k4a_calibration_3d_to_2d() returns invalid results, the valid value is set to 0.
Related Samples
Next Steps
Now that you are familiar with camera calibration, you can also learn about:《Device Synchronization》
