Objective

2

The extrinsic parameter of the RGB-D camera can be used to convert point cloud in camera coordinate to that in world coordinate. This parameter can be applied to 3D reconstruction

3D measurement

robot gesture estimation

and target detection

among others. The RGB-D camera (e.g.

Kinect

PrimeSense

and RealSense) consists of two sensors: RGB sensor and depth sensor. The former sensor retrieves the RGB image

whereas the latter retrieves depth image from the scene. To translate the 3D point cloud in the camera coordinate to the world coordinate

the extrinsic parameters of depth sensor have to be calibrated. The general calibration methods use calibration objects (such as chessboard) to obtain the extrinsic parameter of the RGB-D color sensor

which is regarded as the extrinsic parameter of the depth sensor approximately. These methods do not make full use of depth information

thereby causing difficulty in simplifying the calibration process. Moreover

lack of knowledge on the difference between the depth sensor and color sensor can cause large errors. Thus

to accurately estimate the extrinsic parameter of the depth sensor in the RGB-D camera

some methods have been proposed by using the extrinsic parameters of depth sensor relative to the color sensor. However

these methods complicate the calibration process. To simplify the calibration process of the extrinsic parameter of the depth sensor

the depth information should be fully utilized. Results of the methods are based on the color image with the parameter of the color sensor. However

the majority of applications on the RGB-D camera are based on the depth sensor. Moreover

parameters of the depth sensor should be directly calibrated.

Method

2

We build the spatial constraint relation between the ground plane and the camera

which can be used to select the ground plane from planes detected in the 3D point cloud. The ground plane should satisfy the following conditions: 1) The angle between the z axis of the camera and the ground plane is less than the specified threshold. 2) The z value of the ground plane in the world coordinate is larger than that of the other points

which are not in the ground plane. Moreover

we create the world coordinate based on the detected ground planes automatically. The origin point of the world coordinate is the projection of the origin point of the camera coordinate to the plane and the y axis of the world coordinate is the projection of the z axis of the camera coordinate to the plane. In addition

the direction of the z axis of the world coordinate is toward the origin point of the world coordinate from the origin point of the camera coordinate. We calibrate the extrinsic parameter of the RGB-D camera in the following steps. First

we reconstruct the 3D point cloud from the depth image retrieved from the depth sensor of the RGB-D camera. The reconstructed 3D point cloud is in the camera coordinate

whose subset forms a large number of planes. Second

planes in the 3D point cloud are detected by the MELSAC method. At most

one ground plane exists in the detected planes. Third

the spatial constraint rule between the ground plane and camera is built. The detected planes are filtered by the spatial constraint rule until the ground plane is found or all the planes are iterated. The process stops when a ground plane cannot be found. Finally

by using the relation between the ground plane and the camera

point sets are selected to calculate the extrinsic parameters.

Result

2

In the experiment

the benchmark is the result of checkerboard extrinsic calibration method only processing the RGB image of RGB-D information which is retrieved from the PrimeSense camera. We record an 89.4 s video and use it in the experiment. The videos contain two sub-videos: RGB video whose channel is 3 and depth video whose channel is 1. A 7x7 checkerboard is found in every frame of the RGB video

which is processed by the checkerboard-based method. The input of our proposed method is the frame of the depth video. The result shows that the average tilt angle error is -1.14°

the average pitch angle error is 4.57°

and the average camera height error is 3.96 cm. An experiment to test the robustness of the noise is also performed. The variance of the Gaussian noise in the depth frame is increased

and the result of each variance Gaussian noise is obtained. The stability of calibration decreases with the increase in the variance of Gaussian noise. The result shows that our method performs effectively when the variance of the Gaussian noise is below 0.01.

Conclusion

2

Our proposed method fully utilizes the depth information of the RGB-D camera

and simplifies the process of extrinsic calibration of the depth sensor. Thus

our method can be used in actual application. For convenience

the source code is also published. This method can automatically detect the ground plane and does not require other calibration objects. Therefore

the proposed method can calibrate each frame of the recorded video accurately

and it is not sensitive to the noise in the depth image. In addition

the algorithm has high parallelism. The process of estimating planes in the 3D point cloud and filtering these planes can be implemented in a parallel manner. The proposed method will have real-time performance based on this parallel implementation.