浏览全部资源
扫码关注微信
纸质出版日期： 2021-05-16 ，
录用日期： 2020-09-27
移动端阅览
宋绪杰, 戴孙浩, 林春雨, 詹书涛, 赵耀. 伪点云修正增强激光雷达数据[J]. 中国图象图形学报, 2021,26(5):1157-1168.
Xujie Song, Sunhao Dai, Chunyu Lin, Shutao Zhan, Yao Zhao. LiDAR data enhancement via pseudo-LiDAR point cloud correction[J]. Journal of Image and Graphics, 2021,26(5):1157-1168.
目的
2
激光雷达在自动驾驶中具有重要意义，但其价格昂贵，且产生的激光线束数量仍然较少，造成采集的点云密度较稀疏。为了更好地感知周围环境，本文提出一种激光雷达数据增强算法，由双目图像生成伪点云并对伪点云进行坐标修正，进而实现激光雷达点云的稠密化处理，提高3D目标检测精度。此算法不针对特定的3D目标检测网络结构，是一种通用的点云稠密化方法。
方法
2
首先利用双目RGB图像生成深度图像，根据先验的相机参数和深度信息计算出每个像素点在雷达坐标系下的粗略3维坐标，即伪点云。为了更好地分割地面，本文提出了循环RANSAC（random sample consensus）算法，引入了一个分离平面型非地面点云的暂存器，改进复杂场景下的地面分割效果。然后将原始点云进行地面分割后插入KDTree（k-dimensional tree），以伪点云中的每个点为中心在KDTree中搜索若干近邻点，基于这些近邻点进行曲面重建。根据曲面重建结果，设计一种计算几何方法导出伪点云修正后的精确坐标。最后，将修正后的伪点云与原始激光雷达点云融合得到稠密化点云。
结果
2
实验结果表明，稠密化的点云在视觉上具有较好的质量，物体具有更加完整的形状和轮廓，并且在KITTI（Karlsruhe Institute of Technology and Toyota Technological Institute）数据集上提升了3D目标检测精度。在使用该数据增强方法后，KITTI数据集下AVOD（aggregate view object detection）检测方法的AP
3D
-Easy（average precision of 3D object detection on easy setting）提升了8.25%，AVOD-FPN（aggregate view object detection with feature pyramid network）检测方法的AP
BEV
-Hard（average precision of bird’s eye view on hard setting）提升了7.14%。
结论
2
本文提出的激光雷达数据增强算法，实现了点云的稠密化处理，并使3D目标检测结果更加精确。
Objective
2
Light detection and ranging (LiDAR) plays an important role in autonomous driving. Even if it is expensive
the number of equipped laser beams is still small
which results in a sparse point cloud density. The sparsity of the LiDAR point cloud makes 3D object detection difficult. The camera is another vital sensor used in autonomous driving because of the mature image recognition methods and its competitive price compared with LiDAR. However
it does not perform as well as LiDAR in 3D object detection task. To perceive the surrounding environment better
this study proposes a LiDAR data enhancement algorithm based on pseudo-LiDAR point cloud correction method to increase the density of LiDAR point cloud
thereby improving the accuracy of 3D object detection. This method has a wide application prospect because it is a general method to densify the point cloud and improve the detection accuracy
which does not depend on specific 3D object detection network structures.
Method
2
The algorithm can be divided into four steps. In the first step
the depth map is generated on the basis of the stereo RGB images by using depth estimation methods
such as pyramid stereo matching network (PSMNet) and DeepPruner. The approximate 3D coordinates of each pixel corresponding to the LiDAR coordinate system are calculated according to the camera parameters and depth information. The point cloud formed by approximate 3D coordinates is usually called pseudo-LiDAR point cloud. In the second step
the ground points in the original point cloud are removed by ground segmentation method because they will disturb the surface reconstruction process in the following step. In order to improve the performance of ground segmentation
this study designs an iterative random sample consensus (RANSAC) algorithm. A register is used in the iterative RANSAC algorithm to store the points extracted in each iteration
which are planar point cloud but not ground point cloud. The register ensures that the next iteration of RANSAC will not be affected by these non-ground planar points. The iterative RANSAC algorithm performs better than the normal RANSAC algorithm in complex scenarios where the number of points in the non-ground plane is larger than that in a ground plane or there exist multiple ground planes with different angles of inclination. In the third step
the original point cloud is inserted into a k-dimensional tree (KDTree) after ground segmentation. Thereafter
several neighboring points of each point in the pseudo-LiDAR point cloud are obtained by searching in the KDTree. On the basis of these neighboring points
surface reconstruction is performed to catch the local feature around the pseudo-LiDAR point. The Delaunay triangulation surface reconstruction method
which is considered as the optimal surface reconstruction algorithm by many researchers
is used in this step. The result of surface reconstruction is represented as several tiny triangles that cover the whole surface in 3D space. If the distance between the current and last processed approximate 3D point is within the KDTree searching radius
the surface reconstruction result of the last processed point is directly regarded as the surface reconstruction result of the current point to save time by skipping the KDTree searching and surface reconstruction process. Furthermore
because the process of KDTree searching and surface reconstruction is independent for each pseudo-LiDAR point
parallel computation based on OpenMP is used to speed up this step. In the fourth step
the precise 3D coordinates of pseudo-LiDAR point cloud corresponding to the LiDAR coordinate system are derived by the designed computational geometry method. In the computational geometry method
two different depth values are set for each pixel to obtain two points in the 3D coordinate system. The two points determine the line of the light path of this pixel. Then
the precise 3D coordinate of this pixel is considered as the closest point from the origin among all of the intersection points of the light path line and reconstruction triangle surfaces. This computational geometry method realizes the function of pseudo-LiDAR point coordinate correction. To prevent the loss of accuracy
the method avoids using division in the calculation process based on inequality analysis. Finally
the precise 3D points generated in step four are merged with the origin point cloud scanned by LiDAR to obtain the dense point cloud.
Result
2
After the densification
the objects in the point cloud have more complete shapes and contours than before
which means that the characteristics of objects are more obvious. In order to further verify the validity of this data enhancement method
the aggregate view object detection (AVOD) and aggregate view object detection with feature pyramid network (AVOD-FPN) methods are implemented to check whether the average precision of 3D object detection on the dense point cloud is higher than that of the original point cloud. The data enhancement algorithm was used on KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset to obtain a dataset with dense LiDAR point clouds. Then
3D object detection methods were implemented on the original LiDAR point cloud and dense LiDAR point cloud. After using this data enhancement method
the AP
3D
-Easy of AVOD increased by 8.25%
and the AP
BEV
-Hard of AVOD-FPN increased by 7.14%.
Conclusion
2
A vision-based LiDAR data enhancement algorithm was proposed to increase the density of LiDAR point cloud
thereby improving the accuracy of 3D object detection. The experimental results show that the dense point cloud has good visual quality
and the data enhancement method improves the accuracy of 3D object detection on KITTI dataset.
激光雷达数据增强点云稠密化伪点云地面分割3D目标检测
light detection and ranging(LiDAR)data enhancementpoint cloud densificationpseudo-LiDAR point cloudground segmentation3D object detection
Bogoslavskyi I and Stachniss C. 2017. Efficient online segmentation for sparse 3D laser scans. PFG-Journal of Photogrammetry, Remote Sensing and GeoinformationScience, 85(1): 41-52[DOI:10.1007/s41064-016-0003-y]
Chandra R, Dagum L, Kohr D, Maydan D and McDonald J, Menon R. 2001. Parallel Programming in OpenMP. San Francisco, USA: Morgan Kaufmann Publishers
Chang J R and Chen Y S. 2018. Pyramid stereo matching network//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 5410-5418[DOI: 10.1109/CVPR.2018.00567http://dx.doi.org/10.1109/CVPR.2018.00567]
Duggal S, Wang S L, Ma W C, Hu R and Urtasun R. 2019. DeepPruner: learning efficient stereo matching via differentiable PatchMatch//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea: IEEE: 4383-4392[DOI: 10.1109/ICCV.2019.00448http://dx.doi.org/10.1109/ICCV.2019.00448]
Fischler M A and Bolles R C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381-395[DOI:10.1145/358669.358692]
Fu H, Gong M M, Wang C H, Batmanghelich K and Tao D C. 2018. Deep ordinal regression network for monocular depth estimation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 2002-2011[DOI: 10.1109/CVPR.2018.00214http://dx.doi.org/10.1109/CVPR.2018.00214]
Geiger A, Lenz P and Urtasun R. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE: 3354-3361[DOI: 10.1109/CVPR.2012.6248074http://dx.doi.org/10.1109/CVPR.2012.6248074]
Gopi M, Krishnan S and Silva C T. 2000. Surface reconstruction based on lower dimensional localized Delaunay triangulation. Computer Graphics Forum, 19(3): 467-478[DOI:10.1111/1467-8659.00439]
Himmelsbach M, Hundelshausen F V and Wuensche H J. 2010. Fast segmentation of 3D point clouds for ground vehicles//2010 IEEE Intelligent Vehicles Symposium. San Diego, USA: IEEE: 560-565[DOI: 10.1109/IVS.2010.5548059http://dx.doi.org/10.1109/IVS.2010.5548059]
Ku J, Mozifian M, Lee J, Harakeh A and Waslander S L. 2018. Joint 3D proposal generation and object detection from view aggregation//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid, Spain: IEEE: 1-8[DOI: 10.1109/IROS.2018.8594049http://dx.doi.org/10.1109/IROS.2018.8594049]
Li C Y, Ku J and Waslander S L. 2020. Confidence guided stereo 3D object detection with split depth estimation[EB/OL]. [2020-07-10].https://arxiv.org/pdf/2003.05505.pdfhttps://arxiv.org/pdf/2003.05505.pdf
Ma F C, Cavalheiro G V and Karaman S. 2019. Self-supervised sparse-to-dense: Self-supervised depth completion from liDAR and monocular camera//Proceedings of 2019 International Conference on Robotics and Automation. Montreal, Canada: IEEE: 3288-3295[DOI: 10.1109/ICRA.2019.8793637http://dx.doi.org/10.1109/ICRA.2019.8793637]
Ma F C and Karaman S. 2018. Sparse-to-dense: depth prediction from sparse depth samples and a single image//Proceedings of 2018 IEEE International Conference on Robotics and Automation. Brisbane, Australia: IEEE: 4796-4803[DOI: 10.1109/ICRA.2018.8460184http://dx.doi.org/10.1109/ICRA.2018.8460184]
Ramasubramanian V and Paliwal K K. 1992. Fast k-dimensional tree algorithms for nearest neighbor search with application to vector quantization encoding. IEEE Transactions on Signal Processing, 40(3): 518-531[DOI:10.1109/78.120795]
Shi S S, Guo C X, Jiang L, Wang Z, Shi J P, Wang X G and Li H S. 2020. PV-RCNN: Point-voxel feature set abstraction for 3d object detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE: 10529-10538[DOI: 10.1109/CVPR42600.2020.01054http://dx.doi.org/10.1109/CVPR42600.2020.01054]
Uhrig J, Schneider N, Schneider L, Franke U, Brox T and Geiger A. 2017. Sparsity invariant CNNS//Proceedings of 2017 International Conference on 3D Vision. Qingdao, China: IEEE: 11-20[DOI: 10.1109/3DV.2017.00012http://dx.doi.org/10.1109/3DV.2017.00012]
Velas M, Spanel M, Hradis M and Herout A. 2018. CNN for very fast ground segmentation in velodyne liDAR data//Proceedings of 2018 IEEE International Conference on Autonomous Robot Systems and Competitions. Torres Vedras, Portugal: IEEE: 97-103[DOI: 10.1109/ICARSC.2018.8374167http://dx.doi.org/10.1109/ICARSC.2018.8374167]
Wang Y, Chao W L, Garg D, Hariharan B, Campbell M and Weinberger K Q. 2019. Pseudo-liDAR from visual depth estimation: Bridging the gap in 3D object detection for autonomous driving//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 8437-8445[DOI: 10.1109/CVPR.2019.00864http://dx.doi.org/10.1109/CVPR.2019.00864]
Yin Z F. 2018. Research on 3D Reconstruction Method of Scattered Cloud Data. Taiyuan: Taiyuan University Of Technology
殷志峰. 2018. 散乱点云数据的三维重建方法研究. 太原: 太原理工大学
You Y R, Wang Y, Chao W L, Garg D, Pleiss G, Hariharan B, Campbell M and Weinberger K Q. 2020. Pseudo-liDAR++: Accurate depth for 3D object detection in autonomous driving//Proceedings of the 8th International Conference on Learning Representations. Addis Ababa, Ethiopia: ICLR: 1-22
相关作者
相关机构
京公网安备11010802024621