Current Issue Cover
深度学习点云质量增强方法综述

陈建文1, 赵丽丽2, 任蓝草1, 孙卓群1, 张新峰3, 马思伟4(1.电子科技大学信息与通信工程学院, 成都 611731;2.中国移动通信有限公司研究院, 北京 100032;3.中国科学院大学计算机科学与技术学院, 北京 100190;4.北京大学计算机学院, 北京 100871)

摘 要
随着三维探测技术的发展,点云逐渐成为最常用的三维物体/场景表征数据类型之一,广泛应用于自动驾驶、增强现实及虚拟现实等领域。然而,受限于硬件设备、采集环境以及遮挡等因素,采集的原始点云通常是不完整、稀疏、嘈杂的,为点云的处理和分析带来了巨大挑战。在此背景下,点云质量增强技术旨在对原始点云进行处理以获得结构完整、密集且接近无噪的点云,具有重要意义。本文对现阶段深度学习点云质量增强方法进行了系统综述,为后续研究者提供研究基础。首先,简要介绍了点云数据处理中通用的关键技术;分别介绍了补全、上采样和去噪3类点云质量增强方法,并对3类方法中的现有算法进行了分类、梳理及总结。其中,点云补全与点云去噪算法均可根据是否采用编码器—解码器结构分为两大类,点云上采样算法可根据网络主要结构分为基于卷积神经网络的方法、基于生成对抗网络的方法和基于图卷积神经网络的方法。其次,总结了质量增强任务中常用的数据集与评价指标,并分别对比分析了现阶段点云补全、上采样和去噪中主流算法的性能。最后,通过系统的梳理,凝练出点云质量增强方向所面临的挑战,并对未来的研究趋势进行了展望。此外,本文汇总了涉及的文献及其开源代码,详见链接https://github.com/LilydotEE/Point_cloud_quality_enhancement。
关键词
Deep learning-based quality enhancement for 3D point clouds:a survey

Chen Jianwen1, Zhao Lili2, Ren Lancao1, Sun Zhuoqun1, Zhang Xinfeng3, Ma Siwei4(1.School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China;2.China Mobile Research Institute, Beijing 100032, China;3.School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100190, China;4.School of Computer Science, Peking University, Beijing 100871, China)

Abstract
With the development of 3D detection technologies,point clouds have gradually become one of the most common data representations of 3D objects or scenes that are widely used in many applications,such as autonomous driving, augmented reality(AR),and virtual reality(VR). However,due to limitations in hardware,environment,and occlusion, the acquired point clouds are usually sparse,noisy,and uneven,hence imposing great challenges to the processing and analysis of point clouds. Therefore,point cloud quality enhancement techniques,which aim to process the original point cloud to obtain a dense,clean,and structurally complete point cloud,are of great significance. In recent years,with the development of hardware and machine learning technologies,deep-learning-based point cloud quality enhancement methods,which have great potential to extract the features of point clouds,have attracted the attention of scholars at home and abroad. Related works mainly focus on point cloud completion,point cloud upsampling(also known as super-resolution), and point cloud denoising. Point cloud completion fills the incomplete point clouds to restore the complete point cloud information,while point cloud upsampling increases the point number of the original point cloud to obtain a denser point cloud, and point cloud denoising removes the noisy points in the point cloud to obtain a cleaner point cloud. This paper systematically reviews the existing point cloud quality enhancement methods based on deep learning to offer a basis for subsequent research. First,this study briefly introduces the fundamentals and key technologies that are widely used in point cloud analysis. Second,three types of point cloud quality enhancement technologies,namely,upsampling,completion,and denoising,are introduced,classified,and summarized. According to the types of input data,point cloud completion methods can be divided into voxel- and point-based algorithms,with the latter being further sub-divided into two types depending on whether the encoder-decoder structure is exploited or not. The encoder-decoder-structure-based algorithms can be further divided according to whether the generative adversarial network(GAN)structure is used. Point cloud upsampling methods can be classified into convolutional neural network(CNN)-based algorithms,GAN-based algorithms,and graph convolutional network(GCN)-based algorithms. Point cloud denoising methods can also be divided into two types based on whether the encoder-decoder structure is exploited or not. Third,the commonly used datasets and evaluation metrics in point cloud quality enhancement tasks are summarized. The performance evaluation metrics for geometry reconstruction mainly include chamfer distance,earth mover’s distance,Hausdorff distance,and point-to-surface distance. This paper then compares the state-of-the-art algorithms of point cloud completion and upsampling on common datasets and identifies the reasons for the differences in their performance. The recent progress and challenges in the field are then summarized, and future research trends are proposed. The findings are summarized as follows:1)the point cloud features extracted by existing deep learning-based algorithms are highly global,which means that the local features related to the detailed structure cannot be captured well,thus resulting in poor detail reconstruction. Traditional geometric algorithms are known to effectively represent data features based on geometric information. Therefore,how to combine geometric algorithms with deep learning for point cloud quality enhancement is worth exploring. 2)Most algorithms are for dense point clouds of single objects,and only a few studies have focused on sparse LiDAR point clouds containing large-scale outdoor scenes. 3)Most of the related studies only consider the point cloud processing of a single frame and ignore the temporal correlation of point cloud sequences. Therefore,how to utilize the spatial-temporal correlation to improve quality enhancement performance warrants further investigation. 4)In existing methods,the proposed network models are often complex and the inference speed is relatively slow,which fail to meet the real-time requirements of several applications. Therefore,how to further reduce the scale of the model parameters and improve the inference speed is a research direction worth exploring. 5) Most of the existing methods only process the geometric information(3D coordinates)of point clouds and ignore the attribute information(e. g. ,color and intensity). Therefore,how to simultaneously enhance the quality of geometric and attribute information needs to be explored. Project page:https://github.com/LilydotEE/Point_cloud_quality_enhancement.
Keywords

订阅号|日报