浏览全部资源
扫码关注微信
收稿：2021-07-05，
修回：2021-8-19，
录用：2021-8-26，
纸质出版：2022-02-16
移动端阅览
目的
2
点云分类传统方法中大量依赖人工设计特征，缺乏深层次特征，难以进一步提高精度，基于深度学习的方法大部分利用结构化网络，转化为其他表征造成了3维空间结构信息的丢失，部分利用局部结构学习多层次特征的方法也因为忽略了机载数据的几何信息，难以实现精细分类。针对上述问题，本文提出了一种基于多特征融合几何卷积神经网络（multi-feature fusion and geometric convolutional neural network，MFFGCNN）的机载LiDAR（light detection and ranging）点云地物分类方法。
方法
2
提取并融合有效的浅层传统特征，并结合坐标尺度等预处理方法，称为APD模块（airporne laser scanning point cloud design module），在输入特征层面对典型地物有针对性地进行信息补充，来提高网络对大区域、低密度的机载LiDAR点云原始数据的适应能力和基础分类精度，基于多特征融合的几何卷积模块，称为FGC（multi-feature fusion and geometric convolution）算子，编码点的全局和局部空间几何结构，实现对大区域点云层次化几何结构的获取，最终与多尺度全局的逐点深度特征聚合提取高级语义特征，并基于空间上采样获得逐点的多尺度深度特征实现机载LiDAR点云的语义分割。
结果
2
在ISPRS（International Society for Photogrammetry and Remote Sensing）提供的3维标记基准数据集上进行模型训练与测试，由于面向建筑物、地面和植被3类典型地物，对ISPRS的9类数据集进行了类别划分。本文算法在全局准确率上取得了81.42%的较高精度，消融实验结果证明FGC模块可以提高8%的全局准确率，能够有效地提取局部几何特性，相较仅基于点的3维空间坐标方法，本文方法可提高15%的整体分类精度。
结论
2
提出的MFFCGNN网络综合了传统特征的优势和深度学习模型的优点，能够实现机载LiDAR点云的城市重要地物快速分类。
Objective
2
Airborne laser scanning (ALS) offers a mature structure of point cloud data
which can represent complicated geometric information of the real world. Point cloud classification is a critical task in airborne laser detection and ranging applications
such as topographic mapping
power line detection
building reconstruction
etc. However
unavoidable reasons
such as complicated topographic conditions
sensor noise and sparse point cloud density
make ALS point cloud classification very difficult. Original point cloud classification is dependent on manual features
targeted classification conditions and parameter designation. Deep learning-based methods transform 3D point cloud into other representations such as 3D voxels
2D images and octree structures based on structured networks at the cost of 3D spatial structure information loss. As the requirement for high point cloud density
the lack of adaptability for data and deep features caused in accuracy. Moreover
some networks make multi-modal data fusion or learn multi-level feature representations of points via local structures exploration
but the applications of geometric information of airborne data is constraint to achieve fine-grained classification for geometrically prominent and diverse ALS point cloud. In this paper
we propose a multi-feature fusion and geometric convolutional neural network (MFFGCNN) consisting of ALS data processing
multi-feature fusion and deep geometric features aggregation for point cloud classification.
Method
2
First
an ALS point cloud design module
called APD module
is constructed to organize point cloud structure by balancing classes and scale
partitioning point cloud and processing raw coordinates. Next
discriminative typical features are used to supplement the point cloud information at the input feature level. The applications of echo and intensity has been demonstrated as input along with the coordinates into the point-based network
which can preserve the 3D spatial characteristics while making full use of the effective point cloud features. Then
four types of geometric features of points and their K-nearest neighborhoods are calculated by dividing the neighborhood region at three different scales
with local spatial information. A geometric convolution module based on multi-feature fusion
called multi-feature fusion and geometric convolution(FGC) operator
is to encode the global and local spatial geometric structure of points. This design can obtain the hierarchical geometric structure of large area point clouds. At the end
our method aggregates the discriminative deep global and local geometric features in different levels and the input multi-class features into a hierarchical advanced semantic feature
which enables semantic segmentation of airborne LiDAR(light detection and ranging) point clouds by spatial up-sampling.
Result
2
The comparative analyses are based on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark dataset. For the three typical ground features of buildings
ground and trees
the dataset is further divided on initial contained nine categories such as car
facade and shrub. The ALS point cloud is segmented into blocks of 50 m as a batch input into the network. Each feature is extracted from the layer and then processed using batch normalization. The batch number for training and evaluation is 16 and the maximum epoch during training is 200. The test is implemented via a NVIDIA GTX 2060Ti GPU. Mean intersection-over-union (mIoU)
mean accuracy (mAcc)
and overall accuracy (OA) are evaluated. The results of ablation experiments demonstrate that the FGC module can improve the global accuracy by 8%
which can effectively extract local geometric features. Compared with the 3D spatial coordinate-based method
the overall classification accuracy can improve by 15%. The relative elevation of the neighborhood can reflect elevation directional heterogeneity and ground points can achieve clear classification. Echo features can be validated for advantage in vegetation classification. The introduction of geometric features facilitates the distinction between building points and background points while ensuring the consistency of the building's main body and clear contours. The four classes of geometric features are targeted at the curvature variation
edges
all consistency of the feature and the unique spherical characteristics of the vegetation. It illustrated further that the involvement of multiple classes of features in the semantic segmentation of airborne LiDAR point clouds. The visualization results also demonstrate a stronger model
especially in difficult situations such as buildings surrounded by tall trees and dense buildings with complex roof structures still achieving excellent performance
but there is a potential for improvement in the edges of features
and in the detailed parts of complex scenes.
Conclusion
2
The proposed MFFCGNN network combines the advantages of initial features and deep learning-based models. The demonstrated model can be implemented in 3D city modelling.
Alshehhi R, Marpu P R, Woon W L and Mura Mauro D. 2017. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 130: 139-149[DOI:10.1016/j.isprsjprs.2017.05.002]
Antonarakis A S, Richards K S and Brasington J. 2008. Object-based land cover classification using airborne LiDAR. Remote Sensing of Environment, 112(6): 2988-2998[DOI:10.1016/j.rse.2008.02.004]
Bi K and Gui D Z. 2014. Brief probe into national geographic conditions monitoring and fundamental surveying and mapping. Remote Sensing Information, 29(4): 10-15
毕凯, 桂德竹. 2014. 浅谈地理国情监测与基础测绘. 遥感信息, 29(4): 10-15)[DOI:10.3969/j.issn.1000-3177.2014.04.003]
Blomley R, Jutzi B and Weinmann M. 2016. Classification of airborne laser scanning data using geometric multi-scale features and different neighbourhood types. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Ⅲ-3: 169-176[DOI:10.5194/isprs-annals-III-3-169-2016]
Charles R Q, Su H, Kaichun M and Guibas L J. 2017. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 77-85[ DOI: 10.1109/CVPR.2017.16 http://dx.doi.org/10.1109/CVPR.2017.16 ]
Chehata N, Guo L and Mallet C. 2009. Airborne LiDAR feature selection for urban classification using random forests. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 39: 207-212[DOI:10.13203/j.whugis20130206]
Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251[DOI:10.1109/TGRS.2016.2584107]
Gu L, Ji Y and Liu C P. 2021. Classification method of three-dimensional point cloud based on multiple modal feature fusion. Computer Engineering, 47(2): 279-284
顾砾, 季怡, 刘纯平. 2021. 基于多模态特征融合的3维点云分类方法. 计算机工程, 47(2): 279-284)[DOI:10.19678/j.issn.1000-3428.0057147]
Guo B, Huang X F, Zhang F and Sohn G. 2015. Classification of airborne laser scanning data using JointBoost. ISPRS Journal of Photogrammetry and Remote Sensing, 100: 71-83[DOI:10.1016/j.isprsjprs.2014.04.015]
Hu H Y, Hui Z Y and Li N. 2020. Airborne LiDAR point cloud classification based on multiple-entity eigenvector fusion. Chinese Journal of Lasers, 47(8): #0810002
胡海瑛, 惠振阳, 李娜. 2020. 基于多基元特征向量融合的机载LiDAR点云分类. 中国激光, 47(8): #0810002)[DOI:10.3788/CJL202047.0810002]
Hu X Y and Yi Y. 2016. Deep-learning-based classification for DTM extraction from ALS point cloud. Remote Sensing, 8(9): #730[DOI:10.3390/rs8090730]
Huang Q G, Wang W Y and Neumann U. 2018. Recurrent slice networks for 3D segmentation of point clouds//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake, USA: IEEE[ DOI: 10.1109/CVPR.2018.00278 http://dx.doi.org/10.1109/CVPR.2018.00278 ]
Jiang M Y, Wu Y R, Zhao T Q, Zhao Z L and Lu C W. 2018. PointSIFT: a SIFT-like network module for 3D point cloud semantic segmentation[EB/OL]. [2021-06-05] . https://arxiv.org/pdf/1807.00652.pdf https://arxiv.org/pdf/1807.00652.pdf
Li D L, Shen X, Yu Y T, Guan H Y, Li J, Zhang G and Li D R. 2020. Building extraction from airborne multi-spectral LiDAR point clouds based on graph geometric moments convolutional neural networks. Remote Sensing, 12(19): #3186[DOI:10.3390/rs12193186]
Liu J N and Zhang X H. 2003. Progress of airborne laser scanning altimetry. Geomatics and Information Science of Wuhan University, 28(2): 132-137
刘经南, 张小红. 2003. 激光扫描测高技术的发展与现状. 武汉大学学报(信息科学版), 28(2): 132-137)[DOI:10.3969/j.issn.1671-8860.2003.02.002]
Niemeyer J, Rottensteiner F and Soergel U. 2014. Contextual classification of lidar data and building object detection in urban areas. ISPRS Journal of Photogrammetry and Remote Sensing, 87: 152-165[DOI:10.1016/j.isprsjprs.2013.11.001]
Pan S Y and Guan H Y. 2018. Object classification using airborne multispectral LiDAR data. Acta Geodaetica et Cartographica Sinica, 47(2): 198-207
潘锁艳, 管海燕. 2018. 机载多光谱LiDAR数据的地物分类方法. 测绘学报, 47(2): 198-207)[DOI:10.11947/j.AGCS.2018.20170512]
Qi C R, Yi L, Su H and Guibas L J. 2017. PointNet++: deep hierarchical feature learning on point sets in a metric space//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc. : 5105-5114
Su H, Maji S, Kalogerakis E and Learned-Miller E. 2015. Multi-view convolutional neural networks for 3D shape recognition//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 945-953[ DOI: 10.1109/ICCV.2015.114 http://dx.doi.org/10.1109/ICCV.2015.114 ]
Sun J and Lai Z L. 2014. Airborne LiDAR feature selection for urban classification using random forests. Geomatics and Information Science of Wuhan University, 39(11): 1310-1313
孙杰, 赖祖龙. 2014. 利用随机森林的城区机载LiDAR数据特征选择与分类. 武汉大学学报(信息科学版), 39(11): 1310-1313)[DOI:10.13203/j.whugis20130206]
Wang Z, Zhang L Q, Zhang L, Li R J, Zheng Y B and Zhu Z D. 2018. A deep neural network with spatial pooling (DNNSP) for 3-D point cloud classification. IEEE Transactions on Geoscience and Remote Sensing, 56(8): 4594-4604[DOI:10.1109/TGRS.2018.2829625]
Wen C C, Yang L N, Li X, Peng L and Chi T H. 2020. Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification. ISPRS Journal of Photogrammetry and Remote Sensing, 162: 50-62[DOI:10.1016/j.isprsjprs.2020.02.004]
WidyaningrumE, Bai Q, Fajari M K and Lindenbergh R C. 2021. Airborne laser scanning point cloud classification using the DGCNN deep learning method. Remote Sensing, 13(5): #859[DOI:10.3390/rs13050859]
Wu B C, Zhou X Y, Zhao S C, Yue X Y and Keutzer K. 2018. SqueezeSegV2: improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud[EB/OL]. [2021-06-05] . https://arxiv.org/pdf/1809.08495v1.pdf https://arxiv.org/pdf/1809.08495v1.pdf
Wu Z R, Song S R, Khosla A, Yu F, Zhang L G, Tang X O and Xiao J X. 2015. 3D ShapeNets: a deep representation for volumetric shapes//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 1912-1920[ DOI: 10.1109/CVPR.2015.7298801 http://dx.doi.org/10.1109/CVPR.2015.7298801 ]
Xiong Y, Gao R Q and Xu Z Y. 2018. Random forest method for dimension reduction and point cloud classification based on airborne LiDAR. Acta Geodaetica et Cartographica Sinica, 47(4): 508-518
熊艳, 高仁强, 徐战亚. 2018. 机载LiDAR点云数据降维与分类的随机森林方法. 测绘学报, 47(4): 508-518)[DOI:10.11947/j.AGCS.2018.20170417]
Yang S J, Zhang K S and Shao Y S. 2019. Classification of airborne LiDAR point cloud data based on multiscale adaptive features. Acta Optica Sinica, 39(2): 369-375
杨书娟, 张珂殊, 邵永社. 2019. 基于多尺度自适应特征的机载LiDAR点云分类. 光学学报, 39(2): 369-375)[DOI:10.3788/AOS201939.0228001]
Yang Z S, Jiang W S, Xu B, Zhu Q S, Jiang S and Huang W. 2017. A convolutional neural network-based 3D semantic labeling method for ALS point clouds. Remote Sensing, 9(9): #936[DOI:10.3390/rs9090936]
Yang Z S, Tan B, Pei H K and Jiang W S. 2018. Segmentation and multi-scale convolutional neural network-based classification of airborne laser scanner data. Sensors, 18(10): #3347[DOI:10.3390/s18103347]
Yousefhussien M, Kelbe D J, Ientilucci E J and Salvaggio C. 2018. A multi-scale fully convolutional network for semantic labeling of 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 143: 191-204[DOI:10.1016/j.isprsjprs.2018.03.018]
Zhang J X, Duan M Y, Lin X G and Zang Y. 2017. Comparison and analysis of models for 3D power line reconstruction using LiDAR point cloud. Geomatics and Information Science of Wuhan University, 42(11): 1565-1572
张继贤, 段敏燕, 林祥国, 臧艺. 2017. 激光雷达点云电力线3维重建模型的对比与分析. 武汉大学学报(信息科学版), 42(11): 1565-1572)[DOI:10.13203/j.whugis20150385]
Zhang L Q, Li Z Q, Li A J and Liu F Y. 2018. Large-scale urban point cloud labeling and reconstruction. ISPRS Journal of Photogrammetry and Remote Sensing, 138: 86-100[DOI:10.1016/j.isprsjprs.2018.02.008]
Zhao C, Zhang B M, Yu D H, Guo H T and Lu J. 2019. Airborne LiDAR point cloud classification using transfer learning. Optics and Precision Engineering, 27(7): 1601-1612
赵传, 张保明, 余东行, 郭海涛, 卢俊. 2019. 利用迁移学习的机载激光雷达点云分类. 光学精密工程, 27(7): 1601-1612)[DOI:10.3788/OPE.20192707.1601]
Zhao R B, Pang M Y and Wang J D. 2018. Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network. International Journal of Geographical Information Science, 32(5): 960-979[DOI:10.1080/13658816.2018.1431840]
相关作者
相关机构
京公网安备11010802024621
