浏览全部资源
扫码关注微信
收稿：2019-07-19，
修回：2019-10-24，
录用：2019-10-31，
纸质出版：2020-06-16
移动端阅览
目的
2
深度网络用于3维点云数据的分类分割任务时，精度与模型在全局和局部特征上的描述能力密切相关。现有的特征提取网络，往往将全局特征和不同尺度下的局部特征相结合，忽略了点与点之间的结构信息和位置关系。为此，通过在分类分割模型中引入图卷积神经网络(graph convolution neural network，GCN)和改进池化层函数，增强局部特征表征能力和获取更丰富的全局特征，改善模型对点云数据的分类分割性能。
方法
2
GCN模块通过K近邻算法构造图结构，利用相邻点对的边缘卷积获取局部特征，在深度网络模型中动态扩展GCN使模型获得完备的局部特征。在池化层，通过选择差异性的池化函数，联合提取多个全局特征并进行综合，保证模型在数据抖动时的鲁棒性。
结果
2
在ModelNet40、ShapeNet和S3DIS(stanford large-scale 3D indoor semantics)数据集上进行分类、部分分割以及语义场景分割实验，验证模型的分类分割性能。与PointNet相比，在ModelNet40分类实验中，整体精度和平均分类精度分别提升4%和3.7%；在ShapeNet部分分割数据集和S3DIS室内场景数据集中，平均交并比(mean intersection-over-union，mIoU)分别高1.4%和9.8%。采用不同的池化函数测试结果表明，本文提出的差异性池化函数与PointNet提出的池化函数相比，平均分类精度提升了0.9%，有效改善了模型性能。
结论
2
本文改进的网络模型可以有效获取点云数据中的全局和局部特征，实现更优的分类和分割效果。
Objective
2
The depth feature representation of the 3D model is the key and premise of 3D target recognition and 3D model semantic segmentation. It has broad application prospects in the fields of robot
automatic driving
virtual reality
and remote sensing mapping. Semantic segmentation has achieved great progress with the help of deep learning
but most of the methods are used to process 2D images. Given the large amount of data
uneven density
and irregular shape of unstructured 3D point clouds
their classification and segmentation still have enormous challenges. Traditional convolutional neural networks (CNNs) require regularized data as input. The point cloud needs to be converted into multi-view or a voxel mesh to process. The existing deep learning network used for directly processing point cloud data solves the disorder problem of point cloud through the pooling layer of CNN. Thus
the network model can directly classify and segment the point cloud data. As for the classification and segmentation model dealing with point cloud data
its accuracy is closely related to the ability of the network to describe global and local features. Existing feature extraction networks often combine global features with local features at different scales
ignoring the structural information and position relationship among points. Thus
the global feature vectors with more significant features cannot be generated in the pooling layer
resulting in low classification and segmentation accuracy.
Method
2
To improve the performance of the network model
the graph CNN (GCN) and the improved pooling layer function are introduced in the classification and segmentation model. The method can enhance the ability of local feature representation and obtain more abundant global features. The processing ability of the network model to point cloud data can be improved. In the GCN
a graph structure is constructed by connecting the vertex with the nearest
K
points through the K-nearest neighbor algorithm. The convolution operation is then carried out on the edge and relative position relationship of the adjacent point pairs in the graph structure. Consequently
the more detailed local features implicit in the point cloud data are extracted. The graph structure in the GCN model is not fixed. It is dynamically updated and the graph convolution module can be stacked numerous times in the network to further perceive the local characteristics of point cloud data. In the network pooling layer
a hybrid pooling structure is adopted composed of two parallel pooling channels to obtain the global feature vectors. The maximum pooling channel is used to obtain the maximal feature vector
while another maximum-average pooling channel is used to obtain a synthetic feature concerning the maximal and mean feature vectors. The acquired characteristic vectors are concatenated to obtain the final global feature vector of the network. Consequently
the network provides good robustness for the jittered data.
Result
2
The datasets ModelNet40
ShapeNet
and Stanford 3D indoor semantics (S3DIS) are mostly used for testing the performance of classification
partial segmentation
and semantic scene segmentation. Several experiments are carried out on the above three datasets to validate the performance of the model. In the classification experiment of ModelNet40
the proposed model achieves a better classification effect compared with the other competitive models. The overall accuracy and average classification accuracy are improved by 4% and 3.7%
respectively
compared with PointNet. In the partial segmentation ShapeNet dataset
the mean intersection-over-union (mIoU) is used as the index for evaluating model segmentation performance. In the comparison test
the proposed model in this study also obtains a satisfactory segmentation result. Specifically
our model's mIoU is 1.4% higher than that of PointNet. In S3DIS indoor scene dataset
our model's mIoU is 9.8% higher than PointNet. Furthermore
different pooling functions are tested and investigated to verify the effectiveness of the proposed hybrid pooling function in this study. Results show that the proposed hybrid pooling function in this study improves the average classification accuracy
exhibiting a 0.9% increase compared with the pooling function by PointNet.
Conclusion
2
Experimental results show that the local features of point cloud data can be effectively extracted by introducing GCN into the network model. The hybrid pooling function also yields great improvement in generating global characteristics with additional information. In general
the proposed network model can effectively obtain the global and local features of point cloud data and achieve better classification and segmentation effects.
Armeni I, Sener O, Zamir A R, Jiang H, Brilakis I, Fischer M and Savarese S. 2016. 3D semantic parsing of large-scale indoor spaces//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE: 1534-1543[ DOI: 10.1109/CVPR.2016.170 http://dx.doi.org/10.1109/CVPR.2016.170 ]
Bai J, Si Q L and Qin F W. 2019. Lightweight real-time point cloud classification network LightPointNet. Journal of Computer-aided Design and Graphics, 31(4):612-621
白静, 司庆龙, 秦飞巍. 2019.轻量级实时点云分类网络LightPointNet.计算机辅助设计与图形学学报, 31(4):612-621)[DOI:10.3724/SP.J.1089.2019.17328]
Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2018. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834-848[DOI:10.1109/TPAMI.2017.2699184]
Jaderberg M, Simonyan K, Zisserman A and Kavukcuoglu K. 2015. Spatial transformer networks//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press: 2017-2025
Jiang M Y, Wu Y R, Zhao T Q, Zhao Z L and Lu C W. 2018. PointSIFT: a SIFT-like network module for 3D point cloud semantic segmentation[EB/OL]. https://arxiv.org/pdf/1807.00652.pdf https://arxiv.org/pdf/1807.00652.pdf
Klokov R and Lempitsky V. 2017. Escape from cells: deep Kd-networks for the recognition of 3D point cloud models//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 863-872[ DOI: 10.1109/ICCV.2017.99 http://dx.doi.org/10.1109/ICCV.2017.99 ]
Li Y Y, Bu R, Sun M C, Wu W, Di X H and Chen B Q. 2018. PointCNN: convolution On X -transformed points[EB/OL]. https://arxiv.org/pdf/1801.07791.pdf https://arxiv.org/pdf/1801.07791.pdf .
Maturana D and Scherer S. 2015. VoxNet: a 3D convolutional neural network for real-time object recognition//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg, Germany: IEEE: 922-928[ DOI: 10.1109/IROS.2015.7353481 http://dx.doi.org/10.1109/IROS.2015.7353481 ]
Qi C R, Su H, Kaichun M and Guibas L J. 2017a. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 77-85[ DOI: 10.1109/CVPR.2017.16 http://dx.doi.org/10.1109/CVPR.2017.16 ]
Qi C R, Su H, Nieβner M, Dai A, Yan M Y and Guibas L J. 2016. Volumetric and multi-view CNNS for object classification on 3D data//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE: 5648-5656[ DOI: 10.1109/CVPR.2016.609 http://dx.doi.org/10.1109/CVPR.2016.609 ]
Qi C R,Yi L, Su H and Guibas L J. 2017b. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. https://arxiv.org/pdf/1706.02413.pdf https://arxiv.org/pdf/1706.02413.pdf
Ren M W, Niu L and Fang Y. 2017. 3D-A-Nets: 3D deep dense descriptor for volumetric shapes with adversarial networks[EB/OL]. https://arxiv.org/pdf/1711.10108.pdf https://arxiv.org/pdf/1711.10108.pdf
Su H, Maji S, Kalogerakis E and Learned-Miller E. 2015. Multi-view convolutional neural networks for 3D shape recognition//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 945-953[ DOI: 10.1109/ICCV.2015.114 http://dx.doi.org/10.1109/ICCV.2015.114 ]
Wang R, Xue X Y, Ping X J, Niu S Z and Zhang T. 2018. Steganalysis of JPEG images based on image classification and segmentation. Journal ofImage and Graphics, 23(10):1472-1482
汪然, 薛小燕, 平西建, 牛少彰, 张涛. 2018.分类与分割相结合的JPEG图像隐写分析.中国图象图形学报, 23(10):1472-1482)[DOI:10.11834/jig.180037]
Wang Y, Sun Y B, Liu Z W, Sarma S E, Bronstein M M and Solomon J M. 2018. Dynamic graph CNN for learning on point clouds[EB/OL]. https: //arxiv.org/pdf/1801.07829.pdf
Wen W W, Wen G J, Hui B W and Chen D X. 2019. Model library construction by combining global and local surfaces for 3D object recognition. Journal of Image and Graphics, 24(2):248-257
文威威, 文贡坚, 回丙伟, 陈鼎新. 2019.结合全局与局部信息的点云目标识别模型库构建.中国图象图形学报, 24(2):248-257)[DOI:10.11834/jig.180270]
Wu Z R, Song S R, Khosla A, Yu F, Zhang L G, Tang X O and Xiao J X. 2015. 3D ShapeNets: a deep representation for volumetric shapes//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE: 1912-1920[ DOI: 10.1109/CVPR.2015.7298801 http://dx.doi.org/10.1109/CVPR.2015.7298801 ]
Yi L, Kim V G, Ceylan D, Shen I C, Yan M Y, Su H, Lu C W, Huang Q X, Sheffer A and Guibas L. 2016. A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 35(6):210[DOI:10.1145/2980179.2980238]
Yu B, Yin H T and Zhu Z X. 2017. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting//Proceedings of the 27th International Joint Conference on Artificial Intelligence.[s.l.]: IJCAI: 3634-3640[ DOI: 10.24963/ijcai.2018/505 http://dx.doi.org/10.24963/ijcai.2018/505 ]
相关作者
相关机构
京公网安备11010802024621
