Extended pointwise convolution network model for point cloud classification and segmentation

Xinliang Zhang; Chenlin Fu; Yunji Zhao

doi:10.11834/jig.190508

Image Analysis and Recognition | Views : 0 下载量: 31 CSCD: 3

PDF
Export
Share
Collection
Album

Extended pointwise convolution network model for point cloud classification and segmentation
Vol. 25, Issue 8, Pages: 1551-1557(2020)
Received：04 October 2019，

Revised：2019-12-13，

Accepted：20 December 2019，

Published：16 August 2020
DOI： 10.11834/jig.190508
稿件说明：

移动端阅览

DOI：

Xinliang Zhang, Chenlin Fu, Yunji Zhao. Extended pointwise convolution network model for point cloud classification and segmentation[J]. Journal of Image and Graphics, 2020, 25(8): 1551-1557. DOI： 10.11834/jig.190508.

摘要

目的

点态卷积网络用于点云分类分割任务时，由于点态卷积算子可直接处理点云数据，逐点提取局部特征向量，解决了结构化点云带来的维度剧增和信息丢失等问题。但是为了保持点云数据结构的一致性，点态卷积算子及卷积网络模型本身并不具有描述点云全局特征的结构，因此，对点态卷积网络模型进行扩展，扩展后的模型具有的全局特征是保证分类分割准确性的重要依据。

方法

构造中心点放射模型来描述点云逐点相对于全局的几何约束关系，将其引入到点态卷积网络的特征拼接环节扩展特征向量，从而为点态卷积网络构建完善的局部—全局特征描述，用于点云数据的分类分割任务。首先，将点云视为由中心点以一定方向和距离放射到物体表面的点的集合，由中心点指向点云各点的放射矢量，其矢量大小确定了各点所存在的曲面和对于中心点的紧密程度，矢量方向描述了各点对于中心点的包围方向及存在的射线。进而由点云中的坐标信息得到点云的中心点，逐点计算放射矢量构造中心点放射模型，用以描述点云的全局特征。然后，利用点云数据的坐标信息来检索点的属性，确定卷积中参与特定点卷积运算的邻域，点态卷积算子遍历点云各点，输出逐点局部特征，进一步经多层点态卷积操作得到不同深度上的局部特征描述。最后，将中心点放射模型的全局特征和点态卷积的局部特征拼接，完成特征扩展，得到点态卷积网络的扩展模型。拼接后的局部—全局特征输入全连接层用于类标签预测，输入点态卷积层用于逐点标签预测。

结果

在ModelNet40和S3DIS(Stanford large-scale 3D indoor spaces dataset)数据集上分别进行实验，验证模型的分类分割性能。在ModelNet40的分类实验中，与点态卷积网络相比，扩展后的网络模型在整体分类精度和类属分类精度上分别提高1.8%和3.5%，在S3DIS数据集的分割实验中，扩展后的点态卷积网络模型整体分割精度和，类属分割精度分别提高0.7%和2.2%。

结论

引入的中心点放射模型可以有效获取点云数据的全局特征，扩展后的点态卷积网络模型实现了更优的分类和分割效果。

Abstract

Objective

A pointwise convolution network (PCN) has great potential for the classification and segmentation of point cloud data. The pointwise convolution operator in each layer directly functions on the point cloud data to generate the local feature vectors point-by-point. The PCN can solve the problems caused by dimension increase and information loss because of the avoidance of point data structuration. However

the structure of PCN is responsible for the consistent maintenance of the point cloud during the implementation of pointwise convolution. Consequently

the PCN has no constituent that can describe the global features of point cloud. The PCN essentially needs a characteristic expansion considering the global features in terms of classification and segmentation accuracies for the point cloud data.

Method

A so-called central point radiation model is proposed in this paper to pointwisely describe the global geometric properties. With the introduction of the radiation model into feature concatenation

the result ant extended PCN (EPCN) realizes a complete representation of local and global features for point cloud classification and segmentation. First

the point cloud in the central point radiation model is regarded as a set of projection point on the object surface by a radiation line from the central point. The amplitude of radiation vector determines the owner surface and tightness of each point around the central point

and the direction specifies the pointwise encircling direction and contributing radiation line. The central point is prescribed by the calculation of coordinate information in the point cloud

and the pointwise radiation vector is generated. Thus

the central point radiation model is constructed to describe global features. Second

the local feature description of the point cloud data at each depth is obtained through a multilayer pointwise convolution. The coordinates of the point cloud data provide an index for retrieving point attributes and determining the neighborhood points involved in the convolution of a concerned point. The pointwise convolution traverses all points in the point cloud and completely yields the pointwise local features. Finally

an EPCN is derived through the concatenation of the global features from the central point radiation model and the local features from the pointwise convolution. The complete feature description vector is adopted as the input of the fully connected layer of PCN for the category label prediction

and the input of the pointwise convolution layer is used for the label prediction point-by-point.

Result

The classification and segmentation performance of the proposed EPCN model are validated on the point cloud data sets of ModelNet40 and S3DIS(Stanford large-scale 3D indoor dataset)

respectively. The EPCN classification on ModelNet40 leads to 1.8% and 3.5% increases in mean and

-class accuracies compared with the PCN. The EPCN segmentation on S3DIS provides 0.7% and 2.2% increases in

-mean and

-class accuracies compared with the PCN.

Conclusion

Experimental results verify the effectiveness of the proposed central point radiation model for the global feature extraction of the point cloud data. The EPCN model provides an improved performance in terms of classification and segmentation.

关键词

Keywords

references

Armeni I, Sener O, Zamir A R, Jiang H, Brilakis I, Fischer M and Savarese S. 2016.3D semantic parsing of large-scale indoor spaces//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 1534-1543[ DOI: 10.1109/CVPR.2016.170 http://dx.doi.org/10.1109/CVPR.2016.170 ]

Cavaliere D, Loia V, Saggese A, Senatore S and Vento M. 2019. Semantically enhanced UAVs to increase the aerial scene understanding. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 49(3):555-567[DOI:10.1109/TSMC.2017.2757462]

Charles R Q, Su H, Kaichun M and Guibas L J. 2017. PointNet: deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 77-85[ DOI: 10.1109/CVPR.2017.16 http://dx.doi.org/10.1109/CVPR.2017.16 ]

Davison A J, Reid I D, Molton N D and Stasse O. 2007. MonoSLAM:real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6):1052-1067[DOI:10.1109/TPAMI.2007.1049]

Feng Y T, Feng Y F, You H X, Zhao X B and Gao Y. 2018. MeshNet: mesh neural network for 3D shape representation[EB/OL].[2019-09-27] . https://arxiv.org/pdf/1811.11424.pdf https://arxiv.org/pdf/1811.11424.pdf

Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V and Garcia-Rodriguez J. 2017. A review on deep learning techniques applied to semantic segmentation[EB/OL].[2019-09-27] . https://arxiv.org/pdf/1704.06857.pdf https://arxiv.org/pdf/1704.06857.pdf

HanockaR, Hertz A, Fish N, Giryes R, Fleishman S and Cohen-Or D. 2019. MeshCNN: a network with an edge[EB/OL].[2019-09-27] . https://arxiv.org/pdf/1809.05910.pdf https://arxiv.org/pdf/1809.05910.pdf

Hua B S, Tran M K and Yeung S K. 2018. Pointwise convolutional neural networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 984-993[ DOI: 10.1109/CVPR.2018.00109 http://dx.doi.org/10.1109/CVPR.2018.00109 ]

Huang J and You S Y. 2016. Point cloud labeling using 3D convolutional neural network//Proceedings of the 23rd International Conference on Pattern Recognition. Cancun: IEEE: 2670-2675[ DOI: 10.1109/ICPR.2016.7900038 http://dx.doi.org/10.1109/ICPR.2016.7900038 ]

Klokov R and Lempitsky V. 2017. Escape from cells: deep Kd-networks for the recognition of 3D point cloud models//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 863-872[ DOI: 10.1109/ICCV.2017.99 http://dx.doi.org/10.1109/ICCV.2017.99 ]

Le T and Duan Y. 2018. PointGrid: a deep network for 3D shape understanding//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 9204-9214[ DOI: 10.1109/CVPR.2018.00959 http://dx.doi.org/10.1109/CVPR.2018.00959 ]

Li Y Y, Bu R, Sun M C, Wu W, Di X H and Chen B Q. 2018. PointCNN: convolution on X -transformed points//Proceedings of the Neural Information Processing Systems.[s.l.]: [s.n.] : 820-830

Maturana D and Scherer S. 2015. VoxNet: a 3D convolutional neural network for real-time object recognition//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg: IEEE: 922-928[ DOI: 10.1109/IROS.2015.7353481 http://dx.doi.org/10.1109/IROS.2015.7353481 ]

Qiu Z S, Zhuang Y, Yan F, Hu H S and Wang W. 2019. RGB-DI images and full convolution neural network-based outdoor scene understanding for mobile robots. IEEE Transactions on Instrumentation and Measurement, 68(1):27-37[DOI:10.1109/TIM.2018.2834085]

Su H, Maji S, Kalogerakis E and Learned-Miller E. 2015. Multi-view convolutional neural networks for 3D Shape recognition//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 945-953[ DOI: 10.1109/ICCV.2015.114 http://dx.doi.org/10.1109/ICCV.2015.114 ]

Wang P S, Liu Y, Guo Y X, Sun C Y and Tong X. 2017. O-CNN: octree-based convolutional neural networks for 3D shape analysis[EB/OL].[2019-09-27] . https://arxiv.org/pdf/17112.01537.pdf https://arxiv.org/pdf/17112.01537.pdf

Wu Z R, Song S R, Khosla A, Yu F, Zhang L G, Tang X O and Xiao J X. 2015.3D ShapeNets: a deep representation for volumetric shapes//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 1912-1920[ DOI: 10.1109/CVPR.2015.7298801 http://dx.doi.org/10.1109/CVPR.2015.7298801 ]

Zhou Y and Tuzel O. 2018. VoxelNet: end-to-end learning for point cloud based 3D object detection//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4490-4499[ DOI: 10.1109/CVPR.2018.00472 http://dx.doi.org/10.1109/CVPR.2018.00472 ]

Alert me when the article has been cited

提交

Dynamic graph convolution with spatial attention for point cloud classification and segmentation

Classification network for 3D point cloud based on spatial structure convolution and attention mechanism

Feature extraction of flat target based on point cloud attribute information

Enhanced attention-based joint semantic instance segmentation network for point clouds

Large-model driven test-time adaptation for multi-modal point cloud semantic segmentation