Current Issue Cover
自监督深度残差函数映射网络的3维模型对应关系计算

杨军, 马中昌(兰州交通大学电子与信息工程学院, 兰州 730070)

摘 要
目的 针对传统非刚性3维模型的对应关系计算方法需要模型间真实对应关系监督的缺点,提出一种自监督深度残差函数映射网络(self-supervised deep residual functional maps network,SSDRFMN)。方法 首先将局部坐标系与直方图结合以计算3维模型的特征描述符,即方向直方图签名(signature of histograms of orientations,SHOT)描述符;其次将源模型与目标模型的SHOT描述符输入SSDRFMN,利用深度函数映射(deep functional maps,DFM)层计算两个模型间的函数映射矩阵,并通过模糊对应层将函数映射关系转换为点到点的对应关系;最后利用自监督损失函数计算模型间的测地距离误差,对计算出的对应关系进行评估。结果 实验结果表明,在MPI-FAUST数据集上,本文算法相比于有监督的深度函数映射(supervised deep functional maps,SDFM)算法,人体模型对应关系的测地误差减小了1.45;相比于频谱上采样(spectral upsampling,SU)算法减小了1.67。在TOSCA数据集上,本文算法相比于SDFM算法,狗、猫和狼等模型的对应关系的测地误差分别减小了3.13、0.98和1.89;相比于SU算法分别减小了2.81、2.22和1.11,并有效克服了已有深度函数映射方法需要模型间的真实对应关系来监督的缺点,使得该方法可以适用于不同的数据集,可扩展性大幅增强。结论 本文通过自监督深度残差函数映射网络训练模型的方向直方图签名描述符,提升了模型对应关系的准确率。本文方法可以适应于不同的数据集,相比传统方法,普适性较好。
关键词
Correspondence calculation of 3D models by a self-supervised deep residual functional maps network

Yang Jun, Ma Zhongchang(School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China)

Abstract
Objective Calculating 3D shapes correspondence is a central problem in the field of geometry processing that plays an important role in shapes reconstruction, object recognition and classification, and other tasks. Therefore, finding a meaningful and accurate correspondence among shapes has important research significance and application value. In recent years, deep-learning-based calculation of the correspondence among 3D shapes has attracted the attention of many scholars in the field of geometry processing. Using neural networks to learn the feature descriptors on the surfaces of 3D shapes can help us obtain accurate and comprehensive feature information and provides a solid foundation for building accurate correspondences among 3D shapes. Deep-learning-based methods for calculating the correspondences can be roughly divided into 1) methods based on the end-to-end depth functional maps network, and 2) methods based on other neural networks. The first method uses deep functional map networks to learn the feature descriptors of 3D shapes and then use these descriptors to analyze the spatial structure characteristics of shapes. Afterward, theory of functional maps is applied to solve the shape matching problem, find out the functional maps matrix among shapes, and determine the correspondence among shapes. Meanwhile, the second method uses neural networks to learn the feature descriptors of shapes, takes the calculation of correspondences as part of the learning process, uses the learned shape spatial geometric structure features for shape matching, and obtains the ideal correspondence among shapes. The feature descriptors and the network of learning descriptors of shapes play a crucial role in calculating 3D shapes correspondence calculation methods ignore the important influence of feature descriptors on the representation of 3D shapes, and the calculated shape descriptors contain relatively few information that cannot solve the problems related to shape symmetry and shape boundary descriptor distortion. Moreover, in the subsequent correspondence calculation process, these methods are unable to generate an accurate functional map of the symmetrical part of shapes, thereby leading to inaccurate correspondence calculations. The existing 3D shape correspondence calculation methods based on deep learning all adopt a supervision mechanism, which limits the universality of these methods to a large extent. To address these problems, this paper proposes a self-supervised deep residual functional maps network (SSDRFMN) to calculate 3D shapes correspondence. Method The proposed method involves two steps. First, we calculate the feature descriptor of the 3D shape by combining the local coordinate system with a histogram, which is a signature of the histograms of orientations (SHOT) descriptor. We initially establish a local coordinate on the surface of shapes and then enhance the recognition ability of our descriptor by introducing the geometric information of the feature points. Afterward, we calculate the local histogram at a given point and use the calculated geometric information to form a histogram and a signature. Compared with traditional feature descriptors, hybrid feature descriptors can better represent the spatial structure and surface feature information of 3D shapes and provide high-quality inputs for network learning. Second, we use end-to-end SSDRFMN to calculate the correspondence among shapes. The SHOT descriptors of the source and target shapes are inputted into SSDRFMN, and the feature descriptor iteratively trains the neural network. The deep functional maps (DFM) layer is then used to calculate the function mapping matrix between two shapes. The corresponding relationship problem is transformed into solving the function mapping matrix problem, and the functional map relationship is converted into a point-to-point correspondence relationship through a fuzzy correspondence layer. The self-supervised loss function is then used to calculate the geodesic distance error between shapes and to evaluate the corresponding relationship. The loss function minimizes the geodesic distance error between shapes via network training and replaces the real correspondence between the manually labeled models with geometric constraints to achieve a self-supervised learning of the network. Result Experimental results show that compared with the supervised deep functional maps (SDFM) and spectral upsampling (SU) algorithms, the proposed algorithm reduces the geodesic error of correspondences between the human model and the MPI-FAUST dataset by 1.45% and 1.67%, respectively. Meanwhile, in the TOSCA dataset, the proposed algorithm reduces the geodesic errors of the correspondences of dog, cat, and wolf models by 3.13% (2.81%), 0.98% (2.22%), and 1.89% (1.11%) compared with SDFM (SU), respectively. Therefore, apart from its applicability to different datasets and its high scalability, this algorithm effectively overcomes the shortcomings of extant depth functional maps methods that require the true correspondence among shapes to supervise. Conclusion Experimental results show that the proposed method outperforms the existing methods for calculating the correspondence among 3D shapes. On the one hand, our proposed method trains the neural network through the SHOT descriptor of shapes, there by effectively solving the symmetry and boundary distortion of the model and producing a better representation of the surface features of 3D shapes. On the other hand, the proposed method uses a self-supervised deep neural network to learn the features of descriptors and to accurately calculate the correspondences among shapes. This method also shows excellent universality and accuracy, thereby highlighting its value in shapes matching, model recognition, segmentation, and retrieval.
Keywords

订阅号|日报